Data Engineer - Oxenham Group LLC

Position Overview
We are seeking a Senior Data Engineer with extensive experience in data migrations, particularly in transitioning data environments from Microsoft Azure to Snowflake. The ideal candidate has a strong background in data engineering, cloud-based data storage solutions, and data pipeline automation. You'll play a key role in designing, implementing, and optimizing data migration strategies while ensuring data integrity and performance across platforms.

Key Responsibilities

• Data Migration: Lead the migration of data from Azure (including Azure SQL Database, Data Lake, etc.) to Snowflake, ensuring efficiency, reliability, and scalability throughout the process.
• Data Integration: Develop and maintain ETL (Extract, Transform, Load) processes to integrate data seamlessly across both Azure and Snowflake environments.
• Performance Optimization: Analyze and optimize data flows and performance on both Azure and Snowflake, adjusting resource allocation and data partitioning to enhance performance.
• Data Pipeline Development: Create automated data pipelines, leveraging tools such as Azure Data Factory, dbt, or Snowpipe, to support ongoing data ingestion and transformation requirements.
• Quality Assurance: Ensure data integrity and accuracy throughout the migration by implementing rigorous quality control and validation processes.
• Collaboration & Support: Collaborate with data analysts, data scientists, and business stakeholders to understand data needs and translate them into scalable solutions. Offer support and training to internal teams on the new Snowflake environment.
• Documentation: Maintain clear documentation of data processes, transformations, and migration procedures.

Required Skills & Qualifications

• Experience: Minimum of 5+ years in data engineering, with a proven record in data migration and integration projects.
• Technical Skills:
  • Cloud Platforms: Strong experience with Microsoft Azure or Snowflake (experience with both is a plus).
  • ETL & Data Pipeline Tools: Proficiency in tools such as Azure Data Factory, dbt, or Snowpipe.
  • Programming Languages: Proficient in Python, SQL, and one other language commonly used in data processing (e.g., Java, Scala).
• Database Management: Familiar with both relational databases and modern data lake structures, including experience with Azure SQL Database, Cosmos DB, and Snowflake's unique architecture.
• Problem-Solving: Strong analytical skills with the ability to troubleshoot complex data issues and optimize for performance.
• Collaboration: Excellent communication and collaboration skills, with a track record of working effectively in cross-functional teams.