Senior Data Engineer -W2 - eTek IT Services, Inc.

Key Responsibilities:

Develop and maintain robust data pipelines using technologies such as Apache Spark, Python, and AWS services.

Design and implement data models for efficient storage and retrieval of structured and unstructured data.

Collaborate with cross-functional teams to understand data requirements and provide technical expertise.

Optimize and troubleshoot data processes to ensure high performance and reliability.

Create and maintain documentation for data engineering processes, workflows, and best practices.

Implement ETL processes to transform and load data from diverse sources into data warehouses or lakes.

Participate in code reviews and contribute to the continuous improvement of data engineering practices and standards.

Develop and maintain data quality and governance frameworks to ensure data accuracy and integrity.

Identify opportunities to enhance data infrastructure and propose innovative solutions to address business needs.

Stay current with industry trends and advancements in data engineering to drive continuous innovation.

Required Qualifications:

Bachelor's degree in Computer Science, Engineering, or a related field.

5+ years of professional experience in data engineering or a related role.

Proven expertise in Python and SQL for data manipulation, analysis, and scripting.

Extensive experience with Apache Spark for big data processing and analytics.

Strong understanding of data modeling principles and techniques.

Advanced knowledge of ETL processes and data warehousing concepts.

Experience with cloud platforms such as AWS for building and deploying data solutions.

Proficiency in data pipeline development using tools and frameworks like Apache Airflow or similar.

Solid understanding of distributed computing principles and big data technologies.

Excellent problem-solving skills and the ability to optimize data processes for performance and scalability.

Experience in implementing data governance and quality assurance measures.

Strong communication and collaboration skills to work effectively within a team and across departments.

Ability to multitask and manage priorities in a fast-paced, dynamic environment.

Relevant certifications in data engineering or cloud platforms are a plus.

Deep passion for data-driven decision-making and continuous learning in the field of data engineering.