Cloud Data Engineer - Avispa Technology

Cloud Data Engineer ROCGJP00027944

A leading biotechnology company is seeking a highly skilled and motivated Cloud Data Engineer to join our growing data team. The ideal candidate will have a strong background in developing data pipelines, implementing data models, and executing best practices for developing data products. This role requires expertise in AWS, GitLab CI/CD, dbt, Snowflake, SQL, Python, Git, and DevOps, as well as hands-on experience with orchestrators such as Airflow and AutoMateNow, and data governance technologies like Monte Carlo and Collibra.

Cloud Data Engineer Pay and Benefits:

• Hourly pay: $70-$80/hr (pay varies based on candidate's experience)
• Worksite: Leading biotechnology company (South San Francisco, CA 94080)
• W2 Employment, Group Medical, Dental, Vision, Life, Retirement Savings Program, PSL
• 40 hours/week, 7 Month Assignment

Cloud Data Engineer Responsibilities:

• Develop Data Pipelines: Design, develop, and maintain robust, scalable, and efficient ETL/ELT pipelines to support diverse data sources and large-scale data processing.
• Data Modeling: Create and maintain data models and data architecture to ensure data integrity and optimum performance.
• Best Practices Implementation: Apply industry best practices in data warehousing, data governance, and data lifecycle management.
• Collaboration: Work closely with data scientists, analysts, and other stakeholders to gather requirements and deliver high-quality data products.
• Automation and CI/CD: Implement CI/CD pipelines using GitLab for automated testing, deployment, and integration of data workflows.
• Database Management: Manage and optimize Snowflake databases for performance, scalability, and cost-effectiveness.
• Coding and Scripting: Write efficient SQL queries and Python scripts to extract, load, and transform data.
• Version Control: Utilize Git for version control, ensuring traceable and manageable code changes.
• Orchestration: Utilize orchestration tools like Airflow and AutoMateNow to manage and automate data workflows.
• Data Governance: Implement and manage data governance and data observability solutions using technologies like Monte Carlo and Collibra.
• Monitoring and Optimization: Monitor data pipeline performance and troubleshoot issues to ensure reliability and efficiency.
• Documentation: Maintain comprehensive documentation for data pipelines, models, and processes.

Cloud Data Engineer Qualifications:

• 5+ years of experience in data engineering or a similar role.
• Bachelor’s degree in Computer Science, Information Technology, Data Science, or a related field.
• AWS Certified Data Analytics – Specialty or AWS Certified Solutions Architect.
• Proven experience with AWS cloud services related to data processing, such as S3, Redshift, Lambda, Glue, and Data Pipeline.
• Proficiency in designing and implementing CI/CD pipelines with GitLab.
• Experience with dbt (data build tool) for transforming data within the warehouse.
• Experience with orchestrators like Airflow and AutoMateNow.
• Experience with real-time data processing and streaming technologies (e.g., Kafka, Kinesis).
• Experience in a regulated industry
• Experience in a manufacturing environment
• Strong knowledge of Snowflake with hands-on experience in data warehousing solutions.
• Expertise in SQL for data querying, manipulation, and optimization.
• Proficiency in Python for scripting and automation tasks.
• Familiarity with DevOps practices and tools.
• Knowledge of data governance and observability tools such as Monte Carlo and Collibra.
• Strong working knowledge of Git for version control.
• Strong problem-solving and analytical skills.
• Excellent communication skills, both written and verbal.
• Ability to work collaboratively in a fast-paced, dynamic environment.
• High attention to detail and commitment to producing high-quality work.