Senior Data Engineer - Veear Projects Inc
Menlo Park, CA 94025
About the Job
Data Engineer Responsibilities:
- Partner with leadership, engineers, program managers and data scientists to understand data needs.
- Design, build and launch extremely efficient and reliable data pipelines to move data across a number of platforms including Data Warehouse, online caches and real-time systems.
- Communicate, at scale, through multiple mediums: Presentations, dashboards, company-wide datasets, bots and more.
- Educate your partners: Use your data and analytics experience to 'see what's missing', identifying and addressing gaps in their existing logging and processes.
- Leverage data and business principles to solve large scale web, mobile and data infrastructure problems.
- Build data expertise and own data quality for your areas.
- 5+ years of Python development experience.
- 5+ years of SQL experience.
- 3+ years' experience with Data Modeling.
- 5+ years' experience in custom ETL design, implementation and maintenance.
- Experience with more than one coding language.
- Experience with designing and implementing real-time pipelines.
- Experience with data quality and validation.
- Experience with SQL performance tuning and E2E process optimization.
- Experience with anomaly/outlier detection.
- Experience with notebook-based Data Science workflow.
- Experience with Airflow.
- Experience querying massive datasets using Spark, Presto, Hive, Impala, etc.
Source : Veear Projects Inc
