Lead Data Engineer (GCP) - IVID TEK INC
Irving, TX 75038
About the Job
Position: Lead Data Engineer (GCP,Azure)
Location: Dallas, TX
Job Type: Contract C2C
Rate: 65-70/ Hr
Location: Dallas, TX
Job Type: Contract C2C
Rate: 65-70/ Hr
Roles and Responsibilities:
Key Skills: Vertex AI , GCP , Kubeflow , CI/CD , Python , BigData
Job Description:
- Identify new opportunities to improve business processes and improve consumer experiences, and prototype solutions to demonstrate value with a crawl, walk, run mindset.
- Work with data scientists and analysts to create and deploy new product features on the ecommerce website, in-store portals and the Client's mobile app
- Implement end-to-end solutions across the full breadth of ML model development lifecycle. The specific role includes working hand in hand with the scientists from the point of data exploration for model development to the point of building features, mls and deploying them in production. You will have an opportunity to work on both batch and real time models. The role also involves operational support.
- Establish scalable, efficient, automated processes for data analyses, model development, validation and implementation
- Write efficient and scalable software to ship products in an iterative, continual-release environment
- Contribute to and promote good software engineering practices across the team and build cloud native software for ML pipelines
- Contribute to and re-use community best practices
- Embody the values and passions that characterize client., with empathy to engage with colleagues from multiple backgrounds
About You
- University or advanced degree in engineering, computer science, mathematics, or a related field
- 10+ years' experience developing and deploying machine learning systems into production, and independent contributor.
- Comfortable with Python ecosystem, vscode, jupyternotebooks.
- Experience working with big data tools: Hadoop, Spark, Kafka, etc.
- Experience with at least one cloud provider solution (AWS, GCP) and understanding of serverless code development (GCP preferred)
- Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc. (Python preferred)
- CI/CD expert. And can work on GitHub actions, harness, Jenkins
- Can work with Google Big Query, or similar warehouse.
- Work on Kubeflow pipelines independently and propose standards.
- Knowledge of Feature Engineering, Feature Store, and audit capabilities.
- Expertise in standard software engineering methodology, e.g. unit testing, test automation, continuous integration, code reviews, design documentation
- Working experience with native ML orchestration systems such as Kubeflow, Step Functions, MLflow, Airflow, TFX...
- Relevant working experience with Docker and Kubernetes is a big plus
- Review and integrate all application requirements, including functional, security, integration, performance, quality and operations requirements.
- Experience on Data Analytics Solutions Cloud Architect experience of at least 3-4 years in development using Cloud Technologies and awareness of different processes around Big Data platforms GCP Experience of 1-2 years on BigQuery, GCS, Dataproc, Composer, Scheduler, Pub/Sub, Dataflow, Cloud Function Proficient in Python scripting language 1-2 years of development experience and 1 year of experience Professional Certification Google Cloud Platform
- Hands on experience in Azure Cloud components such as Azure Data Factory, Azure Logic apps service, Data Bricks, AKS, Azure devops or any Cloud technologies
- Provide technical leadership to the project team to perform design to deployment related activities, provide guidance, perform reviews, prevent and resolve technical issues.
- Hands on experience with GCP services using GCP Console or Deployment Manager
- Excellent communication written and oral and interpersonal skills
Source : IVID TEK INC
