Data Engineer - American Technology Consulting
Urbandale, IA
About the Job
*****US CITIZENS ON W2 BASIS ONLY*****Primary Roles• Create and update API interfaces to MySQL backend databases. The API is hostedon AWS ECS and written using Python.• Ingest data into the Enterprise Data Lake from various database/file sources• Transform data and create common tables using Pyspark and SQL• Load data into MySQL from the Enterprise Data Lake• Analyze data using SQL and Pyspark• Create and maintain updates to AWS resourcesSupporting Roles• Work closely with the front end developer on the API specification• Build test scripts in Python to test data replication processes• Work closely with the product manager and end users to understand requirementsRequired Skills * Experience requirements - at least 3 - 5 years of experience with SQL, Python and cloud computing (AWS preferred) * Languages and Frameworks: Python, SQL, Pyspark * Core understanding of AWS services including IAM, Lambda, S3, RDS, ECS and EC2 * Experience developing Databricks jobs and pipelines * Other: Docker, containerization concepts, REST API concepts (calling REST APIs using Python)Additional SkillsTerraform, Github Actions, Datadog, AWS Database Migration Service, ApacheAirflow, REST API development in Python
Source : American Technology Consulting