Big Data Engineer - Detroit, MI - Georgia IT Inc.
Detroit, MI
About the Job
Big Data Engineer
Location: Detroit, MI
Duration: Longterm
Rate: DOE
US Citizen, Green Card, TN, GC-EAD, and H4-EAD only No Third-party agencies corp to corp.
What You Will Do
Education:
Location: Detroit, MI
Duration: Longterm
Rate: DOE
US Citizen, Green Card, TN, GC-EAD, and H4-EAD only No Third-party agencies corp to corp.
What You Will Do
- Develop ELT processes from various data repositories and APIs across the enterprise, ensuring data quality and process efficiency
- Develop data processing scripts using Spark
- Develop relational and NoSQL data models to help conform data to meet users' needs using Hive and HBase
- Integrate platform into existing EDW and various operational systems
- Develop administration processes to monitor cluster performance, resource usage, backup, and mirroring to ensure a highly available platform
- Address performance and scalability issues in a large-scale data lake environment
- Provide big data platform support and issues resolutions to Data Scientists and fellow engineers
- 2 years of experience with a Hadoop distribution and ecosystem tools such as Hive, Spark, Nifi and Oozie
- 2 years of experience developing batch and streaming ETL/ELT processes
- 2 years of experience with relational and NoSQL databases, including modeling and writing complex queries
- Proficiency in at least one programming language such as Python or Java
- Excellent communication, analytical, and problem-solving skills
- Experience with Linux system administration, Linux scripting, and basic network skills
- AWS experience
Education:
- Master's degree in Computer Science, Software Engineering or closely related field
Source : Georgia IT Inc.
