Data Engineer - Chicago, IL - Georgia IT Inc.
Chicago, IL
About the Job
Job Title: Data Engineer
Location: Chicago, IL
Position Type: 12 Months Contract
US Citizen/ GC
Job Description
Responsibilities
Location: Chicago, IL
Position Type: 12 Months Contract
US Citizen/ GC
Job Description
Responsibilities
- Work closely with system architects and distributed computing engineers and reporting/data warehouse developers to create and optimize the architecture of the streaming data pipeline
- Design and build the streaming data pipeline to transform, aggregate and store the data
- Always angle for greater efficiency and robustness of the pipeline to support big volume of data stream
- Graduate degree in Computer Science, Information Systems or equivalent quantitative field and 5+ years of experience in a similar Data Engineer role.
- Demonstrated ability to build processes that support data stream consumption, data transformation, data structures, metadata, dependency and workload management
- Strong interpersonal skills and ability to project manage and work with cross-functional teams
- Experience building and optimizing 'big data' data pipelines, architectures and data sets.
- Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
- Major Hadoop ecosystem distributions such as HDP, Cloudera etc. HDP is preferred
- Public cloud such as Azure, AWS etc. Azure is preferred
- Json document processing
- Apache Hive and HBase
- Apache Kafka
- Stream-processing systems such as Storm and Spark-Streaming
- Object-oriented/object function scripting languages such as Python, Java etc.
Source : Georgia IT Inc.