Data Engineer from Collabera
Irving, TX
About the Job
Responsibilities:
- Design, develop, and maintain big data architectures and pipelines using technologies like Hadoop, Spark, Kafka, and cloud platforms (AWS, Azure, GCP).
- Ingest data from various sources (e.g., databases, APIs, IoT devices) using ETL tools and frameworks.
- Clean, transform, and enrich data to ensure data quality and consistency.
- Develop and optimize data pipelines for real-time and batch processing.
- Collaborate with data scientists and analysts to identify and address data-related challenges.
- Implement data governance and security best practices to protect sensitive data.
- Monitor and optimize the performance of big data systems.
- Stay up-to-date with the latest trends and technologies in the big data field.
Requirements:
- 5+ years of experience as a Big Data Engineer or a similar role.
- Strong proficiency in programming languages like Python, Java, or Scala.
- In-depth knowledge of big data technologies such as Hadoop, Spark, Kafka, HBase, and Hive.
- Experience with cloud platforms (AWS, Azure, GCP) and their big data services.
- Familiarity with data warehousing and ETL concepts.
- Understanding of machine learning and AI algorithms.
- Excellent problem-solving and analytical skills.
- Strong communication and collaboration skills.
- Ability to work independently and as part of a team.
Salary
55 - 65 /hour
