Spark/Scala Developer - Umanist Staffing LLC

Job Title: Spark/Scala Developer

Location:

Brooklyn, OH

Job Function:

Technology

Job Summary:

We are seeking a skilled Spark/Scala Developer with proven experience in big data technologies. The ideal candidate will have a strong background in Scala, Spark, and SQL, with hands-on experience in building and managing data processing pipelines in a production environment.

Key Responsibilities:

Big Data Expertise:

Apply advanced knowledge of big data technologies such as Spark, Hadoop, or Elasticsearch. Demonstrate experience in either implementation or data science.

Scala and SQL Development:

Write efficient code in Scala and advanced SQL queries. Utilize Hive and Parquet for data processing.

GCP Data Lake:

Leverage experience working with Google Cloud Platform (GCP) data lakes.

Agile Environment:

Work effectively within an Agile framework, contributing to iterative development and continuous improvement.

Coding Skills:

Use your strong coding background in Java, Python, or Scala to develop and maintain robust data solutions.

Testing:

Utilize testing libraries such as Scala Test or equivalent for other languages. Understand and explain various test types, including unit and integration tests, and automate testing processes.

Data Pipelines:

Build and maintain data processing pipelines for production environments, including traditional ETL and analytics pipelines.

Desired Skills:

Expertise in Scala and Spark with practical experience in big data technologies.

Advanced SQL skills and experience with data formats such as Parquet.

Experience with GCP data lakes and Agile methodologies.

Proficiency in at least one major programming language (Java, Python, Scala).

Familiarity with testing libraries and concepts, including unit and integration testing.

Experience in developing and managing data pipelines for production environments.

Qualifications:

Bachelor’s Degree in Computer Science or a related field.