Data Engineer Level 2 - Apidel Technologies

Job Description

About Us:

We are a full stack datascience company and a wholly owned subsidiary of The  Company. We own 10 Petabytes of data, andcollect 35%2B Terabytes of new data each week sourced from 62 Million households.As a member of our software engineering team you will use various cutting-edgetechnologies to develop applications that turn our data into actionableinsights used to personalize the customer experience for shoppers at .

What you\'ll do:

As a Senior Data Engineer, youare part of the software development team. We develop strategies and solutionsto ingest, store, and distribute our big data. Our developers use Big Datatechnologies including (but not limited to) PySpark, Hive, JSON, and SQL todevelop products, tools and software features.

Responsibilities:

Take ownership of features anddrive them to completion through all phases of the entire 84.51 SDLC.

This includes external facingand internal applications as well as process improvement activities such as:

Participate in design of BigData platforms and SQL based solutions

Perform development of BigData platforms and SQL based solutions

Perform unit and integrationtesting

Partner with senior resources,gaining insights

Participate in retrospectivereviews

Participate in the estimationprocess for new work and releases

Be driven to improve yourselfand the way things are done

Minimum Skills Required:

Proven Big Data technologydevelopment experience including Hadoop, Spark (PySpark), and Hive

Understanding of AgilePrinciples (Scrum)

Experience developing withPython

Cloud Development (Azure)

Exposure to VCS (Git, SVN)

Position Specific SkillPreferences: Experience in the following:

Experience developing with SQL(Oracle, SQL Server)

Exposure to NoSQL (Mongo,Cassandra)

Apache NiFi

Airflow

Docker

Key Responsibilities

Innovate, develop, and drivethe development and communication of data strategy and roadmaps across thetechnology organization to support project portfolio and business strategy

Drive the development andcommunication of enterprise standards for data domains and data solutions,focusing on simplified integration and streamlined operational and analyticaluses

Drive digital innovation byleveraging innovative new technologies and approaches to renovate, extend, andtransform the existing core data assets, including SQL-based, NoSQL-based, andCloud-based data platforms

Define high-level migrationplans to address the gaps between the current and future state, typically insync with the budgeting or other capital planning processes Lead the analysisof the technology environment to detect critical deficiencies and recommendsolutions for improvement

Mentor team members in dataprinciples, patterns, processes and practices

Promote the reuse of dataassets, including the management of the data catalog for reference

Draft and review architecturaldiagrams, interface specifications and other design documents

Proactively and holisticallylead activities that create deliverables to guide the direction, development,and delivery of technological responses to targeted business outcomes.

Provide facilitation,analysis, and design tasks required for the development of an enterprise\'s dataand information architecture, focusing on data as an asset for the enterprise.

Develop target-state guidance(i.e., reusable standards, design patterns, guidelines, individual parts andconfigurations) to evolve the technical infrastructure related to data andinformation across the enterprise, including direct collaboration with 84.51.

Note to Vendors

Remote candidates accepted (Norestrictions)

Top Skills:

Azure

Databricks

Services Development

Project: This person will beworking on supporting Azure measurement pipelines/workflows for campaignperformance.

No prescreening

Interview: 1-2 rounds, panelformat

Desired start date: ASAP