Azure Data Engineer/Tester - V2Soft

Job Summary:

This position is for an Offshore Azure Data Tester to support FordDirect's OMNI project. This individual will be responsible for creating and executing various test cases that support the processing of data related to Ford and Lincoln consumers. The primary focus on this testing is to verify the accuracy of this data as various business rules are applied against it in an Azure data platform involving Databricks, Event Hub & Data Factory. This individual will be expected to understand the data flow, validate the content, and build an automation framework that supports a number of data quality checks.

Essential Job Functions:

• Help create test automation by following an existing data automation framework that validates business rules for audience creation across multiple social media channels.
• Automate recurring ETL processes using the PyTest framework.
• Validate the flow of data from disparate sources ingested into various data stores, Event Hub and Data Lakes post which data is transformed by workflows built inside Azure Data Factory followed by data getting stored in Azure Databricks.
• Create and execute manual test cases based on details mentioned as part of functional requirements and technical specifications in order to assure quality and accuracy.
• Thoroughly test the ETL rules built for data transformation.
• Develop test queries using SQL, Python or PySpark as per the user stories to validate the data inside various databases including Databricks.
• Experience with automating tests using Python/PySpark.
• Exposure to PyTest framework would be an added advantage.
• Experience building basic workflows or data pipelines inside Azure Data Factory would be a plus. This would be a great add as certain repeated testing of data flows will need to be automated.
• Knowledge on API testing is good to have. Preparing request block and analyzing API responses in JSON/XML formats.
• Log appropriate defects within Jira when product does not conform to specifications.
• Participate in daily stand-ups with project team as part of the agile methodology.
• Coordinate with development team members regarding defect validation and assist development team members with re-creating defects.
• Create appropriate test cases within TestRail Test Management tool.
• Update tasks information in Jira as appropriate to communicate progress with onshore test lead.

Other Responsibilities:

• Maintain technology expertise, keeping current with evolving testing tools, techniques, and strategies to improve the overall testing efficiency, processes, and best practices.
• Maintain a focus on customer-service, efficiency, quality, and growth.
• Safeguard the company's assets.
• Adhere to the company's compliance program.
• Maintain comprehensive knowledge of industry standards, methodologies, processes, and best practices.

Minimum Qualifications and Job Requirements:

• Experience in Azure Databricks is a must.
• Prior experience with building automation tests using Python/PySpark.
• SQL/Python skills and the ability to write complex queries inside Azure Databricks.
• 2+ years working in Azure Databricks, Data Factory, Data Lake.
• Building basic workflows or data pipelines inside Azure Data Factory would be a plus.
• Good understanding of file formats including JSON, Parquet, Avro, and others
• Ability to learn new technologies quickly.
• Excellent problem-solving skills
• Basic understanding of software development principles.
• Knowledge of Jira
• Ability to handle multiple tasks/projects concurrently and meet deadlines.
• Ability to work in a fast-paced team environment. Expectations include a high level of initiative and a strong commitment to job knowledge, productivity, and attention to detail.
• Solid software engineering skills - participated in full lifecycle development on large projects.