Data Engineer - ValiantIQ INC

Client: Vanguard.

Title: Data Engineer

Location: Malvern PA (Hybrid)

Duration: 12+ Months

Visa: NO H1-B

Main skills

SQL

Pyspark

Python

Job Description

As a Data Engineer, you’ll build sophisticated solutions that support a wide range of advanced data driven capabilities. Provides advanced data solutions by using software to process, store, and serve data to others. Tests data quality and optimizes data availability. Ensures that data pipelines are scalable, repeatable, and secure. Utilizes a deep dive analytical skillset on a variety of internal and external data. Provides advanced data solutions by using software to process, store, and serve data to others.

In this role, you will:

• Apply your technological expertise to address the data challenges that our business and our clients face, using insights to turn challenges into opportunities
• Identify data solutions that have business applications and translate them into design specifications and code
• Collaborate with our Advanced Analytics team to collect and run large-scale datasets that support better investment decisions
• Keep up to speed with the latest industry and technology developments, and apply them to solve analytics challenges focused on business performance
• Handle activities such as quality assurance and data migration and integration to ensure clients and investors get the best business benefit
• Writes ETL (Extract / Transform / Load) processes, designs database systems, and develops tools for real-time and offline analytic processing.
• Troubleshoots software and processes for data consistency and integrity. Integrates large scale data from a variety of sources for business partners to generate insight and make decisions.
• Translates business specifications into design specifications and code. Responsible for writing complex programs, ad hoc queries, and reports. Ensures that all code is well structured, includes sufficient documentation, and is easy to maintain and reuse.
• Partners with internal clients to gain an enhanced understanding of business functions and informational needs. Gains expertise in tools, technologies, and applications/databases in specific business areas and company-wide systems.
• Leads all phases of solution development. Explains technical considerations at related meetings, including those with internal clients and less experienced team members.
• Tests code thoroughly for accuracy of intended purpose. Reviews end product with the client to ensure adequate understanding. Provides data analysis guidance as required.
• Designs and conducts training sessions on tools and data sources used by the team and self provisioners. Provides job aids to team members and business users.
• Tests and implements new software releases through regression testing. Identifies issues and engages with vendors to resolve and elevate software into production.
• Writes ETL (Extract / Transform / Load) processes, designs database systems, and develops tools for real-time and offline analytic processing.
• Troubleshoots software and processes for data consistency and integrity. Integrates large scale data from a variety of sources for business partners to generate insight and make decisions.
• Translates business specifications into design specifications and code. Responsible for writing complex programs, ad hoc queries, and reports. Ensures that all code is well structured, includes sufficient documentation, and is easy to maintain and reuse.
• Partners with internal clients to gain an enhanced understanding of business functions and informational needs. Gains expertise in tools, technologies, and applications/databases in specific business areas and company-wide systems.
• Leads all phases of solution development. Explains technical considerations at related meetings, including those with internal clients and less experienced team members.
• Tests code thoroughly for accuracy of intended purpose. Reviews end product with the client to ensure adequate understanding. Provides data analysis guidance as required.
• Designs and conducts training sessions on tools and data sources used by the team and self provisioners. Provides job aids to team members and business users.
• Tests and implements new software releases through regression testing. Identifies issues and engages with vendors to resolve and elevate software into production.
• Participates in special projects and performs other duties as assigned. 

Qualifications

• Minimum of five years data analytics, programming, database administration, or data management experience.
• Undergraduate degree or equivalent combination of training and experience.
• Prior experience working with data management and/ transformation
• Proficiency with languages/tools Python, Hive and Git
• Experience with Cloud computing, Hadoop/Spark and AWS Infrastructure
• Familiarity with CI/CD, Bit bucket/Bamboo process