Data Engineer - hybrid - w2 - Veterans Sourcing Group
Richardson, TX 75081
About the Job
Data Engineer
Richardson, TX - In office 3 days a week
6 years of experience
enterprise data warehouse systems
snowflake (must have) , R and other cloud technologies (AWS, Azure)
Qualifications:
• Bachelor's degree (BA/BS) in a related field such as information systems, mathematics, or computer science.
• Typically has 6 years of relevant work experience. Consideration given to equivalent combination of education and experience.
• Excellent written and verbal communication skills. Strong organizational and analytical skills.
• Expertise in Data Extraction, Transformation, Loading, Data Analysis, Data Profiling, and SQL Tuning.
• Expertise in Relational & Dimensional Databases in engines like SQL Server, Postgres, Oracle…
• Strong experience in designing and developing enterprise scale data warehouse systems using Snowflake.
• Strong expertise in designing and developing reusable and scalable Data products with data quality scores and integrity checks.
• Strong expertise in developing end to end complex data workflows using Data ingestion tools such as Snaplogic, ADF, Matallion etc.
• Experience with cloud platforms AWS / Azure cloud technologies, Agile methodologies and DevOps is a big plus.
• Experience in architecting cloud native solutions across multiple B2B and B2B2C data domains.
• Experience in architecture of modern APIs for the secure sharing of data across internal application components as well as external technology partners.
• Experience in Data orchestration tools like Apache Airflow, Chronos with Mesos cluster etc.
• Expertise in designing and developing data transformation models in DBT.
• Comparing and analyzing provided statistical information to identify patterns, relationships, and problems; and using this information to design conceptual and logical data models and flowcharts to present to management.
• Experience with developing CICD pipelines in Jenkins or Azure DevOps.
• Knowledge of Python for data manipulation and automation.
• Knowledge of data governance frameworks and best practices.
• Knowledge in integrating with source code versioning tools like Git Hub.
Responsibilities:
• Plan & analyze, develops, maintains, and enhances client systems as well as supports systems of moderate to high complexity.
• Participates in the design, specification, implementation, and maintenance of systems.
• Designs, codes, tests, and documents software programs of moderate complexity as per the requirement specifications.
• Design, develop, and maintain scalable data pipelines using Snowflake, dbt, Snaplogic and ETL tools.
• Participates in design reviews and technical briefings for specific applications.
• Integrate data from various sources, ensuring consistency, accuracy, and reliability.
• Develop and manage ETL/ELT processes to support data warehousing and analytics.
• Assists in preparation of requirement specifications, Analyzing the data, design and develop data driven applications including documenting and revising user procedures and/or manuals.
• Involved with resolution of Medium to severe complexity software development issues that may arise in a production environment.
• Utilize Python for data manipulation, automation, and integration tasks.
• Assemble large, complex data sets that meet functional / non-functional business requirements. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL Server, PostgreSQL, SSIS, T-SQL, PL/SQL
• Work with stakeholders including the Product, Data, Design, Frontend and Backend teams to assist with data-related technical issues and support their data infrastructure needs
• Write complex SQL, T-SQL, PL/SQL queries, stored procedures, functions, cursors in SQL Server and PostgreSQL. Peer review other team members code
• Analyze the long running queries/functions/procures, design and develop performance optimization strategy.
• Create and manage SSIS packages and/or Informatica to perform day to day ETL activities. Use variety of strategies for complex data transformations using an ETL tool
• Perform DBA activities like maintaining the systems health and performance tuning, manage database access, deployments to higher environments, on-call support, shell scripting and python scripting is a plus
• Participate in employing the Continuous Deliver and Continuous Deployment (CI/CD) tools for optimal productivity.
• Collaborate with scrum team members during daily standup and actively engage in sprint refinement, planning, review and retrospective.
• Analyzes, reviews, and alters program to increase operating efficiency or adapt to new requirements.
• Writes documentation to describe program development, logic, coding, and corrections.
Richardson, TX - In office 3 days a week
6 years of experience
enterprise data warehouse systems
snowflake (must have) , R and other cloud technologies (AWS, Azure)
Qualifications:
• Bachelor's degree (BA/BS) in a related field such as information systems, mathematics, or computer science.
• Typically has 6 years of relevant work experience. Consideration given to equivalent combination of education and experience.
• Excellent written and verbal communication skills. Strong organizational and analytical skills.
• Expertise in Data Extraction, Transformation, Loading, Data Analysis, Data Profiling, and SQL Tuning.
• Expertise in Relational & Dimensional Databases in engines like SQL Server, Postgres, Oracle…
• Strong experience in designing and developing enterprise scale data warehouse systems using Snowflake.
• Strong expertise in designing and developing reusable and scalable Data products with data quality scores and integrity checks.
• Strong expertise in developing end to end complex data workflows using Data ingestion tools such as Snaplogic, ADF, Matallion etc.
• Experience with cloud platforms AWS / Azure cloud technologies, Agile methodologies and DevOps is a big plus.
• Experience in architecting cloud native solutions across multiple B2B and B2B2C data domains.
• Experience in architecture of modern APIs for the secure sharing of data across internal application components as well as external technology partners.
• Experience in Data orchestration tools like Apache Airflow, Chronos with Mesos cluster etc.
• Expertise in designing and developing data transformation models in DBT.
• Comparing and analyzing provided statistical information to identify patterns, relationships, and problems; and using this information to design conceptual and logical data models and flowcharts to present to management.
• Experience with developing CICD pipelines in Jenkins or Azure DevOps.
• Knowledge of Python for data manipulation and automation.
• Knowledge of data governance frameworks and best practices.
• Knowledge in integrating with source code versioning tools like Git Hub.
Responsibilities:
• Plan & analyze, develops, maintains, and enhances client systems as well as supports systems of moderate to high complexity.
• Participates in the design, specification, implementation, and maintenance of systems.
• Designs, codes, tests, and documents software programs of moderate complexity as per the requirement specifications.
• Design, develop, and maintain scalable data pipelines using Snowflake, dbt, Snaplogic and ETL tools.
• Participates in design reviews and technical briefings for specific applications.
• Integrate data from various sources, ensuring consistency, accuracy, and reliability.
• Develop and manage ETL/ELT processes to support data warehousing and analytics.
• Assists in preparation of requirement specifications, Analyzing the data, design and develop data driven applications including documenting and revising user procedures and/or manuals.
• Involved with resolution of Medium to severe complexity software development issues that may arise in a production environment.
• Utilize Python for data manipulation, automation, and integration tasks.
• Assemble large, complex data sets that meet functional / non-functional business requirements. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL Server, PostgreSQL, SSIS, T-SQL, PL/SQL
• Work with stakeholders including the Product, Data, Design, Frontend and Backend teams to assist with data-related technical issues and support their data infrastructure needs
• Write complex SQL, T-SQL, PL/SQL queries, stored procedures, functions, cursors in SQL Server and PostgreSQL. Peer review other team members code
• Analyze the long running queries/functions/procures, design and develop performance optimization strategy.
• Create and manage SSIS packages and/or Informatica to perform day to day ETL activities. Use variety of strategies for complex data transformations using an ETL tool
• Perform DBA activities like maintaining the systems health and performance tuning, manage database access, deployments to higher environments, on-call support, shell scripting and python scripting is a plus
• Participate in employing the Continuous Deliver and Continuous Deployment (CI/CD) tools for optimal productivity.
• Collaborate with scrum team members during daily standup and actively engage in sprint refinement, planning, review and retrospective.
• Analyzes, reviews, and alters program to increase operating efficiency or adapt to new requirements.
• Writes documentation to describe program development, logic, coding, and corrections.
Source : Veterans Sourcing Group
