Azure Data Engineer (No C2C) - e.biT Consulting

SUMMARY DESCRIPTION:

The Data Engineer will serve in a strategic role managing the infrastructure that supports data storage, transforming, processing, and retrieval enabling efficient data analysis and decision-making within the organization. This position is critical as part of the Database and Analytics team responsible for development and implementation of complex enterprise-level data integration and consumption solutions. It requires a highly technical, self-motivated senior developer who will work with analysts, architects, and systems engineers to develop and unit test solutions based on the functional and technical specifications to meet quality and performance requirements.

PRIMARY DUTIES AND RESPONSIBILITIES:

• Design and optimize data models, warehouses, architectures, schemas, indexing, and partitioning strategies.
• Collaborate with BI and data analysts to understand data requirements and optimize storage for analytical queries.
• Modernize databases and data warehouses and prepare them for analysis, managing for optimal performance.
• Design, build, manage, and optimize enterprise data pipelines ensuring efficient data flow, data integrity, and data quality throughout the process.

• Automate efficient data acquisition, transformation, and integration from a variety of data sources including databases, APIs, message queues, data streams, etc.

• Competently performs advanced data tasks with minimal supervision, including architecting advanced data solutions, leading and coaching others, and effectively partnering with stakeholders.
• Interface with other technical and non-technical departments and outside vendors on assigned projects.
• Under the direction of the IT Management, will establish standards, policies and procedures pertaining to data governance, database/data warehouse management, metadata management, security, optimization, and utilization.
• Ensure data security and privacy by implementing access controls, encryption, and anonymization techniques as per data governance and compliance policies.
• Provide training and consulting to other associates on database structure and standards.
• Document data pipelines, processes, and architectural designs for future reference and knowledge sharing.
• Stay informed of latest trends and technologies in the data engineering field, and evaluate and adopt new tools, frameworks, and platforms (like Microsoft Fabric) to enhance data processing and storage capabilities.
• When necessary, implement and document schema modifications made to legacy production environment.

• Perform any other function required by IT Management for the successful operation of all IT and data services provided to our clients.
• Available nights and weekends as needed for system changes and rollouts.

• Bachelor s or Master s degree in computer science, information systems, applied mathematics, or closely related field.
• Minimum of eight (8) years full time employment experience as a data engineer, data architect, or equivalent required.
• Knowledge of Mortgage and/or Title Industry is helpful.

• Experience in working with large, heterogeneous datasets in building and optimizing data pipelines, pipeline architectures, and integrated datasets using traditional and modern data integration technologies (such as ETL, ELT, MPP, data replication, change data captures, message-oriented data movement, API design, stream data integration and data virtualization)
• Experience working with cloud data engineering stacks (specifically Azure) and tools (Microsoft Fabric, Data Lake, Synapse, Azure Data Factory, Databricks, Informatica, etc.)
• Strong, in-depth understanding of database architecture, storage, and administration utilizing Azure stack.
• Deep understanding of Data architectural approaches, Data Engineering Solutions, Software Engineering principles and best practices.
• Working knowledge and experience with modern BI and ETL tools (Power BI, Power Automate, ADF, SSIS, etc.)
• Experience utilizing data storage solutions including Azure Blob storage, ADLS Gen 2.
• Solid understanding of relational and dimensional database principles and best practices in a client/server, thin-client, and cloud computing environment.
• Advanced working knowledge of TSQL and SQL Server, transactions, error handling, security and maintenance with experience writing complex stored procedures, views, and user-defined functions.
• Experience with .net scripting and understanding of API integration in a service-oriented architecture.
• Communicate effectively and professionally, in both oral and written communications, with all levels of associates.
• Understanding of and experience with agile methodology.
• Strong independent troubleshooting and problem-solving skills.
• Organizational, time management, and motivational skills.
• Must be able to perform in a fast paced and mission critical production environment.
• Ability to analyze and troubleshoot problems, determine solutions, and produce the desired outcome.
• Team Player with a strong work ethic.
• PowerShell scripting experience desired.

• Establish and maintain effective working relationships with Business and IT associates.
• Embrace and champion change as well as advocate for a data-driven culture.
• Improve business productivity by proposing innovative and creative solutions.
• Follow directions from management.
• Understand and follow posted work rules and procedures.
• Communicate in a positive manner to all associates and other business professionals.
• Maintain a positive professional attitude.

• Some travel may be required.
• Must maintain a professional appearance.
• Position requires long periods of work at a desk, utilizing a keyboard and monitor.

• Available to work evenings and/or weekends (as required).
• Workdays and hours are Monday thru Friday 8:30 am to 5:30 pm

This position outlines the basic tasks and requirements for the position noted. It is not a comprehensive listing of all job duties of the associate.

FLSA STATUS: Exempt