Data Bricks Engineer - TrekRecruit LLC
Jersey City, NJ
About the Job
Job Title: Data Engineer
Location:Jersey City, NJ
Hybrid role- Twice in a week
Location:Jersey City, NJ
Hybrid role- Twice in a week
Job Description :
- Design and implement data storage systems using AWS
- EC2,EMR,redshift or Azure services like Azure SQL Database, Azure
- Data Lake Storage, and Azure Synapse.
- Ensure scalability, performance, and cost-effectiveness.
- Develop and implement data integration processes using Azure Data
- Factory.
- Extract data from various sources, transform it, and load it into data
- warehouses or data lakes.
- Utilize big data technologies such as Apache Spark by using Scala/Spark
- Create data processing workflows and pipelines to support data analytics and machine learning applications.
- Build and maintain new and existing applications in preparation for a large-scale architectural migration within an Agile function.
- Monitor and optimize data pipelines and database performance to ensure data processing efficiency.
- Build interfaces for supporting evolving and new applications and accommodating new data sources and types of data.
- Document data engineering processes, data models, and pipelines to ensure transparency and maintainability.
Skill sets needed:
- Bachelor's degree Computer Science or a related field.
- 5+ years of experience in building data and analytics platforms focused on AWS/Azure data and analytics solutions.
- Expertise in Azure services such as Azure SQL Database, Azure Data Factory, Azure
- Synapse Analytics, and Azure Data Lake Proficiency in:
- Software development and scripting languages
- ETL tools (e.g., SSIS, Azure Data Factory, Power BI Dataflow).
- Database management (SQL server ,MSSQL, Azure SQL Database, Azure Data Lake Storage, and Azure Synapse).
- 5+ years of programming experience in Python/Pyspark and Java.
- Experience in Unix shell scripting and scheduling tools like Autosys
- Knowledge of data modeling and data warehousing concepts.
- Excellent problem-solving and troubleshooting abilities.
- Attention to detail and commitment to data accuracy.
- Experience with cloud-based data migration.
- Strong analytical and problem-solving skills, with ability to conduct root cause analysis on system, process or production problems and ability to provide viable solutions.
- Experience working in an Agile environment with Scrum Master/Product owner and ability to deliver.
- Knowledge of Jira, Confluence, SAFe development methodology and DevOps.
Source : TrekRecruit LLC