Senior Data Engineer - SparkCognition
Austin, TX
About the Job
Who are we and why this opportunity?
SparkCognition, Inc. delivers world-class AI solutions that allow a business to solve their most critical problems, empowering them to run a more sustainable, safer, and profitable business. Our award-winning AI solutions predict future outcomes, optimize processes, and prevent cyberattacks. We partner with the world's industry leaders to analyze, optimize, and learn from data. We augment human intelligence, drive profitable growth, and achieve operational excellence.
Drive change and create a footprint. Learn more at: SparkCognition
As a Senior Data Engineer, you will design, implement, and maintain large-scale, high-performance data infrastructure to support our AI models and analytics platforms. You'll work with various data sources to create efficient, scalable data pipelines and ensure data quality, accessibility, and security. This role offers the chance to collaborate with data scientists, software engineers, and business stakeholders to influence data strategy and decision-making.
You Will:
- Architect, build, and maintain robust, scalable data pipelines and ETL processes to handle large volumes of data.
- Ensure high availability, reliability, and quality of data used across the company, optimizing pipelines for performance and scalability.
- Collaborate with data scientists, engineers, and business stakeholders to define data requirements and design solutions that enable advanced analytics and AI models.
- Lead the optimization of data processing workflows, storage solutions, and data models to improve efficiency and performance.
- Develop and implement data governance strategies, ensuring data security, integrity, and compliance.
- Mentor and guide junior data engineers and provide technical leadership in designing data architectures.
- Troubleshoot complex data engineering challenges and drive continuous improvement in data practices.
You'll Have:
- A bachelor's or master's degree in Computer Science, Information Systems, or a related field.
- 5-10 years of experience in designing and maintaining large-scale data infrastructure and pipelines.
- Strong expertise in SQL, data modeling, and experience with modern ETL frameworks.
- Proven experience with cloud platforms like AWS, Google Cloud, or Azure, including data storage and processing tools (e.g., Redshift, BigQuery, Snowflake, Databricks).
- Deep understanding of distributed systems, data architecture principles, and best practices for building data pipelines.
- Proficiency in programming languages such as Python, Scala, or Java.
- Strong problem-solving skills and experience optimizing data workflows for performance and scalability.
- Experience with version control systems (e.g., Git), CI/CD pipelines, and orchestration tools like Airflow.
- Excellent communication and collaboration skills, with a demonstrated ability to work with cross-functional teams.
SparkCognition is an equal opportunity employer, dedicated to diversity, equality, and inclusion, and provides equal employment opportunities to all employees and applicants for employment.
SparkCognition prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state, or local laws.
SparkCognition is committed to providing reasonable accommodations throughout the recruiting process. If you need a reasonable accommodation, please contact us to discuss how we can assist you.