Sr. II Azure Databricks Administrator - NYU Langone Medical Center

NYU Langone Health is a world-class, patient-centered, integrated academic medical center, known for its excellence in clinical care, research, and education. It comprises more than 200 locations throughout the New York area, including five inpatient locations, a children's hospital, three emergency rooms and a level 1 trauma center. Also part of NYU Langone Health is the Laura and Isaac Perlmutter Cancer Center, a National Cancer Institute designated comprehensive cancer center, and NYU Grossman School of Medicine, which since 1841 has trained thousands of physicians and scientists who have helped to shape the course of medical history. At NYU Langone Health, equity, diversity, and inclusion are fundamental values. We strive to be a place where our exceptionally talented faculty, staff, and students of all identities can thrive. We embrace diversity, inclusion, and individual skills, ideas, and knowledge. For more information, go tonyulangone.org, and interact with us onLinkedIn,Glassdoor,Indeed,Facebook,Twitter,YouTubeandInstagram.

Position Summary:
We have an exciting opportunity to join our team as a Sr. II Azure Databricks Administrator.

In this role, as the Azure Databricks Administrator within this department, you will be responsible for managing and maintaining the Databricks platform ensuring optimal performance and security across workspaces clusters and jobs. This role is crucial in overseeing user administration, managing Azure storage solutions and implementing CI/CD pipelines using Azure DevOps. By leveraging a deep understanding of Databricks architecture and proficiency in scripting languages the administrator will automate tasks and enhance the efficiency of data operations. Strong communication skills and a commitment to high quality support will enable effective collaboration with cross functional teams directly contributing to the department s mission of delivering robust data solutions and insights.

Job Responsibilities:

oManage and maintain Azure Databricks Platform workspaces clusters and jobs

oOversee user administration including access controls and permissions

oHandle library installations runtime management and policy enforcement

oImplement/analyze cost control measures and monitor platform usage and performance

oAdminister Unity Catalog for data governance and security

oCollaborate with data engineers data scientists and analysts to optimize and streamline data workflows and analytical pipelines on the Databricks platform

oManage Azure storage solutions including Blob Storage and Data Lake Storage

oAdminister Azure Key Vault for secure storage of secrets and keys

oConfigure and manage Azure Data Factory for data integration and ETL processes

oImplement and manage VNETs firewalls Azure policies and security best practices

oImplement and manage Azure DevOps for CI/CD pipelines and version control

oImplement best practices for automated deployment and continuous integration

oSet up budgets and alerts to monitor and control Azure costs and spend

oImplement monitoring solutions for platform health and performance

oConfigure alerts for proactive issue detection and resolution

Minimum Qualifications:

oBachelor s degree in Computer science, Information systems or Engineering

o7 years of related professional experience administering data platforms

o3 years’ experience with Azure Databricks Lakehouse Platform

oProficiency in scripting languages such as Python/PySpark PowerShell or Bash

oExperience in automating administrative tasks and workflows

oKnowledge of security best practices and compliance requirements

oExperience with ETL processes data pipelines and big data technologies

oExperience with backup and restore procedures for Databricks and Azure services

oAbility to troubleshoot and resolve issues in a timely and efficient manner

oStrong verbal and written communication skills

oAbility to document processes procedures and configurations clearly

oCommitment to providing high quality support to internal/external users and stakeholders

oAbility to understand and address the needs of internal and external customers

oTeam player with the ability to work collaboratively with cross functional teams

oFlexibility to adapt to changing requirements and priorities

oWillingness to learn new technologies and continuously improve skills

Preferred Qualifications:

oDeep understanding of Databricks architecture features and best practices

oExperience with Delta Lake and Databricks SQL for advanced data management

oExperience in performance tuning and optimization for both Databricks and Azure environments

oExperience supporting/administering data science and ML workloads in Databricks

oKnowledge of disaster recovery planning and implementation

oExperience integrating Databricks with other data platforms and tools

oUnderstanding of hybrid and multi cloud environments

oExperience implementing data protection measures and ensuring regulatory compliance

Qualified candidates must be able to effectively communicate with all levels of the organization.

NYU Langone Health provides its staff with far more than just a place to work. Rather, we are an institution you can be proud of, an institution where you'll feel good about devoting your time and your talents.

NYU Langone Health is an equal opportunity and affirmative action employer committed to diversity and inclusion in all aspects of recruiting and employment. All qualified individuals are encouraged to apply and will receive consideration without regard to race, color, gender, gender identity or expression, sex, sexual orientation, transgender status, gender dysphoria, national origin, age, religion, disability, military and veteran status, marital or parental status, citizenship status, genetic information or any other factor which cannot lawfully be used as a basis for an employment decision. We require applications to be completed online.
If you wish to view NYU Langone Health's EEO policies, please click here. Please click here to view the Federal "EEO is the law" poster or visit https://www.dol.gov/ofccp/regs/compliance/posters/ofccpost.htm for more information.

NYU Langone Health provides a salary range to comply with the New York state Law on Salary Transparency in Job Advertisements. The salary range for the role is $81,325.15 - $135,541.73 Annually. Actual salaries depend on a variety of factors, including experience, specialty, education, and hospital need. The salary range or contractual rate listed does not include bonuses/incentive, differential pay or other forms of compensation or benefits.

To view the Pay Transparency Notice, please click here