Azure Databricks Platform Admin

Job Title: Azure Databricks Platform Admin

Location: Raritan, NJ (Onsite)

Job Summary

We are seeking an experienced Azure Databricks Platform Admin responsible for administering, securing, monitoring, and optimizing Azure Databricks environments across development, test, and production landscapes. The ideal candidate will have hands-on experience with workspace administration, cluster management, Unity Catalog, security controls, automation, and Azure platform services.

Key Responsibilities

• Administer and maintain Azure Databricks workspaces, clusters, jobs, notebooks, and platform configurations.
• Configure and manage Unity Catalog metastores, catalogs, schemas, external locations, and storage credentials.
• Implement and enforce security controls, including access management, role-based permissions, service principals, secret scopes, and integration with Microsoft Entra ID.
• Manage cluster policies, autoscaling, runtime versions, job clusters, and cost optimization measures to improve platform efficiency.
• Integrate Azure Databricks with Azure services such as ADLS Gen2, Azure Key Vault, Azure Monitor, Private Link, VNets, and managed identities.
• Monitor platform health, audit logs, usage patterns, and job performance; troubleshoot failures, connectivity issues, and performance bottlenecks.
• Support CI/CD and infrastructure automation using Terraform, Azure DevOps, Databricks CLI, and REST APIs.
• Maintain environment standards across Dev, QA, and Prod, including deployment controls, documentation, and operational runbooks.
• Collaborate with data engineering, architecture, security, and support teams to ensure platform reliability, governance, and compliance.
• Drive continuous improvement in platform operations, observability, security posture, and cost management.

Required Skills

• Azure Databricks administration experience in enterprise environments.
• Strong knowledge of Unity Catalog, Delta Lake, workspace governance, access controls, and Databricks administration concepts.
• Good understanding of Azure services, including ADLS Gen2, Azure Key Vault, Entra ID, networking, private endpoints, and monitoring services.
• Experience with cluster optimization, troubleshooting Spark job issues, workload tuning, and platform monitoring.
• Exposure to automation and DevOps practices using Terraform, Azure DevOps, GitHub Actions, Python, Bash, or PowerShell.
• Knowledge of identity and access management, audit logging, compliance, and data security best practices.
• Working knowledge of SQL and familiarity with Python or scripting for platform automation.
• Strong problem-solving, stakeholder communication, and documentation skills.
• Mandatory Skills: Azure Functions, Databricks.

Preferred Skills

• Certification in Azure Administration, Azure Data Engineering, or Databricks platform technologies.
• Experience supporting platform setup in secure enterprise environments with governance and network controls.
• Familiarity with FinOps practices, tagging strategies, and cloud cost reporting.
• Exposure to data engineering workloads, orchestration frameworks, and performance optimization best practices.
• Experience creating SOPs, knowledge articles, and platform support documentation.

Qualifications

• 10 years of overall experience in cloud platform administration, data platform operations, or infrastructure support.
• Hands-on experience administering Azure Databricks in enterprise environments.

We are an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, sexual orientation, or gender identity), national origin, citizenship status, age, disability, genetic information, protected veteran status, or any other characteristic protected by applicable law.

https://www.e-verify.gov/sites/default/files/everify/posters/IER_RighttoWorkPoster.pdf