Technology Lead - Azure and Databricks Admin

We are seeking an experienced Databricks Platform Administrator to manage, secure, optimize, and support enterprise Databricks environments across cloud platforms. The ideal candidate will be responsible for workspace administration, cluster management, security governance, cloud integrations, platform monitoring, automation, and operational support. This role requires strong expertise in Azure Databricks (preferred), along with exposure to AWS and GCP Databricks environments.

Required Qualifications

• Candidates authorized to work for any employer in Canada without employer-based visa sponsorship are welcome to apply. Infosys is unable to provide immigration sponsorship for this role at this time.
• Candidate must be located within commuting distance of Calgary, Alberta or be willing to relocate to the areas.
• Bachelor’s degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education.
• At least 4 years of Information Technology experience

Key Responsibilities

Platform Administration

• Administer and manage Databricks workspaces across development, test, and production environments.
• Configure and maintain Databricks workspaces, metastore, catalogs, clusters, pools, and policies.
• Manage Unity Catalog implementation, governance, and access controls.
• Perform platform upgrades, maintenance, and health checks.
• Support multi-workspace and multi-region deployments.

Cluster Management

• Create, configure, and manage all-purpose and job clusters.
• Implement cluster policies to control costs and ensure compliance.
• Optimize cluster sizing and auto-scaling configurations.
• Monitor cluster utilization, performance, and cost efficiency.
• Troubleshoot cluster startup, termination, and execution issues.

Security & Governance

• Implement RBAC and workspace access controls.
• Integrate Databricks with Azure AD, AWS IAM, or GCP Identity services.
• Manage service principals, managed identities, and secret scopes.
• Implement Unity Catalog governance and data security policies.
• Support audit logging, compliance, and platform security reviews.

Cloud Integration

• Integrate Databricks with:
• Azure Data Lake Storage (ADLS)
• AWS S3
• Azure Key Vault
• AWS Secrets Manager
• SQL Databases
• Event Hub
• Kafka
• Snowflake
• Power BI

Monitoring & Performance

• Monitor platform health and resource utilization.
• Establish proactive alerting and monitoring mechanisms.
• Analyze Spark job performance and recommend optimizations.
• Troubleshoot platform incidents and performance bottlenecks.
• Support SLA-driven production environments.

DevOps & Automation

• Automate Databricks administration using:
• Terraform
• Azure DevOps
• GitHub Actions
• Databricks CLI
• REST APIs
• Manage CI/CD pipelines for Databricks deployments.
• Support Infrastructure as Code (IaC) initiatives.

Operational Support

• Participate in incident, problem, and change management activities.
• Support patching, upgrades, certificate renewals, and maintenance activities.
• Create and maintain SOPs, runbooks, and technical documentation.
• Collaborate with engineering, infrastructure, security, and application teams.

Required Skills

Databricks

• Databricks Workspace Administration
• Unity Catalog
• Cluster Management
• Cluster Policies
• Instance Pools
• Jobs & Workflows
• SQL Warehouses
• Databricks CLI
• REST APIs
• Delta Lake

Cloud Platforms

• Azure Databricks (Preferred)
• AWS Databricks
• GCP Databricks

Preferred Skill And Experience

• Experience with enterprise-scale Azure Databricks environments.
• Strong understanding of data lake architecture and governance frameworks.
• Experience supporting production environments with strict SLA requirements.

Soft Skills

Strong analytical and troubleshooting skills.

Excellent communication and stakeholder coordination.

Ability to work in a global delivery model.

Documentation and knowledge ‑ sharing mindset.

Experience

• 8-10 years of experience in IT Industry
• 5+ years of Databricks Administration experience.
• Prior experience in large enterprise or managed services engagements preferred.
• Exposure to Infosys‑scale delivery environments is an advantage.

The job entails sitting as well as working at a computer for extended periods of time. Should be able to communicate by telephone, email or face to face. Travel may be required as per the job requirements

About Us

Infosys is a global leader in next-generation digital services and consulting. We enable clients in more than 50 countries to navigate their digital transformation. With over four decades of experience in managing the systems and workings of global enterprises, we expertly steer our clients through their digital journey. We do it by enabling the enterprise with an AI-powered core that helps prioritize the execution of change. We also empower the business with agile digital at scale to deliver unprecedented levels of performance and customer delight. Our always-on learning agenda drives their continuous improvement through building and transferring digital skills, expertise, and ideas from our innovation ecosystem.

EEO

Infosys provides equal employment opportunities to applicants and employees without regard to race; color; sex; gender identity; sexual orientation; religious practices and observances; national origin; pregnancy, childbirth, or related medical conditions; status as a protected veteran or spouse/family member of a protected veteran; or disability.