Databricks Architect

Databricks Architect

Position: Databricks Architect

Location: Flexible / Hybrid / Remote

Employment Type: Full-time

About the Role

We are seeking an experienced Databricks Solutions Architect to lead the design of enterprise data and AI solutions built on the Databricks Lakehouse Platform. This role is responsible for defining technical strategy, creating scalable solution architectures, and partnering with clients to modernize their data ecosystems.

The successful candidate will work closely with executive stakeholders, business leaders, engineering teams, and delivery organizations to define solution roadmaps, establish architecture standards, and ensure successful implementation of enterprise-scale data platforms.

Key Responsibilities

Enterprise Architecture & Solution Design

• Design enterprise-scale data and AI architectures using the Databricks Lakehouse Platform.
• Develop solution blueprints aligned with business objectives and enterprise technology strategies.
• Define scalable, secure, resilient, and cost optimized cloud architectures.
• Design modern data platforms supporting analytics, AI, machine learning, and Generative AI.
• Establish enterprise architecture standards, design patterns, and reusable frameworks.
• Lead architecture workshops and translate business requirements into technical solutions..

Data Platform Strategy

• Define enterprise data modernization roadmaps.
• Develop migration strategies from traditional data warehouses and Hadoop platforms.
• Architect batch, streaming, and real time data processing solutions.
• Design enterprise data models supporting operational and analytical workloads.
• Define data lifecycle, storage, governance, and retention strategies.

AI & Analytics Architecture

• Design architectures supporting enterprise AI and machine learning initiatives.
• Define feature engineering, model management, and MLOps strategies.
• Architect retrieval augmented generation (RAG), vector search, and AI ready data platforms.
• Guide integration of AI capabilities into enterprise data ecosystems.
• Evaluate emerging AI technologies and recommend adoption strategies.

Cloud Architecture

• Design cloud native architectures across Azure, AWS, and Google Cloud Platform.
• Define Infrastructure as Code, CI/CD, and platform automation strategies.
• Establish architecture for high availability, disaster recovery, scalability, and resilience.
• Define monitoring, observability, logging, and operational excellence standards.

Governance & Security

• Define enterprise data governance strategies.
• Architect security models including Unity Catalog, role-based access control, and data protection.
• Establish metadata management, lineage, cataloging, and data quality standards.
• Ensure compliance with enterprise security policies and regulatory requirements.

Client Leadership & Consulting

• Serve as the trusted technical advisor for client executives and technology leaders.
• Lead discovery workshops, architecture assessments, and strategic planning engagements.
• Present solution architectures to executive and technical audiences.
• Support business development through solution design, proposals, estimations, and technical presentations.
• Build long term relationships with client stakeholders.

Required Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• 10+ years of experience designing enterprise data platforms.
• 5+ years of Databricks architecture experience.
• Proven experience leading enterprise cloud modernization initiatives.
• Demonstrated success designing scalable data and AI platforms.
• Strong understanding of enterprise architecture methodologies and distributed computing.

Technical Skills

Databricks

• Databricks Lakehouse Platform
• Delta Lake
• Unity Catalog
• Databricks SQL
• Delta Live Tables
• MLflow
• Databricks Workflows
• Auto Loader
• Structured Streaming
• Photon Engine

Architecture & Design

• Enterprise Architecture
• Solution Architecture
• Data Platform Architecture
• Lakehouse Architecture
• Data Mesh
• Data Fabric
• Reference Architecture Development
• Architecture Governance

Programming

• Python
• PySpark
• SQL
• Scala (preferred)

Cloud Platforms

• Microsoft Azure
• Amazon Web Services (AWS)
• Google Cloud Platform (GCP)

Data Technologies

• Apache Spark
• Kafka
• Airflow
• Snowflake
• Azure Data Factory
• Azure Synapse
• DBT
• Data Lake Storage

DevOps & Automation

• Terraform
• Git
• Azure DevOps
• GitHub Actions
• Jenkins
• Docker
• Kubernetes

Preferred Experience

• Enterprise data modernization programs.
• Data modernization initiatives
• Lakehouse implementations.
• Data mesh or data fabric architectures.
• AI/ML platform implementation.
• Real-time streaming architectures.
• Master Data Management (MDM).
• Data governance and metadata management.
• Multi-cloud architecture.

Certifications (Preferred)

• Databricks Certified Data Engineer Professional
• Databricks Certified Solutions Architect (if available)
• Azure Solutions Architect Expert
• Azure Data Engineer Associate
• AWS Certified Solutions Architect
• Google Professional Data Engineer