Data Platform Owner
Indexed description
Overview
The Data Platform Owner is responsible for the strategic direction, governance, and operational excellence of a cloud-based data platform built on a medallion architecture and powered by Databricks. This role ensures the platform delivers scalable, reliable, and high-quality data products that support analytics, data science, and business decision-making. The Data Platform Owner partners closely with engineering, analytics, and business teams to drive platform adoption, optimize performance, and continuously evolve data capabilities in alignment with organizational goals.
Responsibilities
- Design, manage, and optimize enterprise data platforms and pipelines within AWS and Databricks environments, including ingestion, transformation, storage, and consumption layers.
- Deep hands-on expertise with Databricks administration, workspace management, Delta Lake, Unity Catalog, declarative pipelines, cluster optimization, and cost management.
- Strong experience designing scalable, secure, and high-performing cloud data architectures using technologies such as PySpark, Spark, Python, SQL, Hadoop, and related data engineering frameworks.
- Experience building and supporting end-to-end data pipelines within hybrid cloud architectures, ensuring data quality, consistency, governance, lineage, and compliance.
- Solid understanding of data warehouse and data modeling methodologies, including Kimball and Inmon approaches.
- Experience implementing data governance, metadata management, lineage, cataloging, retention, and security practices using modern governance tools and frameworks.
- Strong knowledge of AWS services including S3, IAM, EC2, Lambda, Glue, and VPC, along with DevOps practices such as CI/CD and Infrastructure as Code.
- Ability to monitor, troubleshoot, automate, and continuously improve cloud-based data platforms for scalability, reliability, performance, and operational efficiency.
- Experience collaborating with data engineers, analysts, data scientists, business stakeholders, and infrastructure teams to define and deliver data solutions aligned with business objectives.
- Strong communication and documentation skills, including the ability to present complex technical concepts to both technical and non-technical audiences.
- Strong interpersonal and leadership skills, with the ability to foster collaboration, resolve conflicts, and maintain a positive team culture.
- 5+ years of experience in data platform management, data architecture, data engineering, or data curation.
- 5+ years of hands-on development experience with Oracle and/or Microsoft SQL Server, including PL/SQL and T-SQL.
- 5+ years of experience developing data solutions using Python and PySpark.
- Strong experience implementing and optimizing ETL/ELT solutions within Databricks and cloud platforms such as AWS and Azure.
- Experience with SSIS, C#, APIs, and microservices development.
- Strong understanding of data transformation, processing, normalization, denormalization, and large-scale event-driven streaming architectures.
- Familiarity with stream-processing technologies, AI/ML concepts, and modern distributed data platforms is preferred.
- AWS Certified Solutions Architect (Associate or Professional) preferred.
- Databricks Certified Data Engineer (Associate or Professional) or Databricks Administrator certification preferred.
- Experience with CI/CD and DevOps tools such as GitHub, Azure DevOps, or Bitbucket.
- Prior leadership or mentoring experience within engineering or platform teams is preferred.
- Strong verbal and written communication skills, with the ability to collaborate effectively across technical and business teams.
What Do We Offer
- Hybrid work (flexibility to work 2 days/week from home)
Create a free Caio profile to unlock more results and save your role and location preferences.
Unlock free search