Senior Data Engineer - Remote Opportunity!
Indexed description
At KinderCare Learning Companies, you’ll use your skills and expertise to support the work (and fun) that happens in our sites and centers every day. From marketers and strategists to financial analysts and data engineers, and so much more, we’re all passionate about crafting a world where children, families, and organizations can thrive.
As Senior Databricks Engineer, you will be a hands-on technical expert and force multiplier on our Databricks-based data platform. You’ll own the design, optimization, and governance of our medallion lakehouse architecture (Bronze/Silver/Gold), the Unity Catalog, and the pipelines that feed enterprise BI and emerging AI/ML products. You'll operate deep in the Databricks ecosystem daily (Delta Lake, Unity Catalog, Workflows, Delta Live Tables / Lakeflow, MLflow) while also shaping how the platform supports next-generation capabilities, including Databricks-native ML/AI features such as Genie spaces, feature engineering, model serving, and vector search. This role partners closely with the Data Engineering Lead and BI Architect and sits in a SOX-governed environment, so a strong instinct for data governance, access control, and auditable engineering practices is essential.
Responsibilities:
Platform & Pipeline Engineering
- Design, build, and optimize production-grade ETL/ELT pipelines across the medallion architecture using Delta Lake, Delta Live Tables / Lakeflow Declarative Pipelines, and Databricks Workflows
- Own performance tuning and cost efficiency across the platform — cluster/job sizing, Photon, partitioning and Z-ordering, Liquid Clustering, Auto Loader, and DBU cost governance
- Architect and enforce data models that support enterprise BI (Microsoft Fabric/Power BI) and downstream analytics products
- Build and maintain CI/CD pipelines for Databricks assets (Databricks Asset Bundles, Repos, Git-based deployment) following Agile and Test-Driven Development practices
- Integrate platform pipelines with middleware and source systems (e.g., Boomi) and cloud-native services
- Administer and evolve Unity Catalog: catalogs/schemas, fine-grained access control, lineage, row/column-level security, and workspace-catalog bindings
- Implement and enforce data quality, observability, and reliability practices (expectations/constraints, monitoring, alerting, SLA management) across pipelines
- Partner with security, compliance, and audit teams to maintain SOX ITGC alignment; access reviews, change control, and auditable engineering practices
- Troubleshoot and resolve complex production data pipeline issues, performing root-cause analysis and implementing preventive fixes
- Create clear, durable documentation of architecture, procedures, and operational runbooks
- Evaluate, pilot, and productionize Databricks-native AI/ML capabilities — including Genie for natural-language data access and MLflow for experiment tracking and model lifecycle management, Feature Store, and Model Serving
- Support the build-out of vector search and retrieval-augmented generation (RAG) patterns on top of governed Unity Catalog data for internal AI use cases
- Collaborate with data science and analytics stakeholders to prepare curated, ML-ready Gold-layer datasets and feature pipelines
- Stay current on the Databricks roadmap (Lakehouse AI, Mosaic AI, Agent frameworks) and recommend adoption where it advances platform maturity and business value
- Help define guardrails and human-in-the-loop controls for AI-assisted and agentic engineering workflows introduced to the platform
- Serve as a technical mentor to mid-level data engineers, raising the bar on Databricks best practices, code quality, and architectural rigor
- Partner with the Data Architect and Data Engineering Lead to translate business requirements into technical specifications for BI and AI products
- Collaborate cross-functionally with technical and non-technical stakeholders, including product, security, and business teams
- Contribute to data governance, data security, and data privacy standards across the platform
- Bachelor’s degree in computer science, information systems, engineering, statistics, or related field or equivalent work experience
- 7+ years of experience as a data engineer with 3+ years specifically architecting and operating production workloads on Databricks
- Strong understanding of data governance, data security, and access control best practices
- Experience with Agile development methodologies, CI/CD automation, and Test-Driven Development
- Excellent problem-solving skills and the ability to lead technical troubleshooting independently
- Strong written and verbal communication skills with the ability to explain technical concepts to non-technical stakeholders
- Databricks certifications (e.g., Databricks Certified Data Engineer Professional, Databricks Certified Machine Learning Associate/Professional) strongly preferred
- Demonstrated experience owning a lakehouse/medallion architecture at scale, including data modeling for BI consumption
- Experience operating in a governed or regulated environment (SOX, HIPAA, or similar) with formal change control and access governance
- Deep, hands-on expertise with the Databricks Lakehouse Platform: Delta Lake, Unity Catalog, Delta Live Tables / Lakeflow, Workflows, and cluster/job optimization (Photon, Auto Loader, Liquid Clustering)
- Advanced SQL and strong Python (PySpark) development skills; comfort with Scala a plus
- Experience with Databricks Asset Bundles, Repos, and CI/CD for lakehouse deployments
- Working knowledge of cloud data services (Azure preferred — ADLS, Azure SQL, Synapse/Fabric; AWS/GCP equivalents acceptable) and cloud migration patterns
- Familiarity with BI integration layers such as Microsoft Fabric/Power BI and enterprise middleware (e.g., Boomi) a plus
- Hands-on experience with MLflow for experiment tracking, model registry, and lifecycle management
- Working knowledge of Databricks AI/ML capabilities — Feature Store, Model Serving, Genie, Mosaic AI, or equivalent lakehouse ML tooling
- Exposure to vector search, embeddings, or RAG architectures and an understanding of how governed data feeds AI/ML products
- Comfort partnering with data science teams on ML-ready data pipelines, even without a formal data science background
- Know your whole family is supported with discounted child care benefits.
- Breathe easy with medical, dental, and vision benefits for your family (and pets, too!).
- Feel supported in your mental health and personal growth with employee assistance programs.
- Feel great and thrive with access to health and wellness programs, paid time off and discounts for work necessities, such as cell phones.
- … and much more.
KinderCare Learning Companies is an Equal Opportunity employer. All qualified applicants will receive consideration for employment without regard to race, national origin, age, sex, religion, disability, sexual orientation, marital status, military or veteran status, gender identity or expression, or any other basis protected by local, state, or federal law.
Create a free Caio profile to unlock more results and save your role and location preferences.
Unlock free search