Data Analyst with Essbase exp

Job Title: Databricks / Python Data Engineer with Essbase exp

Location: Remote

Roles and Responsibilities:

About the Role

You will design, build, and maintain scalable data pipelines and transformation frameworks using Python and Databricks to power enterprise analytics, integrations, and downstream data consumers. You will own the full lifecycle of data engineering — from ingestion and transformation through orchestration, optimization, and deployment automation — within a cloud-hosted, CI/CD-driven environment.

You bring strong opinions on data architecture, move fast without sacrificing reliability, and actively use AI-assisted tooling to accelerate delivery and improve engineering quality. You are comfortable leading technical decisions within a scrum team and mentoring engineers around you.

Responsibilities

• Architect and own scalable data pipelines in Databricks using Python and Spark.
• Design and implement batch and near-real-time ingestion patterns for structured and semi-structured enterprise data.
• Build and maintain medallion-style data models and transformation layers across bronze, silver, and gold datasets.
• Optimize Spark jobs for performance, cost efficiency, and reliability across large-scale datasets.
• Define and enforce data quality, schema validation, lineage, and observability standards.
• Drive integration patterns for enterprise source systems, APIs, flat files, and cloud storage platforms.
• Automate repetitive engineering tasks — pipeline scaffolding, test generation, transformation templates, and documentation — using AI-assisted tooling such as Claude, Copilot, or similar tools.
• Own deployment and orchestration workflows for Databricks assets through CI/CD pipelines and infrastructure automation.
• Partner with analytics, BI, platform, and application teams to deliver trusted and consumable data products.
• Mentor mid-level engineers and participate actively in code review and sprint ceremonies.

Required Skills

• 5+ years of professional data engineering experience with strong Python expertise.
• Deep experience with Databricks, Apache Spark, and distributed data processing concepts.
• Strong hands-on experience building ETL/ELT pipelines using Python and PySpark.
• Proven experience designing data models and transformation pipelines for enterprise-scale analytics use cases.
• Strong understanding of data quality practices, schema evolution, partitioning, and performance tuning.
• Experience working with cloud object storage, Delta Lake, and modern lakehouse architecture principles.
• Familiarity with workflow orchestration and scheduling tools in a CI/CD-driven environment.
• Git-based workflow in a fast-paced engineering environment.
• Daily use of AI coding assistants such as Claude Code, Copilot, Cursor, or similar tools as a core part of the workflow.