Starburst Engineer

Job Description

Primary Skill: Starburst Enterprise Platform

Secondary: Apache Airflow, Python

Experience: Minimum 8 years

Roles & Responsibilities

Starburst

• Design and implement Starburst SQL queries, catalogs, schemas, and connectors.
• Develop data pipelines and federated query solutions across diverse sources (S3, RDBMS etc.).
• Tune and optimize Starburst clusters, catalogs, and worker configurations for performance.

Airflow Development & Orchestration

• Design, develop, test, and deploy Airflow DAGs for complex data workflows.
• Implement best practices for Airflow, including task retries, SLAs, scheduling, and monitoring.
• Build custom Airflow operators, hooks, and plugins using Python.

Data Pipeline Engineering

• Build and maintain data ingestion and transformation pipelines using SQL, Python, and/or Airflow.
• Implement Starburst-integrated ETL/ELT workflows with distributed compute.
• Ensure pipelines are scalable, fault-tolerant, and performance-optimized.
• Implement data quality checks, validation, and lineage tracking.

Cloud & Infrastructure Integration

• Work with cloud platforms to integrate Airflow with services such as Netapp S3
• Deploy and manage Airflow environments like Kubernetes.

Data Governance & Security

• Implement security policies, row/column-level security, and data masking.
• Ensure compliance with enterprise governance frameworks.
• Document data lineage, metadata, and catalog configurations

ThoughtSpot Development & Configuration

• Develop and manage ThoughtSpot worksheets, Liveboards, Answers, and SpotIQ insights.
• Build intuitive, user-friendly search experiences enabling self-service analytics.
• Configure data models, relationships, and joins to ensure high-quality analytical results.
• Implement row-level and column-level security policies.

Required Qualifications

• 3+ years of hand on Experience with Starburst Enterprise Platform.
• Familiarity with Kubernetes deployments.
• Knowledge of cost governance & workload isolation in distributed environments.
• 3+ years of experience working with Apache Airflow (core, providers, DAG authoring).
• Strong proficiency in Python.
• 2–4+ years of experience working with ThoughtSpot
• Solid understanding of data modeling concepts (star schema, relational modeling).
• Experience integrating BI tools with ELT/ETL pipelines.
• Hands-on experience with cloud-based Airflow deployments
• Experience with SQL and relational databases
• Experience with Git and CI/CD frameworks

Preferred Qualifications

• Experience with BI tools (Tableau, Power BI, Oracle OAS/OAC) using Starburst as a source.

Salary Range- $110,000-$130,000 a year

Qualifications: BACHELOR OF COMPUTER SCIENCE