Sr. Data Platform Engineer

We are building the next generation of enterprise data platforms: keeping production systems reliable while productizing query, processing, orchestration, and governance with solid engineering and mainstream AI capabilities.

You Will Bring

• Platform engineering skills — production operations and component evolution for Spark / Flink, Airflow, Trino/Presto, and lakehouse table formats
• Product sense — turn repetitive manual work into durable platform capabilities (APIs, CLIs, control planes, agents), not one-off scripts
• AI fluency — use mainstream AI technologies (LLMs, agents, tool-calling, knowledge bases) to improve platform efficiency and experience, and take responsibility for generated outcomes

• Own reliability, capacity, change management, and incident response for compute and orchestration planes (Spark / Flink / Airflow / Trino·Presto)
• Maintain and evolve lakehouse processing and data-sync components, improving observability, release quality, and operational efficiency
• Help modernize the query plane: cluster deployment, authorization and traffic control, reproducible environments, capacity planning, and slow-query remediation
• Design platform capabilities with a product mindset: configuration contracts, approval boundaries, auditing, self-service onboarding, and closed-loop diagnostics
• Explore and apply mainstream AI technologies to build next-generation data platform capabilities (intelligent troubleshooting, knowledge capture, natural-language interaction, automated operations), with human–AI co-governance rather than black-box automation
• Partner with business, security, and infrastructure teams to improve platform experience and efficiency under compliance constraints

• 3+ years of experience in data platforms / big data engineering, with independent production on-call ownership
• Compute & orchestration
• Deep understanding of Spark / Flink, with hands-on production operations and troubleshooting
• Deep understanding of Airflow, with hands-on scheduling operations and change management
• Deep understanding of Trino / Presto, with hands-on cluster operations (concurrency/queues, authorization, slow queries, capacity)
• Data lake / lakehouse
• Deep understanding of Apache Hudi / Apache Iceberg (write modes, compaction/cleanup, time travel, and read/write paths with Spark/Trino)
• Programming
• Java / Scala required
• Strong Linux troubleshooting and SQL skills
• Product mindset: define platform capabilities from the user journey; prioritize reusable, measurable, operable outcomes over one-off firefighting

• Vertica (or similar MPP / analytical warehouse) operations and performance tuning
• Hands-on experience deploying and operating Spark / Flink / Trino on Kubernetes
• Rust (strong plus for high-performance proxies, control planes, or platform tooling)
• Python (orchestration ecosystem, automation, agent-side work)
• Kafka / CDC experience (e.g., Debezium)
• Metadata, lineage, or data-governance platform experience
• Reverse proxy / gateway / quota-rate-limiting / observability experience (Prometheus, Grafana, logging platforms)
• Agent / tool-calling / RAG / knowledge-base productization experience