Senior Data Engineer

Optomi, in partnership with a leader in the streaming and entertainment company, is looking for a Senior Data Engineer to join the Product Performance Team. We are seeking a passionate Senior Data Engineer to join the Product Performance Team within the company. Our team is vital, owning the development of reliable data solutions that measure the product performance. This role involves designing, building, and testing the core algorithms and data models that generate precise and dependable browse, playback attribution, and search data.

What the right candidate will enjoy:

• Working with a modern data and engineering tech stack, including Airflow, Spark, Databricks, Delta Lake, Snowflake, GitHub Actions, and Jenkins.
• Collaborating with Product Managers, Data Analysts, and cross-functional engineering teams to define requirements and deliver new data features.

What type of experience does the right candidate have:

• 5+ years of software engineering experience developing large data pipelines.
• Strong fundamental Scala and Python software engineering skills.
• Hands-on production environment experience with distributed engineering systems such as Spark and Kafka.

What the responsibilities are of the right candidate:

• Design, implement, and deploy robust data solutions for both batch ingestion and stream processing using Scala and Python with frameworks like Spark.
• Establish and enforce rigorous standards for code development, testing, and deployment.
• Maintain, update, and expand existing source code that powers core datasets.
• Proactively investigate and resolve complex data pipeline failures and quality issues.