Senior Data Engineer - Power

At Kpler, we are dedicated to helping our clients navigate complex markets with ease. By simplifying global trade information and providing valuable insights, we empower organisations to make informed decisions in commodities, energy, and maritime sectors.

Since our founding in 2014, we have focused on delivering top-tier intelligence through user-friendly platforms. Our team of over 850 experts from 69 countries works tirelessly to transform intricate data into actionable strategies, ensuring our clients stay ahead in a dynamic market landscape. Join us to leverage cutting-edge innovation for impactful results and experience unparalleled support on your journey to success.

As a Senior Data Engineer, you will own the design and development of the data pipelines and systems that give the team and analytical models timely access to primary market sources covering Japanese power pricing, availability, and generation data.

You will work at the intersection of data engineering and product development, collaborating closely with analysts, product managers, and the broader backend team to ensure data is reliable, well-structured, and efficiently accessible to end users.

Your mission is to

• Lead the design, development, and maintenance of data scrapers and ingestion pipelines for Japanese power market sources (pricing, availability, generation).
• Own the database architecture that underpins the Japan Power product, ensuring data is well-modelled, performant, and reliable.
• Drive improvements to monitoring, alerting, and observability tooling for data pipelines and APIs.
• Ensure strong API performance for distributing Japan Power data to end users and downstream consumers.
• Lead technical design discussions and architecture reviews for new data features and infrastructure changes.
• Collaborate with the product team to contribute to roadmap planning, ensuring plans are ambitious and technically sustainable.
• Mentor junior data engineers through code reviews, design discussions, and knowledge sharing.
• Identify and address technical debt, data quality issues, and performance bottlenecks proactively.
• Contribute to the continuous improvement of engineering standards and best practices within the Power team.

Experience & Background

• 6+ years of experience as a data-focused software or data engineer.
• Strong Python expertise, including production-grade data pipelines and API development (FastAPI experience is a plus).
• Significant experience with PostgreSQL or similar relational data stores, including schema design and query optimisation.
• Experience building and consuming RESTful APIs at production scale.
• Strong understanding of time-series data, event data, normalisation, and database design.
• Proficiency with Git, code review practices, and Agile methodologies.
• Experience leading or owning the architecture of data systems end-to-end.
• Strong command of written and spoken English.

Desirable

• Experience with AWS or another cloud provider, including infrastructure-as-code (Terraform).
• Experience with Airflow or equivalent pipeline orchestration tools in production.
• Experience with containerisation (Docker) and orchestration (Kubernetes).
• Familiarity with hexagonal or medallion architecture patterns.
• Experience with monitoring and observability tools (Datadog, Grafana, Prometheus).
• Experience with NoSQL databases.
• Familiarity with sqlmesh or dbt for data transformation.
• Japanese language skills (advantageous for working with local market data sources).

We are a dynamic company dedicated to nurturing connections and innovating solutions to tackle market challenges head-on. If you thrive on customer satisfaction and turning ideas into reality, then you’ve found your ideal destination. Are you ready to embark on this exciting journey with us?

We make things happen

We act decisively and with purpose, going the extra mile.

We build
together

We foster relationships and develop creative solutions to address market challenges.

We are here to help

We are accessible and supportive to colleagues and clients with a friendly approach.

Our People Pledge

Don’t meet every single requirement? Research shows that women and people of color are less likely than others to apply if they feel like they don’t match 100% of the job requirements. Don’t let the confidence gap stand in your way, we’d love to hear from you! We understand that experience comes in many different forms and are dedicated to adding new perspectives to the team.

Kpler is committed to providing a fair, inclusive and diverse work-environment. We believe that different perspectives lead to better ideas, and better ideas allow us to better understand the needs and interests of our diverse, global community. We welcome people of different backgrounds, experiences, abilities and perspectives and are an equal opportunity employer.

By applying, I confirm that I have read and accept the Staff Privacy Notice