Senior Engineer

Senior Engineer / Remote or Berlin / 3 – 6 month Freelance Contract / Start ASAP

They have an on CALL ROTAION within the team Out of Hours, you will need to be ok with this.

Background:

This team will work on the clients Evaluations Platform, within the Simulations and Evaluations domain under the Agent Builder Tooling organization, contributing to the Simulations and Evaluations 3.0 roadmap. The focus is on building backend services that automatically generate and run tests for AI agents by orchestrating simulations and executing the evaluations engine programmatically, at enterprise scale.

The domain is about making agent testing fast to set up, scalable to thousands of conversations, and actionable: enabling template-driven and automated test creation (from agent definitions and conversation transcripts), running high volume simulation-based test suites, and producing repeatable evaluation outcomes that help teams detect regressions and continuously improve agents with clear recommendations

Skill Set needed:

Expertise in designing, developing, and operating high-performance, low latency software systems for real-time processing and streaming data.

Deep proficiency in Python and less critical but ideal: TypeScript, with a focus on architecting scalable, maintainable, and testable solutions.

Strong foundation in design patterns, clean coding, and data-intensive, event driven systems (e.g., Kafka or NATS.

Advanced knowledge of API design principles for efficient and scalable dataflow solutions.

Deep understanding of how product analytics can drive user-centric insights and create business value.

Expertise in designing systems and leading complex technical projects across a 9-18 month roadmap.

Thrives in fast-paced, agile environments, consistently delivering high-quality, scalable code while driving impactful projects to successful completion.

Demonstrated strong leadership and collaboration skills, effectively communicating and working seamlessly across cross-functional teams to achieve shared business objectives.

Strong end-to-end ownership of multi-month deliverables, including driving scope, execution, delivery, and operational readiness (quality, observability, reliability), with minimal oversight.

Product engineering mindset: ability to deeply understand the problem space, clarify ambiguous requirements, propose solutions and trade-offs, and partner with product and stakeholders to shape the right outcome (not just implement a handed-down specification).

Operational excellence: proven ability to operate production services, including on-call readiness, monitoring and observability, and effective response to alerts, incidents, and other operational events (triage, mitigation, and follow-up through root-cause analysis and prevention).