Senior Python and Kafka Engineer

Job Description:

Event-Driven Systems, Observability and Data Lineage

Position Summary

We are seeking a hands-on Senior Python and Kafka Engineer to join an established engineering team supporting a correctness-critical, event-driven digital repository processing platform.

The platform performs downstream processing after digital objects have been deposited and archived. Its services create persistent identifiers, update operational metadata and search stores, generate viewable and streamable derivatives, distribute content to image, media and full-text services, and notify depositors. The pipeline is event-driven, with an initial archival event being processed by multiple independent services that publish and consume subsequent events.

The successful candidate will work directly within the existing team to improve structured logging, end-to-end event visibility, reconciliation, completion verification, and Kafka delivery reliability. This is an implementation-focused engineering position. The architecture is owned internally, and the selected engineer will build enhancements into a live system rather than redesigning or replacing it.

Candidates should have strong, production-level experience in all of the following areas:

Apache Kafka

• Significant hands-on experience operating or developing Kafka-based production applications.

• Deep understanding of producer and consumer delivery semantics.

• Experience with Kafka offsets, commit strategies, consumer groups, partition assignment, and rebalancing.

• Demonstrated ability to diagnose missing, delayed, duplicated, or repeatedly retried messages.

• Practical experience implementing retry and dead-letter processing patterns.

Python

• Advanced Python software-engineering experience.

• Strong understanding of concurrency, threading, and runtime behavior.

• Experience building and supporting production services and shared Python libraries.

• Ability to write maintainable, testable, and operationally supportable code.

Observability and Structured Logging

• Experience defining and implementing structured logging standards, not only configuring logging products.

• Ability to establish consistent log fields, severity levels, correlation information, error context, and diagnostic detail.

• Understanding of observability practices for distributed and event-driven systems.

PostgreSQL

• Strong working knowledge of PostgreSQL.

• Experience using PostgreSQL JSON capabilities.

• Ability to design and query operational, tracking, or event-related data structures.