Kafka Migration engineer ( .NET + Kafka)

Tasks

Kafka Migration Engineer (.NET + Kafka)

Location: London

Duration: 4 months

Note: This role requires strong .NET skills in addition to Kafka expertise. All producers and consumers are .NET applications — APIs produce messages, and some services both consume and produce. This is not an infrastructure-only Kafka role; it requires reading, modifying, and validating .NET Kafka client code.

Responsibilities

Migrate all Kafka producers and consumers from Kafka to Confluent Cloud

Update Confluent.Kafka .NET client configuration (SASL auth, Schema Registry, bootstrap servers)

Validate message flows end-to-end for apps acting as both producer and consumer

Ensure consumer group configurations, offset management, and error handling are correct post-migration

Kafka throughput validation and performance testing

Work with .NET engineers to resolve application-level Kafka integration issues

Requirements

Required Skills

C# / .NET — strong coding ability (reading and modifying producer/consumer code)

Confluent.Kafka .NET client library (ProducerBuilder, ConsumerBuilder, configuration)

Confluent Cloud — SASL/PLAIN authentication, Schema Registry, ACLs

Kafka internals: topics, partitions, consumer groups, offset management, rebalancing

Message serialization (Avro, JSON Schema) with .NET-specific serializers/deserializers

Troubleshooting Kafka connectivity and message flow issues in .NET applications

Kafka monitoring (consumer lag, throughput, error rates)

Nice to Have

Experience with apps that are both consumers and producers (event-driven architectures)

Schema evolution and compatibility strategies in Schema Registry

Performance tuning (batch size, compression) for .NET producers

Kafka Connect and MirrorMaker for migration scenari

Find more English Speaking Jobs in United Kingdom on Arbeitnow