Senior Software Engineer - Data Team (AWS, Flink, OpenSearch)
Indexed description
We are seeking a Senior Streaming Platform Engineer (6+ years) with deep expertise in enterprise-scale streaming and cloud data platforms. Based in Atlanta, this role collaborates closely with teams in Salt Lake City and Pune, with reasonable overlapping hours. The ideal candidate brings hands-on experience managing scalable, secure, and highly available streaming ecosystems that support real-time analytics, event-driven architectures, and enterprise data platforms — including exposure to AI-assisted engineering and automation practices.
What You’ll Do
- Administer, optimize, and govern Kafka, AWS MSK, Kinesis, Kafka Connect, and OpenSearch platforms
- Design scalable, fault-tolerant, and secure streaming architectures
- Monitor, troubleshoot, and performance-tune distributed systems; drive capacity planning
- Implement security controls, automation, and operational best practices
- Support integrations with data lakes, Snowflake, analytics platforms, and downstream systems
- Automate infrastructure provisioning using Terraform or similar IaC tools
- Leverage AI-assisted development tools (Claude or equivalent) to improve engineering productivity and platform automation
- Collaborate with engineering and analytics teams to enable real-time data processing and AI-driven data solutions
Required:
- 6+ years of experience with Kafka administration and distributed streaming systems
- Strong AWS cloud services experience (MSK, Kinesis, and related services)
- Linux administration and automation/scripting skills
- Python development and platform reliability engineering
- Desire to learn and implement evolving and new technologies
- Experience with Flink using Spring Boot
- Snowflake integration experience
- Kubernetes orchestration
- Administration knowledge in OpenSearch clusters
- Familiarity with AI-enabled engineering workflows and tools
- Streaming platforms operate with high availability, performance, and reliability at enterprise scale
- Governance, security, and automation practices are consistently applied across the ecosystem
- Engineering teams are unblocked and empowered through well-integrated data pipelines and real-time infrastructure
- AI tooling meaningfully improves operational efficiency and engineering velocity
Create a free Caio profile to unlock the full index and keep your job-search signal for future recommendations.
Unlock free search