AI Python Engineer
Indexed description
We are looking for a Senior Python GenAI Engineer to join internal teams working on AI-driven decision automation solutions. The role focuses on evaluating, testing, and improving Generative AI agents used in risk assessment, workflow automation, and document processing.
Key Responsibilities
- Evaluate GenAI agent outputs for accuracy, consistency, and business relevance
- Design and apply AI evaluation frameworks (quality, hallucination, reliability, bias, latency)
- Test underwriting / risk-assessment AI workflows using real business scenarios
- Improve prompts, agent logic, orchestration flows, and output quality
- Collaborate with engineers, analysts, and stakeholders on production AI systems
- Support continuous optimization of AI agents in regulated environments
- Define KPIs and benchmarks for GenAI performance
- 6+ years of commercial software development experience
- Hands-on experience with GenAI / LLM products in production
- Understanding of AI agents / agentic workflowsKnowledge of banking, fintech, underwriting, insurance, or risk domains is a strong plus
- Familiarity with Microsoft Copilot Studio is beneficial
- Experience with LangChain / LangGraph or similar orchestration tools
- Experience with evaluation frameworks (RAGAS, DeepEval, LangSmith, prompt testing, etc.) Strong Python skills
Create a free Caio profile to unlock more results and save your role and location preferences.
Unlock free search