AI Engineer
Indexed description
The Problem You’ll Own
LLMs are powerful. They are also brittle in ways that matter enormously when a model is making decisions about a multi-million dollar purchase order, a patient record, or a financial transaction. Hallucinations, context failures, retrieval mismatches, and inconsistent outputs are not acceptable in our use case.
Your job is to build the systems that make AI reliable in exactly these environments: validation agents that reason about business rules, evaluation pipelines that catch regressions, and observability tooling that tells you when something has gone wrong before the customer does.
What You’ll Do
- Build and deploy LLM-powered agents for enterprise data validation — reading specs, reasoning about business rules, identifying failure modes, and generating structured outputs
- Design and own evaluation frameworks: automated test suites, LLM-as-judge pipelines, regression detection, and benchmarks that track whether our agents are improving
- Build RAG pipelines that work reliably on real enterprise data — messy schemas, inconsistent formats, mixed structured and unstructured content
- Integrate AI systems with enterprise infrastructure (SAP, Snowflake, Databricks, Postgres, REST APIs) with attention to latency, data residency, and compliance
- Design agentic workflows with tool use, multi-step reasoning, and deterministic guardrails
- Build observability tooling: trace agent reasoning, track output reliability, and detect hallucinations or drift in production
- Work directly with FDSEs to understand real deployment failures and translate them into system improvements
- Languages: Python (primary), Go, Node.js
- AI/ML: LLMs (Claude, GPT-4, Command R+), RAG, vector databases, embeddings, fine-tuning
- Evaluation: LLM-as-judge, automated eval pipelines, custom benchmarks
- Data: Snowflake, Databricks, Postgres
- Infra: containers, Kubernetes / ECS / Cloud Run
- Tools: LangChain, LlamaIndex, OpenAI / Anthropic APIs, LangSmith
Equity: Early-stage equity grant
- Production builder: you’ve shipped LLM-powered features real users depend on and debugged them when they broke
- LLM practitioner: you understand hallucinations, retrieval failures, context limits, and what it takes to make agents deterministic enough for enterprise use
- Systems thinker: you design for latency, failure modes, retry logic, and observability before features
- Enterprise-aware: data residency, compliance, audit trails, and deterministic guardrails are first-class design constraints for you
- 3+ years in AI/ML or backend engineering with strong AI exposure
- Hands-on production experience with LLM APIs (Anthropic, OpenAI, Cohere)
- Experience designing evaluation frameworks: automated evals, regression tests, or LLM-as-judge pipelines
- Strong Python; experience with LangChain, LlamaIndex, or similar agentic frameworks
- Familiarity with RAG architectures: chunking, embedding models, vector DBs, retrieval quality
Create a free Caio profile to unlock more results and save your role and location preferences.
Unlock free search