Senior Engineering Manager, Agentic & Generative AI Benchmarking and Evaluations

ServiceNow Senior Engineering Manager, Agentic & Generative AI Benchmarking and Evaluations 16 Hours AgoSaved Remote or Hybrid Santa Clara, CA, USA 201K-352K Annually Senior level 201K-352K Annually Senior levelArtificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • AutomationLead a team building evaluation infrastructure and benchmarks for agentic and generative AI. Design automated tests, validate RAG and knowledge grounding, benchmark LLMs, optimize model selection, and translate evaluation results into product and risk recommendations while collaborating cross-functionally and growing an AI-native engineering team.Top Skills: AnthropicBleuCmdbsG-EvalGoogle (Llms)Hybrid SearchKnowledge GraphsLlm-As-A-JudgeNumpyOpenaiPandasPythonRetrieval-Augmented Generation (Rag)RougeSemantic Re-RankingSQL