Senior Director, Inference Products and Optimizations

DigitalOcean Senior Director, Inference Products and Optimizations Reposted 23 Hours AgoSaved In-Office San Francisco, CA, USA 274K-343K Annually Senior level 274K-343K Annually Senior levelArtificial Intelligence • Cloud • Software • Infrastructure as a Service (IaaS)Lead and grow an engineering organization building and scaling DigitalOcean's LLM inference product suite. Own product roadmap execution, inference-serving architecture, model optimizations (quantization, KV-cache, speculative decoding), and GPU-aware performance. Recruit and mentor engineers, partner cross-functionally, ensure production reliability and SLAs, institutionalize benchmarking, observability and auto-tuning, and drive contributions to open-source inference engines.Top Skills: Amd GpusContainer RuntimesDisaggregated ServingKubernetesKv-Cache ManagementLlm-DMicroservicesNvidia GpusNvlinkQuantizationServerlessSglangSpeculative DecodingVllm