Product Manager - AI Inference & Model Serving

About the role

Mirantis, a Kubernetes-native AI infrastructure company, is looking for a commercially driven, deeply technical Product Manager to own AI inference and model serving for k0rdent AI, its control plane for GPU infrastructure and distributed AI workloads. The role defines how NeoClouds and Enterprise customers deploy, scale and operate production inference services, covering serverless inference, dedicated endpoints, workload placement, autoscaling, routing, lifecycle management, observability and full-stack performance optimization.

Key responsibilities

• Own the product strategy, roadmap and lifecycle for inference and model serving, including serverless inference, dedicated endpoints, autoscaling, routing, KV cache management and observability.
• Lead technical discovery with NeoClouds, sovereign clouds and enterprise platform teams and turn findings into prioritized requirements and architecture direction.
• Partner with engineering on trade-offs across runtime integration, GPU scheduling, network, storage and serving topology, including disaggregated and multi-model serving.
• Define positioning based on measurable outcomes such as latency distributions, throughput per GPU, utilization, tail reliability and cost per token.
• Drive go-to-market execution: pricing and packaging, reference architectures, sizing guides, PoC playbooks and direct engagement with customers, analysts and ecosystem partners.

Required profile

• 7+ years in product management, technical product management or a senior technical role owning AI/ML and inference products.
• Strong understanding of production AI inference, including model serving, serverless execution, dedicated endpoints, autoscaling, routing, workload placement, observability and reliability.
• Ability to reason about performance trade-offs across GPU, network, storage, orchestration and runtime layers and translate them into business value such as TTFT, throughput per GPU and TCOW.
• Credibility with engineering leaders and infrastructure operators, including production architecture reviews and technical commercial conversations with platform engineering buyers.

Required skills

• Working knowledge of modern inference runtimes: vLLM, SGLang, TensorRT-LLM, Dynamo, Triton.
• Optimization patterns in production: continuous batching, KV cache management, cold starts, prefill versus decode, disaggregated and multi-model serving.

What we offer

• Professional development and training, including conferences and working groups.
• A customized workstation (macOS, Windows).
• A competitive compensation package with a strong benefits plan and stock options.