Senior Software Engineer (AI Inference & Runtime Platform)

About the role

AZX is building an AI inference and runtime platform that powers open‑weight models across managed GPU clouds and customer‑managed Kubernetes clusters. As a Senior Software Engineer you will own the layer where AI workloads run, from the inference control plane to the sandboxed agent environment, ensuring reliability, security and cost‑effective scaling.

Key responsibilities

• Manage the serving tier for open‑weight models, including engine deployment, cold‑start strategy, per‑model SLOs and upgrade/canary processes.
• Administer the Kubernetes layer for inference and sandbox workloads: operators, CRDs, autoscaling (KEDA/Karpenter), GPU scheduling and node lifecycle.
• Own the stateful data plane end‑to‑end with tested restore procedures.
• Oversee the sandbox runtime and host‑side control plane, handling lifecycle, snapshot/fork, metering and threat‑model enforcement.
• Direct FastAPI control‑plane services, infrastructure‑as‑code (Terraform/OpenTofu, Bicep) and monitoring dashboards.

Required profile

• 5+ years shipping production systems in a systems language; strong Rust experience (or deep Go, C/C++, Zig).
• Hands‑on experience operating Kubernetes workloads at scale, including custom operators and autoscaling.
• Proven ability to threat‑model isolation boundaries and implement security best practices for untrusted code execution.
• Experience deploying or operating open‑weight LLM serving infrastructure (e.g., vLLM, SGLang).

Required skills

• Rust (tokio), Go, C/C++, Zig
• Kubernetes operators, CRDs, controller‑runtime/Kubebuilder
• KEDA, Karpenter, GPU device plugins/DRA
• FastAPI, Python
• Terraform/OpenTofu, Bicep
• Firecracker, Kata, gVisor (sandbox technologies)

What we offer

• Competitive early‑stage startup compensation with bonus eligibility.
• Health insurance with meaningful coverage for dependents.
• Flexible paid time off and equity participation.
• Fully remote culture with a cluster of teammates in Seattle.