Staff Site Reliability Engineer - AI Gateway (Dev Platform)
Kraken
Job description
About the role
Kraken, part of Payward, is looking for a Staff Site Reliability Engineer within its Dev Platform team. The team builds the internal platform that Kraken's engineers use to build, ship and run software, and the AI Gateway is the company-wide LLM gateway giving teams self-service access to models from providers such as Anthropic, OpenAI and Google. The role owns the reliability, availability and performance of the AI Gateway as a production-critical service.
Key responsibilities
- Own the reliability, availability and performance of the AI Gateway, including SLOs, error budgets and on-call practices.
- Design resilience: multi-provider routing, failover, retries, rate limiting and graceful degradation when an upstream provider is slow or unavailable.
- Set technical direction for gateway architecture, capacity planning and scaling as LLM traffic grows.
- Build observability for LLM traffic, covering latency, errors, token usage and cost, attributed by team and use case.
- Work with Security, Finance and AI governance on access controls, budgets, audit logging and data-handling requirements.
- Lead incident response and blameless post-incident reviews, turning lessons into reliability improvements.
- Automate operations through infrastructure as code, safe deployments and progressive rollouts.
- Mentor engineers and raise the operating standard across Dev Platform.
Required profile
- Extensive experience in site reliability, infrastructure or platform engineering, owning high-traffic, business-critical services in production.
- Staff-level influence: sets direction across teams, writes clear design documents and balances reliability, cost and developer velocity.
- Good judgment on which reliability work matters most.
Required skills
- Deep hands-on experience with Kubernetes, a major cloud provider and infrastructure as code (e.g. Terraform).
- Strong command of SLOs, observability and incident management.
- Proficiency in Rust or Python, and comfort reading and contributing to a production codebase.
- API gateways, reverse proxies or traffic management: load balancing, rate limiting, failover and caching.
- Consul and Vault for service discovery, configuration and secrets management.
- Authentication and authorization (OAuth 2.0/OIDC, JWT, mTLS, API keys) for multi-tenant services.
- Model evaluation pipelines and familiarity with coding agents and agent memory systems (vector stores, search, relational and key-value stores).
Nice to have: experience with LLM gateways or provider APIs at scale, hosting and serving models (vLLM, Ray), Kubernetes controllers and operators, cost attribution and FinOps, and a background in regulated environments such as finance or crypto. Applications are accepted on an ongoing basis; assessments may be asked for during hiring.
Questions fréquentes
Why are you reporting this job?
Explore further
Salaries, guides and searches in the United States.
Salaries by job title
- Delivery Specialist 330
- Line Cook 264
- Restaurant Host 251
- Dishwasher 242
- Shift Supervisor 226
- Parts Specialist 208
- Retail Service Specialist 176
- Server - Red Robin 140
Apply in 30 seconds
Enter your email to apply. An account will be created automatically.
By continuing, you accept our terms of use.
Already have an account? Login
A question about this job?
Ask it here: you will get the full job summary by e-mail, right away.
Published 3 weeks ago
Expires 1 month from now
12 views · 0 interested
Boost your chances
Upload your CV — we will match you with relevant openings.
Analyzing your CV...
Kraken