Jobiglo

No results.

Staff Site Reliability Engineer - AI Gateway (Dev Platform)

Kraken

Senior 🇬🇧 English
Kubernetes Terraform Rust Python Consul Vault SLOs Observability API Gateways vLLM Ray FinOps

Job description

About the role

Kraken, part of Payward, is looking for a Staff Site Reliability Engineer within its Dev Platform team. The team builds the internal platform that Kraken's engineers use to build, ship and run software, and the AI Gateway is the company-wide LLM gateway giving teams self-service access to models from providers such as Anthropic, OpenAI and Google. The role owns the reliability, availability and performance of the AI Gateway as a production-critical service.

Key responsibilities

  • Own the reliability, availability and performance of the AI Gateway, including SLOs, error budgets and on-call practices.
  • Design resilience: multi-provider routing, failover, retries, rate limiting and graceful degradation when an upstream provider is slow or unavailable.
  • Set technical direction for gateway architecture, capacity planning and scaling as LLM traffic grows.
  • Build observability for LLM traffic, covering latency, errors, token usage and cost, attributed by team and use case.
  • Work with Security, Finance and AI governance on access controls, budgets, audit logging and data-handling requirements.
  • Lead incident response and blameless post-incident reviews, turning lessons into reliability improvements.
  • Automate operations through infrastructure as code, safe deployments and progressive rollouts.
  • Mentor engineers and raise the operating standard across Dev Platform.

Required profile

  • Extensive experience in site reliability, infrastructure or platform engineering, owning high-traffic, business-critical services in production.
  • Staff-level influence: sets direction across teams, writes clear design documents and balances reliability, cost and developer velocity.
  • Good judgment on which reliability work matters most.

Required skills

  • Deep hands-on experience with Kubernetes, a major cloud provider and infrastructure as code (e.g. Terraform).
  • Strong command of SLOs, observability and incident management.
  • Proficiency in Rust or Python, and comfort reading and contributing to a production codebase.
  • API gateways, reverse proxies or traffic management: load balancing, rate limiting, failover and caching.
  • Consul and Vault for service discovery, configuration and secrets management.
  • Authentication and authorization (OAuth 2.0/OIDC, JWT, mTLS, API keys) for multi-tenant services.
  • Model evaluation pipelines and familiarity with coding agents and agent memory systems (vector stores, search, relational and key-value stores).

Nice to have: experience with LLM gateways or provider APIs at scale, hosting and serving models (vLLM, Ray), Kubernetes controllers and operators, cost attribution and FinOps, and a background in regulated environments such as finance or crypto. Applications are accepted on an ongoing basis; assessments may be asked for during hiring.

Questions fréquentes

Le salaire n'est pas communiqué publiquement par le recruteur. Vous pouvez postuler et négocier directement avec Kraken.
Cliquez sur "Postuler maintenant" en haut de la page. Vous pouvez importer votre CV en 1 clic — Jobiglo extrait automatiquement vos informations et postule pour vous.

Why are you reporting this job?

Thank you for your report. We will review this job.

Explore further

Salaries, guides and searches in the United States.

Apply in 30 seconds

Enter your email to apply. An account will be created automatically.

Apply now →

By continuing, you accept our terms of use.

Already have an account? Login

A question about this job?

Ask it here: you will get the full job summary by e-mail, right away.

Published 3 weeks ago

Expires 1 month from now

12 views · 0 interested

Boost your chances

Upload your CV — we will match you with relevant openings.

Analyzing your CV...

Kraken