Senior AI Compute Infrastructure Engineer
Kraken
Job description
About the role
Kraken, founded in 2011 and part of the Payward group, is building a dedicated AI Compute and Infrastructure team within engineering leadership. The team owns the infrastructure layer that lets Kraken run AI workloads with control, speed, reliability and cost discipline, and works closely with AI/ML researchers, platform engineers, security teams and product teams.
Key responsibilities
- Own and operate GPU and accelerator clusters for training, inference, evaluation and experimentation, including drivers, runtimes, kernels, device plugins, node configuration, scheduling primitives and workload isolation.
- Design infrastructure so models can run locally on GPUs where it is strategically and economically preferable, reducing reliance on external providers.
- Build and improve scheduling, orchestration, placement, quota management and utilization systems across heterogeneous accelerator environments.
- Optimize inference pipelines for latency, throughput, reliability, memory efficiency and cost using frameworks such as vLLM, Triton Inference Server, TensorRT or equivalent.
- Partner with ML engineers and researchers to remove bottlenecks in training, evaluation, batch and online inference, deployment and production debugging.
- Build observability for GPU utilization, memory pressure, queue depth, token throughput, request latency, failed workloads, capacity pressure and spend.
- Drive reliability, incident response, alerting, runbooks and post-incident improvements.
- Evaluate new hardware, cloud instance families, accelerators, runtimes and serving frameworks, and build tooling that makes GPU usage visible and easy for internal teams to use.
Required profile
- 5+ years of infrastructure engineering experience, with significant time on GPU compute, ML infrastructure, distributed systems, HPC or large-scale production platforms.
- Hands-on experience operating GPU clusters or accelerator-backed infrastructure in production or production-like environments.
- Strong systems engineering fundamentals across Linux, networking, storage, containers, Kubernetes, distributed runtimes and production debugging.
- Track record of optimizing compute costs while keeping performance, reliability and availability expectations clear.
- Comfortable in high-stakes, always-on environments and able to explain infrastructure tradeoffs to researchers, product teams, security stakeholders and engineering leadership.
Required skills
- Proficiency in Python for infrastructure automation, tooling and debugging.
- Experience with ML serving frameworks such as vLLM, Triton Inference Server, TensorRT, TorchServe, KServe, Ray Serve or equivalents.
- Nice to have: experience at a frontier AI lab, hyperscaler, high-frequency trading firm or research platform; familiarity with custom silicon or specialized accelerators; background in capacity planning or GPU fleet cost management; experience with distributed training frameworks, CUDA debugging, Rust, C++, Go or CUDA.
Questions fréquentes
Why are you reporting this job?
Explore further
Salaries, guides and searches in the United States.
Salary: Senior Software Engineer Based on 42 job offers in the United StatesSalaries by job title
- Delivery Specialist 330
- Line Cook 264
- Restaurant Host 251
- Dishwasher 242
- Shift Supervisor 226
- Parts Specialist 208
- Retail Service Specialist 176
- Server - Red Robin 140
Apply in 30 seconds
Enter your email to apply. An account will be created automatically.
By continuing, you accept our terms of use.
Already have an account? Login
A question about this job?
Ask it here: you will get the full job summary by e-mail, right away.
Published 11 hours ago
Expires 1 month from now
11 views · 0 interested
Boost your chances
Upload your CV — we will match you with relevant openings.
Analyzing your CV...
Kraken