Jobiglo

No results.

Senior Inference Software Engineer

hpe · Spring

New
Hybrid Senior 137,000 - 315,000 USD/year 🇬🇧 English
Go Python Kubernetes vLLM TensorRT-LLM CUDA NCCL RDMA

Job description

About the role

HPE's Private Cloud AI organization is seeking a Senior Software Engineer to build and evolve the model runtime in HPE AI Essentials, the inference platform used by enterprises to run large language models on their own hardware, including air-gapped and sovereign environments. The focus is on execution efficiency: low tail latency and high GPU utilization on customer hardware of varying generations. The role is hybrid, with about two days per week in an HPE office.

Key responsibilities

  • Design, implement and own major components of the LLM serving deployment, including engine integration, continuous batching, KV cache management and reuse, and quantized execution.
  • Partner with inference teams to improve time-to-first-token, inter-token latency, throughput per GPU and P95/P99 tail latency.
  • Build distributed execution capabilities such as disaggregated prefill/decode, tensor and pipeline parallelism, and KV cache offload across GPU memory, host memory and RDMA-attached storage.
  • Evaluate emerging runtimes, quantization schemes, speculative decoding and mixture-of-experts serving and recommend adoption.
  • Contribute to the orchestration layer, including model admission, GPU scheduling and partitioning, cache-aware request routing and autoscaling.
  • Triage and resolve customer issues end-to-end, and review code and mentor team members.

Required profile

  • Minimum of 8 years of software engineering experience, including 1-2+ years on LLM inference runtimes or production model serving.
  • Degree in Computer Science or a related field.
  • Preferred: upstream contributions to vLLM, SGLang, TensorRT-LLM, llm-d, LMCache or KServe, and experience with air-gapped or regulated enterprise software delivery.

Required skills

  • Familiarity with LLM inference engines such as vLLM, SGLang, TensorRT-LLM, TGI or NVIDIA NIM, including engine internals.
  • Understanding of continuous batching, paged attention, KV cache reuse, quantization and speculative decoding.
  • Working knowledge of tensor and pipeline parallelism, NCCL and the GPU memory hierarchy.
  • Advanced Kubernetes knowledge, including operators, custom resources, controllers and scheduling.
  • Strong Go and Python, and the ability to debug and profile C++/CUDA with tools such as Nsight.

What we offer

  • Base salary of USD 144,000-273,000 in Colorado, and USD 137,000-315,000 in North Carolina and Texas; variable incentives may also be offered.
  • Benefits covering health and wellbeing, and programs for professional development.

Questions fréquentes

Le salaire proposé pour ce poste est de 137-315k USD par an. Le détail figure dans l'annonce.
Cliquez sur "Postuler maintenant" en haut de la page. Vous pouvez importer votre CV en 1 clic — Jobiglo extrait automatiquement vos informations et postule pour vous.
Source : ats:workday

Why are you reporting this job?

Thank you for your report. We will review this job.

Apply in 30 seconds

Enter your email to apply. An account will be created automatically.

Apply now →

By continuing, you accept our terms of use.

Already have an account? Login

A question about this job?

Ask it here: you will get the full job summary by e-mail, right away.

Published 6 hours ago

Expires 1 month from now

1 views · 0 interested

Boost your chances

Upload your CV — we will match you with relevant openings.

Analyzing your CV...

hpe

Spring