Jobiglo

No results.

Principal Software Engineer, Inference

hpe · Spring

New
Hybrid Senior 157,000 - 361,000 USD/year 🇬🇧 English
vLLM SGLang TensorRT-LLM TGI NVIDIA NIM Kubernetes Go Python CUDA RDMA GPUDirect Storage InfiniBand RoCE MIG

Job description

About the role

HPE's Private Cloud AI organization is seeking a Principal Software Engineer to lead the model runtime within HPE AI Essentials, the inference platform used by enterprises to run large language models on their own hardware, including air-gapped and sovereign environments. The focus is on sustained execution efficiency: low tail latency and high GPU utilization on customer-owned hardware of varying generations. The role is hybrid, with about two days per week in an HPE office.

Key responsibilities

  • Define and own the technical direction of LLM serving, including engine integration, continuous batching, KV cache management and reuse, and quantized execution.
  • Partner with inference performance teams, accountable for time-to-first-token, inter-token latency, throughput per GPU and P95/P99 tail latency.
  • Define distributed inferencing strategy, including disaggregated prefill/decode, tensor and pipeline parallelism, and KV cache offload across GPU, host memory and RDMA-attached storage.
  • Evaluate emerging runtimes, quantization schemes, speculative decoding and mixture-of-experts serving, and decide whether each is adopted, developed in-house or declined.
  • Define the orchestration layer: model admission, GPU scheduling and partitioning, cache-aware request routing and autoscaling.
  • Mentor engineers, lead design and architecture reviews, and present technical direction to business unit and executive audiences.

Required profile

  • Minimum 12 years of software engineering experience, including more than 1 year on LLM inference runtimes or production model serving.
  • Degree in Computer Science or a related field.
  • Production experience with LLM inference engines such as vLLM, SGLang, TensorRT-LLM, TGI or NVIDIA NIM, including modifying engine internals.
  • Deep understanding of inference internals: continuous batching, paged attention, KV cache reuse, chunked prefill, quantization and speculative decoding.
  • Expert-level knowledge of Kubernetes operators, custom resources, controllers and scheduling.
  • Strong Go and Python, and the ability to debug and profile C++/CUDA with tools such as Nsight.

Preferred

  • Upstream contributions to vLLM, SGLang, TensorRT-LLM, LLM-D, LMCache or KServe.
  • Experience with disaggregated prefill/decode, KV cache offload at scale, RDMA, GPUDirect Storage, InfiniBand or RoCE, MIG and multi-tenant GPU isolation.
  • On-premises, air-gapped or regulated enterprise software delivery.

What we offer

  • Base salary of USD 165,500 to 313,500 in Colorado, and USD 157,000 to 361,000 in North Carolina and Texas; variable incentives may also be offered.
  • Benefits covering health and wellbeing, and professional development programs.

Questions fréquentes

Le salaire proposé pour ce poste est de 157-361k USD par an. Le détail figure dans l'annonce.
Cliquez sur "Postuler maintenant" en haut de la page. Vous pouvez importer votre CV en 1 clic — Jobiglo extrait automatiquement vos informations et postule pour vous.
Source : ats:workday

Why are you reporting this job?

Thank you for your report. We will review this job.

Apply in 30 seconds

Enter your email to apply. An account will be created automatically.

Apply now →

By continuing, you accept our terms of use.

Already have an account? Login

A question about this job?

Ask it here: you will get the full job summary by e-mail, right away.

Published 7 hours ago

Expires 1 month from now

2 views · 0 interested

Boost your chances

Upload your CV — we will match you with relevant openings.

Analyzing your CV...

hpe

Spring