Head of Inference – Edge AI Platform

About the role

We are looking for a visionary, execution‑oriented Head of Inference to define and own the inference architecture for Perimeter Compute’s edge‑AI platform. Reporting directly to the CEO, you will lead the design, proof‑of‑concept and production rollout of low‑latency, distributed inference services across multi‑GPU, multi‑node edge deployments.

Key responsibilities

• Define the overall inference strategy and architecture for edge AI workloads.
• Own the inference serving stack (vLLM, TensorRT‑LLM, Triton or equivalent) and deliver a fast, credible POC for partners and customers.
• Optimize cost‑per‑token, GPU utilization, KV‑cache management and batching for production workloads.
• Establish and monitor SLAs for latency, throughput, observability and reliability.
• Build distributed inference pipelines across multi‑GPU, multi‑node edge environments.
• Set performance baselines, define quantization strategies and guide infrastructure design.
• Lead integration of the software access layer and collaborate with advisors, partners and early customers.

Required profile

• Senior technical leader with hands‑on experience building large‑scale inference systems.
• Strong ability to translate complex AI requirements into robust infrastructure solutions.
• Excellent communication skills to represent the company with investors, partners and customers.

Required skills

• vLLM
• TensorRT‑LLM
• Triton Inference Server
• GPU programming and optimization
• KV‑cache management and batching techniques
• Quantization methods for inference
• Distributed inference pipelines on multi‑GPU, multi‑node edge deployments

What we offer

• Opportunity to shape the core technology of a pre‑seed AI venture.
• Close collaboration with the CEO and a network of seasoned advisors.
• Remote work with a preference for candidates based in New York City.