Machine Learning Research Engineer

About the role

Etched builds AI chips hard-coded for individual model architectures, aiming to deliver throughput and latency far beyond GPUs. Its first product, Sohu, only supports transformers. The Machine Learning Research Engineer proposes and conducts novel research to achieve results on Sohu that are unviable on GPUs, translating core mathematical operations of popular Transformer-based models into highly performant instruction sequences for Sohu.

Key responsibilities

• Propose and conduct novel research to achieve results on Sohu that are unviable on GPUs.
• Translate core mathematical operations from popular Transformer-based models into maximally performant instruction sequences for Sohu.
• Develop deep architectural knowledge to inform best-in-the-world software performance on Sohu, collaborating with hardware architects and designers.
• Co-design and fine-tune emerging model architectures for highest efficiency on Sohu.
• Contribute to the Sohu software stack, performance characterization tools and runtime abstractions by implementing frontier models in Python and Rust.
• Implement a novel test-time compute algorithm that leverages Sohu's capabilities.
• Implement diffusion models on Sohu for real-time image generation.
• Optimize model instructions and scheduling algorithms for utilization, latency, throughput and a mix of them.
• Implement inference-time acceleration techniques such as speculative decoding, tree search, KV cache sharing and priority scheduling.

Required profile

• An ML research background with interest in hardware co-design.
• Experience working cross-functionally in diverse software and hardware organizations.
• Strong candidates: ML systems research and hardware co-design backgrounds, published inference-time compute or efficient ML research, and Rust experience.

Required skills

• Python, PyTorch and/or JAX.
• Familiarity with transformer architectures and/or inference serving stacks (vLLM, SGLang).
• Experience in distributed inference or training environments.
• Preferred: GPU kernel knowledge, the CUDA compilation stack and other hardware accelerators.

What we offer

• Full medical, dental and vision coverage, with 100% of premiums covered.
• Housing subsidy of $2,000 per month for those living within walking distance of the office.
• Daily lunch and dinner in the office.
• Relocation support for those moving to Cupertino.