Research Engineer - Model Evaluation & MLOps

About the role

Sciforium is building next‑generation multimodal AI models and a high‑efficiency serving platform. As a Research Engineer focused on Model Evaluation & MLOps, you will create the tools and infrastructure needed to evaluate, deploy, and operate these models reliably on GPUs.

Key responsibilities

• Rapidly integrate internal and open‑weight language and multimodal models into GPU evaluation and inference environments.
• Build automated benchmarks for model quality and system performance, covering latency, throughput, and memory usage.
• Create reproducible comparisons across Sciforium models, external baselines, and runtime configurations.
• Develop and maintain experiment tracking, model registry, and versioning for models, datasets, and evaluation configurations.
• Automate the path from research checkpoints to validated deployments using CI/CD and reproducible workflows.
• Monitor model quality and system performance, diagnosing failures or regressions across pipelines.
• Build reusable tools that enable researchers to launch evaluations, compare experiments, and reproduce results.
• Profile end‑to‑end model workloads and collaborate with distributed systems, inference, and GPU kernel engineers on deeper performance issues.

Required profile

• 2+ years of professional ML or software engineering experience on production ML systems or MLOps infrastructure.
• Strong Python programming skills and experience building reliable production systems.
• Hands‑on experience with PyTorch, TensorFlow, or JAX and modern language or multimodal model architectures.
• Experience with model evaluation, benchmarking, experiment tracking, versioning, deployment, or monitoring.
• Proficiency running, benchmarking, and debugging models on GPU inference runtimes (e.g., vLLM, SGLang, TensorRT‑LLM) in containerized cloud or on‑prem environments.
• Clear documentation and collaboration skills across research, infrastructure, and product teams.
• MS or PhD in Computer Science, Computer Engineering, Machine Learning, or equivalent practical experience.

Required skills

• Python
• PyTorch
• TensorFlow
• JAX
• GPU inference runtimes (vLLM, SGLang, TensorRT‑LLM)
• Containerization (Docker/Kubernetes)
• CI/CD pipelines
• Experiment tracking and model registry
• Versioning and deployment workflows
• GPU benchmarking and performance profiling

What we offer

• Medical, dental, and vision insurance
• 401k plan
• Daily lunch, snacks, and beverages
• Flexible time off
• Competitive salary and equity
• Equal‑opportunity employment