Senior Backend Engineer

About the role

LiteLLM is seeking a Senior Backend Engineer to own and enhance the guardrail and observability layers of its AI gateway platform. You will ensure that security policies are reliably logged, errors are visible to users, and monitoring scales to billions of requests per month.

Key responsibilities

• Build and scale backend services with a focus on performance, reliability, and continuous improvement.
• Log and trace all guardrail and policy enforcement calls through SpendLogs and related database tables.
• Design CPU‑level guardrails to protect against common attacks on LLM APIs, MCP servers, and agents.
• Identify and fix silent failures in guardrail creation, registration, and policy application.
• Integrate and maintain observability tools such as Datadog, Splunk, Prometheus, and OpenTelemetry.
• Optimize observability for 1B+ requests/month with minimal latency and no memory leaks.
• Collaborate cross‑functionally on performance, reliability, and security priorities.

Required profile

• Bachelor’s or Master’s degree in Computer Science or a related field.
• At least 1 year of professional experience with Python and backend frameworks.
• Strong understanding of logging best practices, error handling, and secure backend development.
• Experience with monitoring, logging, or metrics platforms.
• Familiarity with database integration and troubleshooting.
• Drive to deliver high‑quality code with robust auditing and debugging capabilities.

Required skills

• Python
• FastAPI
• Flask
• Datadog
• Splunk
• Prometheus
• OpenTelemetry
• PostgreSQL
• Redis

What we offer

• High‑impact work on compliance and reliability for enterprise customers.
• Opportunity to shape core features used at global scale.
• Fast‑paced growth environment with technical ownership.
• Competitive salary and health, dental, and vision benefits.