Senior Backend Engineer – AI Platform
careers.azx.io
Job description
About the role
AZX is building an AI platform that provides a governed front‑door to multiple large language models, both hosted and self‑hosted. As a Senior Backend Engineer you will design and operate the core services that power this platform, enabling enterprise customers to track usage, control costs, and enforce policies.
Key responsibilities
- Own the LLM gateway, handling routing, metering, budgeting, guardrails, and adapters for various model providers.
- Develop and maintain the retrieval and knowledge service, including connectors, chunking, hybrid search, reranking, and grounded answers.
- Build an evaluation harness that turns retrieval tuning into evidence with confidence intervals and segment breakdowns.
- Maintain the async framework’s public API, rate‑limit algorithms, and admin tooling, evolving it for durable agent execution.
- Design and enforce authorization across the platform, implementing token hierarchies, capability grammars, and per‑tenant isolation.
Required profile
- 5+ years of production‑grade async Python experience, including cancellation scopes, streaming lifecycles, and connection pooling.
- Deep knowledge of PostgreSQL internals (MVCC, advisory locks, vacuum) and appropriate use of Redis/Dragonfly.
- Proven experience shipping stable API surfaces with versioning, idempotency, error vocabularies, and migration discipline.
- Expertise in at least one LLM‑gateway or retrieval engineering domain, with credibility in the other.
- Cost‑accounting and evaluation‑first mindset, having shipped gates or harnesses that caught regressions.
Required skills
- Python 3.12+ (async, type‑strict), FastAPI/Starlette, Pydantic v2.
- asyncpg, SQLAlchemy, PostgreSQL (including pgvector).
- Redis, Dragonfly, Lua scripting.
- OpenTelemetry, SSE/streaming, OpenAI & Anthropic APIs.
- Job frameworks such as Celery, Dramatiq, Prefect; rate‑limit algorithms (token bucket, GCRA).
What we offer
- Competitive early‑stage startup compensation with equity.
- Bonus eligibility and comprehensive health insurance covering dependents.
- Flexible paid time off and a fully remote work culture.
- Opportunities to work with industry‑leading clients in energy, real‑estate, logistics, and utilities.
Questions fréquentes
Why are you reporting this job?
Explore further
Salaries, guides and searches in the United States.
Salary: Senior Software Engineer Based on 42 job offers in the United StatesSalaries by job title
- Delivery Specialist 330
- Line Cook 264
- Restaurant Host 251
- Dishwasher 242
- Shift Supervisor 226
- Parts Specialist 208
- Retail Service Specialist 176
- Server - Red Robin 140
Apply in 30 seconds
Enter your email to apply. An account will be created automatically.
By continuing, you accept our terms of use.
Already have an account? Login
A question about this job?
Ask it here: you will get the full job summary by e-mail, right away.
Published 1 month ago
Expires 15 hours from now
34 views · 0 interested
Boost your chances
Upload your CV — we will match you with relevant openings.
Analyzing your CV...
careers.azx.io