Senior Backend Engineer - LLM Features

About the role

Deda, a technology group operating in the United States, is hiring an LLM Engineer to build the LLM features in its products. The role focuses on turning credit-union staff questions into queries against a semantic layer, extracting facts from incoming content, writing prompts and agent logic, building retrieval and adding guardrails, and measuring every feature with an evaluation set.

Key responsibilities

• Turn a staff member's question into a query against the semantic layer (metrics, dimensions, filters, time range), run it and explain the answer in plain language.
• Extract facts from incoming content against user-defined rules, with a citation back to the source, so alerts can be checked rather than believed.
• Generate narratives around measurement results that say only what the numbers support, with every figure traced to its source.
• Write prompts and agent logic, build retrieval, and keep prompts versioned in a shared repository.
• Build the shared guardrail library: tenant scoping outside the model, grounding checks, input/output validation and refusal when requests are out of scope.
• Build and run the shared evaluation harness and an evaluation set for every feature, wired into CI so no change ships without a score.
• Trace calls, monitor quality, latency and cost per feature, and identify which model and prompt version produced any answer.

Required profile

• Experience as an LLM, AI or ML engineer, or as a backend or data engineer who has shipped at least one LLM feature to real users and measured it.
• Ability to design an evaluation set and choose the right metric, and to calibrate an LLM-as-judge against human labels.
• Production experience with structured outputs, tool calling and multi-step agent flows, and the judgment to replace an agent step with plain code when appropriate.
• Solid SQL (PostgreSQL preferred) and an understanding of how a semantic layer defines metrics and dimensions.
• Treats prompt injection, tenant leakage and invented figures as real threats, designing against them with authorization outside the model and validated outputs.
• Solid Python, Git, code review, tests, CI and containers, with models treated as swappable components behind an interface.
• Can explain to a product manager why a change raised or lowered a score, and speaks up early when a feature cannot reach the needed quality.

Nice to have

• Text-to-SQL or semantic-layer querying in production (dbt Semantic Layer, Cube, LookML).
• Information extraction from documents and web content, and open-weight models served in-house (vLLM, TGI, Ollama).
• LLM observability and tracing (Langfuse, LangSmith, OpenTelemetry), search engines and vector stores (OpenSearch, Elasticsearch, pgvector).
• Financial services or regulated data (GLBA, GDPR), and orchestration tools such as Dagster.
• Java/Spring Boot, which runs part of the existing LLM code and the backend serving these features.