Cyber Evaluations Engineer

About the role

Anthropic is looking for a Cyber Evaluations Engineer to design, run, and analyze evaluations that measure cyber‑related capabilities and safeguard robustness of its AI models. The role sits at the intersection of AI safety, security research, and policy, helping ensure that new models resist jailbreaks and other forms of misuse.

Key responsibilities

• Design and execute capability, uplift, and safety evaluations to assess cyber‑risk in emerging models.
• Perform per‑release robustness testing ahead of major model launches.
• Analyze evaluation data, including jailbreak and prompt‑bypass incidents, and communicate findings to engineering and policy teams.
• Prototype and tune detection probes for cyber abuse, contributing to a layered abuse‑detection architecture.
• Collaborate with the cyber policy team to translate policy requirements into technical safeguards and measure their precision over time.
• Build and maintain internal tooling for running and scoring evaluations.
• Work with cross‑functional partners to turn evaluation insights into concrete safeguard improvements.

Required profile

• Proven experience building or running evaluations, benchmarks, or test suites for software or ML systems on short timelines.
• Hands‑on cybersecurity background (CTF participation, vulnerability research, exploit development, or security research).
• Strong ability to communicate evaluation results to technical and policy stakeholders.
• Eligibility for a secret security clearance or ability to obtain one.

Required skills

• Python programming.
• Experience with detection content authoring (e.g., Sigma, YARA, Suricata, SIEM rules).
• Familiarity with AI/ML evaluation frameworks.