Staff Software Engineer, Foundations

About the role

Vanta is looking for a Staff Software Engineer to lead the re‑platforming of the data layer that powers its compliance product. You will own a complex migration from a Mongo‑centric architecture to a schema‑aware, Postgres‑backed system while ensuring zero data loss and preserving all public API contracts.

Key responsibilities

• Lead the migration of Vanta’s resource data model from MongoDB to PostgreSQL, running both generations in parallel without breaking customer integrations.
• Drive cross‑team solutions that depend on the Foundations platform.
• Design for correctness under eventual consistency, using idempotent session handling, conditional writes, and explicit back‑pressure.
• Design and evolve the data ingestion and pipeline architecture to handle terabyte‑scale streams with high throughput.
• Take the internal Query API to production with schema versioning, joins, exports, per‑tenant isolation, and predictable latency.
• Build data infrastructure whose correctness can be externally audited for compliance evidence.
• Diagnose production‑only failures such as hot partitions, unbounded fan‑out, and online rewrites of continuously written tables.
• Set architectural direction for streaming (Kafka), caching (Redis), and persistent storage (PostgreSQL, MongoDB) across the Foundations stack.
• Mentor senior engineers and raise the technical bar across the team.

Required profile

• Proven ability to lead large‑scale data platform migrations.
• Experience working across multiple engineering teams to deliver platform‑wide solutions.
• Strong focus on data correctness, auditability, and compliance requirements.
• Capability to mentor senior engineers and influence architectural decisions.

Required skills

• PostgreSQL
• MongoDB
• Kafka
• Amazon S3
• Redis
• High‑throughput data pipelines
• Schema‑aware data modeling
• Eventual consistency design
• Idempotent session handling
• Conditional writes and back‑pressure mechanisms
• Multi‑tenancy architecture
• Terabyte‑scale data stream processing
• Query API design with schema versioning, joins, and exports
• Auditable data infrastructure for compliance evidence