Staff Engineer – Data Platform

About the role

We are seeking a Staff Engineer to lead the design and implementation of Shield AI’s Data Platform, the core knowledge‑graph‑based API that powers both human and autonomous workflows across the AI Factory.

Key responsibilities

• Architect and build a unified Graph API that serves as the backbone for data discovery, relationship traversal, and payload access.
• Own the DataOps stack – evaluate, optimize, and maintain storage, indexing, query, ingestion, and compute infrastructure.
• Define and enforce best‑practice patterns for schema modeling, lineage, and evolution.
• Deliver first‑party integrations for simulation, test, training, and edge‑device data streams.
• Provide self‑service SDKs, tools, and diagnostics to improve developer experience.

Required profile

• Extensive experience designing and operating distributed data solutions or storage systems.
• Proven ability to deliver production‑grade services in Go and/or Python.
• Deep knowledge of data modeling, API design, schema evolution, and identity management.
• Hands‑on expertise with Kubernetes, Linux, networking, security, and observability.
• Experience deploying data infrastructure on cloud or customer‑managed environments using IaC practices.

Required skills

• Go
• Python
• Kubernetes
• Linux
• Relational and graph databases
• Object and columnar storage (e.g., S3, Parquet)
• Infrastructure as Code tools (Terraform, Helm, GitOps)
• Streaming platforms (Kafka, NATS, Redpanda)
• Distributed execution frameworks (Ray)
• Data formats and query engines (Apache Arrow, Parquet)