Lead Data Mesh Engineer

About the role

Cloudera is building an enterprise-grade, Everything-as-Code Data Mesh that turns corporate IT from a manual ticket-driven delivery model into an automated, machine-executable software factory. As Lead Data Mesh Engineer, you will own the architectural execution and hands-on delivery of high-throughput, asynchronous event-streaming infrastructure and hybrid data pipelines, where every asset, route and policy lives as declarative code in Git.

Key responsibilities

• Architect, provision and maintain multi-AZ AWS MSK clusters and Apache NiFi runtimes on Amazon EKS (Graviton ARM64) using Terraform and GitOps
• Integrate vendor-neutral schema registries (AWS Glue / Apicurio) to enforce Apache Avro and AsyncAPI data contracts defined in the central Governance Hub
• Implement partition strategies, dynamic storage auto-scaling (gp3/io2) and local compression (Snappy/Zstd) to reduce cross-cloud data movement costs
• Build fault-tolerant dead letter queue replay loops and ensure streaming payloads carry out-of-band tracing context

Required profile

• BSc/MSc in a related field or equivalent experience
• 6+ years of hands-on experience architecting and operating Apache Kafka (preferably AWS MSK) and Apache NiFi in containerized Kubernetes environments (EKS)
• Expertise in Apache Avro, AsyncAPI and schema registry management for MDM enforcement
• Practical understanding of OpenTelemetry, W3C distributed tracing and observability tools such as Datadog

Required skills

• Terraform and GitOps tools (ArgoCD or Flux) under a pull-based model
• Apache Kafka, Apache NiFi, AWS MSK, Apache Avro, AsyncAPI, AWS Glue, Apicurio
• Nice to have: Policy-as-Code tools (OPA / Rego) and experience connecting event meshes to data lakehouses (Cloudera CDP, Hadoop, Hive)

What we offer

• Generous PTO and Unplugged Days, flexible WFH policy
• Mental and physical wellness programs, phone and internet reimbursement
• Career development access, comprehensive benefits and paid volunteer time