Software Engineer – Kafka (Data Platform)

About the role

As a member of the Data Engine team, you will help build the core data and machine‑learning platform that powers autonomy across all of Applied Intuition’s verticals. The role focuses on ingesting, processing, and serving massive streams of sensor data from real‑world fleets.

Key responsibilities

• Design and implement large‑scale data platforms that handle petabytes of multimodal sensor data.
• Develop data curation and tagging systems to enable efficient dataset discovery, labeling workflows, and quality assessment.
• Build high‑performance data processing pipelines using modern distributed computing frameworks.
• Collaborate with product and research teams to define data‑infrastructure requirements and ensure scalability.
• Operate and maintain services on Kubernetes and related cloud‑native environments.

Required profile

• Bachelor’s degree in Computer Science, Software Engineering, or a related field.
• At least 2 years of professional backend engineering experience.
• Strong problem‑solving skills and ability to work in a fast‑moving, early‑stage team.
• Experience building distributed systems and handling large data volumes.

Required skills

• Apache Spark
• Apache Hudi
• Trino
• Apache Kafka
• Flyte
• Kubernetes
• Python
• Golang
• Java