AWS Lakehouse Data Engineer

About the role

We are seeking an AWS Lakehouse Data Engineer to design, implement, and operate a cloud‑native data platform that powers AI/ML, analytics, reporting, and data visualization. The role focuses on building a modern lakehouse on Amazon S3 using AWS‑native services and open table formats while ensuring strong governance, cost efficiency, and operational control.

Key responsibilities

• Design and implement batch and streaming ingestion pipelines from APIs, relational databases, file drops, event streams, and external partners using Python and PySpark.
• Build and manage a Delta‑Lakehouse‑style platform on Amazon S3 with Apache Iceberg, providing ACID transactions, schema evolution, time‑travel, and fast querying via Athena, EMR, Glue, and Redshift.
• Implement data governance, fine‑grained access control, metadata cataloging, lineage, and data quality checks using AWS Lake Formation, Glue Data Catalog, IAM, KMS and related services.
• Automate AWS provisioning with Infrastructure as Code, develop CI/CD pipelines for data workflows, and establish observability with metrics, logs, alerts, and runbooks.

Required profile

• Bachelor's degree in a related field or 4 years equivalent experience.
• At least 6 years of relevant experience, including hands‑on work with AWS data lake/lakehouse architectures and production ETL/ELT pipelines.
• Demonstrated ability to troubleshoot distributed data‑processing workloads and optimize performance, reliability, and cost.

Required skills

• AWS services: S3, Glue, Athena, EMR, Lake Formation, Redshift.
• Programming: Python, PySpark.
• Data formats and engines: Apache Iceberg, Apache Parquet, Delta Lake concepts.
• SQL and analytical query optimization.
• Infrastructure as Code: Terraform, CloudFormation, CDK.
• CI/CD tools: Git, Jenkins, CodePipeline, GitHub Actions.