Data Engineer (Core & Analytics)

About the role

The Wakapi team is expanding and seeks a Data Engineer to build performant data pipelines, manage relational databases, and create cloud‑portable data lake environments using modern open storage formats and embedded analytical engines.

Key responsibilities

• Design and implement resilient data processing pipelines in Python following Medallion architecture (Bronze/Silver/Gold).
• Standardize data storage with open formats such as Parquet, Apache Iceberg, and Delta Lake for cross‑cloud compatibility.
• Integrate DuckDB with Python for fast, lightweight processing within ETL workflows.
• Architect and optimise production PostgreSQL schemas and use SQLite for lightweight embedded storage.
• Scale heavy ETL, transformation, and analytics workloads using Databricks or Apache Spark.

Required profile

• Advanced SQL proficiency with deep PostgreSQL expertise and working knowledge of SQLite.
• Hands‑on experience building robust analytical pipelines using Python and DuckDB.
• Proven track record designing data lakes with Medallion architecture.
• Familiarity with the Databricks / PySpark ecosystem for distributed processing.

Required skills

• Python
• SQL (PostgreSQL, SQLite)
• DuckDB
• Parquet, Apache Iceberg, Delta Lake
• Databricks, Apache Spark, PySpark