Senior Data Engineer

About the role

The Senior Data Engineer will design, build, and optimise scalable data pipelines and platform infrastructure within a medallion architecture on Microsoft Fabric and OneLake. The role supports enterprise‑grade ingestion, transformation, and enrichment of healthcare, legal and insurance data for analytics and operational reporting.

Key responsibilities

• Design and implement ingestion pipelines from structured and unstructured sources into the Bronze layer, ensuring reliability, throughput and HIPAA compliance.
• Develop transformation logic for Silver and Gold layers, including normalization, deduplication, entity resolution and feature‑store pipelines.
• Enforce data governance, multi‑tenant isolation, and document data lineage and contracts for audit readiness.
• Build data‑quality validation scripts, monitor pipeline health, and establish alerting for failures.
• Collaborate with ML Engineering, Data Science and Software Engineering teams to deliver production‑ready data solutions.

Required profile

• B.S. or B.A. in Computer Science, Information Systems, Mathematics or related field.
• 7+ years of data engineering experience, preferably on Azure/Microsoft Fabric platforms.
• Strong problem‑solving, organisational skills and ability to work independently.

Required skills

• Python programming
• Advanced SQL
• Microsoft Fabric and OneLake
• Azure Data Factory or equivalent orchestration tools
• CI/CD practices for data pipelines
• AI‑assisted development tools (e.g., GitHub Copilot)
• Familiarity with dbt, PySpark (preferred)