Databricks Developer

About the role

We are seeking a highly skilled Databricks Developer to design, build, and maintain modern data platforms using the Databricks Lakehouse architecture. The role focuses on creating scalable data pipelines that enable analytics, reporting, and AI/ML initiatives.

Key responsibilities

• Design and implement scalable data solutions on the Databricks Lakehouse architecture.
• Develop and maintain PySpark, Python, and SQL data pipelines.
• Build and optimise ETL/ELT workflows for batch and near‑real‑time processing.
• Implement Delta Lake features such as ACID transactions, time travel, schema evolution and data versioning.
• Create and maintain enterprise data models (star, snowflake, dimensional, data vault).
• Ensure data quality through validation, monitoring, reconciliation and governance controls.
• Manage data catalogs, metadata, and lineage processes.
• Collaborate with stakeholders, architects and analytics teams to translate requirements into technical solutions.
• Optimise Databricks workloads for performance, scalability and cost efficiency.
• Implement security, access controls and governance best practices within Databricks.
• Support troubleshooting, root‑cause analysis and production issue resolution.
• Contribute to data platform modernisation and cloud migration initiatives.

Required profile

• 4–8 years of experience in data engineering and analytics.
• Bachelor’s or Master’s degree in Computer Science, IT, Data Engineering or related field.
• Minimum 3 years of hands‑on experience with Databricks Lakehouse architecture.
• Proven experience with Azure services such as Azure Databricks, ADLS, Data Factory and Synapse.
• Strong background in building robust ETL/ELT pipelines and data models.

Required skills

• Databricks platform and Lakehouse architecture
• Delta Lake (ACID, time travel, schema evolution)
• Unity Catalog / data catalog and metadata management
• Databricks Workflows, Jobs, Clusters and performance tuning
• PySpark for large‑scale data processing
• Python programming
• Advanced SQL development and query optimisation
• Data modelling: star schema, snowflake, dimensional modelling, data vault
• Data quality frameworks, data lineage and governance
• Azure Databricks, Azure Data Lake Storage (ADLS), Azure Data Factory, Azure Synapse Analytics
• CI/CD pipelines (Azure DevOps, GitHub Actions)