Roles and Responsibilities

• Data Platform Architecture: Design and implement scalable data platform architectures on Azure and Databricks, including medallion architecture (Bronze/Silver/Gold), data lakehouse patterns, and integration layers that serve analytics, reporting, and downstream consumers.
• Pipeline Development: Build and maintain robust ETL/ELT pipelines using Databricks (PySpark/Spark SQL), and related Azure data services. Ensure pipelines handle batch and streaming workloads reliably at scale.
• Legacy Modernisation: Lead the migration of legacy data pipelines to modern cloud-native solutions on Azure and Databricks. Define migration strategies, manage cutover plans, and ensure zero data loss during transition.
• Data Quality & Governance: Establish and enforce data quality frameworks, validation rules, and monitoring across all pipelines. Implement data lineage tracking, cataloguing (Unity Catalog), ...