Responsibilities:

• Build and optimise scalable data pipelines using PySpark and SQL
• Design and implement ETL/ELT processes for batch and streaming data
• Develop data solutions using Databricks Lakehouse and Delta Lake
• Ingest and integrate data from internal and external sources (e.g. Kafka, CDC)
• Optimise Spark jobs and data workflows for performance, scalability, and cost efficiency
• Manage infrastructure and environments using Terraform (IaC)
• Ensure data quality, monitoring, and reliability
• Implement governance and access controls (e.g. Unity Catalog)
• Deliver clean, structured, and accessible data for analytics and business use
• Collaborate with cross-functional teams to support analytics, reporting, and AI/ML initiatives

Qualifications:

• Demonstrated experience in data engineering, with a proven ability to build scalable data solutions