Job Role : Databricks Data Platform Engineer

Job Location : Abu Dhabi, UAE

Experience : 5+ Years

Role Summary : Design and build scalable, high-performance, and governed data pipelines on Azure Databricks to support analytics, reporting, and enterprise data intelligence initiatives.

• Key Responsibilities – Develop batch pipelines using Delta Lake with proper table design, partitioning, optimization, and lifecycle management
• Implement incremental ingestion using Autoloader with schema evolution and checkpointing
• Build Structured Streaming pipelines with watermarking, state management, and late data handling
• Develop declarative pipelines using Lakeflow standards
• Ensure pipelines support idempotency, replayability, and safe backfills
• Optimize Spark workloads (AQE, skew handling, shuffle tuning, join optimization)
• ...