Mandatory Skill:

Azure Databrick Structured Streamin Delta Lake Delta Live Tables (DLTSpark Declarative Pipelines (SDP DatabricksAsset Bundle Unity Catalo Auto Loade Cloud File Apache SparK

Preferred Experience

Migration of traditional ETL pipelines to Spark Declar ative PipelineEnterprise-scale Lakehouse implementation Data Quality frameworks and governance solutionLarge-scale streaming architectures processing millions of events per daExperience implementing Medallion Architecture

Streaming & Real-Time Data Processing

• Design and develop real-time streaming pipelines using Databricks Structured Streaming.
• Build and maintain Kafka-based ingestion frameworks.
• Handle late-arriving events using watermarks, event-time processing, and stateful streaming concepts.
• Implement exactly-once processing and checkpointing mechanisms.
• Mo...