JD -
5+ years hands-on Azure Databricks experience.
Build real-time ingestion pipelines with Azure Event Hub and Databricks Structured Streaming.
Design scalable streaming pipelines following best practices.
Familiarity with Unity Catalog and data governance principles.
Implement structured logging and metadata augmentation.
Unit testing and Dev Ops practices for data workloads.
Proficient in SQL and Python programming and advanced software development and Dev Ops concepts.
Implement Azure Functions for XML/XSD validation.
Optimize Spark performance in Databricks.
Proficient in Python XML/XSD parsing/validation and modular packages.
Hands-on with Azure Dev Ops/Git Hub.
Set up monitoring/alerting via Azure Monitor, Databricks Jobs, Log Analytics.
Basic Terraform/Ia C knowledge.
Collaborate cross-functionally; adopt new tech; apply data engineering principles.
Databricks Certified Associate/Professional Data Engineer (strong plus).