Drive data innovation as a Senior Data Engineer skilled in Databricks, Apache Spark, and Python. Construct scalable, secure data platforms on Microsoft Azure tailored for analytics.
This crucial role focuses on building a robust Lakehouse architecture while managing data pipelines and ensuring production reliability. You will lead efforts in data quality, enforce governance, and implement effective processing solutions. Mentorship and collaboration are key in this position.
Key Responsibilities:
• Design and maintain data pipelines with Databricks
• Build Delta Lake architectures and manage data models
• Develop real-time and batch processing solutions using Python
• Implement governance and access control measures
• Optimize Spark workloads for performance gains
Requirements:
• 8+ years of Data Engineering experience
• 5+ years of expertise in Databricks and Apache Spark
• Strong Python programming skills
• Familiarity with Azure services and SQL