Take the stage as a Senior Databricks Architect and drive the optimization of ETL pipelines on cloud platforms. Your strong experience in Databricks, Spark, SQL, and real-time data processing will be essential.
This role is designed for leaders in data engineering who excel in building efficient ETL pipelines. Utilize your skills with Python and PySpark while integrating APIs and supporting real-time streaming solutions. Collaborate with cross-functional teams to manage projects and improve data workflows using Azure DevOps and Airflow.
Key Responsibilities:
• Build and manage ETL pipelines with Databricks
• Develop data processing strategies using SQL and Python
• Implement CI/CD processes via Azure DevOps
• Utilize Airflow for effective workflow management
• Support real-time data integration and streaming solutions
Requirements:
• Extensive experience with Databricks and Spark
• Strong capabilities in SQL and Python for ETL development
• Experience w...