Roles and Responsibilities

• Design, develop, test, deploy, and maintain large-scale data pipelines using Azure Data Factory (ADF) to extract, transform, and load data from various sources into Azure storage solutions.
• Collaborate with cross-functional teams to gather requirements for data processing needs and design scalable architectures that meet business needs.
• Develop complex SQL queries to optimize database performance and troubleshoot issues in relational databases like SQL Server or Postgres.
• Implement machine learning models using PySpark on Azure Databricks for advanced analytics capabilities.