About Position:
We are seeking Sr. Data Engineer with hands-on experience in azure data bricks, DTL, Pyspark, etc.,
- Role: Senior Data Engineer
- Location: Pune, Mumbai, Bangalore, Hyderabad, Delhi
- Experience: 6 to 12 Years
- Job Type: Full Time Employment
What You'll Do:
- Design, build, and maintain DLT pipelines to process structured and unstructured data in Databricks.
- Implement ETL/ELT workflows using Delta Lake, Spark Streaming, and Auto Loader.
- Ensure data pipeline quality, reliability, observability, and performance using DLT expectations and monitoring tools.
- Develop and optimize batch and streaming data pipelines using Py Spark or Scala.
- Work with data architects to define and enforce data modeling, governance, and lineage standards.
- Integrate data from multiple sources including relational databases, APIs, cloud services, and files.
- Optimize storage and compute usage using Delta Lake best practices.
- Collaborate with ...