IN-Senior Associate_ Databricks Senior Data Engineer_Data and Analytics_Advisory_Bangalore
This job is in your area. Enjoy a short commute and work close to home.
Job Description
Description
& Summary:We are seekinghands-on Databricks Data Engineerwith 5–7 years of experience who can design, build, andoptimizescalable data pipelines on theDatabricks Lakehouse. The ideal candidate is strong inSQL, Python, andPySpark, understandsDelta Lakeinside out, and is experienced withproduction-grade ETL/ELT,orchestration,cost/performance optimization, anddata governance(e.g., Unity Catalog).
Candidate should have hands-on knowledge of AzureADFandADLS.
Responsibilities:
You will work on:
Build reliable, scalablebatch and streamingpipelines usingPySparkandDatabricks Workflows/Jobs.
ImplementDelta Lakebest practices: schema enforcement/evolution, ACID transactions, time travel, OPTIMIZE/ZORDER, and VACUUM.
DevelopDelta Live Tables (DLT)orStructured Streamingpipelines wit...