Country/Region:
IN
Requisition ID:
33626
Work Model:
Position Type:
Salary Range:
Location:
INDIA - PUNE - BIRLASOFT OFFICE - HINJAWADI
Title: Technical Lead-Data Engg
Description:
Area(s) of responsibility
Job Description: Azure Data Engineer
Key Responsibilities
1. Data Solution Design & Architecture
- Design end‑to‑end data engineering solutions using Azure Data Factory, Azure Databricks, PySpark, SQL, and other Azure-native services.
- Architect and implement scalable and secure Modern Data Warehouse (MDW) and Lakehouse solutions leveraging Azure Data Lake Storage and Databricks.
2. Databricks & PySpark Engineering
- Develop scalable transformation scripts using PySpark on Databricks, applying advanced optimizations like caching, partitioning, and Delta Lake capabilities.
- Implement Delta Lake features—ACID transactions, schema enforcement, schema evolution, and time travel—across the data lifecycle.
- Perform performance tuning, handling bottlenecks related to cluster configuration, shuffle operations, joins, and parallelization.
- Collaborate with platform teams to manage Databricks clusters, jobs, notebooks, and CI/CD integrations.