Title:  Technical Lead-Data Engg

Area(s) of responsibility

Develop scalable data processing solutions using Azure Databricks, ensuring efficient data workflows and optimized performance.

•            Build and maintain ETL/ELT pipelines, leveraging PySpark/Spark and SQL to transform and process large datasets.

•            Implement data security, access controls, and governance standards, ensuring data quality and integrity throughout ETL processes

•            Cloud Lead with experience in Azure ADF, Databricks, PySpark

•            Responsible for designing and implementing secure, scalable, and highly available cloud-based solutions and estimation on AWS and Azure Cloud

•            Experience in Azure Databricks and ADF, Azure Synapse and PySpark

•            Experience with integration of different data sources with Data Warehouse and Data Lake is required

•            Experience in creating Data warehouse, data lakes for Reporting

•            Understanding of data modelling and data architecture concepts

•            To be able to clearly articulate pros and cons of various technologies and platforms

•            Define and implement cloud governance and best practices.

•            Identify and implement automation opportunities to increase operational efficiency.