Title: Technical Specialist-Data Engg
Area(s) of responsibility
• Develop scalable data processing solutions using Azure Databricks, ensuring efficient data workflows and optimized performance.
• Build and maintain ETL/ELT pipelines, leveraging PySpark/Spark and SQL to transform and process large datasets.
• Implement data security, access controls, and governance standards, ensuring data quality and integrity throughout ETL processes
• Cloud Lead with experience in Azure ADF, Databricks, PySpark
• Responsible for designing and implementing secure, scalable, and highly available cloud-based solutions and estimation on AWS and Azure Cloud
• Experience in Azure Databricks and ADF, Azure Synapse and PySpark
• Experience with integration of different data sources with Data Warehouse and Data Lake is required