ADF, Databrick

Responsibilities

Design and develop ETL/ELT pipelines using Azure Data Factory (ADF).

Build and optimize data processing workflows using Azure Databricks (PySpark/Spark SQL).

Integrate data from various sources such as SQL Server, Azure SQL, ADLS, APIs, and third-party applications.

Implement data transformation, cleansing, and validation processes.

Develop and maintain data lakes and data warehouse solutions on Azure.

Monitor and troubleshoot pipeline failures, performance bottlenecks, and data quality issues.

Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.

Implement security, governance, and best practices for data management.

Create technical documentation and support deployment activities.

Required Skills

Azure Services

Azure Data Factory (ADF)

Azure Databricks

Azure Data Lake Storage (ADLS Gen2)

Azure SQL Database / SQL Server

Azure Synapse Analytics (preferred)

Preferred Skills

• Azure Data Factory (ADF)
• Databricks

Educational Requirements

MCA,MSc,MTech,BCA,BSc,BTech