Azure Data Factory

Key Responsibilities:

• Design, develop, and manage end-to-end data pipelines using Azure Data Factory (ADF).

• Build scalable and efficient ETL/ELT workflows for data ingestion, transformation, and loading.

• Integrate data from on-premises and cloud-based sources, including Azure SQL, Blob Storage, Data Lake, and REST APIs.

• Collaborate with data architects and business analysts to understand data requirements and translate them into robust data solutions.

• Implement data validation, error handling, and logging mechanisms within pipelines.

• Work with Azure Data Lake, Azure Synapse Analytics, Azure SQL Database, and Azure Functions as part of the data ecosystem.

• Ensure performance optimization and cost-efficient design of data workflows.

• Participate in code reviews, testing, and deployment of data solutions.

• Document pipeline architecture, processes, and configurations.

• Stay updated with Azure services, features, and best practices.