| Job Description |
Key Responsibilities:
Design, develop, and maintain data pipelines using Azure Data Factory, ensuring data is ingested, transformed, and loaded across diverse data sources and targets. Work with structured and unstructured data sources such as SQL Server, Azure Blob Storage, Data Lake, REST APIs, and more. Develop and optimize ETL/ELT processes using ADF, Data Flows, and Databricks (if applicable). Manage and monitor pipeline performance, troubleshoot failures, and ensure data quality and integrity. Integrate and orchestrate data workflows with Azure services like Azure Data Lake Storage (ADLS), Azure Synapse Analytics, Azure SQL Database, and Azure Functions. Collaborate with data architects, analysts, and stakeholders to gather data requirements and implement solutions. Implement and manage CI/CD pipelines for data pipeline deployments (using Azure DevOps preferred). Ensure compliance with security and governance standards. Maintain technical documentation for data pipelines, workflows, and architecture.
|
