Data Engineer

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Factory (ADF).
• Build and optimize big data solutions using Azure Databricks and Apache Spark.
• Develop data transformation and processing workflows using Python and SQL.
• Implement and manage Delta Tables for reliable and high-performance data storage.
• Configure and administer Unity Catalog for data governance and access control.
• Design and manage Azure Data Lake Storage (ADLS) for enterprise data solutions.
• Optimize data pipelines for performance, scalability, and reliability.
• Collaborate with cross-functional teams to deliver business-driven data solutions.
• Ensure data quality, security, and compliance with organizational standards.
• Troubleshoot production issues and support continuous improvement initiatives.

Required Skills

• 6+ years of experience in Data Engineering.
• Strong knowledge of SQL and Python.
• Hands-on experience with Azure Databricks.
• Expertise in Azure Data Factory (ADF).
• Strong experience with Apache Spark.
• Experience with Delta Tables and Delta Lake.
• Hands-on knowledge of Unity Catalog.
• Experience with Azure Data Lake Storage (ADLS).
• Strong understanding of ETL/ELT, data warehousing, and data modeling.
• Familiarity with Git, CI/CD, and Azure cloud services.

Preferred Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
• Microsoft Azure Data Engineer certification is a plus.
• Excellent analytical, problem-solving, and communication skills.