Azure Databricks Data Engineer

Job Title : Azure Databricks Data Engineer

Contract - Hybrid 3 days a week

Location - Hyderabad ONLY

Data Engineer with experience in the Consumer-Packaged Goods (CPG) domain to design, build, and maintain scalable data pipelines on the Microsoft Azure platform. The ideal candidate will have strong hands-on expertise in Azure Databricks, PySpark, and Azure Data Factory (ADF), along with experience working with CPG datasets such as sales, retail, inventory, and supply chain data.

Key Responsibilities

• Design, develop, and maintain ETL/ELT pipelines using Azure Data Factory and Azure Databricks
• Process and transform large-scale CPG datasets (e.g., POS sales, inventory, promotions, pricing, retailer data)
• Build scalable data processing workflows using PySpark
• Integrate data from multiple sources including retailer feeds (e.g., Nielsen, IRI), APIs, and flat files
• Develop data models to support sales analytics, demand forecasting insights
• Ensure data quality, validation, and consistency across CPG data pipelines
• Optimize pipeline performance and cost on Azure
• Collaborate with business stakeholders
• Implement monitoring, logging, and troubleshooting for data workflows

Required Skills & Qualifications

• Bachelor's degree in engineering, or related field
• 4+ years of experience in data engineering
• Strong hands-on experience with:
• Azure Databricks
• PySpark
• Azure Data Factory (ADF)
• Proficiency in Python and SQL
• Experience working with CPG or retail data (sales, inventory, promotions, product hierarchy)
• Strong understanding of data warehousing and data lake architectures (ADLS)
• Experience building scalable ETL pipelines
• Familiarity with version control tools (Git)

Preferred Qualifications

• Experience with retail/CPG data providers (e.g., Nielsen, IRI)
• Knowledge of Delta Lake and data Lakehouse architecture
• Familiarity with demand forecasting and supply chain analytics
• Exposure to streaming data pipelines
• Experience with CI/CD pipelines (Azure DevOps)

Domain-Specific Knowledge (CPG)

• Understanding of:
• Point-of-Sale (POS) data
• Product hierarchies (SKU, category, brand)
• Trade promotions and pricing strategies
• Retailer and distributor data flows
• Ability to translate business requirements into scalable data solutions for sales and supply chain analytics

Key Competencies

• Strong analytical and problem-solving skills
• Ability to work with both technical teams and business stakeholders
• Strong communication skills
• Attention to detail and data quality focus
• Agile mindset

Nice to Have

• Azure Certification (DP-203/DP-700: Azure Data Engineer Associate)
• Experience in FMCG/CPG analytics platforms
• Knowledge of Power BI or other BI tools

#AzureDatabricks #AzureDataEngineer #PySpark #AzureDataFactory #DataEngineering #DeltaLake #MicrosoftAzure #CPG #RetailAnalytics #SupplyChainAnalytics #AzureDatabricks

#AzureDataEngineer #PythonSQL #DeltaLake #ADLS #fmgc/cpg #powerbi

Interested resources may please share your cv to [email protected]