Senior Azure Data Engineer – Client Digital Twin & Databricks

Location: Bengaluru (Hybrid)

Employment Type: 6 Months Contract

Work Schedule:

• Monday & Friday – Work from Home

• Tuesday to Thursday – Work from Office

Shift Timing: 1:30 PM – 10:30 PM IST

About the Role

We are looking for an experienced Senior Azure Data Engineer – Client Digital Twin & Databricks to join a global engineering and digital transformation team supporting large-scale Oil & Gas operations. The ideal candidate will have strong experience in Azure-based data engineering, Digital Twin solutions, and cloud data platforms, with hands-on exposure to the Client or Oil & Gas industry.

In this role, you will design, develop, and support scalable data pipelines, cloud-based data solutions, and Digital Twin initiatives that improve engineering workflows, operational efficiency, and enterprise analytics. You will work closely with data engineers, product owners, engineering teams, and business stakeholders in an Agile environment.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure Data Factory, Databricks, and Azure cloud services.

• Build and optimize ETL/ELT workflows for engineering and operational data.

• Develop data integration solutions across structured and unstructured data sources.

• Support Digital Twin initiatives by integrating engineering data, operational systems, and enterprise data platforms.

• Build API connectors for engineering applications and operational systems.

• Develop data transformation processes using Python and Azure technologies.

• Work with Delta Lake and cloud-based data storage solutions to improve data accessibility and performance.

• Implement data quality rules, validation processes, and governance standards.

• Support data modeling, data profiling, and data enrichment activities.

• Develop and maintain CI/CD pipelines and follow DevOps best practices for deployment and version control.

• Perform unit testing, functional testing, and troubleshoot data pipeline issues.

• Collaborate with cross-functional teams during requirement gathering, solution design, development, testing, and deployment.

• Participate in Agile ceremonies including sprint planning, daily stand-ups, sprint reviews, and retrospectives.

• Prepare technical documentation, implementation guides, and solution presentations for business stakeholders.

Required Skills

Technical Skills

• Azure Data Factory (ADF)

• Azure Databricks

• Azure Cloud Services

• Python

• CI/CD Pipelines

• Delta Lake

• SQL

• ETL / ELT Development

• Data Pipelines

• REST APIs

• Azure Data Lake

• Data Modeling

• Data Integration

• Data Quality

• Data Governance

Domain Experience

• Client industry experience is mandatory.

• Strong experience in Oil & Gas engineering data environments.

• Experience working on Digital Twin projects or engineering data platforms.

• Understanding of facilities engineering, asset data, and operational data is highly preferred.

Nice to Have

• PySpark

• Power BI

• Azure DevOps

• Data Profiling

• Industrial IoT

• Engineering Data Management

• Asset Information Management

Soft Skills

• Excellent communication and presentation skills.

• Experience working directly with business stakeholders and cross-functional teams.

• Strong analytical and problem-solving abilities.

• Experience working in Agile/Scrum teams.

• Ability to manage multiple priorities in a fast-paced environment.

• Strong documentation and technical writing skills.

• Self-motivated with a collaborative approach to delivering high-quality solutions.

Required Experience

• 8+ years of experience in Data Engineering.

• Strong hands-on experience with Azure Data Factory, Databricks, and Python.

• Proven experience building enterprise-scale data pipelines and cloud data solutions.

• Experience working within the Client or Oil & Gas industry is mandatory.

• Experience supporting Digital Twin initiatives or engineering data platforms is highly preferred.

Preferred Qualifications

• Experience with Azure-based analytics platforms.

• Experience with Delta Lake architecture.

• Knowledge of DevOps and CI/CD implementation.

• Experience integrating engineering systems with cloud data platforms.

• Familiarity with Industrial IoT and Digital Transformation initiatives.

• Microsoft Azure certifications are an added advantage.

Why Join?

• Opportunity to work on enterprise-scale Digital Twin initiatives.

• Exposure to modern Azure cloud and data engineering technologies.

• Work alongside global engineering and digital transformation teams.

• Hybrid work model with collaborative project delivery.

• Opportunity to contribute to innovative projects supporting the Client and Oil & Gas industry.