Posted 01 July, 2026
Senior Data Engineer Databricks
NR Consulting - India
Pan India : Pune | Bangalore | Chennai | Kolkata | Gurugram, Uttar Pradesh, IN
Full Time
Reference: 26-15246-2220-1
Title: Senior Data Engineer Databricks
Location: Pan India : Pune | Bangalore | Chennai | Kolkata | Gurugram
Exp: 5-8 Years
Job Description:
Design, develop, and maintain scalable data pipelines using Databricks, Apache Spark, and Python.
Build and manage feature engineering frameworks and reusable feature pipelines for machine learning use cases.
Develop and maintain Feature Stores to enable training and inference consistency.
Implement ETL/ELT processes for structured and unstructured data from multiple sources.
Optimize Spark jobs, Databricks workloads, and data models for performance and cost efficiency.
Collaborate with Data Scientists, ML Engineers, and Business stakeholders to translate business requirements into data solutions.
Ensure data quality, governance, security, and compliance standards are met.
Design and implement batch and near real-time data processing solutions.
Mentor junior engineers and contribute to architecture and design discussions.
Support CI/CD, automated testing, and deployment of data engineering solutions.
Required Skills
5+ years of experience in Data Engineering.
Strong hands-on experience with Databricks and Apache Spark (PySpark/Scala Spark).
Extensive experience in Feature Engineering for Machine Learning use cases.
Strong programming skills in Python.
Expertise in SQL and data modeling.
Experience with Delta Lake, Databricks Workflows, Unity Catalog, and Lakehouse Architecture.
Experience with cloud platforms such as Azure, AWS, or GCP.
Knowledge of data orchestration tools (Airflow, ADF, etc.).
Understanding of MLOps concepts and ML lifecycle management.
Experience working with large-scale distributed datasets.
Preferred Qualifications
Experience with Databricks Feature Store and MLflow.
Exposure to real-time processing using Kafka/Event Hub.
Knowledge of predictive analytics, recommendation systems, or customer analytics.
Databricks certifications are highly preferred.
Experience in Retail, Client, Healthcare, Banking, or Supply Chain analytics domains.
Location: Pan India : Pune | Bangalore | Chennai | Kolkata | Gurugram
Exp: 5-8 Years
Job Description:
Design, develop, and maintain scalable data pipelines using Databricks, Apache Spark, and Python.
Build and manage feature engineering frameworks and reusable feature pipelines for machine learning use cases.
Develop and maintain Feature Stores to enable training and inference consistency.
Implement ETL/ELT processes for structured and unstructured data from multiple sources.
Optimize Spark jobs, Databricks workloads, and data models for performance and cost efficiency.
Collaborate with Data Scientists, ML Engineers, and Business stakeholders to translate business requirements into data solutions.
Ensure data quality, governance, security, and compliance standards are met.
Design and implement batch and near real-time data processing solutions.
Mentor junior engineers and contribute to architecture and design discussions.
Support CI/CD, automated testing, and deployment of data engineering solutions.
Required Skills
5+ years of experience in Data Engineering.
Strong hands-on experience with Databricks and Apache Spark (PySpark/Scala Spark).
Extensive experience in Feature Engineering for Machine Learning use cases.
Strong programming skills in Python.
Expertise in SQL and data modeling.
Experience with Delta Lake, Databricks Workflows, Unity Catalog, and Lakehouse Architecture.
Experience with cloud platforms such as Azure, AWS, or GCP.
Knowledge of data orchestration tools (Airflow, ADF, etc.).
Understanding of MLOps concepts and ML lifecycle management.
Experience working with large-scale distributed datasets.
Preferred Qualifications
Experience with Databricks Feature Store and MLflow.
Exposure to real-time processing using Kafka/Event Hub.
Knowledge of predictive analytics, recommendation systems, or customer analytics.
Databricks certifications are highly preferred.
Experience in Retail, Client, Healthcare, Banking, or Supply Chain analytics domains.
