Posted 20 August, 2026
Senior Data Engineer
Diverse Lynx
Bengaluru,560001
Full Time
Reference: 365_569689_26-03347
Experience
Key Skills Required
Expected Responsibilities
- 5–10 years of experience in Data Engineering
Key Skills Required
-
Strong hands-on experience with Databricks:
- PySpark
- SQL
- Delta Lake
- Unity Catalog
- Databricks Workflows / Jobs orchestration
- Experience with Azure Data Factory (ADF)
- Experience building Lakehouse architectures using Bronze, Silver & Gold layers
-
Good knowledge of Databricks Lakeflow:
- Declarative Pipelines
- Lakeflow Connect
- CDC (Change Data Capture)
- Data quality validations
-
Experience with AI/BI and Genie:
- Semantic layers
- Certified datasets
- Self-service analytics solutions
-
Experience building AI Agents/Copilots using:
- LLMs
- RAG (Retrieval-Augmented Generation)
- Vector Search
- Prompt Engineering
-
Good understanding of Unity Catalog governance:
- Access controls
- Data masking
- Audit logging
- Lineage
- Experience with CI/CD, Git, Databricks Repos & Asset Bundles
Expected Responsibilities
- Design, develop, and support end-to-end data pipelines in Databricks
- Build and maintain Delta tables under Unity Catalog
- Develop Lakeflow pipelines with incremental processing and CDC
- Enable AI/BI use cases through semantic models and Genie spaces
- Integrate SharePoint and external data sources into the Lakehouse
- Ensure data quality, governance, and operational reliability
- Drive delivery independently and proactively resolve issues
