Sr Cloud Data Engineer

We are seeking a highly skilled Senior Data Engineer to lead the design, development, and optimization of scalable data solutions that power enterprise analytics and business decision-making. The ideal candidate will bring deep expertise in Azure and Databricks, strong hands-on experience in building robust ETL/ELT pipelines, and a solid understanding of data quality, governance, and performance tuning. This role will be instrumental in enabling reliable, high-quality data flows across complex systems, with a strong focus on supporting insurance domain use cases, regulatory needs, and data-driven business transformation.

• 6-10 years of hands-on experience in data engineering, with strong expertise in Azure cloud and Databricks-based data platform development.
• Bachelor's degree in computer science, Information Technology, Engineering, or a related discipline.
• Strong programming skills in PySpark, Python, and SQL, with proven experience in building scalable and maintainable data pipelines.
• Solid experience in data modeling, ETL/ELT pipeline design, Delta Lake, and enterprise data warehousing concepts.
• Hands-on expertise with Azure Data Factory, Azure Databricks, Azure Data Lake Storage, and related Azure data services.
• Experience with Databricks capabilities such as Delta Live Tables, Auto Loader, Unity Catalog, and workflow orchestration.
• Strong understanding of performance tuning, optimization of Spark workloads, and pipeline reliability in large-scale data environments.
• Working knowledge of data governance, data quality, and secure data handling practices within enterprise platforms.
• Preferred exposure to insurance domain data models, business processes, and regulatory reporting requirements.
• Excellent analytical, communication, and problem-solving skills, with the ability to collaborate effectively across cross-functional teams and deliver under tight deadlines.

• Lead the design, development, and optimization of scalable, high-performance data pipelines, ensuring reliable data ingestion and transformation across distributed systems, while proactively identifying and resolving performance bottlenecks and system inefficiencies.
• Define and enforce enterprise-grade data quality frameworks, implementing advanced validation rules, anomaly detection, and reconciliation processes to ensure data accuracy, completeness, and integrity across large-scale datasets.
• Own end-to-end monitoring and reliability of data platforms, establishing observability practices (alerts, SLAs, dashboards) and driving root-cause analysis to minimize downtime and ensure seamless continuity of critical data workflows.
• Govern and continuously improve data pipelines through periodic audits and reviews, ensuring adherence to internal standards, data governance policies, and regulatory compliance, while driving standardization and best practices across teams.
• Collaborate with cross-functional stakeholders (architects, analysts, product owners) to align data solutions with business objectives, and actively contribute to backlog prioritization, sprint planning, and technical problem-solving in Agile environments.
• Mentor junior engineers and drive technical excellence, promoting reusable frameworks, coding standards, and best practices to improve overall team productivity and solution quality.