Data Engineer

As a Senior Data Engineer, you will bridge the gap between complex insurance business logic and modern cloud data architecture. You will be responsible for designing and implementing end-to-end data solutions that handle vast amounts of policy, claims, and billing data.

Working in a fast-paced environment, you will leverage the Medallion Architecture to ensure data quality and reliability. Your expertise in Unity Catalog will be essential in maintaining a secure and governed environment, ensuring that sensitive policyholder information is managed with the highest standards of compliance. You will mentor junior engineers and collaborate closely with Data Scientists and Actuaries to turn raw data into a strategic asset.

• Experience: Minimum 5+ years of professional experience in Data Engineering, with a significant portion dedicated to the Insurance domain (Life, P&C, or Health).
• Technical Core: Expert-level proficiency in SQL and Python (PySpark).
• Platform Expertise: Deep hands-on experience with Databricks (Workflows, Delta Live Tables, SQL Warehouse).
• Governance: Proven experience implementing Unity Catalog for data discovery and access management.
• Cloud Ecosystem: Strong experience with the Azure stack, including Azure Data Lake Storage (ADLS Gen2), Azure Data Factory, and Key Vault.
• Data Modeling: Advanced understanding of data warehousing concepts and dimensional modeling tailored for insurance metrics (Loss Ratios, GWP, IBNR).
• Soft Skills: Strong analytical mindset with the ability to explain complex technical concepts to non-technical stakeholders.
• Education: Bachelor's or Master's degree in Computer Science, Information Systems, or a related quantitative field.
• Certification (Preferred): Databricks Certified Data Engineer Associate or Azure Data Engineer Associate (DP-203).

• Pipeline Development: Design, develop, and maintain robust ETL/ELT pipelines using PySpark and Databricks to ingest data from various insurance source systems.
• Data Governance: Implement and manage data governance, lineage, and fine-grained access control using Azure Unity Catalog.
• Performance Tuning: Optimize complex SQL queries and Spark jobs for performance and cost-efficiency within the Azure environment.
• Architecture: Lead the transition from legacy data warehouses to a modern Lakehouse architecture, ensuring seamless integration of structured and unstructured data.
• Collaboration: Partner with Actuarial and Finance teams to translate complex insurance business rules into technical data models (e.g., Star Schema, Data Vault).
• Automation: Build and maintain CI/CD pipelines for data workloads using Azure DevOps or GitHub Actions.
• Monitoring: Implement proactive monitoring and alerting for data quality and pipeline health to ensure "single source of truth" reliability.