Principal Data Engineer

- Architect Data based solutions (foundational and analytical data product) based on business need and in line with the Mastercard technology portfolio, patterns and standards.

- Collaborate closely with finance, business and technical stakeholders to translate requirements into technical data architecture and solution.

- Handson, with the ability to write / develop complex code and also help the engineering teams.

- Exposure to building Data contracts and Data products.

- Partner with data and engineering teams to build robust data pipelines, data modeling, and ensure seamless integration across diverse systems and domains.

- Facilitate trade-off discussions with both business and technical stakeholders to balance priorities and make informed decisions.

- Ability to choose between different technical options considering all aspects of engineering principles.

- Ensure alignment between business goals and technical execution, making sure features and solutions meet business requirements and customer needs.

- Participate in sprint planning, retrospectives, and other agile ceremonies to ensure the team is aligned and delivering efficiently.

- Drive the adoption of best practices in software engineering, including code reviews, testing, and continuous integration/continuous delivery (CI/CD).

- Optimize the cost/benefit of software features and architecture, ensuring scalability, performance, and operational efficiency.

- Act as leader of Data platform / technology, providing guidance on complex technical challenges and driving resolution, ensuring solutions align with Mastercard's engineering and data principles, and technical policies.

- Identify opportunities for process improvements, helping to streamline workflows and enhance team productivity.

- Mentor and guide engineers across various experience levels, helping them grow technically and improve their software and data engineering skills.

- Proven experience as data architect and engineer, both on on-prem and cloud-based data platforms (traditional to modern platforms) with a strong focus on delivering large-scale projects in an agile environment.

- Work with the engineering team to help design, develop and implement large scale, high-volume, high-performance, highly available, scalable data platform and pipelines for the Lakehouse being built on on-prem Cloudera Data Platform - CDP. Ability to handle large-scale data processing and distributed computing across massive datasets.

- Must - Hands-on experience with deep understanding and experience with modern Data Platforms - Cloudera Data Platform (CDP).

- Strong hands-on experience with PySpark

- Experience with Cloudera Data Platform (CDE, CDW, Ozone, Airflow, SDX), Apache Ranger. Deep understanding of distributed data systems and Hive Metastore

- Experience and understanding of cataloging, lineage, and governance

- Experience / understanding Open Data Contract Standard (ODCS) and its implementation

- Experience working with SQL, Iceberg open table and Parquet file format, and partitioning/bucketing strategies.

- Experience with data lifecycle management, including ingestion, ETL, pruning, modelling, and governance, within highly regulated environments.

- Experience with performance engineering, ensuring systems are built to scale and meet varying demands.

- Knowledge of security best practices and experience in ensuring the secure development of applications.

- Nice to have:

• Experience modernising enterprise finance systems or regulated environments
• Knowledge of CI/CD, data engineering best practices
• Understanding of financial data structures, accounting processes, or reconciliation workflows
• Prior experience with financial systems, such as Oracle Financials, Oracle Fusion Cloud, and Hyperion, with experience optimising their integration into broader data ecosystems will be a plus.