Data Lead (Databricks & Power BI) | Offshore

Responsibilities
API Integration & Data Extraction

• Lead the integration of Fenergo APIs to extract relevant KYC and AML data, ensuring seamless connectivity and data flow between systems
\n
• Design, develop, and maintain scalable data pipelines and ETL processes to support data ingestion from various sources, including databases, APIs, and flat files
\n
• Ensure robust data extraction processes that maintain data quality and compliance with regulatory requirements
\n

Data Processing & Pipeline Development

\n
• Utilize Databricks and Apache Spark to design and implement robust data processing pipelines, ensuring high data quality and performance
\n
• Work with DataFrames for transforming data and implementing the Medallion Architecture
\n
• Execute SQL queries for data extraction, manipulation, and complex data operations
\n
• Join datasets and add fields to reports to provide comprehensive analytical insights
\n
• Leverage AI tools in Databricks to assist with data workflows and optimization
\n
• Use notebooks as data transformation pipelines for efficient data processing
\n

Data Analysis & Interpretation

\n
• Analyze and interpret complex data sets to identify trends, patterns, and anomalies that can inform business decisions related to client and investor lifecycle management
\n
• Understand and navigate the data model to ensure accurate data representation and reporting
\n
• Conduct regular data quality assessments and audits to ensure data integrity and compliance with industry standards
\n
• Perform root cause analysis to swiftly identify data issues and collaborate with relevant teams to implement effective solutions
\n

Data Visualisation & Reporting

\n
• Architect and develop interactive dashboards and reports in Power BI, translating complex data into clear, actionable insights for clients, leadership, and stakeholders
\n
• Develop and maintain dashboards and reports to provide insights into key performance indicators (KPIs) and operational metrics for KYC/AML processes
\n
• Create visual representations that highlight critical data points for regular reporting to clients and senior management
\n
• Ensure reports meet the needs of both technical and non-technical stakeholders
\n

Collaboration & Stakeholder Management

\n
• Collaborate with cross-functional teams, including IT, Compliance, Risk Management, business analysts, and senior management, to gather data requirements and deliver strategic insights
\n
• Engage with clients and internal stakeholders to understand their reporting needs and ensure alignment with business objectives
\n
• Work closely with KYC/AML operations teams to ensure data solutions support compliance and regulatory requirements
\n
• Act as a bridge between technical teams and business users, translating complex data concepts into actionable business insights
\n

Documentation & Governance

\n
• Maintain comprehensive documentation of data processes, API integrations, data flows, data management processes, and reporting solutions for future reference and compliance
\n
• Document data governance practices and ensure adherence to data quality best practices
\n
• Ensure all data handling complies with regulatory standards and internal policies
\n

Continuous Improvement & Problem-Solving

\n
• Recommend long-term product solutions to enhance data quality, accessibility, and usability
\n
• Identify opportunities for process optimization and automation in data workflows
\n
• Stay up-to-date with industry trends and best practices in data engineering, analysis, and management
\n
• Proactively identify and resolve data-related issues, ensuring timely and accurate reporting
\n
• Demonstrate creativity and insightfulness in developing dynamic approaches to complex data challenges
\n

Quality Assurance

\n
• Ensure data integrity throughout all pipelines and reporting mechanisms
\n
• Implement data validation and quality control measures
\n
• Monitor data processes and implement control mechanisms to ensure reliability
\n

Skills

\n

Core Data Engineering Skills (Required):

\n
• Proficiency in Databricks and Apache Spark for data processing and pipeline development
\n
• Strong knowledge of Power BI for data visualization and reporting, with ability to create executive-level dashboards
\n
• Expert-level proficiency in SQL for data querying, manipulation, and complex analytical operations
\n
• Experience with programming languages such as Python or R for data analysis and automation
\n
• Strong understanding of data warehousing concepts and ETL processes
\n
• Knowledge of data modeling concepts and best practices for data management
\n
• Understanding of the Medallion Architecture and data lakehouse principles
\n
• Experience working with DataFrames for data transformation
\n
• Ability to leverage AI tools in Databricks to optimize data workflows
\n

API & Integration Skills (Required):

\n
• Strong experience in API integration for data extraction and system connectivity
\n
• Ability to ensure seamless data flow between multiple systems
\n

Cloud & Infrastructure (Required):

\n
• Experience with cloud platforms, particularly Azure or AWS
\n
• Knowledge of Git connection to Databricks for version control
\n
• Experience with AWS/Azure and Databricks integration/mounting
\n
• Understanding of data governance and data quality best practices
\n

Additional Technical Skills (Preferred):

\n
• Databricks administration skills
\n
• Familiarity with machine learning concepts and their application in data analysis
\n
• Experience with graph data models
\n
• Understanding of data governance and compliance standards (GDPR, AML regulations,
\n
• Knowledge of secure data handling practices