Assistant Manager/ Consultant 2

• 4+ Years of hands-on, progressive experience in Cloud Data Engineering, Data Architecture
• Deep conceptual understanding of core data engineering principles, ETL/ELT patterns, and metadata management
• Insurance industry experience preferred but not mandatory
• Tools:
  • Cloud Environment: AWS (S3, IAM, VPC, Glue, Lambda etc.)
  • Experience with dbt Core, Iceberg, AWS Glue Catalog, AWS lambda, Redshift, Athena, Splunk, AmazonQ, AWS streaming services, Git
  • Strong SQL, Pyspark and Python

B. Tech or MCA with 4+ years of cloud data engineering experience. Hands-on experience with cloud ecosystems AWS, Redshift, SQL, Python, Pyspark, Data migration into Cloud, pipeline building, CI/CD pipeline automation. Glue, Lambda Amazon Q or Codex AI. Good, comunication Skill, Building ELT/ETL pipelines and using workflow orchestration tools like Apache Airflow or dbt. Ability to translate complex business needs into actionable technical requirements. Strong troubleshooting and problem-solving skills in distributed systems. Agile/Scrum teamwork capabilities and version control practices

• Platform Development
  • Knowledge of best practices for our end-to-end data pipelines in Cloud, ensuring scalability, reliability, and security across the platform.
  • Aggressively folowing the project timeline to ensure rapid delivery of data migration in AWS RDS.
  • Working with Data Analysts, and business stakeholders to translate complex requirements into robust, production-ready ETL data solutions.
  • Architect and maintain reliable data pipelines to make raw data usable for analysts and data scientists.
  • Optimize large-scale data processing algorithms and cloud data warehouse performance.
  • Ensure data security, governance, and access control across cloud environments
• Data Pipeline Development
  • Ingestion & Transformation: Design, build, and optimize high-volume data ingestion and transformation jobs using tools like dbt Core, iceberg, AWS Glue, ensuring data quality and integrity.
  • Workflow: Develop and maintain sophisticated data pipelines using AWS glue/lambda, focusing on modularity and reusability.
  • Streaming & Real-time Integration: Implement and manage real-time data flows utilizing native AWS streaming services (e.g., Kinesis) for immediate data availability.
  • Data Security and Privacy: Data Anonymization, Compliance with Regulations
  • Good knowledge of CI/CD for ETL pipeline automation and Data migration
• Drive the Data Ecosystem Management & Monitoring
  • Open Table Formats & Management: Implement and maintain the Iceberg open table format, utilizing tools for efficient schema evolution and data management.
  • Compute Engine Optimization: Optimize query performance and cost efficiency across our primary compute engines: Amazon Redshift, and AWS Athena.
  • rapidly identify bottlenecks, and troubleshoot production issues