AWS Pyspark, Redshift, Airflow, IceBerg

PU

DNACMT

Number of Openings

2

Country

India

Detailed JD (Roles and Responsibilities)

Roles & Responsibilities:

• Design, develop, and maintain robust ETL pipelines using PySpark and Airflow.
• Architect and manage data infrastructure on AWS using services like S3, Redshift, Glue, Lambda, EMR.
• Implement and optimize SQL queries and database schemas for efficient data storage and retrieval.
• Ensure data quality through unit testing and validation frameworks.
• Monitor and troubleshoot data pipeline performance and reliability.

Qualification

• 5+ years of experience in data engineering or data warehouse development.
• Strong programming skills in Python and SQL.
• Hands-on experience with PySpark for distributed data processing.
• Proven expertise in Airflow for workflow orchestration.
• Deep understanding of AWS services relevant to data engineering.
• Experience with Amazon Redshift for data warehousing.

Total Experience

10+ years

Relevant Experience

5+ years

Mandatory skills

AWS Pyspark, Redshift, Airflow, IceBerg, Lambda

Desired skills

Domain (Industry)

Semicon

Client Name

GlobalFoundries