AWS Data Engineer

Key Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL/ELT workflows on AWS.
• Build and optimize data ingestion, transformation, and processing solutions for structured and unstructured data.
• Develop and manage data lakes and data warehouses using AWS services.
• Implement data integration solutions using AWS Glue, Lambda, EMR, and other AWS services.
• Optimize data processing performance, scalability, and reliability.
• Ensure data quality, governance, security, and compliance across the data platform.
• Collaborate with data scientists, analysts, and application teams to deliver data solutions.
• Monitor data pipelines, troubleshoot production issues, and implement preventive measures.
• Document data models, ETL processes, and technical solutions.

Required Skills

• Minimum 5+ years of relevant experience as a Data Engineer.
• Strong hands-on experience with AWS data services (Mandatory).
• Expertise in AWS Glue, Amazon S3, Amazon Redshift, AWS Lambda, EMR, Athena, and IAM.
• Strong SQL skills and experience with relational databases.
• Proficiency in Python and PySpark for data engineering.
• Experience with Apache Spark and ETL/ELT pipeline development.
• Knowledge of workflow orchestration tools such as Apache Airflow or AWS Step Functions.
• Experience with version control (Git) and CI/CD pipelines.
• Strong understanding of data warehousing, data modeling, and big data concepts.
• Excellent analytical, problem-solving, and communication skills.

Preferred Skills

• Experience with Kafka, Snowflake, or Databricks.
• Knowledge of containerization technologies such as Docker and Kubernetes.
• Exposure to DevOps practices and Infrastructure as Code (Terraform or CloudFormation).
• AWS Certifications such as AWS Certified Data Engineer – Associate or AWS Certified Solutions Architect are preferred.