Design, develop, and maintain large-scale data processing workflows using big data technologies
Develop ETL/ELT pipelines to ingest, clean, transform, and aggregate data from various sources
Work with distributed computing frameworks such as Apache Hadoop, Spark, Flink, or Kafka
Optimize data pipelines for performance, reliability, and scalability
Collaborate with data scientists, analysts, and engineers to support data-driven projects
Implement data quality checks and validation mechanisms
Monitor and troubleshoot data processing jobs and infrastructure
Document data workflows, architecture, and processes for team collaboration and future maintenance
