Design, develop, and optimize big data pipelines and workflows using tools such as Apache Spark, Hadoop, Kafka, and Hive.
Work closely with data scientists, analysts, and business teams to understand data requirements.
Build and maintain data lakes or data warehouses on cloud or on-prem platforms.
Develop ETL/ELT solutions for structured and unstructured data ingestion and processing.
Ensure data quality, consistency, security, and compliance across platforms.
Monitor performance, diagnose issues, and implement scalable solutions for big data environments.
Automate workflows and integrate data pipelines with external systems or APIs.
Collaborate in Agile/Scrum teams and contribute to continuous delivery practices
