Design, develop, and optimize large-scale data processing solutions using PySpark.
Build scalable ETL pipelines to ingest, process, and transform structured and unstructured data.
Work with data architects and analysts to define and implement data models and flows.
Optimize Spark jobs for performance and reliability in production environments.
Integrate with cloud-based data platforms such as AWS, Azure, or GCP.
Write efficient and reusable Python code for data transformation and analytics.
Implement data quality, validation, and monitoring tools.
