Experience- 4+ years
Please find the job description below:
About the Role:
We are looking for an experienced Big Data Engineer with proven hands-on expertise in Apache Spark. The role involves designing, developing, and optimizing large-scale data processing pipelines. Proficiency in Scala is preferred; however, experience in any JVM-based language for Spark development is acceptable.
Key Responsibilities:
1. Design, develop, and maintain data processing pipelines using Apache Spark.
2. Optimize Spark jobs for performance and scalability in distributed environments.
3. Work with large datasets to perform ETL (Extract, Transform, Load) operations.
4. Collaborate with data scientists, analysts, and engineers to deliver robust data solutions.
5. Ensure adherence to best practices for data quality, security, and governance.
6. Troubleshoot and resolve issues related to data processing and performance tuning.
