Hadoop Developer- Chennai
Experience: 4+ Years
Location: Chennai
Department: Data Engineering / Big Data
Job Overview:
We are looking for an experienced Automation Hadoop Developer with a strong expertise in Scala to join our growing Data Engineering team. As an Automation Hadoop Developer, you will be responsible for designing, developing, and automating data pipelines in a large-scale Hadoop ecosystem. You will leverage your skills in Scala, Hadoop, and related technologies to build efficient and scalable data solutions that meet the needs of our organization.
The ideal candidate will have hands-on experience with Hadoop ecosystem tools like Hive, Pig, HBase, and Spark and will be highly proficient in Scala for automating and optimizing ETL processes, data pipelines, and big data workflows.
Required Skills & Qualifications:
4+ years of experience in Hadoop development, with a strong focus on automation and Scala.
Strong proficiency in Scala and experience using it for building automated ETL pipelines and data processing tasks.
Hands-on experience with Hadoop ecosystem tools such as Spark, Hive, HBase, Pig, and MapReduce.
Solid understanding of HDFS and other distributed storage systems.
Experience with stream and batch processing frameworks like Apache Spark Streaming and Apache Flink.
Strong knowledge of SQL for querying and manipulating large datasets within Hadoop.
Experience with Apache Kafka or similar tools for real-time data streaming and messaging.
Proficiency in Linux/Unix environments and scripting languages like Bash.
Familiarity with Cloud platforms like AWS or Azure for deploying and managing Hadoop clusters (preferred).
Experience with containerization tools such as Docker is a plus.
Understanding of CI/CD pipelines and how to automate deployment processes.
Proficient in version control using Git.
Familiarity with data warehousing concepts and technologies like Redshift or Google BigQuery is a plus.
