Senior Data Engineer

Key Responsibilities: • Design, develop, and maintain scalable ETL/ELT pipelines for large-scale data processing. • Optimize and troubleshoot existing data pipelines for improved efficiency and reliability. • Build, manage, and optimize databases using MongoDB, Elasticsearch, Redis, and SQL systems.

• Ensure high availability, data integrity, and scalability of all database environments. • Implement cloud-based data infrastructure on AWS, Azure, or Google Cloud as per project requirements. • Configure and manage CI/CD pipelines for automated deployment and testing of data workflows.

• Work with containerization tools like Docker; exposure to Kubernetes will be advantageous. • Develop and maintain RESTful APIs using Flask for internal and external data integration. • Collaborate with frontend, backend, data science, and analytics teams to deliver robust data solutions.

• Contribute to architectural discussions, sprint planning, code reviews, and process improvements. • Monitor system performance, manage logging, and address production issues proactively. • Support continuous improvement by mentoring junior engineers and sharing best practices.

Required Skills & Qualifications: • Strong proficiency in Python for building data pipelines and APIs. • Hands-on experience with MongoDB, Elasticsearch, Redis, and SQL databases. • In-depth understanding of ETL/ELT processes, distributed data systems, and data engineering frameworks.

• Experience in developing and maintaining APIs using Flask. • Working knowledge of cloud platforms such as AWS, GCP, or Azure. • Familiarity with CI/CD tools like Jenkins or GitHub Actions.

• Knowledge of Docker; Kubernetes knowledge is a plus. • Strong analytical, debugging, and problem-solving abilities. • Excellent communication and collaboration skills, with the ability to work independently.

Preferred Skills: • Experience working with data streaming technologies such as Kafka. • Understanding of machine learning workflows or MLOps. • Familiarity with Infrastructure as Code tools including Terraform or Ansible.

Experience: • 5 to 8 years of experience in Data Engineering or a similar technical role. • Proven experience managing large-scale, production-grade data systems. Educational Qualification: • Bachelor’s or Master’s degree in Computer Science, Information Technology, Data Engineering, or a related field.