Lead II - Data Engineering (Python, AWS, SQL)

About The Job:

• We are looking for Senior ETL / Data Integration Engineers to reinforce an existing 3-person ETL team.
• The team is responsible for designing, developing, and maintaining all data flows between the E-commerce platform, SAP, the OMS, and several third-party systems.
• The technical environment is primarily based on Python, AWS Lambda, AWS ECS, and event-driven pipelines triggered through a Kafka-like messaging system.
• The objective is to onboard senior, hands-on engineers who can quickly contribute to existing developments, improve integration patterns, and bring technical expertise to support the team’s roadmap.

Essential Job Functions:

• Design, develop, and maintain ETL pipelines and data integration flows.
• Build and improve Python-based data processing components.
• Develop and operate AWS Lambda functions and containerized services on ECS.
• Integrate data flows between E-commerce platforms, SAP, OMS, and external systems.
• Work with event-driven mechanisms to trigger and orchestrate ETL pipelines.
• Ensure reliability, scalability, data consistency, and proper error handling.
• Contribute to CI/CD pipelines and delivery processes using GitLab and Terraform.
• Participate in code reviews, technical discussions, and architecture decisions.
• Improve observability, logging, and monitoring of ETL and integration workflows.
• Document technical solutions, standards, and integration patterns.

Qualifications:

• Senior hands-on engineer with strong Python development experience.
• Solid experience working in AWS cloud-native environments.
• Good knowledge of AWS Lambda, ECS, and containerized workloads.
• Experience with ETL, data integration, or event-driven architectures.
• Strong understanding of messaging systems such as Kafka or equivalent technologies.
• Ability to analyze existing systems and propose pragmatic improvements.
• Strong autonomy, ownership mindset, and focus on delivery.
• Excellent communication skills with both technical and non-technical stakeholders.
• Comfortable working in Agile environments using GitLab, Jira, and Confluence.

Technical Skills

• Python.
• AWS Lambda.
• AWS ECS.
• ETL Development.
• Data Integration.
• Event-Driven Architecture.
• Kafka or Equivalent Messaging Platforms.
• Terraform.
• GitLab CI/CD.
• Monitoring, Logging, and Observability.

Nice to Have

• Experience in E-commerce environments.
• Experience integrating with SAP systems.
• Experience integrating with Order Management Systems (OMS).
• Knowledge of data quality frameworks and controls.
• Experience implementing retry mechanisms and idempotency patterns.
• Knowledge of monitoring and operational best practices for distributed systems.

How to Apply: Interested candidates are encouraged to respond/submit their updated resumes, and for additional job opportunities, please visit Jobs In India – VARITE.

Unlock Rewards: Refer Candidates and Earn.

If you're not available or interested in this opportunity, please pass this along to anyone in your network who might be a good fit and interested in our open positions.

About VARITE: VARITE is a global staffing and IT consulting company providing technical consulting and team augmentation services to Fortune 500 Companies in USA, UK, CANADA and INDIA.

Equal Opportunity Employer:

VARITE is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. We do not discriminate based on race, color, religion, sex, sexual orientation, gender identity or expression, national origin, age, marital status, veteran status, or disability status.