Data Engineer II

Role Overview

We are seeking a highly skilled Data Engineer II to design, build, and scale robust data platforms that power analytics and product use cases. This role requires strong ownership in developing data pipelines, optimizing infrastructure, and ensuring high standards of data quality, governance, and performance. The ideal candidate brings deep hands-on experience with modern data technologies and thrives in a fast-paced, product-driven environment.

Key Responsibilities

Data Pipeline Development

• Design, build, and maintain scalable data pipelines for ingesting, transforming, and loading data across batch and real-time (streaming) systems.
• Integrate data from diverse sources while ensuring reliability, scalability, and efficiency.

Data Modeling

• Design and implement optimized data models, warehouse schemas, and tables for analytics and reporting.
• Apply best practices in data architecture to support scalable data consumption.

Platform & Infrastructure Management

• Manage and optimize data platforms and infrastructure including Spark, Snowflake, Kafka, Airflow, and Kubernetes.
• Ensure high availability, scalability, and fault tolerance of data systems.

Data Quality & Observability

• Implement data validation frameworks, automated testing, and monitoring systems.
• Establish alerting mechanisms to ensure data freshness, accuracy, and completeness.

Governance & Security

• Enforce data governance standards including access controls, data lineage, and compliance.
• Ensure secure handling of sensitive data (PII) aligned with regulatory requirements.

Performance & Cost Optimization

• Optimize queries, partitioning strategies, file formats, and compute resources.
• Continuously improve system performance while managing infrastructure costs.

Tooling & Automation

• Develop and maintain CI/CD pipelines for data workflows.
• Build internal tools, reusable frameworks, and automation solutions to improve engineering productivity.

Required Skills & Qualifications

• 5-8 years of experience in data engineering or a related field.
• Strong programming skills in Python, Scala, or Java.
• Advanced proficiency in SQL and data modeling concepts.
• Hands-on experience with:
  • Apache Spark (distributed data processing)
  • Snowflake or similar cloud data warehouses
  • Kafka or other streaming platforms
  • Airflow or similar orchestration tools
  • Kubernetes (container orchestration)
• Proven experience building both batch and real-time data pipelines.
• Strong understanding of data quality, observability, and governance frameworks.