Senior Data Engineer (5-7 Years)

Lead Data Engineer

About the Role:

Xtelify Ltd is looking for a Data Engineer to spearhead our Data Platform team. In

this role, you will architect, scale, and oversee the deployment of mission-critical data

pipelines and data-products handling a massive scale of ~5B daily events and a concurrency

of 5M active users . Our platform is built on a cloud-native modern data stack (AWS/GCP),

enabling real-time reporting and deep data exploration from first principles.

As a Lead, you will not only write high-performance code but also define technical

roadmaps, mentor engineers, and ensure our infrastructure supports Xtelify's aggressive

growth.

• Experience: 6+ Years (with 4+ years dedicated to data engineering, software

development, or distributed systems)

• Job Location: Gurgaon

Responsibilities:

• Architecture & Scaling: Architect, build, and optimize robust, fault-tolerant, and

high-performance data pipelines capable of processing multi-terabyte streaming data

daily across Xtelify's video streaming platforms.

• SLA Ownership: Define and maintain the infrastructure required to extract,

consume, and analyze live streaming data within a high-availability, 99.999% SLA

Big Data environment.

• Data Strategy & Optimization: Enhance data models, workflows, and processing

frameworks to improve performance, scalability, and cost-efficiency across both

structured and unstructured datasets.

• Data Governance: Drive data governance, data security, and compliance best

practices across large-scale, complex data environments.

• Cross-Functional Collaboration: Partner with data scientists, product teams, and

business stakeholders to translate complex requirements into scalable, data-driven

solutions.

• Technical Leadership & Mentorship: Serve as the technical anchor for the

engineering team. Provide mentorship, promote engineering best practices in pipeline

and API design, and manage the full lifecycle of data services.

• Innovation: Stay ahead of emerging technologies to drive innovation in analytics

pipelines, real-time data serving, and infrastructure for machine learning applications.

Desired Profile:

• Education: B.E / B.Tech / M.E / M.Tech / M.S in Computer Science or Software

Engineering from a premier institute.

• Core Expertise: Deep, hands-on experience with Spark and Scala/Java with a

proven track record of deploying production-grade streaming and batch applications

at scale.• Streaming & Real-Time: Advanced, production-level experience with real-time and

streaming data architectures such as Kafka, Flink, or Spark Streaming for low-

latency data processing.

• System Design & Modeling: Strong grasp of advanced data structures, algorithms,

distributed system design, ETL pipeline design, and dimensional modeling.

• Big Data & Cloud Ecosystem: Deep proficiency in the Hadoop ecosystem and cloud

platform ecosystems (AWS/GCP) including tools like S3/GCS, EMR/Dataproc,

Cloud SQL, Nifi, Trino and query optimization.

• Delivery & Execution: Proven ability to work in complex, unstructured

environments and take end-to-end technical ownership of complex, enterprise-wide

data modules.

Good to Have:

• API Development: Hands-on experience building APIs using REST, GraphQL, or

gRPC to serve real-time and batch data.

• OLAP Engines: Experience with real-time OLAP engines like Apache Druid,

Apache Pinot, or ClickHouse.

• NoSQL Solutions: Exposure to high-throughput NoSQL solutions like Cassandra,

MongoDB, Redis, or Elasticsearch at scale.

• Cloud Infrastructure: Familiarity with Infrastructure as Code (Terraform) and

containerization (Docker, Kubernetes).

• AI Agents for Data Ops: Experience building intelligent AI agents or LLM-driven

tools to automate BAU data operations (e.g., automated root-cause analysis for

pipeline failures, self-healing workflows, semantic parsing of logs, or intelligent

optimization of query costs and resource allocation).