Senior Research Engineer

This role is for one of Weekday's clients
Salary range: Rs 5000000 - Rs 9000000 (ie INR 50 - 90 LPA)

Min Experience: 1+ years
Location: Bengaluru
JobType: full-time

We are looking for a highly motivated Senior Research Engineer with 1-8 years of experience to join our research and engineering team. The ideal candidate will work at the intersection of LLM research, model evaluation, and large-scale experimentation, helping develop rigorous methods to understand, measure, and improve the capabilities of modern language models.

You will design and implement evaluation frameworks, build high-quality datasets and test suites, analyze model behavior, and translate research findings into practical improvements. This role is ideal for someone who enjoys solving open-ended research problems while also being comfortable building production-quality systems.

Requirements

Key Responsibilities

• Design, develop, and maintain comprehensive LLM evaluation (evals) frameworks to measure model capabilities, reliability, reasoning, instruction following, safety, and task performance.
• Conduct research on large language models, including model behavior, capabilities, limitations, prompting, fine-tuning, and evaluation methodologies.
• Develop novel evaluation methodologies and experiments for emerging LLM capabilities and use cases.
• Create high-quality evaluation datasets, test cases, rubrics, and automated evaluation pipelines.
• Analyze model outputs using quantitative and qualitative methods to identify performance gaps and behavioral patterns.
• Design controlled experiments to compare models, prompts, training approaches, and inference strategies.
• Build scalable tooling for running evaluations across large numbers of prompts, models, and datasets.
• Collaborate with researchers, ML engineers, and product teams to convert research insights into measurable model improvements.
• Investigate failures and edge cases and develop targeted evaluations to capture previously undetected model weaknesses.
• Contribute to technical documentation, research reports, internal benchmarks, and presentations of findings.
• Stay current with developments in LLM research, evaluation techniques, reasoning systems, and AI benchmarks.

Required Skills & Qualifications

• 1-8 years of experience in machine learning, AI research, software engineering, data science, or a related technical field.
• Strong hands-on experience designing and implementing LLM evals or model evaluation systems.
• Solid understanding of LLM research, including model capabilities, prompting, fine-tuning, inference, and evaluation methodologies.
• Strong Python programming and experience working with ML/AI frameworks and data-processing pipelines.
• Ability to formulate research questions, design experiments, interpret results, and communicate technical findings clearly.
• Strong analytical and problem-solving skills with attention to experimental rigor and reproducibility.
• Experience working with large datasets, automated testing, and evaluation pipelines.

Good-to-Have Skills

• Experience developing or working with LLM benchmarks and standardized evaluation suites.
• Familiarity with benchmark design, dataset curation, scoring methodologies, and statistical analysis.
• Experience with open-source LLMs, model APIs, Hugging Face, PyTorch, or similar frameworks.
• Exposure to reinforcement learning, RLHF/RLAIF, fine-tuning, synthetic data generation, or agentic systems.
• Research publications, technical blogs, open-source contributions, or demonstrated independent AI research work.