QA Manager (Playwright and LLM - DeepEval)

Job Description – QA Manager (Playwright + LLM - DeepEval)

Position: QA Manager

We are looking for a highly driven and technically strong QA Manager to lead quality engineering initiatives across modern web applications and AI-powered products. The ideal candidate should have strong expertise in test automation using Playwright with JavaScript, hands-on understanding of GenAI/LLM evaluation methodologies, and excellent stakeholder communication skills. This role requires someone who can thrive in a fast-paced, high-growth environment while building scalable quality processes and collaborating across engineering, product, and client teams.

Key Responsibilities

\n
• Lead and manage end-to-end Quality Assurance activities across multiple products and releases.
\n
• Define and implement QA strategies, test plans, automation frameworks, and quality standards.
\n
• Drive automation initiatives using Playwright and JavaScript for UI, API, and end-to-end testing.
\n
• Establish scalable automation practices integrated with CI/CD pipelines.
\n
• Validate and evaluate GenAI/LLM-based applications using metrics such as relevancy, faithfulness, contextual accuracy, hallucination detection, bias, and response quality.
\n
• Collaborate with Product Managers, Engineering teams, Data/AI teams, and clients to ensure high-quality product delivery.
\n
• Participate in requirement discussions and provide quality-focused inputs during early development stages.
\n
• Manage defect lifecycle, root cause analysis, risk assessment, and release sign-offs.
\n
• Build and mentor high-performing QA teams while promoting a strong quality culture.
\n
• Work closely with clients and stakeholders to provide testing updates, quality reports, risk analysis, and release readiness communication.
\n
• Adapt quickly to evolving priorities and support rapid product iterations in a fast-growing environment.
\n
• Drive continuous improvement initiatives across testing processes, tools, and delivery practices.
\n
• Ensure application performance, reliability, security, usability, and compliance standards are met.
\n
• Track and report QA metrics, automation coverage, release quality, and operational efficiency.
\n

Required Skills & Qualifications

\n
• Bachelor’s degree in Computer Science, Engineering, or related field.
\n
• 10+ years of experience in Software Testing/Quality Engineering with leadership experience.
\n
• Strong hands-on experience with Playwright and JavaScript/TypeScript automation frameworks.
\n
• Good understanding of API testing, UI testing, regression testing, and CI/CD integration.
\n
• Practical understanding of Large Language Models (LLMs), GenAI workflows, prompt validation, and AI evaluation concepts.
\n
• Experience with LLM evaluation tools/frameworks such as DeepEval or equivalent is preferred.
\n
• Excellent client-facing communication and stakeholder management skills.
\n
• Strong collaboration and cross-functional coordination abilities.
\n
• Ability to work effectively in fast-paced startup or high-growth environments.
\n
• Experience in Agile/Scrum development methodologies.
\n
• Strong analytical, problem-solving, and decision-making capabilities.
\n

Preferred Qualifications

\n
• Experience testing AI-powered healthcare, SaaS, or enterprise applications.
\n
• Exposure to cloud platforms such as AWS, Azure, or GCP.
\n
• Knowledge of performance testing and security testing practices.
\n
• Familiarity with accessibility testing and modern quality engineering practices.
\n
• Experience building QA processes from scratch in scaling organizations.
\n

What We’re Looking For

\n
• Ownership mindset with strong execution capabilities.
\n
• Passion for innovation and modern quality engineering practices.
\n
• Ability to balance technical depth with people leadership.
\n
• Strong focus on delivering exceptional product quality and customer experience.
\n

Job Description – QA Manager (Automation + LLM ( Deep)

Position: QA Manager

We are looking for a highly driven and technically strong QA Manager to lead quality engineering initiatives across modern web applications and AI-powered products. The ideal candidate should have strong expertise in test automation using Playwright with JavaScript, hands-on understanding of GenAI/LLM evaluation methodologies, and excellent stakeholder communication skills. This role requires someone who can thrive in a fast-paced, high-growth environment while building scalable quality processes and collaborating across engineering, product, and client teams.

Key Responsibilities

\n
• Lead and manage end-to-end Quality Assurance activities across multiple products and releases.
\n
• Define and implement QA strategies, test plans, automation frameworks, and quality standards.
\n
• Drive automation initiatives using Playwright and JavaScript for UI, API, and end-to-end testing.
\n
• Establish scalable automation practices integrated with CI/CD pipelines.
\n
• Validate and evaluate GenAI/LLM-based applications using metrics such as relevancy, faithfulness, contextual accuracy, hallucination detection, bias, and response quality.
\n
• Collaborate with Product Managers, Engineering teams, Data/AI teams, and clients to ensure high-quality product delivery.
\n
• Participate in requirement discussions and provide quality-focused inputs during early development stages.
\n
• Manage defect lifecycle, root cause analysis, risk assessment, and release sign-offs.
\n
• Build and mentor high-performing QA teams while promoting a strong quality culture.
\n
• Work closely with clients and stakeholders to provide testing updates, quality reports, risk analysis, and release readiness communication.
\n
• Adapt quickly to evolving priorities and support rapid product iterations in a fast-growing environment.
\n
• Drive continuous improvement initiatives across testing processes, tools, and delivery practices.
\n
• Ensure application performance, reliability, security, usability, and compliance standards are met.
\n
• Track and report QA metrics, automation coverage, release quality, and operational efficiency.
\n

Required Skills & Qualifications

\n
• Bachelor’s degree in Computer Science, Engineering, or related field.
\n
• 10+ years of experience in Software Testing/Quality Engineering with leadership experience.
\n
• Strong hands-on experience with Playwright and JavaScript/TypeScript automation frameworks.
\n
• Good understanding of API testing, UI testing, regression testing, and CI/CD integration.
\n
• Practical understanding of Large Language Models (LLMs), GenAI workflows, prompt validation, and AI evaluation concepts.
\n
• Experience with LLM evaluation tools/frameworks such as DeepEval or equivalent is preferred.
\n
• Excellent client-facing communication and stakeholder management skills.
\n
• Strong collaboration and cross-functional coordination abilities.
\n
• Ability to work effectively in fast-paced startup or high-growth environments.
\n
• Experience in Agile/Scrum development methodologies.
\n
• Strong analytical, problem-solving, and decision-making capabilities.
\n

Preferred Qualifications

\n
• Experience testing AI-powered healthcare, SaaS, or enterprise applications.
\n
• Exposure to cloud platforms such as AWS, Azure, or GCP.
\n
• Knowledge of performance testing and security testing practices.
\n
• Familiarity with accessibility testing and modern quality engineering practices.
\n
• Experience building QA processes from scratch in scaling organizations.
\n

What We’re Looking For

\n
• Ownership mindset with strong execution capabilities.
\n
• Passion for innovation and modern quality engineering practices.
\n
• Ability to balance technical depth with people leadership.
\n
• Strong focus on delivering exceptional product quality and customer experience.
\n