Posted 29 July, 2026
AI Agent Performance Manager
WILLWARE TECHNOLOGIES PRIVATE LIMITED
Pune, MH, MH, IN
Full Time
Company name : Willware Technologies
Role : AI Agent Performance Manager
Experience : 7+ Years
Location : Pune
JD
Mandatory Skills
- 5-7 years of experience in AI/ML, Generative AI, or AI Engineering.
- Strong experience with AI Agents / Agentic AI.
- Hands-on experience with LLMs (GPT, Gemini, Claude, Llama, etc.) and Generative AI.
- Proficiency in Python.
- Experience with Google Cloud Platform (GCP).
- Experience in Machine Learning model deployment and production environments.
- Strong knowledge of AI testing, validation, and performance evaluation.
- Experience defining AI performance metrics, KPIs, and operational guardrails.
- Experience with production monitoring, issue analysis, troubleshooting, and AI performance optimization.
- Understanding of MLOps/LLMOps, CI/CD, and AI model lifecycle management.
- Excellent communication and stakeholder management skills.
Key Responsibilities
- Define and monitor AI agent performance metrics and operational KPIs.
- Design and execute testing and validation strategies for AI agents.
- Deploy AI agents and AI models on GCP.
- Monitor production performance, troubleshoot issues, and optimize AI systems.
- Ensure operational readiness and production stability before releases.
- Collaborate with product, engineering, and business teams to deliver reliable AI solutions.
- Drive continuous improvement in AI agent quality, scalability, and reliability.
Preferred Skills
- LangChain, LangGraph, CrewAI, AutoGen, or similar AI agent frameworks.
- Experience with Prompt Engineering, RAG, Vector Databases, and AI Observability.
- Experience in cloud-native AI deployments and monitoring.
