Java / AI Developer

Job Description

Role Overview

We are looking for a Mid-Level Java Developer (4-9 years) with hands-on experience in AI integration to join our product engineering team. In this role, you will be responsible for embedding AI capabilities - including converting the user-provided natural language inputs, or database metadata information into valid product configurations, Retrieval-Augmented Generation (RAG), LLM-powered features, and agentic workflows - directly into our Java/Spring Boot-based enterprise product. You will work at the intersection of traditional backend engineering and modern AI/LLM technologies, owning the design, development, and maintenance of AI-powered components end-to-end.

Key Responsibilities:

AI Feature Development

• Capture user intent via a conversational/chat interface and translate it into valid product configurations.
  
• Extract metadata from heterogeneous sources - file formats (CSV, Excel, PDF), relational database schemas, and API contracts (OpenAPI, WSDL) and convert them to valid product configurations.
  
• Design and implement RAG pipelines that retrieve contextually relevant information from internal knowledge bases and surface it through LLM-generated responses.
  
• Integrate LLM-powered features such as intelligent chat, document summarization, content generation, and semantic search into the existing product.
  
• Build and maintain AI agent workflows - including tool use, multi-step reasoning chains, and orchestrated LLM calls - to automate complex product-level tasks.
  
• Author, version, and iterate on system prompts and user prompt templates to reliably steer LLM behavior.
  
• Apply advanced prompting techniques: chain-of-thought, few-shot examples, output format enforcement, and instruction following.
  
• Integrate with multiple LLM providers - OpenAI/Azure OpenAI, Anthropic Claude, AWS Bedrock, and open-source models (LLaMA, Mistral, etc.) - through their APIs.
  
• Build provider-agnostic abstraction layers using Spring AI and LangChain4j to allow model switching without core code changes.
  

Backend Engineering


• Write clean, maintainable, production-grade Java code following SOLID principles and enterprise design patterns.
  
• Integrate AI services with existing relational databases via JPA/Hibernate, including schema design for storing conversation history, embedding metadata, and audit trails.
  
• Manage build pipelines and dependency governance using Maven or Gradle.
  

Testing & Quality


• Write comprehensive unit, integration, and contract tests for AI-integrated components.
  
• Handle non-deterministic LLM output gracefully - implement fallback logic, retry strategies, and output validation.
  
• Monitor AI feature performance in production; instrument logging and tracing for LLM calls, including token usage, latency, and error rates.
  

Mandatory Skills & Experience


Java & Backend


• 4–9 years of professional Java development experience.
  
• Strong proficiency in Spring Boot - REST controllers, service layer design, dependency injection, and configuration management.
  
• Experience building and consuming RESTful Microservices .
  
• Working knowledge of JPA/Hibernate - entity modelling, JPQL, transaction management.
  

AI / LLM Engineering


• Hands-on experience integrating with at least one major LLM provider API (OpenAI, Azure OpenAI, Anthropic, or AWS Bedrock) from Java.
  
• Practical experience with Prompt Engineering - not just calling APIs, but crafting, testing, and iterating prompts to achieve reliable, structured outputs.
  
• Experience building RAG pipelines - document ingestion, chunking strategies, embedding generation, similarity retrieval, and context injection into prompts.
  
• Working knowledge of LangChain4j and/or Spring AI frameworks for LLM orchestration within Java applications.
  
• Understanding of AI agent concepts - tool/function calling, agent loops, multi-step task orchestration.
  

Good-to-Have Skills


• Experience with vector databases such as Pinecone, Weaviate, Qdrant, pgvector, or Chroma for embedding storage and semantic search.
  
• Exposure to multiple LLM providers (OpenAI, Anthropic Claude, AWS Bedrock, LLaMA/Mistral via Ollama or similar).
  
• Familiarity with open-source LLM deployment - running models locally or on self-hosted infrastructure.
  
• Knowledge of token budgeting , context window management, and cost optimization strategies for LLM APIs.
  
• Experience with observability tooling for AI systems - LangSmith, Helicone, or custom logging pipelines.
  
• Familiarity with streaming responses from LLMs (Server-Sent Events / WebSocket delivery to frontend).
  
• Understanding of embedding models - how they differ from generative models, when to use which, and how to evaluate embedding quality.
  
• Exposure to CI/CD pipelines and containerization (Docker, Kubernetes).