An AI agent that answers student queries over both voice calls and text chat, grounded in real institutional docs.
Role: AI Engineer
Voice → understanding → actionSystem study ↗
“Can you help with my coursework?”Natural speech, understood.
Grounded in institutional docs
Voice / text → Retrieval → Reasoning → Response
AI Engineer
Conceptual system illustration
An AI support agent for IIT Madras students that handles questions over both voice calls and text chat. It runs retrieval-augmented generation over an ingested knowledge base, with LLM tool-calling so it can act, not just answer. Built on FastAPI with a provider-agnostic LLM layer (LiteLLM — GPT-4.1, Claude, Gemini, Groq or local Ollama), a Qdrant vector store, async PostgreSQL and Redis caching; voice runs through a Deepgram STT/TTS pipeline on Vapi/LiveKit, with Sentry monitoring and a Dockerized stack.
Highlights
Built a dual-channel agent — phone voice calls and text chat over one reasoning core — with LLM tool-calling so it can take actions, not just reply.
Implemented a RAG pipeline: document ingestion and chunking into a Qdrant vector store, so every answer is grounded in real IITM material.
Designed a provider-agnostic LLM layer via LiteLLM (GPT-4.1 / Claude / Gemini / Groq / local Ollama), making models swappable without touching app logic.
Wired a real-time voice pipeline (Deepgram STT/TTS on Vapi/LiveKit) on a FastAPI backend with async PostgreSQL, Redis caching, Sentry monitoring and a Dockerized stack.