Jac-GPT is a RAG-based chatbot built for Jaseci to help users retrieve accurate information from the Jaseci and Jac programming language documentation. The assistant is designed to make it easier for developers to understand Jac concepts, explore documentation, and get documentation-grounded answers while working with the language.
Key Contributions
- Vector Store Migration - Switched the vector store from Chroma to FAISS for faster similarity search and retrieval.
- Retrieval Pipeline Optimization - Improved retrieval quality using cross-encoder re-ranking and better chunk metadata.
- RAG Evaluation - Tested different retrieval configurations, including chunk size, chunk overlap, number of retrieved chunks, and final context size.
- Prompt Engineering - Refined prompts to generate clearer, more accurate, and documentation-grounded responses.
- Response Streaming - Added streaming support using Server-Sent Events so responses could appear in real time.
- UI Improvements - Helped redesign and improve the chatbot interface.
A demonstration of Jac-GPT answering documentation queries
RAG Pipeline Process
- Similarity Search - The query is searched against the FAISS vector store to retrieve the top 30 most relevant documentation chunks.
- Re-ranking - The retrieved chunks are scored using a cross-encoder re-ranker by comparing each chunk against the query.
- Context Selection - The top 5 highest-scoring chunks are selected as the final context.
- Response Generation - The selected chunks are passed to the model along with the query to generate a documentation-grounded response.
I also tested a hybrid search approach combining semantic search and keyword search, but it produced weaker responses with insufficient context and bias toward certain documentation sections.