Rosid, Mochamad Alfan and Bustomy, Muhammad Saddam Heykal and Suprianto, and Setiawan, Hamzah (2026) Implementation of an Academic Information Chatbot Based on Retrieval-Augmented Generation (RAG) Using LLaMA 3.1. International Journal on Human-Computing Studies, 8 (1). ISSN 2615-8159

Text IJHCS_Implementation+of+an+Academic.pdf Download (501kB)

Abstract

Providing fast and accurate academic information is a fundamental requirement for universities. However, users often experience difficulties in efficiently searching for specific information on study program websites. This study aims to develop a Retrieval-Augmented Generation (RAG)-based academic information chatbot that is directly integrated with the Informatics Study Program website at Muhammadiyah University of Sidoarjo. The system was built using the LangGraph architecture, ChromaDB vector database, and LLaMA 3.1 model through the Groq API. Testing was conducted using the RAGAS framework to assess the quality of answers and load testing for system performance. The results showed that the system achieved a Context Precision score of 0.86 and a Faithfulness score of 0.77, indicating high relevance and accuracy of answers. In addition, the implementation of the Groq API resulted in an average response latency of 2.11 seconds with a 94% success rate in load testing. These findings indicate that the RAG-based chatbot is effective in overcoming website limitations in delivering academic information at the Informatics Study Program at Muhammadiyah University of Sidoarjo.

Item Type:	Article
Uncontrolled Keywords:	Academic Chatbot, Retrieval-Augmented Generation (RAG), LLaMA 3.1; Groq API, LangGraph
Subjects:	H Social Sciences
Depositing User:	admin eprints
Date Deposited:	19 Jul 2026 11:44
Last Modified:	19 Jul 2026 11:44
URI:	http://eprints.umsida.ac.id/id/eprint/16810

Actions (login required)

View Item