1. What Is Retrieval-Augmented Generation?
RAG is a way to leverage already trained LLMs such as GPT-4 Gemini, Bard and Llama when building applications powered by AI. By adding local knowledge (such as university policy or protocol information), context (such as professor profile information) or history (such as student data), RAG augments LLMs to avoid common AI problems, such as a lack of specific information and LLM hallucinations.
2. How Does RAG Work?
RAG essentially “wraps” an LLM by adding relevant information to the prompt (query) that is sent to the LLM. For example, suppose an administrator wants to ask a question about a student’s graduation status. RAG processes the query first to understand the general issue and specifics being addressed. Then, the RAG tool might retrieve the university protocol, recommendations, student history and grades, sending all of this to the LLM along with the question from the administrator. This gives the LLM local knowledge, context and history to help answer the question. All of this is invisible to the administrator because the RAG wrapper is doing the work of choosing what to send to the LLM.
DIVE DEEPER: How can organizations avoid LLM bias and inaccuracy using data governance?
3. How Does RAG Compare to Fine-Tuning?
Fine-tuning an existing LLM adds information to the model, usually private data. This can be useful to make the LLM better at specific tasks. RAG improves the LLM with up-to-date and contextually useful information at the moment the LLM is queried. The student data isn’t saved in the model, yet the model always has the latest information it needs to help answer questions, and there’s no security issue of leaking confidential data.
4. What Are the Benefits of Using RAG?
RAG extends the value of LLMs by giving the model additional information: local, relevant documents and protocols as well as real-time information from academic databases. Administrators and school officials can ask questions based on what is happening this minute, not when the LLM was trained. This additional information lets the LLM deliver answers that are both more relevant and more accurate. IT teams can also build in higher levels of security and tighter access controls, only feeding in information that the person asking the question is allowed to know.
5. What Are the Challenges of RAG?
RAG applications must preprocess user prompts to decide what additional information to send down. That can be a difficult job, and there’s a chance that the RAG application will send down the wrong data. Also, just because you’ve given an LLM additional information doesn’t mean that it is going to properly understand that data and incorporate it into the response.
