What is Retrieval-Augmented Generation (RAG) and Why Does It Matter To Me?

Important note: This training may demonstrate AI tools that are not approved for use with Stanford data. Inclusion in this session does not imply institutional approval. Participants should refrain from entering Stanford data into unapproved tools. An up-to-date list of approved and reviewed tools is available on the GenAI Evaluation Matrix page.

Have you ever asked an AI tool a question and gotten a confident but completely wrong answer? You are not alone. The way AI systems retrieve and use information has a big impact on the quality of the responses they generate, and understanding that process can help you work with AI tools more effectively.

This introductory workshop explores Retrieval-Augmented Generation (RAG), the architecture behind many of the AI tools you may already be using at work. Through plain-language explanations, live demonstrations, and guided activities, participants will explore how AI tools find, read, and use information to respond to questions, why they sometimes get things wrong, and what practical steps can improve results. No technical background is required.

You will have the opportunity to: 

1. Explore what RAG is and how it powers many common AI tools
2. Understand in plain terms how AI retrieves and uses information to generate responses
3. Identify what makes an AI response trustworthy vs. unreliable
4. Practice prompting strategies that produce more grounded, accurate results
5. Build a personal RAG in ChatGPT

1. How Does AI Know What It Knows?

• Plain-language explanation of RAG using everyday analogies
• Live comparison of a basic AI response vs. a RAG-powered response to the same question

2. How AI Reads and Uses Your Information

• Conceptual walkthrough of how AI breaks documents into pieces and retrieves relevant sections
• How uploaded documents change the quality of AI responses
• What kinds of documents work well vs. poorly in a RAG system

3. Why AI Gets Things Wrong

• Accessible explanation of hallucination, retrieval gaps, and low-quality source material
• Side-by-side examples of a RAG system responding well and poorly to the same question
• Discussion on what caused the difference in response quality

4. Prompting for Grounded Results

• Practical prompting strategies for getting accurate, sourced responses
• Guided practice using provided examples
• Applying prompting strategies to participant-selected use cases

5. Build Your Personal RAG in ChatGPT

• Uploading documents or a provided sample set into ChatGPT
• Configuring a basic custom knowledge setup
• Interacting with the knowledge base through structured prompts
• Comparing results with and without the knowledge base active

6. Where Does This Fit in Your Work?

• Reflection activity: each participant identifies one realistic use case from their own role

None.