Advanced Technologies: Beyond Prompting -- Retrieval Augmented Generation(RAG)

RAG: giving it additional knowledge beyond what it may have learned from data on the Internet or other open sources

Retrieval Augmented Generation (RAG) example

• General Chatbot
• Chatbot with RAG
  • Give the LLM additional information

1. Given a question, search relevant documents for answer (Company documents, most relevant to this question)
2. Incorporate retrieved text into an updated prompt (take the relevant text from documentation and put that into my prompt;)
   1. limitations to the prompt length or the input length for a large language model
   2. Pull out just the part of the document that's most relevant to the question
3. Generate answer from the new prompt with additional context
   1. generate an answer to this, but we're going to augment how we generate text by retrieving the relevant context or the relevant information and augmenting the prompt with that additional text
   2. also add a link to the original source document that led to this answer being generated
   3. can go back and read the original source document and double-check the answer for themseleves

Example of RAG applications

• Chat with PDF files
  • upload pdf and then ask questions
  • use RAG to generate answers for you
• Answer questions based on a website's articles
• New form of web search

Big Idea: LLM as a reasoning engine: which is to think of the LLM not as a knowledge store, but instead as a reasoning engine

• LLMs have a lot of general knowledge, but they don't know everything
• By providing relevant context in the prompt, we ask an LLM to read a piece of text, then process it to get an answer
• We're using it as a reasoning engine to process information, rather than using it as a source of information