Quick 2 minute Tutorial to understand LLM (RAG)s

What are 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝗔𝘂𝗴𝗺𝗲𝗻𝘁𝗲𝗱 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻 (𝗥𝗔𝗚) 𝗦𝘆𝘀𝘁𝗲𝗺𝘀?

Here is an example of a simple RAG based Chatbot to query your Private Knowledge Base.

First step is to store the knowledge of your internal documents in a format that is suitable for querying. We do so by embedding it using an embedding model:

𝟭: Split text corpus of the entire knowledge base into chunks — a chunk will represent a single piece of context available to be queried. Data of interest can be from multiple sources, e.g. Documentation in Confluence supplemented by PDF reports.

𝟮: Use the Embedding Model to transform each of the chunks into a vector embedding.

𝟯: Store all vector embeddings in a Vector Database.

𝟰: Save text that represents each of the embeddings separately together with the pointer to the embedding (we will need this later).

Next we can start constructing the answer to a question/query of interest:

𝟱: Embed a question/query you want to ask using the same Embedding Model that was used to embed the knowledge base itself.

𝟲: Use the resulting Vector Embedding to run a query against the index in the Vector Database. Choose how many vectors…