NirDiamant/RAG_Techniques

This repository showcases various advanced techniques for Retrieval-Augmented Generation (RAG) systems. RAG systems combine information retrieval with generative models to provide accurate and contextually rich responses.

Summary

RAG_Techniques is a community-driven collection of 42+ Jupyter notebooks that walk through retrieval-augmented generation patterns, from foundational pipelines to specialized improvements for chunking, query rewriting, and context handling. Each technique ships as a self-contained tutorial with explanations, runnable code, and a one-click Colab launch. The codebase is primarily Python, built on LangChain and LlamaIndex, and draws on OpenAI models in its examples.

What it is useful for

The repository gives engineers a working reference for moving past textbook RAG demos. Each notebook targets a concrete failure mode or improvement, grouped into categories such as Foundational, Query Enhancement, and Context Enrichment. Specific examples include "Optimizing Chunk Sizes" for tuning the chunking stage, "HyDE (Hypothetical Document Embedding)" and "HyPE" for closing the query-to-document gap, "Contextual Chunk Headers" and "Context Window Enhancement" for adding metadata and surrounding text to retrieved chunks, and "Query Transformations" for rewriting user queries before search. Recently added notebooks cover "MemoRAG," "End-to-End RAG Evaluation," "Open-RAG-Eval," and "JSON RAG."

Engineers can use it to compare approaches side by side on their own corpora rather than rebuilding each variant from research papers. The taxonomy makes it easier to pick a starting point based on which stage of the pipeline is weak, and techniques like "RAG with CSV Files" and "JSON RAG" target structured sources that vanilla RAG handles poorly.

How engineers can use it

The notebooks are designed to run on Google Colab, with a Colab badge next to each entry in the main techniques table. A typical workflow is to open a notebook, supply an OpenAI API key where the code requests it, and run cells in order, then lift the pattern into a LangChain or LlamaIndex pipeline in their own codebase.

The README does not document a unified install command or environment file, so readers should check each notebook's imports for specific dependency versions. The repository's license is listed as NOASSERTION on GitHub, meaning reuse terms are not formally declared; teams shipping production code based on these notebooks should confirm licensing with the maintainer before redistribution. A CONTRIBUTING.md is linked for those wanting to add techniques.