NVIDIA-NeMo/Guardrails

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.

Summary

NVIDIA NeMo Guardrails is a Python toolkit that adds programmable guardrails between application code and large language models. It lets engineers define how an LLM should respond, what topics to avoid, and what data to mask, by writing YAML configurations and a modeling language called Colang. It targets LLM-based conversational systems, including RAG and chatbot applications.

What it is useful for

The toolkit constrains LLM behavior beyond what a single system prompt can express. It supports five rail types: input rails (reject or alter user input, e.g., masking PERSON or EMAIL_ADDRESS entities), dialog rails (steer the conversation along predefined flows), retrieval rails (filter RAG chunks before they reach the LLM), execution rails (wrap custom actions or tools the LLM calls), and output rails (check generated content for moderation, hallucinations, or facts).

Documented use cases include question answering with fact-checking, domain-specific assistants that must stay on topic, and LLM endpoints exposed to customers. A built-in library provides self-checking, jailbreak and injection detection, topic safety, and integrations with ActiveFence, PolicyAI, and AlignScore. The README notes that built-in guardrails "may or may not be suitable for a given production use case" and recommends internal validation against your own requirements.

How engineers can use it

Install with pip install nemoguardrails on Python 3.10, 3.11, 3.12, or 3.13. To use it:

1. Create a config/ folder with config.yml, config.py, actions.py, and .co Colang files. The config.yml lists the model and active flows, e.g., check jailbreak or self check hallucination.
2. Load it in Python:

from nemoguardrails import LLMRails, RailsConfig

config = RailsConfig.from_path("PATH/TO/CONFIG")
rails = LLMRails(config)

completion = rails.generate(
    messages=[{"role": "user", "content": "Hello world!"}]
)

The generate and generate_async methods mirror OpenAI's Chat Completions API. Colang files hold dialogue rules, e.g., matching a user insult and triggering a calm reply. Colang 1.0 is the default; 2.0 is also supported.

For server deployment, run nemoguardrails server --config PATH/TO/CONFIGS --port PORT to expose /v1/chat/completions, or use the provided Dockerfile. A CLI offers chat, evaluate, and actions-server subcommands. LangChain integration is opt-in: set NEMOGUARDRAILS_LLM_FRAMEWORK=langchain or call set_default_framework("langchain").

One documented limitation: the library emits anonymous telemetry by default (version, platform, configured rail names, active features). Disable it with NEMO_GUARDRAILS_NO_USAGE_STATS=1, DO_NOT_TRACK=1, or a do_not_track file in ~/.config/nemoguardrails/, but the opt-out must be set before the library starts. Licensed under Apache 2.0; the latest release noted in the README is 0.24.1.