Guardrails

Guardrails are the checks and constraints placed around a large language model in production to keep its behavior within acceptable bounds: blocking disallowed inputs, filtering unsafe or off-topic outputs, validating that a response matches an expected format, or restricting what actions an AI agent is permitted to take.

They can sit at different points in the pipeline: input guardrails screen what reaches the model, output guardrails check what the model produces before it reaches a user, and some systems add a secondary model whose only job is to review the primary model's output, a pattern related to LLM-as-a-judge. This differs from LLM evaluation, which measures quality before deployment; guardrails act continuously, in real time, on live traffic.

Guardrails matter because a language model's raw output cannot be assumed safe, accurate, or on-policy, particularly for hallucination, leaked sensitive information, or an agent taking an unintended action. A common pitfall is over-relying on guardrails as a single line of defense, when they are best combined with careful prompt design, monitoring, and scoped permissions; another is guardrails strict enough to block legitimate requests, which pushes teams to weaken them and reintroduces the original risk, an ongoing concern in Responsible AI practice.