Guardrails: What Runs Before the Model and What Runs After

A guardrail is a check that runs outside the model, on the text going in or the text coming out. The input side defends against instructions that were never meant to be instructions. The output side defends against content that was never meant to leave. They fail differently and they need separate owners, because a team that builds the input filter tends to assume the output is now covered, which does not follow. Put both in your architecture diagram. A reviewer who sees one will assume the other was forgotten, and will usually be right.

Prompt injection is the class where instructions arrive inside content the application treated as data. A retrieved document. An email body, a support ticket, a filename. The model reads it all as one stream, so a sentence in a PDF telling it to ignore prior instructions carries the same weight as your system prompt. Jailbreaking is the narrower case where the user does it directly. Classifiers catch the obvious attempts and miss rephrased ones, which is why the input filter is a layer rather than a fix. Design so that a defeated filter is survivable: constrain what the model can reach, and assume some hostile text gets through.

Output checks look for content that would be a disclosure if it shipped. Personal data pulled from a record the user was never entitled to see. A credential that ended up in a log the model read. A verbatim block from a licensed document. In Japan the regulated categories come from APPI, and the review will want to know where the check runs and what it does when it fires, so decide now whether a violation means redact, refuse, or escalate to a person. Silence is the worst of the three. The user retries, rephrases, and eventually finds wording that works.

A model that can call tools has a second surface. Reading the wrong document is a disclosure; issuing the wrong write is an incident. Keep the permission decision outside the prompt, in the layer that executes the call, so an injected instruction cannot grant itself an action the user never had. This is ordinary access control applied one layer lower than usual, and Japanese enterprise reviewers recognise it on sight when it is there. Its absence reads as an application that trusts the model's judgement about authorisation.

The obvious metric is the violation rate: how often something got through that should not have. On its own it drives the filter tighter and tighter until the product stops being useful. The second number is the false refusal rate, how often a legitimate request was blocked. Report both, on the same page, from the first pilot onward. A guardrail tuned to zero violations by refusing a fifth of real questions has failed in a way that a single-metric dashboard will never show, and the customer will feel it long before anyone measures it.

Guardrails implemented inside one application protect one application. Put them in the shared layer between your applications and your models and every caller gets them, including the service someone ships next quarter without asking. That layer is the same place model routing and cost accounting belong, which has its own article here. For a Japanese buyer the practical gain is auditability: one place to inspect, one log to sample, one configuration to hand a reviewer.

A guardrail is a check that runs outside the model, on the text going in or the text coming out. Input guardrails screen for injected instructions. Output guardrails stop personal data, credentials, or licensed text appearing in a reply. They are separate systems with separate owners, and a Japanese security review asks about both by name.

Prompt injection is an instruction that arrives inside content the application treated as data: a retrieved document, an email body, a filename. A classifier catches obvious attempts and misses rephrased ones. Treat the filter as one layer, and constrain what the model can reach on the assumption some hostile text gets through.

Run an output check before the reply ships, looking for personal data the user was not entitled to see, credentials, and verbatim licensed text. Japan's APPI defines the regulated categories, so decide in advance whether a violation redacts, refuses, or escalates to a person. Silent failure teaches users to keep rephrasing.

In the layer that executes the call, never in the prompt. A model that can act has a second attack surface: reading the wrong document is a disclosure, issuing the wrong write is an incident. Keeping the permission decision outside the prompt means an injected instruction cannot grant itself an action.

Two numbers, on the same page. The violation rate says how often something got through that should not have. The false refusal rate says how often a legitimate request was blocked. A filter tuned to zero violations by refusing a fifth of real questions fails in a way a single-metric dashboard never shows.