Harmful Response Prevention

Note: When the action is Warn or Block, this guardrail may appear to delay streaming responses. There is no actual delay of the entire response. The complete response is delivered in the same amount of time it would take for the entire streaming response to be delivered. The apparent delay is because WitnessAI withholds the initial and incremental partial responses until the entire response has been received and evaluated by the guardrail. The console shows a Response buffering notice when you choose one of these actions.

Learn more.

Harmful Response Prevention is WitnessAI’s response analysis and control guardrail. The purpose of this guardrail is to analyze the responses back from AI models to user prompts, and then detect, and optionally prevent these responses from being sent to the user. These harmful responses are evaluated in three broad categories; harm to self, harm to others, and illegal activity. When this guardrail detects these activities, it provides the option to Allow, Warn, or Block the response from the AI model with a customizable message.

How it works

Unlike guardrails that inspect the user's prompt, Harmful Response Prevention checks the model's reply on its way back to the user. You configure it in two parts:

An action and a Message that apply whenever any rule matches. You choose the action from the drop-down at the start of the rule sentence, for example Warn the user with the following message.
One or more rules. Each rule selects the risk types to detect and sets its own severity (High, Medium or Low).

This lets you watch for several groups of risk types, each at its own severity, under one action and message. Risk Analysis uses the same Select Risk Types panel to check prompts.

Use Cases

Harm to self or others

Block or warn Users about dangerous AI responses, such as the possibility that AI Model responses may contain unqualified or inaccurate medical or financial information.

Illegal activity

Block or warn Users about dangerous AI responses, particularly common activities that may expose individuals or the Organization to legal risks.

Using Harmful Response Prevention Step-by-Step

This section covers details on the Harmful Response Prevention guardrail, and how to add it to a policy. Refer to the Policy Creation page for how to create Policies.

Add a Harmful Response Prevention guardrail

The Harmful Response Prevention guardrail enabled in a policy, showing a Warn rule with its Set Message (Required) field, a Generate Alert rule with selected risk types and severity, Add Rule, and the Response buffering note.

After creating a Policy

Click the Guardrails tab in the policy editor.
Select the Harmful Response Prevention guardrail from the available options.
Click the slide button to enable the guardrail.
Open the action drop-down at the start of the rule sentence and choose Allow, Warn, or Block.
Under Set Message (Required), enter or choose the message to show the user when a harmful response is detected. The guardrail can't be saved without one.
Add a rule: click Add Risk Type to open the Select Risk Types panel, expand a category, choose the risk types this rule should detect, and click Save in the panel.
Under THEN Set severity to, choose the rule's severity: High, Medium or Low.
To watch for another group of risk types at a different severity, click Add Rule and repeat steps 6 and 7. The top-level action and message apply to every rule.

The Select Risk Types panel, with categories such as Malicious Intent and Harmful Content expanded to show their child risk types.

When you're done, click Save to save the policy.

Harmful Response Prevention guardrail How it works Use Cases Harm to self or others Illegal activity Using Harmful Response Prevention Step-by-Step Add a Harmful Response Prevention guardrail