Guardrails & Governance

Constrain it and prove it

What stops this from causing harm, and who signs off?

15 entries · 3 categories · 2 equations

The arithmetic

Layered defense, attack success rate

\mathrm{ASR}_{\text{total}} \;=\; \prod_{i=1}^{k} \mathrm{ASR}_i \quad \text{only if } \mathrm{ASR}_i \perp \mathrm{ASR}_j

Three filters at 30% ASR each multiply to 2.7% — on paper. Independence is the assumption that fails: an adaptive attacker who defeats one classifier usually defeats correlated ones, which is why measured layered ASR against adaptive attacks stays above 85% instead of collapsing.

False blocks per day

B_{\text{false}} \;=\; \mathrm{FPR} \times Q_{\text{day}}

A filter with 1% FPR on a million-request/day product blocks 10,000 legitimate requests. Guardrail vendors quote recall; the number that decides whether you can ship is FPR at your traffic volume.

Guardrails & PII

Filters for injection, toxicity and personal data.

Microsoft Presidio

NER and regex PII detection, redaction and anonymisation

Deterministic, which makes it one of the few components here that behaves predictably.

piideterministic

MITopen sourcewidely deployedself-host

Azure Prompt Shields

Managed moderation and prompt-injection filters

Published work reports character-injection evasion approaching total bypass.

managedevasion-risk

Proprietarycommercialproduction viablesaas

Guardrails AI

Output validators and schema enforcement via a validator hub

validation

Apache-2.0open sourceproduction viableself-host

Lakera Guard

Commercial injection and jailbreak detection API

managedevasion-risk

Proprietarycommercialproduction viablesaas

Llama Guard / Prompt Guard

Hazard-taxonomy classifier plus a fast injection pre-filter

Defeated by prompt-overflow inputs that it catches in short context.

classifierevasion-risk

Meta communitysource availableproduction viableself-host

LLM Guard

Scanner suite covering PII, toxicity and injection heuristics

scanner

MITopen sourceproduction viableself-host

NeMo Guardrails

Colang dialog-flow rails and topic control that orchestrate other classifiers

dialog-rails

Apache-2.0open sourceproduction viableself-host

Governance tooling

Platforms that emit risk registers and compliance artifacts.

Credo AI

Risk registers, control mappings and Annex IV technical documentation

compliance-artifacts

Proprietarycommercialproduction viablesaas

Holistic AI

AI risk assessment, bias auditing and governance reporting

compliance-artifactsaudit

Proprietarycommercialproduction viablesaas

IBM watsonx.governance

Model inventory, risk workflows and compliance evidence inside the IBM stack

compliance-artifactsenterprise-incumbent

Proprietarycommercialproduction viableboth

Trustible

Regulatory mapping and AI use-case registry for compliance teams

compliance-artifacts

Proprietarycommercialearlysaas

Regulation & frameworks

Obligations and certifications an enterprise roadmap must absorb.

Digital Omnibus on AI

First AI Act amendment; Regulation (EU) 2026/1744, in force 27 July 2026

Deferred Annex III high-risk obligations to 2 Dec 2027 and Annex I to 2 Aug 2028. GPAI enforcement on 2 Aug 2026 was NOT deferred.

euamendmentverify-dates

Regulationcommercial

EU AI Act

Risk-tiered obligations with extraterritorial reach; Regulation (EU) 2024/1689

In force 1 Aug 2024. Prohibitions and AI literacy live 2 Feb 2025. GPAI obligations live 2 Aug 2025.

eubaseline

Regulationcommercial

ISO/IEC 42001:2023

Certifiable AI management system standard

The only auditable certificate in this space. Takes 6-12 months and increasingly appears in due-diligence questionnaires.

certifiableprocurement

Standardcommercial

NIST AI RMF 1.0

Voluntary US risk framework with a Generative AI profile

The cheapest credible starting point, and it maps onto ISO 42001. Roughly 3-6 months.

usvoluntaryon-ramp

Publicopen source

Field notes

Prompt-injection classifiers are filters, not boundaries. Published 2025-2026 work shows character injection and algorithmic evasion approaching total evasion against several commercial and open detectors.
Over-length prompt-overflow inputs defeat detectors that catch the same payload in short context. Layered defenses cut naive attack success substantially, but adaptive attackers still exceed 85% success.
Put the real control at the architecture layer: least-privilege tool scopes, human approval on state-changing actions, no implicit trust in retrieved content.
Deterministic components — Presidio redaction, schema validation, allow-lists — are the only parts that behave predictably. Anything sold as blocking prompt injection is theatre.
The Annex III deferral to December 2027 is a resequencing opportunity, not a cancellation.
Start ISO 42001 now if you sell to enterprise or government. It takes 6-12 months and increasingly appears in due-diligence questionnaires. NIST AI RMF first is the standard on-ramp.
Verify regulatory dates against primary sources before citing them internally. This is a starting point, not legal advice.