Skip to content

Content rules engine ​

The content rules engine checks AI traffic. It inspects what flows between users, models and tools, decides what to do with it, and records why. The engine lives in @ai-orchestrator/guard-sdk as a standalone library: it knows nothing about workflows, nodes or HTTP, and it never makes a network call of its own.

Concepts: Prompt injection and the OWASP Top 10 for LLM apps explain the attacks these checks address.

This page describes the engine. Content rules in workflows covers the Content Rules node, the agent rules port, stored rule sets and bindings, the protection port and rule events. What is still to come is listed under Not yet available.

Concepts ​

Channels ​

Every piece of content is inspected on one channel. Rule sets configure each channel separately because the same text means different things in different places: a phrase that is a harmless question in a prompt is an attack inside a tool result.

ChannelWhat it carriesTypical concerns
promptWhat the user, or a template, sends to the modeldirect injection, jailbreaks, credentials pasted by mistake
model_outputWhat the model produced before anything acts on itleaked system prompts, exfiltration links, sensitive data
tool_argumentsWhat the model wants to pass to a toolSQL and command injection, path traversal, outbound URLs, schema violations
tool_resultWhat a tool, document or web page returnedindirect injection, hidden characters, sensitive data
final_outputThe answer about to leave the runleaked canaries, sensitive data, exfiltration

Three more channels exist only in stored rule sets; the Content Rules node never offers them:

ChannelWhat it carriesSee
egressWhat a step sends out (a URL, headers, a body)Data labels and destinations
responseWhat an external webhook, chat or MCP invoke caller receivesResponses to external callers
trigger_inputA webhook payload, at the route, before the run startsResponses to external callers

Detector input ​

A detector receives the content (text, or a JSON value inspected through its canonical serialisation), the channel, provenance labels (trusted, user, untrusted_external, sensitive), the run subject (role, user id, project) and, for tool traffic, the tool context: server, tool name, input schema and annotations.

Findings ​

A finding names the detector, a category, a score from 0 to 1, the character spans it refers to, masked evidence and, for data rules, a data class such as card_number or credential. Evidence is always masked: a card number appears as •••• •••• •••• 1111, an email as a••@e••••••.org. Raw values never appear in a finding, a trace or a log line.

Detectors and tiers ​

Each detector belongs to a latency tier. Evaluation runs tiers in order, cheapest first, so a rule can stop a request before a classifier or judge is ever consulted.

TierCostBuilt in
rulemicrosecondsinjection rule packs, hidden characters, exfiltration URLs, canaries, data rule patterns, command and path checks, egress check, schema validation
parsermillisecondsSQL check (parses to an AST per dialect)
classifierlocal modelinterfaces only (injection and content safety); rule-based stand-ins run when no model is registered. Topic checks sit here too.
judgean LLM callinterface only; nothing is registered by default. Output checks can call a chat model of their own.

createDefaultGuardRegistry() returns every built-in detector. Deployments add their own with registry.register(detector) or swap a built-in one with replace.

Rule sets ​

A rule set is a named, versioned JSON document. Per channel it lists the detectors that run, their thresholds and the action for each finding category or data class. A data-class entry wins over its category; findings with neither fall back to the channel's defaultAction, which defaults to block.

Rule sets are validated with zod and serialised with a stable key order and a SHA-256 hash, so they can be stored as policy-as-code and compared across environments. Every verdict carries the rule set id, version and hash it was produced with.

Actions ​

ActionEffect
allowNothing. The finding is still recorded.
blockThe content does not pass. Remaining detectors are skipped.
redactThe spans are replaced with [REDACTED:<class>].
maskThe spans are replaced with their masked form (last four digits kept, and so on).
protectThe spans are replaced with tokens from a protection provider.
transformThe detector rewrites the content, for example stripping hidden characters.
require_approvalThe content passes to the approval flow.
reviewThe content is routed to review instead of continuing.

Injection findings (prompt_injection, indirect_injection, canary_leak) are never redacted, masked, protected or transformed. A rule set that asks for one of those gets block instead, and the verdict says why.

Shadow mode ​

Any action can run in shadow mode, per action for the whole rule set (shadow: { block: true }) or per rule (actions: { prompt_injection: { action: "block", shadow: true } }). A shadow action is recorded in the verdict as if it had applied, but the content passes unchanged and evaluation continues. Use it to observe a new rule set on real traffic before enforcing it.

Latency budget: fail open or fail closed ​

A rule set sets a latency budget in milliseconds and what happens when it runs out. The budget is checked before each detector, and each detector is raced against the time left.

  • fail_closed (the default): over budget, the content is blocked.
  • fail_open: over budget, the content passes with whatever was found so far.

A detector that throws, or a detector named by the rule set that is not registered, is treated the same way as an exceeded budget.

Protection providers ​

A protection provider turns a sensitive value into a token (protect) and, when the provider is reversible, back again (reveal). Providers declare whether they are deterministic (equal values give equal tokens, so joins keep working) and which data classes they cover. createDefaultProtectionRegistry() returns the built-in ones:

ProviderReversibleDeterministicNotes
masknoyesthe masked form of the value
keyed_hashnoyesHMAC-SHA-256 pseudonym under an injected key
aes_gcmyesnoAES-256-GCM, random nonce, key id in the token, data class bound as authenticated data
surrogatenoyeskeyed, format-preserving stand-in: an email, card, IBAN, US SSN, UK National Insurance or phone number keeps its shape (cards stay Luhn-valid); credentials, private keys and JWTs are fully redacted. Pseudonymisation: whoever holds the key can test a guessed value
token_vaultyesoptionaltokens kept in an injected store; random, deterministic, or format-preserving (card tokens stay Luhn-valid)

Key material arrives through an injected key resolver. The library never reads environment variables or files; in L2M the resolver reads stored secrets through SecretService. The vault's storage is an interface too, with an in-memory implementation for tests and single-process use; L2M stores it encrypted in the database.

Fallback ​

The rule set's protection.dataClasses maps each data class to a provider. When a finding should be protected but no provider is mapped for its class, the provider does not cover the class, or the provider fails, the rule set's protection.fallback applies. It defaults to block and may be redact or mask; it may not be protect.

Detectors ​

IdTierChannelsFinds
injection.directruleprompt, model output, final outputinstruction override, refusal of prior rules, role markers, system-prompt extraction, unrestricted personas, known jailbreak personas, safety overrides, exfiltration instructions. Also matched after leetspeak and base64 decoding and with zero-width characters removed.
injection.indirectruletool result, tool arguments, untrusted promptseverything above plus text addressing the model, embedded tasks, instructions in hidden markup, planted tool calls, secrecy demands
injection.hidden_unicoderulealltag characters (decoded as evidence), bidi overrides, zero-width runs, soft-hyphen runs; transform strips them
exfiltration.urlsrulemodel output, final output, tool arguments, tool resultmarkdown and HTML images and links whose URLs carry data (long or high-entropy query values, data-like parameter names, data-like path segments or hostnames); URLs outside an allowlist in tool arguments
leak.canaryrulemodel output, final output, tool argumentscanary tokens from the system prompt
injection.classifierclassifierprompt, tool result, model output, final outputthe registered classifier's score; the rule packs stand in
dlp.patternsruleallemail, phone, card numbers (Luhn), IBAN (mod-97), US SSN and a pluggable national-id list (UK NI number built in), credentials with known key prefixes, JWTs, private-key blocks, high-entropy values next to credential-like names, custom dictionaries and patterns
tool.sqlparsertool argumentsper dialect (postgres, mysql, mariadb, sqlite, mssql): read, write and DDL classification; stacked queries, tautologies, comment tricks and union probes; table and column allowlists; WHERE required on mutations; row cap rewrite (LIMIT or TOP) or check; unparseable SQL fails closed
tool.commandruletool argumentsshell metacharacters, command substitution, redirection, destructive commands; option-shaped arguments from untrusted input
tool.coderuletool argumentsdangerous imports and calls in Python and JavaScript code-node source
tool.pathruletool argumentspaths outside the allowed roots, traversal, encoded traversal, null bytes and symlink escapes through an injectable realpath
tool.egressruletool argumentsURLs whose host is not allowlisted, local or metadata hostnames, credentials in URLs, disallowed schemes and ports, and any address that is loopback, private, link-local, carrier NAT, cloud metadata, multicast or reserved, for IPv4 and IPv6 including IPv4-mapped IPv6 and decimal, octal and hex IPv4 spellings; each hop of a redirect chain
tool.argumentsruletool argumentsstrict JSON Schema validation (undeclared properties rejected) with size, depth, string, array and property limits
output.schemaparsermodel output, final output, responsean answer that is not JSON or does not match a JSON Schema (schema_violation)
output.topicclassifiermodel output, final output, responsean answer about a denied topic, or about none of the allowed ones (off_topic); phrases, or meaning with an embedding model
output.safetyclassifierprompt, model output, final output, responseself-harm, violence, sexual content, sexual content involving minors, hate (unsafe_content); built-in rules or a chat model
output.groundingparsermodel output, final output, responseclaims the retrieved documents do not support, and citations that name no document (ungrounded); an optional chat model judges borderline claims

The four output.* detectors are the output checks. They need options to mean anything, so the built-in rule set has none of them.

Limits ​

  • The rule packs are pattern based. They catch the phrasings in the corpus and their encoded variants; a paraphrase the patterns do not cover needs the classifier tier, which this release leaves as an interface.
  • The egress check judges addresses, not traffic. A host that resolves to a public address at check time and to a private one a moment later (DNS rebinding) is only caught when the caller resolves once and connects to the resolved address.
  • The SQL check parses whole statements. A fragment pasted into a template is reported as unparseable, which blocks, but the check cannot tell what the surrounding template does.
  • The data rule detectors are patterns with checksums. Names, addresses and free-text health information need the NER model planned for a later release.
  • Spans refer to the canonical text. For JSON content that is the serialisation with sorted keys; the transformed content is parsed back to JSON when it still parses.

Spotlighting helpers ​

wrapWithDelimiters(content) wraps untrusted content between random boundary lines and returns the matching system-prompt text; datamark(content) replaces whitespace with a marker character and does the same. PROMPT_HARDENING_TEXT is a system-prompt paragraph that tells the model what counts as data. generateCanaryToken() and canaryInstruction(token) plant a canary that leak.canary detects in output.

Sample rule set ​

json
{
  "id": "customer-support",
  "name": "Customer support agent",
  "version": "3",
  "latencyBudget": { "ms": 1500, "onExceeded": "fail_closed" },
  "shadow": { "review": true },
  "protection": {
    "dataClasses": {
      "card_number": { "providerId": "token_vault", "options": { "formatPreserving": true } },
      "email": { "providerId": "keyed_hash", "options": { "keyId": "guard-pseudonym-key" } }
    },
    "fallback": "block"
  },
  "dictionaries": [
    { "dataClass": "project_codename", "terms": ["bluebird", "osprey"], "patterns": [], "score": 0.9 }
  ],
  "channels": {
    "prompt": {
      "detectors": [
        { "id": "injection.direct", "threshold": 0.6 },
        { "id": "injection.hidden_unicode", "threshold": 0.5 },
        { "id": "dlp.patterns", "threshold": 0.6 }
      ],
      "actions": {
        "prompt_injection": { "action": "block" },
        "hidden_payload": { "action": "transform" },
        "credential": { "action": "redact" },
        "card_number": { "action": "protect" },
        "sensitive_data": { "action": "allow" }
      },
      "defaultAction": { "action": "block" }
    },
    "tool_result": {
      "detectors": [
        { "id": "injection.indirect", "threshold": 0.6 },
        { "id": "injection.hidden_unicode", "threshold": 0.5 },
        { "id": "exfiltration.urls", "threshold": 0.7 },
        { "id": "dlp.patterns", "threshold": 0.6 }
      ],
      "actions": {
        "indirect_injection": { "action": "block" },
        "hidden_payload": { "action": "transform" },
        "exfiltration": { "action": "review" },
        "card_number": { "action": "protect" },
        "email": { "action": "protect" },
        "project_codename": { "action": "redact" },
        "sensitive_data": { "action": "allow" }
      },
      "defaultAction": { "action": "block" }
    },
    "tool_arguments": {
      "detectors": [
        { "id": "tool.arguments", "threshold": 0.5 },
        { "id": "tool.sql", "threshold": 0.5, "options": { "dialect": "postgres", "allowedTables": ["tickets", "customers"], "maxRows": 200, "allowWrites": false } },
        { "id": "tool.egress", "threshold": 0.5, "options": { "allowedHosts": ["api.example.com", "*.cdn.example.com"] } },
        { "id": "tool.path", "threshold": 0.5, "options": { "roots": ["/srv/attachments"] } },
        { "id": "tool.command", "threshold": 0.6 }
      ],
      "actions": {
        "schema_violation": { "action": "block" },
        "sql_injection": { "action": "block" },
        "sql_policy": { "action": "require_approval" },
        "egress": { "action": "block" },
        "path_traversal": { "action": "block" },
        "command_injection": { "action": "block" }
      },
      "defaultAction": { "action": "block" }
    },
    "final_output": {
      "detectors": [
        { "id": "leak.canary", "threshold": 0.5 },
        { "id": "exfiltration.urls", "threshold": 0.6 },
        { "id": "dlp.patterns", "threshold": 0.6 }
      ],
      "actions": {
        "canary_leak": { "action": "block" },
        "exfiltration": { "action": "block" },
        "credential": { "action": "redact" },
        "card_number": { "action": "mask" },
        "sensitive_data": { "action": "allow" }
      },
      "defaultAction": { "action": "block" }
    }
  }
}

createDefaultGuardProfile() returns the built-in rule set, which blocks injection and tool-argument attacks, strips hidden characters, redacts credentials and masks card numbers in output, and leaves ordinary personal data alone.

Using the library ​

ts
import {
  createDefaultGuardProfile,
  createDefaultGuardRegistry,
  createDefaultProtectionRegistry,
  evaluateGuard
} from "@ai-orchestrator/guard-sdk";

const registry = createDefaultGuardRegistry({ resolver: lookupAddresses });
const protection = createDefaultProtectionRegistry({ keys: resolveKey, defaultKeyId: "guard" });
const profile = createDefaultGuardProfile();

const verdict = await evaluateGuard(
  profile,
  { channel: "tool_result", content: toolOutput, labels: ["untrusted_external"] },
  { registry, protection }
);

if (verdict.blocked) throw new Error(verdict.findings[0]?.message);
useContent(verdict.content);

A verdict carries the overall action, the findings at or above threshold, the suppressed ones, the action applied per finding (requested, effective, shadow, reason), the transformed content, the tokens issued by providers, per-detector timings and statuses, the budget outcome and a decision trace.

Benchmark ​

packages/guard-sdk/corpus/ holds a labelled red-team corpus of content-rule attacks and benign look-alikes in eleven categories, with a main split and a held-out split. runGuardBenchmark(ruleSet) runs it through any rule set. pnpm --filter @ai-orchestrator/guard-sdk test and the repository's CI fail when the default rule set drops below its measured results in any category. See Content rules benchmark for the categories, the OWASP mapping, the measured rates, the Security page, the API and l2m guard benchmark. The item format is in corpus/README.md.

OWASP Top 10 for LLM applications ​

What this release of the engine covers, as a library:

RiskCoverage
LLM01 Prompt injectiondirect and indirect rule packs, hidden-character detection and stripping, spotlighting helpers, canary tokens, classifier tier interface
LLM02 Sensitive information disclosuredata rule detectors with checksums, credential and key detection, masking, redaction, protection providers, canary leak detection
LLM03 Supply chainnot covered by the engine
LLM04 Data and model poisoningnot covered by the engine
LLM05 Improper output handlingexfiltration URL detection in output, hidden-character stripping, schema validation of tool arguments and of answers
LLM06 Excessive agencySQL, command, path and egress checks on tool arguments; the require_approval and review actions
LLM07 System prompt leakageprompt-extraction patterns, canary tokens
LLM08 Vector and embedding weaknessesnot covered by the engine
LLM09 Misinformationgrounding of answers in the retrieved documents (lexical, with an optional model judge); topic limits
LLM10 Unbounded consumptionargument size and depth limits; the row cap in the SQL check

Not yet available ​

The engine is wired into the Content Rules node, the agent rules port, automatic rule sets, the model boundary, responses to external callers, the Reveal node, both gateways, the protection port and rule events; see Content rules in workflows. Still to come:

  • git sync for rule sets (export and import JSON in the meantime);
  • local classifier models, and an LLM judge tier for every detector (the output checks can already call a chat model);
  • the SQL check (tool.sql) on SQL connector nodes and the address check (tool.egress) on HTTP Request nodes without a Content Rules node in front of them.

This website (not the L2M product) uses Google Analytics, only after you accept cookies. Cookie settings