A formal specification of what your AI system is permitted to do, must do, and must never do — and how failures are handled. The operational counterpart to a model card.
Most production AI failures are not failures of the model — they are failures of the deployment context. The model behaves exactly as trained, but the deployment context has assumptions that were never made explicit. Behaviour contracts make those assumptions explicit before deployment, not after an incident.
A behaviour contract serves four functions simultaneously:
The engineering team knows exactly what the system should and should not do. Ambiguity is eliminated at design time.
Legal, compliance, and risk teams can review and sign off on the system's behaviour without reading source code.
When something goes wrong, the behaviour contract is the reference point for determining whether the system behaved as specified or not.
Any change that would cause the system to behave outside the contract requires a formal contract amendment — creating a forcing function for governance.
A complete behaviour contract has six required sections. Each section addresses a different dimension of operational behaviour. Missing sections create gaps that will eventually cause incidents.
Implementation note: The scope section is the most important. Every behaviour contract should include a list of what the system must NOT be used for — not just what it is designed for. Misuse begins where scope clarity ends.
Implementation note: Most prompt injection attacks succeed because there is no input contract. If the system has no defined behaviour for unexpected inputs, attackers will supply unexpected inputs.
Implementation note: The output contract must specify what happens at the boundaries — when the system has low confidence, when it encounters an edge case, when it detects a request it cannot satisfy. Unspecified edge case behaviour is where incidents happen.
Implementation note: The autonomy section must specify the never-autonomous list — the set of actions that require human approval regardless of model confidence. This list should be agreed between technical, legal, and business stakeholders before deployment.
Implementation note: Failure protocols must be written when the system is working, not during an incident. Every production AI deployment must have a tested runbook that the on-call team can execute under pressure.
Implementation note: The monitoring section closes the loop between deployment and governance. Specify the exact metrics that will be monitored, the thresholds that trigger review, and the process by which the contract is updated when thresholds are breached.
The following is a minimal YAML-format behaviour contract template. This format is machine-readable, version-controllable, and can be validated automatically against a schema.
system:
name: "Invoice Classification Agent"
version: "2.1.0"
owner: "AP Engineering Team"
accountable_party: "VP Finance"
deployment_context: "Internal accounts payable workflow"
scope:
purpose: "Classify incoming invoices by vendor category and urgency"
out_of_scope:
- "Approving payment of any invoice"
- "Modifying invoice data"
- "Communicating with vendors"
input_contract:
schema: "InvoiceDocument v3"
valid_range: "Invoices from approved vendor list only"
invalid_input_behaviour: "Reject with reason code, route to AP team"
sensitive_fields: ["vendor_banking_details", "tax_id"]
output_contract:
schema: "ClassificationResult v2"
required_fields: ["category", "urgency", "confidence_score", "reason"]
prohibited: ["vendor_banking_details", "PII not in original invoice"]
low_confidence_threshold: 0.75
low_confidence_behaviour: "Flag for human review, do not auto-route"
autonomy:
level: "L1 — Human Approves"
never_autonomous:
- "Routing invoices above $50,000"
- "Routing invoices from new vendors"
- "Any invoice flagged as duplicate"
failure_protocol:
circuit_breaker: "Suspend routing if error rate > 5% over 10 minutes"
fallback: "Route all invoices to AP team manual queue"
escalation_chain:
- "AP Team Lead (immediate)"
- "Engineering On-Call (if technical failure)"
incident_severity:
P1: "Data exposed or incorrect payment initiated"
P2: "Sustained accuracy degradation > 15%"
monitoring:
accuracy_floor: 0.88
drift_check: "Weekly PSI on vendor category distribution"
audit_log_retention: "7 years (SOX requirement)"
review_cadence: "Quarterly"
retraining_trigger: "PSI > 0.2 on any input feature"A behaviour contract is only as useful as the governance process surrounding it. The contract itself is a document — the governance process is what gives it operational force.
Technical team drafts the contract during system design, before implementation begins. The contract is part of the design specification, not a post-deployment artefact.
Legal, compliance, security, and business stakeholders review and sign off. Any section where reviewers cannot reach consensus is a design problem that must be resolved before deployment.
The contract is formally approved, version-controlled, and stored in a location accessible to all stakeholders including incident responders.
Production behaviour is continuously compared against the contract. Deviations — whether the system behaves outside the contract, or the contract no longer matches business requirements — trigger amendment.
Any material change to system behaviour (model update, scope expansion, autonomy level change) requires a formal contract amendment with the same review and ratification process as the original.
The behaviour contract is live documentation. A contract that is written once and never updated is a liability, not a protection. Build amendment reviews into your operational calendar — quarterly for stable systems, after every significant model update for actively developed ones.
This record is maintained by PAI and free to cite. If something is wrong or missing, tell us. Corrections and source suggestions keep the record honest.