Insights / Reference Article
Published: 2026-04-29 · License: CC BY 4.0
Cite as: Production AI Institute. (2026). How to Write an AI Behaviour Contract.
How to Write an AI Behaviour Contract
An AI behaviour contract is a formal specification of what an AI system is permitted to do, what it must do, what it must never do, and how failures are handled. It is the operational counterpart to a model card — where a model card describes what a system is, a behaviour contract describes how a system must behave in a specific deployment context.
Why Behaviour Contracts Exist
Most production AI failures are not failures of the model — they are failures of the deployment context. The model behaves exactly as trained, but the deployment context has assumptions that were never made explicit. Behaviour contracts make those assumptions explicit before deployment, not after an incident.
A behaviour contract serves four functions simultaneously:
The engineering team knows exactly what the system should and should not do. Ambiguity is eliminated at design time.
Legal, compliance, and risk teams can review and sign off on the system's behaviour without reading source code.
When something goes wrong, the behaviour contract is the reference point for determining whether the system behaved as specified or not.
Any change that would cause the system to behave outside the contract requires a formal contract amendment — creating a forcing function for governance.
The Six Sections of a Behaviour Contract
A complete behaviour contract has six required sections. Each section addresses a different dimension of operational behaviour. Missing sections create gaps that will eventually cause incidents.
System Identity and Scope
- System name and version
- Deployment context (environment, user population, geography)
- Stated purpose in plain language
- Out-of-scope uses explicitly enumerated
- Owner (technical) and accountable party (business)
Implementation note: The scope section is the most important. Every behaviour contract should include a list of what the system must NOT be used for — not just what it is designed for. Misuse begins where scope clarity ends.
Input Contract
- Expected input schema (structure, format, data types)
- Valid input range — what constitutes a valid request
- Behaviour on invalid or malformed input (reject, sanitise, escalate)
- Sensitive input handling (PII, regulated data categories)
- Input validation and preprocessing specification
Implementation note: Most prompt injection attacks succeed because there is no input contract. If the system has no defined behaviour for unexpected inputs, attackers will supply unexpected inputs.
Output Contract
- Output schema (format, structure, permitted content types)
- Confidence or uncertainty expression requirements
- Required citations or source attribution
- Prohibited output types (personal data, regulated content, etc.)
- Behaviour when the system cannot produce a valid output
Implementation note: The output contract must specify what happens at the boundaries — when the system has low confidence, when it encounters an edge case, when it detects a request it cannot satisfy. Unspecified edge case behaviour is where incidents happen.
Autonomy and Oversight
- Autonomy level (L0–L4 per PSF framework)
- Human approval requirements and thresholds
- Escalation paths and conditions
- Override procedures — how humans can intervene
- Actions the system may never take autonomously regardless of confidence
Implementation note: The autonomy section must specify the never-autonomous list — the set of actions that require human approval regardless of model confidence. This list should be agreed between technical, legal, and business stakeholders before deployment.
Failure and Incident Protocol
- Failure detection mechanisms (what signals indicate failure)
- Immediate response procedures (circuit breaker, fallback behaviour)
- Escalation chain with named individuals and contact information
- Incident classification and severity thresholds
- Communication requirements (who must be notified, within what timeframe)
Implementation note: Failure protocols must be written when the system is working, not during an incident. Every production AI deployment must have a tested runbook that the on-call team can execute under pressure.
Monitoring and Compliance
- Performance metrics and acceptable ranges
- Drift detection and retraining triggers
- Audit logging requirements (what must be logged, retention period)
- Applicable regulatory frameworks and compliance requirements
- Review cadence and contract amendment process
Implementation note: The monitoring section closes the loop between deployment and governance. Specify the exact metrics that will be monitored, the thresholds that trigger review, and the process by which the contract is updated when thresholds are breached.
A Minimal Behaviour Contract Template
The following is a minimal YAML-format behaviour contract template. This format is machine-readable, version-controllable, and can be validated automatically against a schema.
system:
name: "Invoice Classification Agent"
version: "2.1.0"
owner: "AP Engineering Team"
accountable_party: "VP Finance"
deployment_context: "Internal accounts payable workflow"
scope:
purpose: "Classify incoming invoices by vendor category and urgency"
out_of_scope:
- "Approving payment of any invoice"
- "Modifying invoice data"
- "Communicating with vendors"
input_contract:
schema: "InvoiceDocument v3"
valid_range: "Invoices from approved vendor list only"
invalid_input_behaviour: "Reject with reason code, route to AP team"
sensitive_fields: ["vendor_banking_details", "tax_id"]
output_contract:
schema: "ClassificationResult v2"
required_fields: ["category", "urgency", "confidence_score", "reason"]
prohibited: ["vendor_banking_details", "PII not in original invoice"]
low_confidence_threshold: 0.75
low_confidence_behaviour: "Flag for human review, do not auto-route"
autonomy:
level: "L1 — Human Approves"
never_autonomous:
- "Routing invoices above $50,000"
- "Routing invoices from new vendors"
- "Any invoice flagged as duplicate"
failure_protocol:
circuit_breaker: "Suspend routing if error rate > 5% over 10 minutes"
fallback: "Route all invoices to AP team manual queue"
escalation_chain:
- "AP Team Lead (immediate)"
- "Engineering On-Call (if technical failure)"
incident_severity:
P1: "Data exposed or incorrect payment initiated"
P2: "Sustained accuracy degradation > 15%"
monitoring:
accuracy_floor: 0.88
drift_check: "Weekly PSI on vendor category distribution"
audit_log_retention: "7 years (SOX requirement)"
review_cadence: "Quarterly"
retraining_trigger: "PSI > 0.2 on any input feature"Governance Process for Behaviour Contracts
A behaviour contract is only as useful as the governance process surrounding it. The contract itself is a document — the governance process is what gives it operational force.
Technical team drafts the contract during system design, before implementation begins. The contract is part of the design specification, not a post-deployment artefact.
Legal, compliance, security, and business stakeholders review and sign off. Any section where reviewers cannot reach consensus is a design problem that must be resolved before deployment.
The contract is formally approved, version-controlled, and stored in a location accessible to all stakeholders including incident responders.
Production behaviour is continuously compared against the contract. Deviations — whether the system behaves outside the contract, or the contract no longer matches business requirements — trigger amendment.
Any material change to system behaviour (model update, scope expansion, autonomy level change) requires a formal contract amendment with the same review and ratification process as the original.
The behaviour contract is live documentation. A contract that is written once and never updated is a liability, not a protection. Build amendment reviews into your operational calendar — quarterly for stable systems, after every significant model update for actively developed ones.