PAI
Production AI Institute
Get Certified
HomeAboutExam Development

Exam development process

PAI examinations are developed by practitioners, reviewed by independent subject matter experts, and validated against real-world production AI deployment scenarios. This page describes how the AIDA and AIMA examinations are built and maintained.

Development principles

Practitioner-authored
All examination items are authored by practitioners with verifiable production AI deployment experience — not by academics or vendor trainers. Items are grounded in real deployment decisions.
Framework-anchored
Every item maps to a specific PSF domain and competency level. The exam blueprint (published at /certify/blueprint) shows the weighting of each domain. Passing the exam demonstrates coverage across all eight domains, not just the ones a candidate finds comfortable.
Scenario-based
Items present realistic production scenarios — an AI system behaving unexpectedly, a data governance decision, an architecture choice — rather than testing vocabulary recall. The correct answer requires applying PSF principles, not memorising definitions.
Regularly reviewed
The item bank is reviewed annually. Items with poor discrimination (where high performers and low performers choose answers at similar rates) are flagged for revision or removal. Items are also reviewed when the PSF is updated to ensure continued alignment.

Development stages

01
Domain mapping
The Assessment Board defines the competencies to be tested within each PSF domain, drawing on the framework specification and practitioner-reported incident data.
02
Item authoring
Practitioners with relevant domain experience author candidate items. Each item is submitted with a rationale explaining the correct answer, the distractor logic, and the PSF reference.
03
Expert review
Items are reviewed by two independent subject matter experts who were not involved in authoring. Reviewers assess accuracy, scenario authenticity, and freedom from vendor-specific assumptions.
04
Sensitivity review
Items are reviewed for unintended cultural assumptions, ambiguous language, and any content that might disadvantage candidates from particular backgrounds or regions.
05
Pilot administration
New items are piloted within live examinations as unscored items. Pilot performance data is collected before items enter the scored item pool.
06
Statistical validation
Pilot items with acceptable discrimination and difficulty indices enter the scored pool. Items that perform poorly are revised or retired.
07
Ongoing maintenance
The scored item pool is reviewed annually. Items are retired if the PSF is updated such that the item no longer accurately reflects the standard, or if statistical performance degrades.

Examination specifications

AIDA
AI Deployment Associate
Items20 scored items
TimeNo time limit (typical: 15–20 minutes)
FormatMultiple choice, single correct answer
Pass mark75% (15/20)
DomainsAll eight PSF domains
AIMA
AI Management Associate
Items20 scored items
TimeNo time limit (typical: 15–20 minutes)
FormatMultiple choice, single correct answer
Pass mark75% (15/20)
DomainsDomains 05–08 (management focus)
View all certifications →