Skip to content
NTPU AI Learn

GoAI 101 · Module 8

AI System Acceptance, Auditability, and Operations

How does a government agency judge whether an AI system is really ready to go live?

Start Section 1
  • Duration60 min
  • LevelCore
  • PrerequisiteModules 1-7 recommended first
0 / 5 completed
Sync my progressJoin a session to keep progress across devices and share it with your instructor

Info

After you join, your completed lesson numbers are sent to your instructor together with your learner code, so they can see your progress. Use the codes your instructor gave you, and do not enter your name, company secrets or other personal data. Progress is kept with the rest of that session's data.

Learning objectives

  • Explain the progression from "can be demoed" to "can be operated."
  • List the 14 acceptance dimensions and check each one.
  • Connect use case, workflow, data, governance, procurement requirement, and test case into a full acceptance narrative.
  • Make an Accept / Accept with conditions / Reject decision based on evidence, and complete an integrated deliverable.

Accept: from demo to real system

10 min

From demo to a real system

Acceptance progression
  1. Can be demoed
  2. Functions can run
  3. Works on real data
  4. Handles abnormal cases
  5. Permissions are controlled
  6. Behavior can be traced
  7. Results can be verified
  8. The system can be operated

Warning

A system being demo-able for leadership does not mean it is ready to go live, six more stages still need to be confirmed one by one, and none can be skipped.
Check yourself

A system has passed the "functions can run" and "works on real data" stages, but "handles abnormal cases" has not yet been confirmed. What does this mean?

A system has passed the "functions can run" and "works on real data" stages, but "handles abnormal cases" has not yet been confirmed. What does this mean?

Key takeaway: The eight stages of the acceptance progression build cumulatively; good performance in early stages is never a reason to skip the later confirmations.

The 14 acceptance dimensions

12 min

The 14 acceptance dimensions

14 acceptance dimensions
Dimension
Function
Workflow
Accuracy / Quality
Knowledge / RAG
Permission
Security / Privacy
Tool Use
Human Approval
Logging / Traceability
Error Handling
Reliability
Version Control
Regression Test
Operations & Maintenance
  • The first four dimensions (Function, Workflow, Accuracy/Quality, Knowledge/RAG) check whether the system "does the right thing," mapping to the design from Modules 1-3.
  • The middle four (Permission, Security/Privacy, Tool Use, Human Approval) check whether the system is "properly controlled," mapping to the governance design from Modules 4-5.
  • The last six (Logging/Traceability, Error Handling, Reliability, Version Control, Regression Test, Operations & Maintenance) check "what happens when something goes wrong, and can it be operated long-term," mapping to Module 6's testing and evaluation.
Check yourself

An acceptance team makes its decision after checking only the "Function" and "Accuracy / Quality" dimensions. What is the main problem with this?

An acceptance team makes its decision after checking only the "Function" and "Accuracy / Quality" dimensions. What is the main problem with this?

Key takeaway: The 14 dimensions are not an arbitrary checklist, they organize the design and test outputs of Modules 1-6 into a complete list that can be checked item by item at acceptance.

Capstone scenario: connecting the first seven modules

12 min

Capstone scenario: connecting the first seven modules

Acceptance is not new work that starts at Module 8, it is connecting the output of the first seven modules and checking whether they are consistent and where the gaps are.

Connecting the local-government AI Agent project
  1. Business requirement
  2. Use case (Module 1)
  3. Workflow (Module 2)
  4. Data (Module 3)
  5. Governance (Module 5)
  6. Procurement requirement (Module 7)
  7. Test case (Module 6)
  8. Acceptance criteria (Module 8)

Info

If a dimension lacks evidence at acceptance time, tracing back often reveals that Module 3's context design was incomplete, or Module 5's governance design missed a gate, or Module 7's requirement never defined a verification method. Acceptance checks the whole chain, not Module 8 in isolation.
Check yourself

At acceptance, the "Security / Privacy" dimension lacks evidence, and tracing back shows Module 3's Context & Data Canvas never specified how sensitive data should be handled. What does this show?

At acceptance, the "Security / Privacy" dimension lacks evidence, and tracing back shows Module 3's Context & Data Canvas never specified how sensitive data should be handled. What does this show?

Key takeaway: The quality of an acceptance decision depends on whether the output of the first seven modules is complete and consistent; chain them together and the gaps surface on their own.

The acceptance decision and evidence

12 min

The acceptance decision: Accept, Accept with conditions, Reject

Scenario: accepting a local-government AI Agent project

A local government outsourced development of a complaint-classification AI Agent, and the vendor requests acceptance after one successful demo. The acceptance team checks all 14 dimensions and finds two gaps: abnormal-case handling (no proper fallback when a complaint can't be classified) and regression testing (no retained test cases to re-verify future updates). The decision is "Accept with conditions": most dimensions, function, permissions, human approval, are met, but the vendor must close the abnormal-handling and regression-test gaps before go-live, with evidence, before the conditions are lifted.

  • Accept: every key dimension has supporting evidence, ready to go live.
  • Accept with conditions: most dimensions are met, with a few items requiring time-bound fixes before full acceptance.
  • Reject / require remediation: a key dimension (e.g. permission control, human approval) is missing, it cannot go live and must be sent back for improvement.

Info

An acceptance decision must cite concrete evidence (test results, logs, permission-configuration records, etc.), not just an impression or the vendor's verbal assurance.
Check yourself

A vendor's demo runs smoothly, and the leader on the spot says "approve it now, let's go live right away." Which judgment is most correct?

A vendor's demo runs smoothly, and the leader on the spot says "approve it now, let's go live right away." Which judgment is most correct?

Key takeaway: The bar for acceptance is "is there evidence," not "did the demo look good", Accept with conditions is often closer to reality than a flat Accept or Reject.

Practice and application

14 min

Practice: Would You Accept This System?

Activity 8 is this course's integrated capstone exercise: check the 14 acceptance dimensions against a simulated evidence pack or vendor demo, make an acceptance decision, and complete the Government AI Procurement & Acceptance Checklist.

  1. Gather the outputs from the first seven modules
    Use case, workflow, context, tool permissions, governance, tests, procurement requirements.
  2. Check all 14 acceptance dimensions
    Find the matching evidence for each dimension.
  3. Make the acceptance decision
    Accept / Accept with conditions / Reject, with the reasoning stated.
  4. Complete the integrated deliverable
    Submit the Government AI Governance & Acceptance Canvas.
  • Government AI Procurement & Acceptance Checklist minimum dimensions: Business Requirement, Functional Requirement, Workflow, Data, Knowledge / RAG, Model, Tool, Permissions, Security, Privacy, Human Approval, Logging, Traceability, Reliability, Failure Recovery, Testing, Version Control, Regression, Operations.
Check yourself

After finishing Activity 8, what complete set of deliverables should you have accumulated?

After finishing Activity 8, what complete set of deliverables should you have accumulated?

Key takeaway: This course's eight modules converge into one complete chain: from "is this work suited to AI" to "is this system really ready to go live," with every judgment along the way recorded, checkable, traceable, and accountable.

Save as PDF

What next?