AI Reliability, Testing, and Failure Handling

10 min

A good demo is not the same as reliable

This module uses one simulated scenario: a simulated city office is considering a "petition classification and reply-drafting assistant". The AI suggests a category and drafts a reply; a caseworker reviews it before anything is sent. In the demo, three cases went well, and a manager asks: "Can we go live?" The answer depends on evidence, not on the impression the demo left.

• Hallucination: AI produces content that sounds plausible but is incorrect or ungrounded, such as citing a regulation number that does not exist.
• Grounding: giving the output checkable data or sources, with a clear origin and version.
• Verification: a person or a systematic method confirms the output is correct, rather than trusting it because it reads smoothly.

Key takeaway: The test model turns "is this system reliable?" from a subjective impression into a series of steps that can be checked and that leave evidence.

10 min

Normal, edge, failure: three types of test case

• Write both expected behavior and prohibited behavior for every case: what it should do and what it must never do are equally important.
• Write the required evidence in advance, such as output records, cited sources, and refusal or hand-back logs, so reasons are not invented afterwards.
• The failure/restricted case maps directly to Module 4 risks such as prompt injection and data leakage: governance design and test design must echo each other.
• A set with only easy-to-pass cases overstates reliability; the test set must deliberately include edge and restricted situations.

Key takeaway: The three case types check whether the system has a clear, safe expected behavior when things are normal, uncertain, and manipulated or restricted.

10 min

Reading simulated outputs: judging Pass / Fail with evidence

This course does not ask you to build or operate a system. You learn to read provided simulated test outputs and make an evidence-based judgment. Below are two simulated records; compare each with its expected behavior.

• Cite only what the record actually contains: a field with no evidence is a gap, not a pass.
• For a Fail, write the failure type (for example misunderstanding, data error, permission error) and a suggested remediation so it can be tracked.

Key takeaway: Pass / Fail can rest only on evidence from tests that were really run; a test not yet run is only a test design.

10 min

Failure handling, logging, and regression testing

Regression Test: after a model change, prompt adjustment, or knowledge-base update, re-run the existing test cases to confirm that what used to work has not been broken. For example, if a ruling in the knowledge base is updated, retest every case that cites it.

Key takeaway: Reliability is not a one-time pre-launch test: it needs a handling path for failures, records that can be reconstructed, and regression tests after every update.

10 min

Practice: three types of test case

Activity 5 asks you to use a template to complete three test cases for the course's simulated case. Your output goes into the "Reliability tests" field of the integrated Canvas and, in Module 6, serves as one source of evidence for the acceptance judgment. Use simulated or de-identified data only; do not enter real personal data, secrets, or unauthorized sensitive information.

1. 1Reuse the case
   Use the case background already in the integrated Canvas; there is no need to choose a new topic.
2. 2Design three cases
   One normal, one edge/ambiguous, one failure/restricted, each with an Input.
3. 3Write expected and prohibited behavior
   Every case needs expected behavior, prohibited behavior, and required evidence.
4. 4Judge or label
   For the provided simulated output, judge Pass / Fail and cite the evidence; label any case you designed but did not run as "test design".

• Go to the practice page: Activity 5, three types of test case

Key takeaway: These three test cases go into the Canvas and, in Module 6, are used as the basis for "is there evidence?", so keep test design and actual evidence clearly apart.