Skip to content
NTPU AI Learn

IL 202 · Module 3

Incidents and Operations

Drill bad output, tool outages, and out-of-scope requests, and write a runbook that actually works.

Start Section 1
  • Duration180 min
  • LevelAdvanced
  • PrerequisiteModule 2 access matrix and integration test log completed
0 / 3 completed

Learning objectives

  • Drill how staff notice and take over when the AI tool produces a wrong answer.
  • Drill the fallback process for a tool outage or timeout.
  • Write a runbook that can actually be followed, covering escalation, handling, and a post-incident review.

Drilling bad output and human handoff

60 min

Drill: the AI gives a clearly wrong answer. Who catches it?

"A human will review it" is not a verifiable control unless you have actually drilled it: when the AI gives a plausible-looking but wrong answer, does the current process have a real chance of catching it? How fast? Who handles it once it is caught?

Drill scenarios (reusing the failure_log.csv error types from SYN-CS-001)
ScenarioDrill question
The AI cites an expired document (outdated_source)Can the agent see the document's valid-through date during review?
The AI guesses an answer for a missing-information case (missing_context)Does the process require the AI to state "cannot answer" when information is insufficient, instead of filling the gap itself?
The AI cites two contradicting documents (conflicting_sources)Does the agent see a "conflict" flag, or only a single confident draft?
The AI handles a case outside its permission scope (restricted_access)Does the system check the case's access level before drafting a reply?

Warning

Common mistake: only drilling scenarios where the AI behaves normally. An incident drill is valuable precisely because it deliberately triggers known error types to see whether the existing safeguards actually work, not because it re-confirms the happy path.
  1. Step 1
    Pick at least three known error types from failure_log.csv or an existing error log.
  2. Step 2
    Actually simulate each scenario, and record whether the current process caught it, when, and who caught it.
  3. Step 3
    For scenarios that were not caught, write down exactly what new control needs to be added.

Key takeaway: "A human will review it" is only a real control once you have drilled a specific error and confirmed the review actually catches it.

Tool outages and out-of-scope requests

60 min

Tool outages and out-of-scope requests: fallback and refusal

Beyond bad output, drill two more scenarios: the AI tool being completely unavailable, and someone asking the AI to do something outside its authorized scope. Confirm both the system and staff know how to react.

  1. Scenario 1: tool timeout or outage
    Simulate the AI tool not responding past the configured timeout, and confirm the system prompts the agent to fall back to a fully manual process instead of waiting indefinitely or freezing.
  2. Scenario 2: out-of-scope request
    Simulate someone asking the AI to handle a case category outside the currently approved usage scope (e.g. a not-yet-approved, permission-restricted category), and confirm the system or staff refuse and log it instead of complying.
  3. Scenario 3: unavailable data source
    Simulate the knowledge-base document source being temporarily unreachable, and confirm the AI tool explicitly reports "data unavailable" instead of fabricating an answer.

Info

The fallback process is not about whether a fallback exists on paper; it is about whether the agent, in the moment, knows to switch to it. If the switch-over is not written down clearly, drills usually reveal everyone improvising differently.

Warning

An out-of-scope request is not necessarily malicious: often a user simply does not know the current approved scope and casually asks the AI to handle something outside it. Both the system and staff need a clear "this is outside the current approved scope" response.

Key takeaway: Outage and out-of-scope drills confirm everyone knows what to do in the moment, rather than discovering the gap after the fact.

Writing a runbook that works

60 min

Write a runbook someone else could actually follow

A runbook's real test is: could someone who was never on this project handle the incident using only this document? If the answer is "no, they would still have to ask the original team," the runbook is not specific enough yet.

Required runbook fields
FieldWhat it covers
Incident typeMatches a scenario drilled in this module (bad output / tool outage / out-of-scope request / unavailable data source)
Detection methodHow the incident is noticed (an alert, a user report, a periodic spot check)
Immediate stepsThe concrete actions front-line staff can take right away, including how to fall back to a fully manual process
Escalation and deadlineWho to notify, how fast, and through which channel
Post-incident reviewHow the incident is logged afterward, and whether the failure log or this module's drill results need updating

Info

Carry this module's drill results (the incident drill) together with this runbook forward as evidence for the operations-readiness domain in Module 4's Readiness Review.
  1. Step 1
    Write one runbook section for each incident scenario drilled in the first two slides.
  2. Step 2
    Ask someone who was not part of the drill to describe the handling steps using only the runbook, and note anything unclear.
  3. Step 3
    Revise the runbook until that person can state the correct steps without asking follow-up questions.
Check yourself

A team wrote a runbook full of practices the team already knows well, but never had anyone outside the project test it. What is the most likely problem?

A team wrote a runbook full of practices the team already knows well, but never had anyone outside the project test it. What is the most likely problem?

Key takeaway: A runbook's quality is judged by whether someone uninvolved can follow it, not by whether the team itself understands it.

Save as PDF

What next?