Module 5 | Execution, Verification & Human-in-the-Loop | AI 104

Execution and verification

6 min

After the next step, how do we know we can continue?

Once a step is decided, it is executed. But a workflow that only decides and executes has no way to notice a bad result. Verification is the step that asks: is this output acceptable, and can the workflow continue?

Core question

After the agent completes the next step, how do we know the result is good enough to continue?

Execution model

Decision: the next step has been chosen
Execute: the step produces an Output
Verify: check the Output
Accept?: if yes, Continue to the next step
If no: Retry, Route Elsewhere, or send to Human Review

Info

Text version of the diagram: Decision leads to Execute, then Verify, then a question, Accept? A yes continues the workflow. A no leads to one of three paths: Retry the step, Route Elsewhere (a different branch), or Human Review.

Warning

A Stop condition matters too. Decide in advance when the workflow should stop instead of retrying forever, for example after a set number of attempts.

Key takeaway: Every executed step needs a check, and every failed check needs a defined next path.

6 min

Verification: how do we check a result?

There is more than one way to verify an output. Pick the method that matches the risk of the step.

Common verification methods
Method	How it works	Suited to
Rule	Check format, required fields, ranges or length	Outputs with a clear structure
Reference source	Compare with a trusted document or record	Facts, figures and policy statements
Comparison	Compare with a previous result or a second answer	Spotting unexpected changes
Second model pass	A separate model call reviews the first output	A quick screen, never a final guarantee
Human review	A person reads and judges the output	High-impact, ambiguous or sensitive results

Confidence is how sure the system seems about its own output. It is a useful signal for routing, for example low confidence goes to a person, but a confident answer can still be wrong. Do not treat confidence as proof.

Warning

A second model pass can share the same blind spots as the first. For important results, add a rule, a reference source or a person.

Check yourself

An agent drafts a summary of a policy and says it is "very confident". What is the best next design?

Accept it, since high confidence means it is correctVerify against the source policy, and add human review if the content is high-impactAsk the same agent to say it is correct again

Key takeaway: Choose a verification method to match the risk. Confidence is a signal, not proof.

Human-in-the-Loop

6 min

Human-in-the-Loop: review, approval, escalation, override

Human-in-the-Loop (HITL) means a person has a defined role inside the workflow. It is not a vague promise to "have someone look at it". Four patterns cover most cases.

Four Human-in-the-Loop patterns
Pattern	What the person does	Example (simulated)
Human Review	Reads the output and corrects or accepts it	A staff member checks a drafted reply before it is sent
Human Approval	Gives an explicit yes or no before an action happens	A manager approves a payment before it is made
Human Escalation	Takes over when the workflow cannot proceed or is unsure	A hard case is passed to a specialist
Human Override	Stops or replaces what the workflow was doing	A person cancels an automatic action or changes its outcome

Info

Human, AI, Human: a person defines the task, AI assists or executes, and a person reviews and remains responsible. Where you place the human step depends on the impact of the action, not on how well the AI usually performs.

Check yourself

A workflow is about to send a payment. Which HITL pattern fits best before the action happens?

Human Approval: a person gives an explicit yes before the paymentHuman Review after the payment was already madeNo human step, because the amount is usually correct

Key takeaway: Place a person by impact: review, approval, escalation or override, each with a clear role.

Tool steps, failures and practice

6 min

Tool Steps and six simple failure modes

Some steps need something outside the agent: a search, a database, an email service, a calendar, an API or an enterprise system. In AI 104 you only need to notice these steps and mark them.

Tool Step placeholder

Write "Tool Step" on the Canvas wherever an external tool or data source is needed, and note what it should do. How an agent connects to a tool safely, and how tools are described, is the subject of AI 201 (MCP). It is not taught here.

Six simple failure modes
Failure mode	What happens	Basic fallback
Missing input	A required piece of information is not there	Ask the requester, or stop with a clear message
Low confidence	The system is unsure of its own output	Route to Human Review
Invalid output	The output fails a format or content check	Retry a limited number of times, then Human Review
Unavailable tool	An external service does not respond	Stop or wait, and tell a person. Do not guess the result
Ambiguous request	The request can be read in more than one way	Ask a clarifying question
Unsafe action	The next action could cause harm or exceed permission	Stop and escalate to a person for approval

Info

Boundary of this course: these are simple failure paths. Designing full failure-recovery architecture, such as saved progress and automatic resumption, belongs to a future advanced course.

Check yourself

A step needs to read a record from an enterprise system, but the system does not respond. What is the safest basic fallback?

Let the agent guess the missing record and continueStop or wait, and notify a person, without inventing the resultSkip the check and mark the step as done

Key takeaway: Mark Tool Steps, and give each simple failure mode a basic fallback.

6 min

Activity 5: Where should a human step in?

Take a workflow (a simulated one, or your own task from earlier modules) and mark where each of five elements belongs.

The five elements
Element	Question to ask
AI execution	Which steps can the AI carry out?
Automatic verification	Which results can a rule or reference source check?
Human review	Which results need a person to read them?
Human approval	Which actions need an explicit yes first?
Stop / fallback	When should the workflow stop, retry or hand over?

Mark AI execution
Show the steps where the AI produces an output.
Add verification
For each output, choose how it is checked.
Place the human
Add review, approval or escalation where the impact is high.
Write the fallback
State what happens when a check fails, and when the workflow stops.

Warning

Privacy reminder: use simulated or already-public workflows. Do not enter company secrets, client personal data or unauthorized sensitive information.

Go to the AI 104 practice: In the "Execution & HITL" activity, place the human steps.

Check yourself

In a workflow that emails a customer, where should human approval go?

After the email is sent, as a reminderBefore the email is sent, when it commits the organization or contains sensitive contentNowhere, because verification by a second model is enough

Key takeaway: Put human approval before irreversible actions, and define when to retry, hand over or stop.

Execution, Verification & Human-in-the-Loop

Learning objectives

Execution and verification

After the next step, how do we know we can continue?

Verification: how do we check a result?

Human-in-the-Loop

Human-in-the-Loop: review, approval, escalation, override

Tool steps, failures and practice

Tool Steps and six simple failure modes

Activity 5: Where should a human step in?

What next?