Module 5 | Evaluation, Verification & Self-Reflection | AI 202

Verification and evaluation

10 min

Verification is not Reflection

After an agent takes an Action, it gets a Result. Two different questions then follow, and mixing them up causes many design mistakes.

Core question

After the agent finishes a step, how does it judge whether the result can be trusted and whether the strategy needs to change?

Two different questions
	Verification	Reflection
Question	Is the result correct or acceptable?	Why did the strategy succeed or fail, and what should change?
Looks at	The Result of one step	The approach across steps
Typical output	Accept, reject or retry	Keep the plan, adjust it, or stop
Simulated example	The drafted reply cites a policy number that must match the policy document	Three replies in a row missed the policy, so change the search source

Evaluation layers

Action: the agent does something
Result: what came back
Verify Result: is this result correct or acceptable?
Evaluate Progress: are we closer to the Goal?
Reflect on Strategy: why is it working or not, and what should change?

Info

Text version of the diagram: five layers in order, Action, Result, Verify Result, Evaluate Progress, Reflect on Strategy. Verification looks at the result, evaluation looks at progress toward the Goal, and reflection looks at the strategy.

Key takeaway: Verification asks whether this result is acceptable. Reflection asks why the strategy works or fails.

9 min

Outcome and process evaluation, and who evaluates

Evaluation can look at what was produced or at how it was produced. It can also be done by different evaluators, and the choice affects how far you can trust it.

Outcome versus process evaluation
Kind	What it checks	Simulated example
Outcome evaluation	Whether the final result meets the Goal	The reply answers the employee question and matches the policy
Process evaluation	Whether the steps taken were sound, safe and within limits	The agent searched an approved source and did not skip a required approval

Who or what evaluates
Evaluator	Strength	Limit
The agent itself	Fast and always available	Shares its own blind spots and may be confident but wrong
External evaluator	A rule, a reference source or a separate check gives independent evidence	Only as good as the rule or source
Human evaluator	Brings judgment, context and accountability	Slower and costs attention, so use it where impact is high

Warning

Confidence is not evidence. Evidence is something that can be checked, such as a source, a rule result or a reviewer decision.

Check yourself

The final answer is correct, but the agent skipped a required approval on the way. Which kind of evaluation would catch this?

Process evaluation, because it checks whether the steps were sound and within limitsOutcome evaluation only, because the answer was rightNo evaluation, because the result was good

Key takeaway: Check both what was produced and how. Prefer external or human evidence for important results.

Reflection, limits and failures

9 min

Self-Reflection: what it is and its limits

Self-Reflection means the agent looks back at its own approach: what it tried, what happened, and what it should do differently. Done well, it turns a failed attempt into a better next attempt. Done badly, it burns effort without improving anything.

What Reflection can and cannot do
Reflection can	Reflection cannot
Notice a pattern across attempts	Create facts the agent never observed
Suggest a change of source, order or approach	Prove that its own answer is correct
Decide that more effort is not paying off	Replace human judgment on high-impact choices

A Reflection loop is one or more rounds of look back, adjust, try again. Every loop needs a limit, because each round costs time and can drift.

Info

Text version: a useful Reflection starts from new evidence, such as a failed check or an unexpected Observation. Without new evidence, there is little for the agent to learn from.

Check yourself

Which is a sound reason to reflect on the strategy?

Three verified attempts failed the same check, which is new evidence that the approach may be wrongThe agent feels like reconsidering, though nothing has changedThe result passed every check, but reflection is always required after each step

Key takeaway: Reflect when there is new evidence, and always bound the number of reflection loops.

9 min

Six Reflection failure modes

Evaluation and Reflection can fail in predictable ways. Knowing them helps you design guards in advance.

Failure modes and simple guards
Failure mode	What goes wrong	Simple guard
Confident but wrong	The agent sounds sure while being incorrect	Require a source, a rule check or a human for important results
Reflection without new evidence	The agent rethinks but nothing new was observed	Trigger reflection only after a failed check or new Observation
Endless self-critique	The agent keeps finding flaws and never finishes	Set a maximum number of reflection rounds and a Stop Condition
Changing a good plan unnecessarily	A working plan is dropped because of vague doubt	Change the plan only when evidence shows a real problem
Optimizing for evaluator preference	The agent shapes output to please the checker, not to meet the Goal	Use several evaluators and keep the Goal visible
Ignoring external constraints	Policy, budget, time or permissions are forgotten	Check constraints as part of every evaluation

Warning

Human, AI, Human still applies. When evaluation is uncertain on a high-impact step, use Human Escalation instead of letting the agent argue with itself.

Check yourself

An agent keeps critiquing its own draft and never sends it. Which failure mode is this, and what is a simple guard?

Endless self-critique; set a maximum number of rounds and a Stop ConditionConfident but wrong; add more self-critique roundsIgnoring external constraints; remove all limits

Key takeaway: Name the failure modes in advance and add a guard for each: evidence, limits, stop conditions and humans.

9 min

Designing Self-Reflection without uncontrolled recursion

Reflection should be a designed step with clear boundaries, not an open loop where the agent reflects on its reflection forever. Five design choices keep it under control.

Set a trigger
Reflect only on named triggers, such as a failed verification or a surprising Observation.
Require evidence
A reflection must point to the evidence that prompted it.
Limit the rounds
Set a maximum number of reflection rounds, for example two, before moving on.
Define the outputs
A reflection ends in one of a few outcomes: keep the plan, adjust the plan, ask a human, or stop.
Add a Stop Condition and escalation
When the limit is reached without progress, stop or use Human Escalation.

A bounded reflection step

Trigger: a check failed or something unexpected was observed
Reflect: what went wrong and what is the evidence?
Decide: keep the plan, adjust it, ask a human, or stop
Count: one round used, and stop if the limit is reached

Info

Text version of the diagram: a trigger starts one round of reflection. The round ends in a decision and increases a counter. If the counter reaches the limit, the agent stops or escalates to a person.

Check yourself

Which design keeps Self-Reflection from becoming uncontrolled recursion?

A named trigger, required evidence, a maximum number of rounds and a defined stop or escalationLet the agent reflect until it feels satisfiedAsk the agent to reflect on its reflection each time

Key takeaway: A designed reflection step has a trigger, evidence, a round limit, defined outputs and a stop or escalation.

Practice and deliverable

9 min

Activity 5: Verify or Reflect?

For each simulated scenario, choose the best response. Then use your choices to write an Evaluation & Reflection Policy.

The five choices
Choice	Use when
Verify	You need to know whether one result is correct or acceptable
Evaluate	You need to know whether progress toward the Goal is being made
Reflect	New evidence suggests the strategy itself may need to change
Ask a human	The impact is high, the evidence is unclear or judgment is needed
Stop	The Goal is met, or the limit is reached, or continuing is unsafe

Read the scenario
Note the latest Result and any new evidence.
Choose one response
Pick Verify, Evaluate, Reflect, Ask a human or Stop, and give a short reason.
Write the policy
Summarize your triggers, evaluators, reflection limits, human escalation rules and stop conditions as an Evaluation & Reflection Policy.

Warning

Privacy reminder: use simulated or already-public scenarios. Do not enter company secrets, client personal data or unauthorized sensitive information.

Go to the AI 202 practice: In the "Verify or Reflect?" activity, choose a response for each scenario.
Continue to Module 6: Next: when the world changes, how does the plan change?

Check yourself

A simulated agent has just drafted a reply that quotes a policy number. What is the best first response?

Verify, by comparing the number with the policy documentReflect on the whole strategy before checking anythingStop, because drafting is finished

Key takeaway: Choose Verify, Evaluate, Reflect, Ask a human or Stop, and record your rules in an Evaluation & Reflection Policy.

Evaluation, Verification & Self-Reflection

Learning objectives

Verification and evaluation

Verification is not Reflection

Outcome and process evaluation, and who evaluates

Reflection, limits and failures

Self-Reflection: what it is and its limits

Six Reflection failure modes

Designing Self-Reflection without uncontrolled recursion

Practice and deliverable

Activity 5: Verify or Reflect?

What next?