Skip to content
NTPU AI Learn

AI 302 · Module 8

Agent Harness & Reliability Canvas

From an agent demo to a reliable agent system

Start Section 1
  • Duration45 min
  • LevelArchitect
  • PrerequisiteAI 202 recommended, plus Modules 1 to 7 of this course
0 / 5 completed
Sync my progressJoin a session to keep progress across devices and share it with your instructor

Info

After you join, your completed lesson numbers are sent to your instructor together with your learner code, so they can see your progress. Use the codes your instructor gave you, and do not enter your name, company secrets or other personal data. Progress is kept with the rest of that session's data.

Learning objectives

  • Name the 16 sections of the Agent Harness & Reliability Canvas and what each one records
  • Follow the recommended Canvas flow from Trigger to Final Output
  • Connect the Canvas sections to the deliverables from Modules 2 to 7
  • Answer the final reflection question about explaining, resuming, avoiding duplicate actions and keeping a human in control
  • List the seven deliverables of AI 302 and see how the course maps to M5276 Week 14 and to AI 104, 202 and 301

Goal, trigger, state and data

9 min

The Canvas, part 1: Goal, Runtime Trigger, State Model, State Data

The Agent Harness & Reliability Canvas is the main artifact of AI 302. It puts the state, control, checkpoints, verification, recovery, context and observability of an agent system on one page, so the design can be reviewed and handed to the next stage. It has 16 sections. This slide covers the first four.

Core question

How can State, Checkpoint, Verification, Recovery, Context and Observability be combined into an Agent Harness that can be implemented?

Canvas sections 1 to 4
#SectionWhat it records
1Agentic Workflow / System GoalWhat the agent system should accomplish
2Runtime TriggerWhat starts execution: a user request, a scheduled task, an event, an incoming document or an API event
3State ModelThe initial state, working states, verification states, human states, error states and terminal states
4State DataWhat must be stored for each state, classified as ephemeral, persistent, sensitive, derived or externally referenced

Info

Human, AI, Human: a person defines the goal and the limits, the agent works inside a controlled harness, and a person reviews and remains responsible for the result.

Key takeaway: Start with a clear goal, a defined trigger, a state model and a plan for what data each state holds.

Loop, checkpoints, verification and human control

9 min

The Canvas, part 2: Execution Loop, Checkpoints, Verification Gates, Human-in-the-Loop

Sections 5 to 8 describe how execution is controlled. They bring in your work from Modules 2, 3 and 4.

Canvas sections 5 to 8
#SectionWhat it records
5Execution LoopNext-step logic, loop condition, stop condition, re-plan condition and human escalation condition
6CheckpointsFor each checkpoint: when, what is saved, why it is safe, resume behavior and replay risk
7Verification GatesFor each critical step: validation method, success criterion, failure route and evidence required
8Human-in-the-LoopReview point, approval point, escalation, override and the context a human needs to decide

Info

Your Agent State & Transition Map from Module 2 fills sections 3 to 5. Your Checkpoint & Resume Plan from Module 3 fills section 6. Your Verification & HITL Gate Map from Module 4 fills sections 7 and 8.

Key takeaway: Sections 5 to 8 say what runs next, where it can safely pause, how results are checked and where a person is involved.

Failure, retry, recovery and compaction

9 min

The Canvas, part 3: Failure Taxonomy, Retry Policy, Recovery & Fallback, Context Compaction

Sections 9 to 12 describe what happens when things go wrong and how a long run stays manageable. They bring in Modules 5 and 6.

Canvas sections 9 to 12
#SectionWhat it records
9Failure TaxonomyLikely failures: transient, tool, data, logical, permission, policy, safety and human rejection
10Retry PolicyRetryable failures, retry limit, the delay or backoff concept and the non-retryable actions
11Recovery & FallbackFallback path, alternative tool, re-plan, compensation, safe stop and human takeover
12Context CompactionWhat stays in working context, what is summarized, what is archived, what can be retrieved later and what must keep source references

Info

Your Failure & Recovery Policy from Module 5 fills sections 9 to 11. Your Context Compaction Policy from Module 6 fills section 12.
Check yourself

Which Canvas sections does your Failure & Recovery Policy from Module 5 mainly fill?

Which Canvas sections does your Failure & Recovery Policy from Module 5 mainly fill?

Key takeaway: Sections 9 to 12 hold your failure, retry, recovery and context compaction design.

Observability, metrics, boundaries, output and the flow

9 min

The Canvas, part 4: Observability, Reliability Metrics, Operational Boundaries, Final Output, and the flow

Canvas sections 13 to 16
#SectionWhat it records
13ObservabilityRun ID, current state, state transitions, tool calls, verification results, retries, errors, human interventions and final status
14Reliability MetricsCompletion rate, recovery success rate, retry rate, escalation rate, average steps, latency, cost and invalid-output rate
15Operational BoundariesStep limit, retry limit, runtime limit, budget, allowed actions, prohibited actions and mandatory human approval
16Final OutputWhat result or artifact the system should produce
Recommended Canvas flow
  1. Trigger
  2. Initialize State
  3. Agent / Workflow Executes
  4. Update State
  5. Checkpoint? If yes, Persist
  6. Verify: Pass, Retry, Human or Fail
  7. Pass: Continue. Retry: apply Retry Policy. Human: Review or Approve. Fail: Recover
  8. Context Too Large? If yes, Compact
  9. Observe / Log
  10. Continue, Re-plan or Stop
  11. Final Output

Info

Text version of the diagram: a trigger starts the run and the harness initializes state. The agent or workflow executes and the state is updated. If this is a checkpoint, the state is persisted. The result is then verified. A pass continues. A retry follows the retry policy. A human outcome goes to review or approval. A failure goes to recovery. Next, if the context is too large it is compacted. The step is observed and logged. Then the run continues, re-plans or stops, and finally produces the final output.

Info

Your Reliability & Observability Scorecard from Module 7 fills sections 13 to 15.
Check yourself

In the recommended flow, what happens after a verification step returns a failure?

In the recommended flow, what happens after a verification step returns a failure?

Key takeaway: Sections 13 to 16 make the run observable and bounded, and the flow ties all 16 sections together.

Complete your Canvas

9 min

Complete your Canvas, and where AI 302 fits

Use your own task from the course, or a simulated one, to complete the Canvas. Reuse your work from Modules 1 to 7 and fill the gaps.

Final reflection question

If this Agent fails halfway through a long-running task, can we explain what happened, resume safely, avoid duplicate actions, and keep a human in control?

  • Agent State & Transition Map
  • Checkpoint & Resume Plan
  • Verification & HITL Gate Map
  • Failure & Recovery Policy
  • Context Compaction Policy
  • Reliability & Observability Scorecard
  • Agent Harness & Reliability Canvas (the primary artifact)

Warning

Privacy reminder: use a simulated or already-public task. Do not enter company secrets, client personal data or unauthorized sensitive information in the Canvas.

Done

Human, AI, Human: you define the goal and the limits, the harness controls execution, and a person reviews and remains responsible.
  • M5276 Agentic AI Innovative Applications, Week 14: this course is the main NTPU AI Learn course aligned to Agent Harness and Loop Engineering for durable workflow automation, covering state machines, checkpointing, context compaction, verification, human-in-the-loop and failure recovery. LangGraph is one implementation example (graph state, nodes, edges, a checkpointer and interrupt and resume), and the concepts do not depend on it.
  • AI 104, Agentic AI Workflow & Execution Design: asks how an agentic workflow should execute. AI 302 asks how that execution is controlled, persisted, verified, recovered and observed.
  • AI 202, Agentic AI Planning & Reasoning (a related course, at /courses/202): AI 202 is decision intelligence, choosing what to do next. AI 302 is execution reliability.
  • AI 301, Multi-Agent Collaboration & Orchestration (a related course, at /courses/301): asks how several agents divide work and coordinate. AI 302 can support a multi-agent system with state, checkpoints and observability, and does not reteach role design or orchestration.

Key takeaway: Your Canvas is a reviewable design, and its key test is whether you can explain a failure, resume safely, avoid duplicate actions and keep a human in control.

Save as PDF

What next?