Skip to content
NTPU AI Learn

AI 302 Practice

Seven short activities that build the judgment behind a good Agent Harness & Reliability Canvas

Info

Self-check (not scored, not stored). Every activity gives immediate feedback and explains why. If you miss, pick again; an activity is done when every item is right.

Agent Logic or Harness?

Decide who or what is responsible for each simulated design element: Agent reasoning (deciding what to do), Workflow logic (the defined sequence of steps), Harness control (running, tracking, pausing, resuming and limiting execution), Human control (a person decides or approves) or Tool responsibility (what a tool itself does).

0 of 8 correct

Item 1

Choosing which of two search queries is more likely to find the missing figure.

Item 2

Saving the run state to storage after each completed step so the run can resume after a crash.

Item 3

The step order fixed in advance: collect, draft, verify, send.

Item 4

A manager decides whether a simulated refund above the limit is granted.

Item 5

A currency service rejecting a request that has a missing field and returning an error code.

Item 6

Stopping the run after 25 steps or a set budget, whichever comes first.

Item 7

Deciding, from the retrieved facts, that the plan needs a different approach.

Item 8

Pausing a run at the approval state until a named reviewer responds.

Build the State Model

A simulated document-processing agent runs as a state machine. Classify each item as a State (where the run is), a Transition (what moves it between states), an Error state (a state for a failure) or a Human review state (a state that waits for a person).

0 of 8 correct

Item 1

EXECUTING: the agent is carrying out the current step.

Item 2

VERIFYING to EXECUTING when a check fails and retries remain.

Item 3

FAILED: the retry limit was reached and the run stopped safely with its reason recorded.

Item 4

REVIEW: the run is waiting for a reviewer to approve the draft.

Item 5

PLANNING: the agent is breaking the goal into steps.

Item 6

VERIFYING to REVIEW when the confidence is below the threshold.

Item 7

ESCALATED: an unrecoverable tool error was passed to a person and the run is on hold.

Item 8

TOOL_ERROR: the required data source is unavailable and the run cannot continue as planned.

Where Should We Checkpoint?

A simulated workflow reads data, drafts a summary, sends an email (an external action) and updates a record. For each point, decide whether it is a good Checkpoint, a point where a checkpoint is not useful or not safe (No checkpoint), or a Duplicate execution risk: a replay here could repeat an external action unless completion is checked first.

0 of 8 correct

Item 1

The data has been read and stored. The next step drafts a summary.

Item 2

The email API call has started, but no reply has come back yet.

Item 3

The email was confirmed as sent (message ID saved). The record update has not started.

Item 4

The agent is in the middle of writing the third sentence of the draft.

Item 5

The payment submission was sent, the timeout fired and the outcome is unknown.

Item 6

The draft is complete and stored, and the run is about to enter the approval state.

Item 7

A temporary variable was just assigned inside a calculation that takes a millisecond.

Item 8

A file deletion command was issued and the harness crashed before recording the result.

Automatic or Human Gate?

For each simulated action decide the gate: Auto-continue (low risk, no check needed), Auto-verify (continue after an automatic check), Human review (a person looks before it goes on), Human approval (a person must approve before an irreversible or external action) or Prohibit (the agent must not do it).

0 of 8 correct

Item 1

Reformatting a scratch table used only inside the run.

Item 2

Extracted invoice data before it is written to the next step.

Item 3

A drafted reply to a complaint that may contain a policy promise.

Item 4

Sending the finished report to an external client.

Item 5

Sending a payment above the approved limit without a named approver.

Item 6

Checking that a generated file matches the required JSON schema.

Item 7

Changing the status of a named employee's leave request.

Item 8

Writing a log line that records a completed step.

Retry, Recover, Escalate or Stop?

For each simulated failure choose the best response: Retry (try the same thing again within a limit), Fallback (use another route or tool), Escalate (pass it to a person) or Safe stop (halt and keep the state and reason). Remember: retry is not always safe.

0 of 8 correct

Item 1

A read-only lookup timed out once. The service is normally reliable.

Item 2

The primary data source is down for the whole hour, and an approved backup source exists.

Item 3

The output failed schema validation three times. The agent cannot tell which field is wrong.

Item 4

The agent is asked to email personal data to an address outside the approved list.

Item 5

A search API returned a 503 error twice and is now answering normally.

Item 6

The agent does not have permission to open the file. A different, approved file holds the same figures.

Item 7

A payment submission timed out and nobody knows whether the money was sent.

Item 8

The run has used its whole budget and repeated the same failing step for an hour.

Compact the Context

A simulated agent has run for 300 steps and its context is nearly full. For each item in the execution history decide whether it should be kept in the working context (Keep), Summarized into a short state note, or Dropped from the working context (it can stay in the archive).

0 of 8 correct

Item 1

The goal of the run and its deadline.

Item 2

Forty earlier search results that were reviewed and no longer needed, apart from the three that were used.

Item 3

A failed attempt whose full error text is already in the log.

Item 4

The human reviewer's instruction: "Do not contact the supplier until I approve."

Item 5

Twenty intermediate reasoning notes that led to the current plan.

Item 6

The current state of the run and the list of pending actions.

Item 7

The full text of a fetched web page, with a link to its source saved.

Item 8

A list of six decisions already made with the evidence that supports each.

What Should We Observe?

A minimum observability plan records what happened, where and why. For each simulated log entry choose which kind of information it is: Run (the identity and status of one execution), State (where the run is), Tool call (a call and its result), Error (what failed and where), Recovery (what was done about it) or Human intervention (what a person did).

0 of 8 correct

Item 1

run-2041 started at 02:00, final status: completed with one retry.

Item 2

Transition: VERIFYING to REVIEW at step 14, reason: confidence below threshold.

Item 3

search_api(query "budget 2026") returned 4 results in 820 ms.

Item 4

Step 9 failed: source file not found (data-source-2).

Item 5

After the failure, the harness switched to the backup source and continued.

Item 6

Reviewer A approved the draft at 09:12 with a comment.

Item 7

Retry 2 of 3 for the export step, waiting 4 seconds before the attempt.

Item 8

The run was paused and resumed by an operator at step 22.

Sync my progressJoin a session to keep progress across devices and share it with your instructor

Info

After you join, your completed lesson numbers are sent to your instructor together with your learner code, so they can see your progress. Use the codes your instructor gave you, and do not enter your name, company secrets or other personal data. Progress is kept with the rest of that session's data.

What next?