Government AI Governance, Security, and Human Oversight

10 min

Where governance starts: who is accountable, how to trace

The core of governance is accountability: however much AI helps, every result has a person or unit who can be held responsible. Human oversight and human-in-the-loop are the mechanisms that make accountability real, but "a human looked at it" does not mean governance is complete. This module reuses the petition triage and draft-reply case from earlier modules (a simulated teaching scenario) and asks what controls it is still missing.

• Accountability: every step and every permission maps to a named owner, not "the system runs by itself and nobody in particular is responsible".
• Human oversight: the person must have enough information, time, and authority to actually judge, correct, or stop, not just sign off in form.
• Traceability: you can trace back how a result was produced, including which data, tools, and versions were used, and who approved it and when.
• Explainability: reviewers can understand why the AI produced a result, and so judge whether it is reasonable.

Key takeaway: The goal of governance is not to eliminate risk, but to ensure that whatever the outcome, someone can explain it, trace it, and answer for it.

10 min

Three control levels: how much human involvement

Key takeaway: Control levels describe how dense human involvement is; moving up a level always requires assessment and approval and never happens automatically.

10 min

Human approval gates, system control gates, and four security risks

• Prompt injection: input content carries instructions that try to push the AI off its rules and into unauthorized actions.
• Data leakage: the AI exposes data it should not, in an answer, a log, or an outbound message.
• Tool misuse: the AI uses a tool with improper parameters or at an improper time, causing unintended queries or writes.
• Over-authorization (excessive agency): the AI or its tools are given more permission than the task needs, so any error has an outsized impact.

Key takeaway: A human approval gate guards a single action and a system control gate guards ongoing behavior; recognizing the four risks is the starting point for designing both.

10 min

Finding governance gaps in one case, and incident response

1. 1Stop
   When something abnormal appears, pause the AI's execution and any outbound action first, to keep the impact from spreading.
2. 2Report
   Report to the owner and the security contact under the agency's procedure, recording when and what was noticed.
3. 3Human takeover
   Designated staff handle the affected cases manually and keep records for later review.

Key takeaway: Write every high-impact step as risk, control, owner, and evidence; when an incident occurs, the order is stop, report, human takeover.

10 min

Practice: governance and control checklist

This practice reuses the workflow from Module 2 and the data and tool permission table from Module 3. You check them for governance gaps and add risk, control, owner, and evidence. Your result goes into the "governance controls" and "risk and mitigation" fields of the integrated Canvas (the governance and control checklist).

1. 1Review the workflow and permission table step by step
   Find steps with no owner, no human approval, excessive permission, or no record.
2. 2Mark control level and gates
   Mark each high-impact step as Level 1, 2, or 3, and distinguish human approval gates from system control gates.
3. 3Write risks and controls
   Cover whichever of prompt injection, data leakage, tool misuse, and over-authorization apply to this case.
4. 4Assign owners and evidence
   Every control needs an owner and evidence that can be checked afterwards; also write how stop, report, and human takeover would work in an incident.

• Go to the practice page: governance and control checklist

Key takeaway: This checklist adds responsibility and evidence to the case from the first three modules, setting up the test design in Module 5 and the acceptance judgment in Module 6.