Security, Grounding & Provenance

10 min

The Five Security Questions

Connecting AI to tools and data is powerful, and it is also where things go wrong. Before you trust or build any tool, it must be able to answer five questions. If a tool cannot answer them, it is not ready to be used.

1. 1Who can call it?
   Anyone, signed-in users, or only specific roles? Enforced on the server, not just hidden in the interface.
2. 2What data can it access?
   Public pages, internal records, or personal data? Access should be the minimum the tool needs.
3. 3Is it read-only or write-capable?
   A tool that only reads is a very different risk from one that can change, delete, publish or send.
4. 4What happens when it fails?
   Does the user get an understandable message, or does the AI quietly fill the gap with a guess?
5. 5What evidence accompanies the result?
   Source, timestamps and a link, so a human can check where the answer came from.

Key takeaway: A trustworthy tool can answer: who calls it, what data, read or write, what on failure, and what evidence.

10 min

Data Classification and Risk Level

Higher classification and higher risk call for stronger authentication, tighter permissions, more logging and, for high risk, a human confirmation step.

Key takeaway: Classify the data, rate the risk, and let both raise the level of protection.

10 min

Permission Levels and Human Confirmation

The following actions should, in principle, always require explicit confirmation from a human:

• Writing data
• Deleting data
• Publishing content externally
• Sending a message
• Changing an account or its permissions
• Any operation behind a high-impact decision

Key takeaway: Write, delete, publish, send and permission changes need authorization plus explicit human confirmation.

10 min

Grounding and Provenance

A grounded answer is based on an authoritative source rather than on the model's guess. Provenance is the evidence trail that lets a human check it. A good result carries fields such as these:

{
  "results": [
    { "name": "Example Researcher", "expertise": ["AI in education"] }
  ],
  "provenance": {
    "source": "Example University Research Directory",
    "source_id": "expert-0042",
    "retrieved_at": "2026-03-01T09:30:00Z",
    "last_updated": "2026-02-10",
    "canonical_url": "https://example.edu/directory/expert-0042",
    "completeness": "partial (3 of 5 fields available)"
  }
}

• source and source_id: which system and which record the answer came from
• retrieved_at: when the tool fetched it
• last_updated: when the source itself last changed
• canonical_url: where a human can open the original
• confidence / completeness: an indicator of how complete the result is, when applicable

Key takeaway: Provenance (source, id, timestamps, link, completeness) lets a human verify where an answer came from.

10 min

Fail Visibly, Not Deceptively

Showing only "Error 500" helps nobody. Just as bad is the opposite: a tool fails and the model answers anyway from memory. The principle is: fail visibly, not deceptively.

Unable to retrieve course data.

Reason:
The authoritative course service is temporarily unavailable.

What you can do:
- Try again later
- Check the official course system

No answer was generated from unverified data.

• Say clearly that the retrieval failed and why, in plain language
• Give the user next steps they can take
• Never include secrets or internal details in the message
• State that no answer was generated from unverified data

Key takeaway: Fail visibly, not deceptively: explain the failure and never fill the gap with a guess.

10 min

Verification Habits and Shared Responsibility

Even a well-built tool does not remove the human from the loop. Build five habits, and use them every time an AI answer relies on a tool.

1. 1Check the source
   Is it an authoritative system, and can you open the original?
2. 2Check the freshness
   Compare retrieved_at and last_updated with what the question needs.
3. 3Check the provenance
   Are source, id and link present, and do they match the claim?
4. 4Look for missing information
   Is anything marked partial, or simply absent?
5. 5Note the uncertainty
   Separate what the tool returned from what the AI inferred, and say which is which.

Key takeaway: Check source, freshness, provenance, gaps and uncertainty; the human stays responsible.