Skip to content

AD 103 · Module 1

Evaluate

Build portfolio thinking, and evaluate Value & Outcome, Feasibility & Readiness, and Risk & Governance separately.

Start Section 1
0 / 4 completed
Sync my progressJoin a session to keep progress across devices and share it with your instructor

Info

After you join, your completed lesson numbers are sent to your instructor together with your learner code, so they can see your progress. Use the codes your instructor gave you, and do not enter your name, company secrets or other personal data. Progress is kept with the rest of that session's data.

Learning objectives

  • Under limited resources, think in terms of a portfolio rather than a single case, weighing dependency, capacity, and learning value.
  • Separate activity from outcome; value can include time, quality, capacity, cost, service, and strategic dimensions.
  • Record technical feasibility and organizational readiness separately.
  • Keep risk visible rather than hiding it as a deduction in a total score; escalate to governance review when needed.

Portfolio Thinking

25 min

Portfolio Thinking

Learning objective: Under limited resources, think in terms of a portfolio rather than a single case, weighing dependency, capacity, and learning value.

With several candidate Use Cases in hand, you cannot evaluate them one at a time and just pick whichever scores "highest". Portfolio thinking considers them together: are there dependencies between cases (e.g. one must finish before another can start), what is the team's current execution capacity, and what learning value comes from doing a smaller case first (e.g. validating an assumption that reduces risk for a later case).

Three dimensions of portfolio thinking
FieldMeaning
DependencyDo cases depend on each other's sequence?
CapacityHow many cases can the team run at once right now?
Learning valueWhich case, done first, reduces risk for the others?
Workplace example

Three candidate Use Cases: A (leave-process automation), B (financial-report summarization), C (procurement quote-comparison assistance). B needs financial-data authorization confirmed first (dependency), the team has only one squad this quarter (capacity), and C is small and can quickly validate whether "AI-assisted comparison is trustworthy" (learning value). So C might go first, with A or B decided based on the result.

  • Dependency, extended case: Use Case D (cross-system report integration) must wait for Use Case E (data-warehouse rollout) to finish, a clear sequencing relationship.
  • Capacity, extended case: the team can only deeply support one pilot at a time; forcing three cases live simultaneously means each gets stretched thin and slips.
  • Learning value, extended case: running one small classification case first to confirm "can AI classification's manual-correction rate stay under 10%" produces a lesson that applies to every larger case after it.
  • When the three dimensions conflict (e.g. the case with the highest learning value happens to have the lowest capacity), honestly record the trade-off instead of forcing a single ranking.

Score each case, decide separately

  • Every case gets its own score; the highest-scoring one goes first.
  • No account is taken of how cases relate to each other.
  • Too many cases easily launch at once, and none of them gets done well.

Portfolio thinking

  • All candidate cases are examined together as a set.
  • Dependencies are identified first, to set a reasonable order.
  • How many run at once is set by team capacity, and the case with the highest learning value is prioritized to validate assumptions.

Info

Human, AI, Human: a person makes the final prioritization and governance decisions; AI may help organize comparison data, but never makes the final call for the organization.

Warning

Data safety reminder: every exercise in this course uses course-provided synthetic scenarios only, never real institutional or personal data.
Check yourself

You have five candidate Use Cases. Which approach best fits portfolio thinking?

You have five candidate Use Cases. Which approach best fits portfolio thinking?

Key takeaway: Facing multiple candidate Use Cases, look at them together as a portfolio, not by scoring each in isolation.

Value & Outcome

25 min

Value & Outcome

Learning objective: Separate activity from outcome; value can include time, quality, capacity, cost, service, and strategic dimensions.

When assessing Value, first confirm whether what is written is an outcome or an activity. Value is not one dimension, it may include time (shorter processing), quality (fewer errors), capacity (more cases handled with the same staff), cost (lower spend), service (higher satisfaction), and strategy (supporting long-term organizational goals). The same Use Case may bring several kinds of value at once, and each should be written out separately rather than vaguely claiming "it has value".

Six dimensions of value
FieldMeaning
TimeHow much faster?
QualityHow much less error or rework?
CapacityHow much more can the same staff handle?
CostHow much lower spend?
ServiceHow much better satisfaction or experience?
StrategyWhich long-term organizational goal does it support?
Workplace example

Value of leave-process automation: time (approval drops from 3 days to 1), capacity (HR can handle 30% more cases with the same staff), quality (fewer errors from manual entry). Written out separately, not just "efficiency improves".

Workplace example (contrast case: procurement quote-comparison assistance)

Breaking down the Value of procurement quote-comparison assistance: time (comparison work drops from an average of 2 hours to 30 minutes), quality (fewer amount errors from manually transcribing quotes), service (faster vendor-quote turnaround, fewer complaints), strategy (supports the organization's annual "digitize administrative processes" goal). Writing the four dimensions out separately lets reviewers judge which one most deserves investment.

  • Time: how long does handling one case drop from, and to? Use a concrete number, not just "faster".
  • Quality: do error rate, rework rate, or complaint volume drop?
  • Capacity: how much more can the same staff handle? Or does overtime that used to be needed go away?
  • Cost: is outsourcing spend, material waste, or some other direct cost reduced?
  • Service: does satisfaction or perceived wait time improve for the user or applicant?
  • Strategy: does it support a long-term goal the organization has already announced (e.g. digital transformation, sustainability, service quality)?

Info

Human, AI, Human: a person makes the final prioritization and governance decisions; AI may help organize comparison data, but never makes the final call for the organization.
Check yourself

A Use Case's Value is written as: "people will feel their work is easier." Which rewrite best fits this lesson's requirement?

A Use Case's Value is written as: "people will feel their work is easier." Which rewrite best fits this lesson's requirement?

Key takeaway: Value should be broken into concrete dimensions and described separately, not waved away with a vague feeling.

Feasibility & Readiness

20 min

Feasibility & Readiness

Learning objective: Record technical feasibility and organizational readiness separately.

Feasibility asks about technical and integration complexity: can the data be obtained, can systems be connected, is AI capability sufficient. Readiness asks about the organizational side: is there a clear owner, can the workflow accommodate change, does the team have the capacity to execute, does it fit existing policy. The two are easily conflated, but being technically feasible does not mean the organization is ready, and vice versa, they must be recorded separately.

Two separate questions
FieldMeaning
FeasibilityIs the data, system, and AI capability sufficient?
ReadinessAre the owner, workflow, team capacity, and policy ready?
Workplace example

Leave-process automation: Feasibility is high (data is already in the existing system, integration is not complex); Readiness is moderate (HR's owner is clear, but managers are used to paper sign-off and need change management). Recording them separately reveals that the real investment needed is in readiness, not technology.

  1. Ask Feasibility first
    Where does the needed data live? Is the format clean? Does the system to be integrated have an open interface?
  2. Then ask Readiness
    Who is the owner? Can that unit's way of working accommodate the change? Does the team have the time and ability to execute?
  3. Mark each dimension separately
    Rate each as high/medium/low on its own; do not let one dimension color the other.
  4. Find where the real investment is needed
    Whichever dimension scores low tells you where to spend time communicating or preparing next.

High feasibility, low readiness

  • Data is complete and the system is easy to integrate; it is technically ready any time.
  • But no unit is willing to be the owner, or managers are used to the old process and resist change.
  • The investment needed is change management and communication, not technical resources.

Low feasibility, high readiness

  • The unit is highly motivated to change, the owner is clear, and the team is well prepared.
  • But the data is scattered across multiple systems in inconsistent formats, making integration costly.
  • The investment needed is data cleanup and technical assessment, not persuading the organization.

Info

Human, AI, Human: a person makes the final prioritization and governance decisions; AI may help organize comparison data, but never makes the final call for the organization.
Check yourself

A Use Case is technically easy to implement (data is complete, APIs are ready), but no one is willing to own it after launch. What does this reflect?

A Use Case is technically easy to implement (data is complete, APIs are ready), but no one is willing to own it after launch. What does this reflect?

Key takeaway: Being technically feasible does not mean the organization is ready, always record feasibility and readiness separately.

Risk & Governance

20 min

Risk & Governance

Learning objective: Keep risk visible rather than hiding it as a deduction in a total score; escalate to governance review when needed.

The point of risk assessment is not to deduct points, but to let decision-makers clearly see what risk exists and what the consequence would be. Folding risk into a negative number buried inside a total score can let a genuinely high-risk case get averaged away and become hard to spot. When risk is clearly high (e.g. involving personal data, financial decisions, regulatory compliance), it should go straight to a governance review rather than letting the Use Case pass on its own.

Risk-handling principles
FieldMeaning
Risk descriptionWhat is the consequence, how likely is it?
VisibilityIs the risk clearly shown, not hidden in a total score?
EscalationDoes it need to go to governance review?
Workplace example

A Use Case involves auto-approving purchase requests above a certain amount. Risk: a wrong call could cause financial loss and audit issues. Approach: rather than deducting points and letting it pass with an averaged total, explicitly flag it "high risk, needs governance review" and hand it to finance/compliance for review.

Workplace example (contrast case: student grade data)

Another Use Case wants AI to analyze grade trends and suggest advising actions for academic advisors. This touches student personal data and academic records. Risk: a wrong analysis or a leak could affect student rights and institutional trust. Approach: instead of letting a high "advising benefit" score mask the risk, explicitly flag it "involves student personal data, requires joint review by data-protection and academic affairs before proceeding", and hold it out of any pilot until that review clears.

  • Personal-data risk: does it involve sensitive information about a student, employee, or other individual?
  • Financial risk: does it involve approving amounts, allocating budget, or a possible financial loss?
  • Regulatory-compliance risk: does the decision need to satisfy a specific law or policy?
  • Safety risk: could a wrong output affect someone's physical safety or a major right?
  • If any one of these is clearly present, route it to governance review first, rather than waiting until every dimension's score is tallied.

Info

Human, AI, Human: a person makes the final prioritization and governance decisions; AI may help organize comparison data, but never makes the final call for the organization.
Check yourself

Which approach best fits this lesson's "never hide risk in a total score" principle?

Which approach best fits this lesson's "never hide risk in a total score" principle?

Key takeaway: Risk must be clearly visible and recorded independently; high-risk cases should escalate straight to governance review, not get averaged away in a total score.

What next?