Module 7 | Conflict Resolution, Human Oversight & Evaluation | AI 301

Conflicts and how to resolve them

10 min

When agents disagree: kinds of conflict

Once several agents share a goal, they will sometimes disagree, repeat each other or pull in different directions. A conflict is not a sign that the design is broken. It is something the design must plan for, because an unhandled conflict either stalls the system or is settled by accident.

Core question

When Agents disagree, duplicate work or produce conflicting results, what does the system do?

Common kinds of multi-agent conflict (simulated examples)
Conflict	What it looks like	Simulated example
Contradictory outputs	Two agents return answers that cannot both be true	One agent says a policy allows the request, another says it does not
Duplicate work	Two agents do the same task	Both a Research Agent and an Analysis Agent fetch the same sources
Priority conflict	Agents disagree on what comes first	One agent wants speed, another wants a full check first
Goal conflict	Agents optimize for different aims	A cost-saving agent and a quality-checking agent want opposite things
Authority conflict	Agents both claim, or both refuse, a decision	Two agents each believe they own the final approval
Evidence conflict	Agents cite different or clashing sources	One relies on last year's document, one on this year's

Info

Human, AI, Human: a person defines the goal, the roles and how conflicts are settled, the agents work inside those rules, and a person reviews the cases the rules cannot settle and remains responsible.

Key takeaway: Name the kinds of conflict your agent team can have before you design how to resolve them.

10 min

The conflict-resolution flow

A good design settles conflicts in a fixed order, from the cheapest and most predictable method to the most expensive. Most conflicts should be resolved early, and only the hard ones should reach a person.

Conflict-resolution flow

Conflict detected
Can policy resolve it? If yes, Apply Rule
If no: Can evidence resolve it? If yes, Compare Evidence
If no: Is there a Supervisor or Evaluator Agent that can decide? If yes, Resolve
If no: Human Escalation

Info

Text version of the diagram: a conflict is detected. First ask whether a policy or rule settles it, and if so apply the rule. If not, ask whether evidence settles it, and if so compare the evidence. If not, ask whether a Supervisor or Evaluator Agent may decide, and if so let it resolve the conflict. If none of these works, escalate to a person.

The four steps
Step	What it relies on	Simulated example
Apply Rule	A written policy or priority order	Compliance findings always outrank speed
Compare Evidence	The quality, source and freshness of evidence	The newer official document wins over an older summary
Supervisor / Evaluator	A role that is allowed to decide	The Supervisor picks between two drafts against stated criteria
Human Escalation	A person with authority	A reviewer decides a high-impact disagreement

Check yourself

Two agents give contradictory answers. A written policy already says which source wins. What should the system do first?

Apply the rule from the policyEscalate to a person immediatelyAsk both agents to keep debating until they agree

Key takeaway: Resolve in order: policy, evidence, Supervisor or Evaluator, then Human Escalation.

9 min

Tie-breaking, arbitration, consensus and Supervisor resolution

Several ideas are used to close a conflict. They are different tools, and mixing them up causes confusion about who decides.

Four ways to close a conflict
Idea	How it works	Use it when
Tie-breaking	A pre-agreed rule picks one option	Two options are equally valid and a quick, predictable answer is enough
Arbitration	A neutral party, an Evaluator Agent or a person, weighs both sides and decides	Agents make competing claims and neither is clearly right
Consensus concept	Agents converge on a shared answer, for example by comparing views until they agree	The stakes are low and agreement is cheap to reach
Supervisor resolution	The Supervisor, who has the wider view, decides	A central role already owns the outcome

Agents settle it among themselves

Consensus needs no extra role
It can loop for a long time
Best for low-stakes disagreements

A designated role settles it

A Supervisor or Evaluator decides
It is faster and easier to audit
Best when a clear owner exists or stakes are higher

Warning

Consensus without a limit can turn into endless discussion, which is exactly the coordination overhead this course warns about. Give any agreement process a limit, and treat reaching it as a reason to escalate.

Check yourself

Two agents keep exchanging arguments about a high-impact decision without converging. What is the best design response?

Let them continue until they agreeSet a limit on the exchange, then hand it to an arbiter or a personLet the agent with the longer answer win

Key takeaway: Pick the closing method deliberately: tie-break for equals, arbitrate for competing claims, keep consensus for low stakes, and bound every exchange.

Human oversight

9 min

Human oversight and escalation

Human-in-the-Loop coordination means deciding in advance where a person takes part. It is a design choice, not a last resort. Four kinds of oversight cover most needs.

Four forms of human oversight
Form	What the person does	Simulated example
Review	Checks a result before it is used	A reviewer reads a report drafted by several agents
Approval	Gives an explicit yes before an action	A manager approves before anything commits the organization
Override	Pauses, redirects or cancels the system at any time	A person stops the team when it heads the wrong way
Escalation	Receives a case the agents cannot settle	An unresolved authority conflict lands on a named owner

Escalate when a conflict is high-impact, cannot be settled by policy, evidence or a designated resolver, or reaches its exchange limit
Escalate when the goal, the authority or the risk is unclear
Always tell the person what was disputed, the options, the evidence and what is needed from them
Name who receives the escalation, so it never falls between two agents

Warning

Do not design the system so that high-impact output is accepted without review. Human oversight must be easy to use, or it will be skipped.

Check yourself

What should a good escalation to a person contain?

The full raw conversation between the agents and no summaryWhat is disputed, the options, the evidence and the decision needed, sent to a named ownerOnly the word "escalated"

Key takeaway: Design review, approval, override and escalation in advance, with a named owner and a clear summary.

Multi-agent evaluation

9 min

Evaluating a multi-agent system

Extra agents add cost and coordination. Evaluation asks whether they earn it: did collaboration improve the outcome compared with a simpler design?

Multi-agent evaluation metrics
Metric	What it tells you
Task success	Whether the team achieved the shared goal
Role adherence	Whether each agent stayed inside its role and boundaries
Delegation quality	Whether work went to the right agent with the right context
Communication efficiency	Whether messages were useful, or just noise
Duplicate-work rate	How often two agents did the same task
Handoff quality	Whether handoffs were complete and clear
Conflict rate	How often agents disagreed or collided
Evidence quality	Whether results were backed by good sources
Latency	How long the whole team took
Cost	What the collaboration cost overall
Human escalation frequency	How often a person had to step in

Read metrics together. A very low escalation rate can mean the team is working well, or that it is hiding problems. A high conflict rate with high duplicate work often points to unclear role boundaries.

Info

Boundary of this course: this module chooses what to measure at the design level. Building monitoring infrastructure and production observability belongs to AI 302 and is not taught here.

Check yourself

A team shows a high duplicate-work rate and frequent conflicts. Which is the most likely design problem to check first?

Role and responsibility boundaries that overlapThe team is too fastHumans review too often

Key takeaway: Measure whether collaboration beats a simpler design, and read the metrics together.

Practice and next steps

8 min

Activity 7: Resolve the Multi-Agent Conflict, and your policy

You will be given a scenario in which agents conflict over responsibility, evidence or a decision. For each scenario, decide how the system should handle it.

Identify the conflict type
Contradictory outputs, duplicate work, or a priority, goal, authority or evidence conflict?
Choose the resolution path
Policy, evidence, Supervisor or Evaluator, or Human Escalation, in that order.
Decide the human requirement
Is a review, approval or escalation needed, and who owns it?
Explain why
Name the conflict, the path and the boundary that support your choice.

Done

Deliverable of this module: a Multi-Agent Evaluation & Escalation Policy. It states the conflict types you expect, the resolution path for each, the metrics you will watch, and when a person is involved.

Warning

Privacy reminder: use simulated or already-public scenarios. Do not enter company secrets, client personal data or unauthorized sensitive information.

In Module 8 you will place this policy on the Multi-Agent Architecture Canvas, in the Conflict Resolution, Human Oversight and Evaluation sections.

Go to the AI 301 practice: In the "Resolve the Multi-Agent Conflict" activity, choose and explain.
Back to Module 6: Orchestration Patterns & Collaboration Topologies
Continue to Module 8: Multi-Agent Architecture Canvas

Check yourself

Two agents each claim final approval of a high-impact action, and no written rule says who owns it. What is the best handling?

Let the faster agent approve itTreat it as an authority conflict, and escalate to a named person while fixing the ownership ruleIgnore it, since both agents approved

Key takeaway: Identify the conflict, choose the path, decide the human role, and record it in your Evaluation & Escalation Policy.

Conflict Resolution, Human Oversight & Multi-Agent Evaluation

Learning objectives

Conflicts and how to resolve them

When agents disagree: kinds of conflict

The conflict-resolution flow

Tie-breaking, arbitration, consensus and Supervisor resolution

Agents settle it among themselves

A designated role settles it

Human oversight

Human oversight and escalation

Multi-agent evaluation

Evaluating a multi-agent system

Practice and next steps

Activity 7: Resolve the Multi-Agent Conflict, and your policy

What next?