Repair the claim. Recheck the consequences.

An AI answer can sound reasonable and still send the work sideways. A return window changes. “Send for review” becomes “approved.” Now the answer has a problem, and so does the work that followed it.

I’d rather find the bad claim or failed task than ask for another version and hope. Fix the part that’s wrong. Then check what leaned on it. These are my own working notes, not a description of a system at work or a claim that this framework has been validated.

A made-up example

A store’s policy: unused items returned within 14 days may get a refund, and a person has to approve it. The assistant’s job is to explain that, ask for the order number and hand the case to a reviewer.

Here’s the reply, with two things worth separating:

01 / Inspect the artifactFictional example

One claim to repair. Another decision to question.

Source / supplied policy

Unused item. Returned within 14 days. A refund may be available.

A person must approve it.

Candidate / assistant reply

“Your unused item is within the 30-day return window. Your refund is approved.”

01Wrong fact: 30 days conflicts with 14.

02Unsupported decision: the assistant cannot grant approval.

The selected claim is the return window. The approval is a separate error; this reply does not have only one mistake.

Then it marks the handoff “Approved for payment” and closes the case.

The wrong number needs a source check. Closing the case needs a task check: the job was to pass it to a person. “Approved” fails both. That’s an unsupported claim and a decision the assistant wasn’t allowed to make.

Four steps

Start by pointing at the exact sentence or output and the source or requirement it should match. “This feels off” doesn’t give you much to repair.

Then check it. Where the rule is exact, use a plain test: this case must stay open for review. Another model can help review the answer, but agreement isn’t independent confirmation. Judge models have documented biases (Zheng et al.); a good score isn’t proof that the answer is right.

For the reply, the repair can be small:

An unused item returned within 14 days may be eligible for a refund. Please share your order number so a reviewer can check it.

Give that repair a limit. Here, the original attempt plus one repair. If the repair still fails, or the source is missing or conflicting, stop and hand the unresolved issue to a person. Anthropic describes an evaluate-and-refine loop as useful when criteria are clear and improvement can be assessed (“Building effective agents”). That supports the approach; it doesn’t validate this whole workflow.

Now comes the easy bit to miss. The handoff still says “Approved for payment.” The case is still closed. Fixing the reply hasn’t touched either one.

02 / Follow what depends on itOne possible workflow

A local fix has a wider check.

Source + task requirements

14 days · conditional eligibility · human approval

Repair the reply

Correct the window. Ask for the order number. Request review.

Revalidate / handoff

“Approved for payment” must become a pending-review summary.

Revalidate / case state

“Closed” must return to awaiting human review.

Whole-result check

Read the reply, handoff and case state against the original requirements.

This map is an illustration, not a universal DAG. A changed source or human decision can send work back to an earlier step. Check the affected dependencies and the overall result.

Put the handoff and case back into pending review, then check all three together. The correction has to reach the places that used the original decision. A correct sentence doesn’t make a correct workflow.

03 / The checking loopBounded, not automatic

Inspect. Evaluate. Repair. Revalidate.

01InspectFind the claim or task and its source.
02EvaluateCheck evidence and explicit requirements.
03Repair locallyIf it fails, change the faulty unit within the bound.
04RevalidateRecheck the repair, affected work and overall requirements.

Original + one repair, in this example.Still failing, missing evidence or conflicting sources? Stop and escalate to a person.

The bound is an illustrative choice, not an optimum. A critic or retry limit does not prove correctness. If the checks cannot be completed, the result stays unresolved.

Why this matters to me

I’m building Fact Riot around the same idea: claims you can trace to sources, dispute and correct in the open. I want to put my own writing through that process, too. It’s still in development; this made-up return is not a product demo or a claim that Fact Riot runs refund workflows.

That habit, checking an answer against its source and knowing when to ask a person, is what I want to teach through Autarkic: an early idea for accessible, practical AI education for lower-income people, local businesses and farmers. It isn’t a running program yet.

So pick one claim you’d want checked. What backs it up? And if it changes, what else needs another look?