# Six errors planted in a fake month of books: custom agents caught five unprompted, missed one sign-off rule, and refused to guess

自建 Agent · Reddit · Nothing is disclosed. The author says the self-test cost about 5.5 dollars and under half an hour; that is a self-reported cost, not income or verified savings.

## Business summary

Reddit user u/Speedydooo wrote in r/AIToolsForSMB about a destructive self-test of his own agent setup. He built a realistic-looking month of bookkeeping for a business that does not exist - invoices, vendor payments, a churn curve - quietly planted six specific errors, and wrote the answer key before handing the data to the agents: a duplicate invoice, a vendor rate that had drifted without being updated, a threshold that needed sign-off and did not get one, a customer segment whose margin looked fine but was actually losing money, revenue leakage visible only in the churn month, and a vendor payment that was never approved. He gave the agents no hints. They caught five of six unprompted. The miss was not arithmetic: it was a sign-off rule his own grading logic did not account for, and two agents flagged that gap on their own. He says the part that built trust was not the catches but three refusals - one would not say whether the business was solvent without a number it needed, one would not estimate customer acquisition cost without real spend data instead of guessing at something reasonable-sounding, and one would not compute a ratio from an ambiguous input, asking which definition he meant instead. Because agent output did not match his answer key, he went back and found two real errors in the key itself: a typo in his math and a rule that did not match how the business actually worked. He says the whole exercise cost about five and a half dollars and took under half an hour - his cost, not a result. The data is fabricated, there are no real clients, no income, no verified savings, no subscription or token breakdown, and no performance claim.

## Business value

The reusable idea is that refusal is an output, not a failure. Most self-tests only count hits, so a model quietly fills uncertain gaps with a plausible number. This author counted 'I do not have enough to answer that' as a finding - and because the output disagreed with his key, he found two mistakes in the key itself. Write the key first, plant known errors, then allow the key to be overturned. The workflow, prompt and synthetic sample here are AgentFlow proposals and were not run.

## Workflow

Write the answer key before handing the data over, then plant the known errors. (AgentFlow's working order; the post does not state whether the key came before or after the errors.)

Hand over the material with no hints about what to look for. (The post says the five catches were unprompted.)

Score each expected finding as caught, missed, or refused - with evidence row numbers. (AgentFlow's formalisation.)

Record refusals and clarifying questions as their own output, stating what is missing and who must supply it. (The author's own observation, made explicit here.)

When output disagrees with the answer key, re-check the key before blaming the agents. (The author found two errors in his own key this way.)

Stop at a synthetic self-test: no real ledger, no external report, no sending. (AgentFlow's boundary; the post does not describe a confirmation step.)

## Preparation and limitations

The post does not say which agents or models were used, how many agents ran, what the six errors looked like in the raw data, what the two answer-key errors were in detail, how the 5.5 dollars was calculated, or what subscription or token prices applied. It gives no real client data, no income, no savings figure, no accuracy over real books, and no time series. The six-error hit rate cannot be generalised to a real ledger. All execution steps, the prompt and the sample here are synthetic proposals.

## Starter prompt

````text
You are my bookkeeping review assistant. You may conclude only from the data I give you. If a field is missing, a definition is ambiguous or evidence is insufficient, write 'cannot determine, missing X' instead of supplying a plausible number. You do not send anything externally and do not modify the source ledger.

[INPUT]
- Synthetic ledger file: [CSV or table path], with at least date / amount / counterparty / receipt id / category
- Period and scope: [start and end dates], [which accounts or lines]
- Review rules: [rules to check, e.g. duplicate receipts, vendor rate consistency, approval thresholds, segment margin, cross-period revenue, payment authorisation]
- Output format: [required table fields]
- Note: my answer key is kept separately and is NOT part of your input. I am giving you no hint about where errors might be.

[STEPS]
1. Report how many rows you actually read, whether the date range is fully covered, and which fields are missing. If fields are missing, stop here and say which step is blocked.
2. Check each rule in turn. For every rule state what you checked, which rows matched, and the evidence. Write 'none found' where applicable; never skip a rule.
3. Write 'cannot determine' and list it separately whenever a key number is missing, real evidence is absent, a definition has more than one reading, or a rule requires approval that is not recorded. State what is missing, which conclusion it affects, and who must supply it.
4. List clarifying questions separately: which definition or scope do you need me to confirm before you can compute.
5. Produce one result table: rule / outcome (found, none found, cannot determine) / evidence row ids / missing item / who confirms.
6. End with 'what I could not cover', including periods not checked, fields not read, and any rule you think is itself ill-defined.

[OUTPUT]
1) read report with row count and coverage; 2) per-rule findings with evidence row ids; 3) cannot-determine list; 4) questions for me; 5) result table; 6) coverage gaps.

[NEEDS MY CONFIRMATION]
- Any judgement involving approval, authorisation or a definition: stop and let me confirm before computing.
- If my rule list is ambiguous: ask which reading applies, do not pick one.
- Connecting this to a real ledger, issuing an external conclusion or sending a report: stop at the synthetic self-test and wait.

[IF MATERIAL IS MISSING OR A STEP FAILS]
Unreadable file, mismatched fields, missing period, duplicate receipt ids that cannot be resolved: say exactly what is missing and which step is blocked. Never substitute rounding, general knowledge or older data. Prefer an extra 'cannot determine' over a number with no evidence.
````

## Costs and pitfalls

The post does not say which agents or models were used, how many agents ran, what the six errors looked like in the raw data, what the two answer-key errors were in detail, how the 5.5 dollars was calculated, or what subscription or token prices applied. It gives no real client data, no income, no savings figure, no accuracy over real books, and no time series. The six-error hit rate cannot be generalised to a real ledger. All execution steps, the prompt and the sample here are synthetic proposals.

## Source and attribution

Speedydooo（Reddit 用户 u/Speedydooo，原帖作者本人，自述对自己的 Agent 配置做了一次破坏性自检） (u/Speedydooo)
https://www.reddit.com/r/AIToolsForSMB/comments/1uxrkgc/i_planted_six_errors_into_a_month_of_fake/
2026-07-16
Results come from the source author. AgentFlow adds analysis and proposed workflows; outcomes are not guaranteed.
