Imagine an ERP team builds a script to match invoices to goods receipts. It could reduce manual checks, but the first sample contains no partial deliveries or returns. Running it in production immediately could leave the accounting team with a batch of incorrect matches to unwind.
As a team lead, I’d ask people proposing an experiment to say what happens if their assumption is wrong, who can see the effects, and where the team should stop. Clear limits give people room to suggest an idea and take responsibility for the risks they can identify.
Agree on how to run the experiment
Use approved test data without sensitive details. Have the script show proposed matches without writing them back to the ERP. Ask accounting to check ordinary orders, partial deliveries, and returns. The hypothesis is that the proposals will match the reviewer’s answers on the agreed sample. If the script gets even one of those cases wrong, pause any expansion, keep the example, and investigate the conditions that led to the mismatch.
Before starting, name who authorizes the test and who reviews the results. Decide what to record, including correct matches, mismatches, and cases the script can’t resolve. The sample informs what to test next. Moving from preview to production writes needs its own authorization, reconciliation plan, and way to reverse the change.
Review what happened and assign the corrective work
Suppose the script matches a return to the wrong receipt. Start by asking what evidence the team had when it chose the test set, why the first sample lacked returns, and what could detect a similar error next time. Google SRE’s blameless postmortem practice examines the conditions in which people made decisions, then uses the findings to improve the system and assign owners for follow-up. Google SRE’s postmortem guide
A lead could say, “What examples did we have when we approved the test? Can the data team add returns to the test set by Thursday and have accounting review the results before we decide what’s next?” Record the owner, due date, and evidence required to close the action, such as a repeatable test result. If the test still fails, keep the script in preview and revisit it when the evidence is ready.
In the next round, check whether the team stopped at the agreed threshold, reported failures promptly, and finished corrective work that can detect the original problem. Use those checks to decide whether the script can take on more work or should remain a review aid.