NOTE 01Why separate execution from improvement?
Doing a task and changing the system carry different risks. Keeping them separate makes it clearer who can change what, and when it takes effect.
Read the note
The run loop uses a fixed version to carry out tasks. The improvement loop collects issues and proposes candidates. Each candidate needs independent evaluation and a release decision; one successful example is not permission to rewrite production workflows. Monitoring and a rollback version remain necessary after release.
Classify failures before optimizing.
Missing information, unsupported inference, unit conversion and lost sources call for different checks.
Read the note
Record the source location, extracted value and validation status for each field. Missing does not mean negative; uncertain does not mean wrong. Identify the failure type before deciding whether to change a rule, add context, adjust a tool or ask for human judgment.
What would count as evidence of improvement?
Better results on development examples may not carry over to the next batch. Independent tests and regression checks matter just as much.
Read the note
For a pilot, we recommend separating development examples from independent evaluation examples and considering quality, cost, latency and human review effort together. Without a baseline, an understood sample distribution and a defined evaluation scope, we do not report percentage gains.