docs(bundle): record the cross-version run and what it does not prove

All 229 runnable recipes were executed against a second tree one major version
newer. Nothing was one-sided -- no recipe answered in one version and fell
silent in the other. 176 matched exactly, 14 differed in size, 39 returned
nothing in either tree.

The README states two cautions with the numbers, because the numbers alone
would overclaim. Matching counts mean the search surface did not move, not that
a claim still holds; anything semantic is untested by counting lines. And the
comparison was worthless before the trees were made comparable: 41 of the 55
apparent differences in the first pass were generated build output in one tree
and a tooling plugin in the other, a distortion large enough that one query
reported 122 files before exclusions and 1 after.

Full method, per-entry data and the follow-up list live in the research archive
(note 27 and _tools/run_detect_recipes.py), not here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
ue-toolchain
2026-09-06 01:50:53 +07:00
parent ee80ce6338
commit 9d52cfa5b9
+20
View File
@@ -71,6 +71,26 @@ command.
If a recipe returns nothing in your tree, widen it before concluding the problem
is absent. **An empty search result proves the pattern, not the absence.**
### They have also been run against the next engine version
Every runnable recipe -- 229 of them -- was executed against a second tree one
major version newer, and the two outputs compared line for line. Nothing was
one-sided: no recipe answered in one version and fell silent in the other. 176
matched exactly, 14 differed in size rather than in kind, and 39 returned nothing
in either tree.
Two cautions come with that, and they matter more than the numbers.
**Matching counts mean the search surface did not move, not that the claim still
holds.** Anything that depends on meaning -- who owns a fact, what replicates,
what is validated -- is not tested by counting lines.
**The comparison is worthless until the trees are made comparable.** In the first
pass 55 recipes appeared to differ; 41 of those differences were generated build
output present in one tree and a tooling plugin present in the other. One query
reported 122 files before the exclusions and 1 after. If you compare two versions
of anything this way, exclude build artefacts first and say that you did.
## What this bundle does not claim
- It is not a defect list for any current engine version. It is a worked example