From 9d52cfa5b90a6108932251b4057b79624b75edfb Mon Sep 17 00:00:00 2001 From: ue-toolchain Date: Sun, 6 Sep 2026 01:50:53 +0700 Subject: [PATCH] docs(bundle): record the cross-version run and what it does not prove All 229 runnable recipes were executed against a second tree one major version newer. Nothing was one-sided -- no recipe answered in one version and fell silent in the other. 176 matched exactly, 14 differed in size, 39 returned nothing in either tree. The README states two cautions with the numbers, because the numbers alone would overclaim. Matching counts mean the search surface did not move, not that a claim still holds; anything semantic is untested by counting lines. And the comparison was worthless before the trees were made comparable: 41 of the 55 apparent differences in the first pass were generated build output in one tree and a tooling plugin in the other, a distortion large enough that one query reported 122 files before exclusions and 1 after. Full method, per-entry data and the follow-up list live in the research archive (note 27 and _tools/run_detect_recipes.py), not here. Co-Authored-By: Claude Opus 5 --- plugins/ue-design-skills/README.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/plugins/ue-design-skills/README.md b/plugins/ue-design-skills/README.md index 79befae..9e20b10 100644 --- a/plugins/ue-design-skills/README.md +++ b/plugins/ue-design-skills/README.md @@ -71,6 +71,26 @@ command. If a recipe returns nothing in your tree, widen it before concluding the problem is absent. **An empty search result proves the pattern, not the absence.** +### They have also been run against the next engine version + +Every runnable recipe -- 229 of them -- was executed against a second tree one +major version newer, and the two outputs compared line for line. Nothing was +one-sided: no recipe answered in one version and fell silent in the other. 176 +matched exactly, 14 differed in size rather than in kind, and 39 returned nothing +in either tree. + +Two cautions come with that, and they matter more than the numbers. + +**Matching counts mean the search surface did not move, not that the claim still +holds.** Anything that depends on meaning -- who owns a fact, what replicates, +what is validated -- is not tested by counting lines. + +**The comparison is worthless until the trees are made comparable.** In the first +pass 55 recipes appeared to differ; 41 of those differences were generated build +output present in one tree and a tooling plugin present in the other. One query +reported 122 files before the exclusions and 1 after. If you compare two versions +of anything this way, exclude build artefacts first and say that you did. + ## What this bundle does not claim - It is not a defect list for any current engine version. It is a worked example