diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d6d263c..238c5d8 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -63,8 +63,10 @@ claude plugin validate . --strict The gate is not advisory. It enforces, among other things: no absolute paths, no source citations, no donor project name outside a Provenance section, no Cyrillic, no vendor or -harness name anywhere under `skills/`, and the six required fields on every failure-mode -entry. +harness name anywhere under `skills/`, no engine toolset, meta-tool or endpoint name +either (P17, see +[ADR-0004](docs/architecture/decisions/0004-engine-tool-surface-out-of-skill-content.md)), +and the six required fields on every failure-mode entry. If you add a rule to the gate, **add a poisoned fixture that makes it fire.** The self-test fails if any declared rule has no fixture. This is deliberate: the validator diff --git a/README.md b/README.md index 2f3d2df..5ec4d47 100644 --- a/README.md +++ b/README.md @@ -22,6 +22,7 @@ - [ADR-0001: Discovery over PID file](docs/architecture/decisions/0001-discovery-over-pid-file.md) — поиск запущенного редактора через UDP-broadcast, а не через файл-PID. - [ADR-0002: Harness-neutral skill bundle](docs/architecture/decisions/0002-harness-neutral-skill-bundle.md) — репозиторий как маркетплейс, `catalog.json` как источник истины, манифест вендора как адаптер. Нейтральность проверяется гейтом, а не обещается. - [ADR-0003: Split licensing](docs/architecture/decisions/0003-split-licensing-prose-and-code.md) — проза бандла под CC BY-ND 4.0, код и метаданные под Apache-2.0; правообладатель назван, атрибуция в титрах — просьба, а не условие. +- [ADR-0004: Engine tool surface out of skill content](docs/architecture/decisions/0004-engine-tool-surface-out-of-skill-content.md) — имена тулсетов, мета-тулов и эндпоинтов движка не попадают в `skills/`: слой экспериментальный, а его отказ молчалив. Проверяется правилом гейта P17. ## Лицензия diff --git a/docs/architecture/decisions/0004-engine-tool-surface-out-of-skill-content.md b/docs/architecture/decisions/0004-engine-tool-surface-out-of-skill-content.md new file mode 100644 index 0000000..13b291c --- /dev/null +++ b/docs/architecture/decisions/0004-engine-tool-surface-out-of-skill-content.md @@ -0,0 +1,97 @@ +--- +status: accepted +date: 2026-09-05 +deciders: project owner +consulted: live measurement of the first-party agent-tool layer in UE 5.8 +informed: future contributors +--- + +# The engine's own tool surface stays out of skill content + +## Context and Problem Statement + +The engine now ships an agent-tool layer of its own: an MCP server, a registry of +toolsets, and skill assets discovered from the project. The obvious move is to write +recipes against it — name the toolset, name the tool, and the reader runs it. + +Two measurements taken against a live editor say why that is a trap. + +**The surface is larger and differently shaped than its own manifest suggests.** The +aggregator plugin lists 21 dependencies. The running server reported **53 registered +toolsets** from 22 plugins, one aggregated dependency contributing none at all and two +non-aggregated plugins contributing two. Any number written into a recipe is a number +about one project's plugin set, not about the engine. + +**The number of visible tools is a project setting, not a property of the engine.** With +tool search on — the default — `tools/list` returns exactly three meta-tools and every +real tool is reached indirectly. With it off, every toolset tool registers natively. A +recipe that says "call tool X" is correct or nonsense depending on one checkbox. + +Both layers are marked Experimental by their author, with an explicit notice that APIs +and data formats may change at any time. That is the vendor telling us the surface is +not a stable address. + +## Decision Drivers + +- **This bundle's subject is what fails silently.** A recipe naming a tool that no longer + exists does not error: the tool is absent, nothing is found, and "nothing found" reads + as "no problem here" — the exact confusion the bundle exists to prevent. +- **ADR-0002 already forbids naming one harness.** An engine build is the same category + of dependency, and the argument for P13 is the argument here. +- **The first party asks for the same thing.** Its published skill-authoring norm lists + *Durable* and *Agnostic* among six properties; this rule is those two, enforced. +- **A promise that lives in prose rots.** P13 was made checkable the hour it was decided. + +## Considered Options + +1. **Write recipes against the native tools.** Cheapest to read, dates fastest. +2. **Name tools but mark them version-bound.** A warning next to a command is read as a + command; and the failure is silent, so the warning arrives too late. +3. **Keep the tool surface out of skill content, and enforce it.** + +## Decision Outcome + +Chosen: **option 3**, as gate rule **P17**. + +Nothing under `skills/` may name a toolset, a meta-tool, an endpoint host:port, or the +Experimental class names of the engine's skill-asset API. Detect recipes may say *what* +to look for; they may not say *which first-party tool to look with*. Everything outside +`skills/` — the catalog, adapters, the gate, the research archive — is unaffected, and +that is where any tool-specific knowledge belongs. + +### Consequences + +- Good: a recipe written today still means something on the next engine build, or fails + loudly by being unrunnable rather than quietly by finding nothing. +- Good: the bundle's neutrality claim now covers both axes it can drift on — the harness + that runs the agent, and the engine that hosts the tools. +- Bad: recipes stay one step further from executable. The reader translates "find fields + with no writer" into whatever their build offers. This is the accepted price. +- Bad: the rule cannot distinguish a tool name from an ordinary word that ends in the + same token. Checked against the whole bundle before landing: zero matches, so the cost + today is zero and any future collision is visible at the moment it is introduced. +- Neutral: knowledge about the native layer is not lost, only relocated. It lives in the + research archive, where it can name builds, ports and classes freely. + +## Confirmation + +Checked at the time of writing, and required for every later change: + +- `python _gate/gate.py .` reports zero violations. *(passing, 17 rules)* +- `python _gate/test_gate.py` reports every rule reddening on its own poisoned fixture, + with a clean baseline and full surface coverage. *(passing, 17/17)* +- The P17 fixture names a toolset, a meta-tool and a `host:port` in one line, and the + rule fires on it. *(verified)* +- The token list produces no match anywhere under `skills/` today. *(verified before the + rule landed; a match would be a new violation, not a false positive to be excused)* + +## Deferred + +- **Whether the bundle ever ships an engine-side adapter** that maps a recipe to whatever + tools the current build offers. That is the natural home for the knowledge this rule + keeps out, and it should be written when a build is worth targeting — not before. +- **Whether skill assets inside the engine become a second delivery channel.** Measured as + workable: an asset created in a live session appeared in the project's skill list with + no editor restart. Also measured as fragile: an asset created with an empty description + is created successfully and then never listed, with no error on either call. A channel + whose failure mode is silent deletion needs a gate of its own before it carries anything. diff --git a/plugins/ue-design-skills/_gate/gate.py b/plugins/ue-design-skills/_gate/gate.py index e06e3b6..0951836 100644 --- a/plugins/ue-design-skills/_gate/gate.py +++ b/plugins/ue-design-skills/_gate/gate.py @@ -49,6 +49,22 @@ HARNESS_TOKENS = ( r'(?= 16, 'rule set shrank; a rule was lost' +assert len(RULES) >= 17, 'rule set shrank; a rule was lost' assert len({r.id for r in RULES}) == len(RULES), 'duplicate rule id' @@ -300,6 +318,40 @@ def check_harness_neutral(root: Path) -> list[Violation]: return out +def check_engine_tool_surface(root: Path) -> list[Violation]: + """P17: nothing under skills/ may name the engine's own agent-tool surface. + + P13 keeps the content free of one *harness*; this keeps it free of one + *engine build*. The distinction matters because the failure looks different: + a harness token tells the reader to run something they do not have, while a + toolset name tells them to run something that used to exist. The second is + quieter -- the tool is missing, the search returns nothing, and "nothing + found" is indistinguishable from "no problem here", which is the exact + failure this bundle exists to document. + + Detect recipes may still say what to look for. They may not say which + first-party tool to look with. + """ + out: list[Violation] = [] + base = root / 'skills' + if not base.is_dir(): + return out + patterns = [re.compile(p) for p in ENGINE_TOOL_TOKENS] + for path in sorted(base.rglob('*')): + if not path.is_file() or path.suffix.lower() not in TEXT_SUFFIXES: + continue + rel = path.relative_to(root).as_posix() + for n, line in enumerate(path.read_text( + encoding='utf-8', errors='replace').splitlines(), 1): + for pat in patterns: + m = pat.search(line) + if m: + out.append(Violation('P17', rel, n, + f'engine tool surface: {m.group(0)}')) + break + return out + + def check_catalog(root: Path) -> list[Violation]: """P14: catalog.json describes every skill without vendor vocabulary. @@ -555,7 +607,8 @@ def check_manifest(root: Path) -> list[Violation]: CHECKS = (check_text_rules, check_top_level, check_junk, check_skills, - check_manifest, check_harness_neutral, check_catalog, + check_manifest, check_harness_neutral, check_engine_tool_surface, + check_catalog, check_id_prefixes) diff --git a/plugins/ue-design-skills/_gate/test_gate.py b/plugins/ue-design-skills/_gate/test_gate.py index 5866cff..39ce196 100644 --- a/plugins/ue-design-skills/_gate/test_gate.py +++ b/plugins/ue-design-skills/_gate/test_gate.py @@ -58,6 +58,14 @@ POISONS: dict[str, tuple] = { # shipped entries vanish from the archive resolver without any count # disagreeing loudly enough to notice. 'P16': ('copyskill', 'skills/sample-skill', 'sample-twin'), + # The engine's own agent-tool layer is Experimental and its surface moves + # between builds: one measured session exposed 53 toolsets, and the count + # of visible tools flips between 3 and hundreds on a single project + # setting. A recipe naming a toolset does not error on the next build, it + # finds nothing -- which reads as "no problem here". + 'P17': ('append', FM, + '\nRun this through the EditorAppToolset via call_tool at ' + 'localhost:8000.\n'), }