ADR-701: Neighbourhood matching: masking, relative scores, lookup on request and a learning loop from run logs¶
Summary¶
- Decided: the ways engine moves toward matching within an intentionally designed neighbourhood, queried and judged as needed, with run logs as its tuning corpus. This record decides the parts the measurements in ADR-700 support, on today's identity (path ids), today's parent relation (directories) and today's reference edges (See Also). Disabled ways are masked at scan time before any competition. Every scan logs its top candidates with share and margin, and the event log rotates. Lint checks that See Also targets resolve, refuses symlinks in ways roots and keeps basenames unique. The judge can receive share, margin and a band label, adopted after an A/B evaluation.
ways-mcpgains lookup on request. A body corpus stored as a binary sidecar serves body confirmation and, if evaluation confirms it, a fused ranking score. The run logs produce reviewed proposals; nothing refits on its own. Stable identity, extra parents and frontmatter edges are proposed separately in ADR-702. - Trades away: more logging per scan, a lint pass that can flag existing user and project ways, a second corpus file once the body sidecar ships, and a judge input that must earn its place in an A/B.
- One-way? No. Every element is additive. With the new flags off and the sidecar absent, behaviour matches today apart from masking.
- Probes: Confident (non-breaking): existing way files, configurations and installs keep working unchanged, and nothing new is required of a way author outside the core corpus. Not confident (judge-scores): the judge should see share and margin, not raw cosine, and only if an A/B shows it judges better with them.
- Inversion: between scoring a flat list of ways independently and holding the graph in a database the engine owns. The decision keeps files as the only source and builds every vector, parameter and log-derived proposal from them.
Context¶
The matcher scores each prompt against one alias vector per way (description + vocabulary) and thresholds a calibrated cosine (ADR-156), with late interaction and body confirmation on multi-sentence surfaces (ADR-160) and a relevance judge on the result (ADR-196). ADR-127 rejected body text in place of the alias and named graph-aware routing as the forward path. ADR-125 named the corpus an authored disclosure graph.
ADR-700 measures the current matcher on a 332-row golden set. Body content in place of the alias loses 19 points of top-1; a light fusion beside the alias gains about 2.5, short of significance. Raw cosine is weak in its middle range, while margin and softmax share separate right from wrong cleanly. Routing loses 6 to 7 points of top-1 per doubling of the corpus. Disabled ways still compete in the late-interaction pass. The event log records fires, near-misses and judge verdicts, but not the ranked candidates.
agent-ways is moving toward knowledge work beyond software, which means more domains and a larger corpus. A review of an earlier draft of this record found its identity and multi-parent machinery speculative at today's corpus size and move rate, and found two defects: masks computed at build, though project toggles are only known at scan time, and per-way toggles in user scope, which ADR-131 rejected. That machinery moved to ADR-702.
Decision¶
1. Masking and toggles¶
- Disabled ways are removed before any scoring competition, single-vector or late-interaction. In late interaction, rows for ways outside the enabled set are dropped before softmax share is computed and before survivors are chosen for confirmation.
- The enabled set is computed at scan time from
disabled_domains(user scope) and per-way toggles (project scope, ADR-131). - The project
ways:map also accepts a path prefix, such assoftwaredev/code/supplychain/*: false, which disables every way under it. A toggle on a way itself overrides a prefix. - Both corpora embed every way, disabled or not, so enabling one again needs no rebuild.
2. Logging and history¶
- Every scan, on the prompt and the task lane, logs its top 5 candidates with cosine, share and margin, and once per event the lane, the basis of the scores and whether the body sidecar was used.
- History is archived, not deleted. Before the size cap or the age rotation removes lines from
events.jsonl, they are written to a compressed archive file dated by day in the same state directory. The size cap bounds only the live file. ways.event_retention_days(user scope only, default 365) expires archive files, not live lines.- If archiving keeps failing (a full disk, no write permission), preservation wins until the live file reaches four times the size cap; past that the cap drops its oldest lines without archiving them and logs one
event_log_droppedevent with the byte count and the reason. - The readers of the event log (introspection, tuning, stats) read the archives as well as the live file.
- The unit of history is the turn. What history is worth keeping for is the decisions made over turns, not its size in bytes. Each scan writes one decision record holding the context once (session, agent, lane, surface, turn position), its top candidates with their scores, and what became of each: fired, re-disclosed, held by the refire window, held by the keyword floor, a near miss, the judge's probability and verdict, and pulls. This record absorbs the
scan_candidatesevent. - Decision records are kept by count of turns, not by bytes:
ways.decision_retention_turns(user scope only, default 50,000, chosen from measured turns per day to cover at least a year) sets how many turns the live file holds. They are archived like the event stream when they leave the live file. - Tuning, stats and the learning loop of §8 read decision records. The detailed event stream stays a debugging trail for introspection, bounded by the size cap and archived as above.
- Rotation claims its daily slot atomically, anchors its cutoff to the newest line already in the log, and never lets a
judge_callline end its scan.
3. Lint¶
- See Also targets resolve to a way, looked up across every root a session reads (project, user, core), so a project or user way may point at a core way. Linting a subtree or a single file resolves against the ways root that contains it.
- No symlinks inside a ways root: the scanner would read a linked way as a second way.
- Basenames are unique within a root (ADR-110 §7); the two colliding basenames are renamed.
- These checks are errors for the core corpus and warnings for user and project ways, so an upgrade alone turns no one's lint red.
4. Scores for ranking and the judge¶
- Whether a way fires stays decided by calibrated cosine (ADR-156), with one calibration until a fused score ships.
- Share is a softmax at τ 0.08 over the top 8 enabled candidates of the scan's lane, and margin is the gap to the next enabled candidate.
- Behind a flag, the judge receives for each candidate its share, margin, a band label (strong, uncertain, weak, with fixed bounds from ADR-700 §4) and the route already in its input. The flag turns on by default only after an A/B on the golden set and on logged verdicts shows the judge rejects more irrelevant fires without losing relevant ones.
5. Lookup on request¶
ways-mcp (ADR-501) gains, running in-process until the daemon's search service exists:
ways_search(query): candidates with route, share, margin and matched sectionways_read(id)ways_neighbors(id): the parent, children, See Also edges and nearest semantic neighbours fromways author siblings, each labelled with its kind
A pull always returns the way, including inside its re-disclosure suppression window (ADR-126), where injection would hold it back, and regardless of the way's scope:, since the server cannot tell which agent called it; the result names the way's scope. A pull needs no judge, because the agent chose it. A pulled way stamps disclosure for the agent that pulled it, so injection does not repeat it on that agent's next turn. The stamp is applied by a PostToolUse hook on the read tool, whose payload names the session and agent; the server itself stamps nothing. Way ids are validated as relative path segments and must resolve inside a ways root, so a way reached through a symlink is refused, consistent with §3's refusal of symlinks inside a ways root. A failed read stamps nothing, and a refused stamp is logged with its reason. A project switched off with enabled: false serves no pulls. Lookup resolves the project as the hooks do. Each pull is logged as way_pulled. A pull inside the suppression window is logged as an out-of-band read, with out_of_band: true and the epoch distance since the last disclosure.
6. Body corpus¶
- Each way's prose split at headings, with frontmatter, code blocks, tables, HTML comments and the See Also section removed, and sections longer than 120 words split. About 4.7 vectors per way.
- Stored as a binary sidecar beside the alias corpus, with a manifest recording each way's content hash, so the in-process path can load it. At today's size it is about 1 MB.
- Body confirmation reads it in place of embedding the first 8 sentences on every prompt, once a measurement shows confirmation is no worse with it.
- Body score in ranking (
alias + 0.25 × best section) is adopted only after an evaluation on held-out real prompts shows a gain with no loss in separating relevant prompts from unrelated ones. Once adopted, confirmation uses a section other than the one that contributed to the ranking score, and a way with one section or none is confirmed against its alias.
7. Degradation¶
| State | Matching | Body confirmation | Precision filter |
|---|---|---|---|
| Embedder, sidecar complete, judge reachable | keyword lane and alias matcher, with the fused score once adopted | sidecar | judge |
| Judge unreachable or past its deadline | as above | sidecar | matcher gates |
| Sidecar missing or incomplete | keyword lane and alias matcher | embedded per call | judge if reachable |
| No embedder | keyword lane only | none | none |
The sidecar is used only when its manifest covers every enabled way the alias corpus holds, at the alias corpus's content hashes, and the installed embedder can return chunk vectors (a way absent from the alias corpus cannot win a chunk, so it never needs confirming); otherwise the scan uses alias scores alone. Fused and alias-only scores are never mixed in one ranking. Each scan logs which state it ran in, and ways status reports the current state and the reason for any drop. The daemon's search service (ADR-502 §2, #668) holds the same files when it lands; until then the hook loads them.
8. Run logs as the learning corpus¶
- Pulled and injected disclosures are told apart. A pull of a way injection did not fire on that turn is a recall miss. An out-of-band read is evidence that the way's
refire:fraction is longer than the work needs. - Logs produce proposals that land as pull requests: vocabulary and body changes from repeated near-misses, boundary or edge changes from repeated confusions and from semantic neighbours with no See Also between them,
refire:changes from out-of-band reads, and changes to the calibration probe set. - Nothing refits on its own. Calibration stays fitted from the reviewed probe set at corpus build, so the fire decision does not drift per install.
- Prompt text stays out of the event log. When the operator opts in, a local consumer of the ways sensor (ADR-199) carries it to an evaluation store on the machine.
- Judge verdicts are weak labels. Fitting and evaluation hold out by session or date.
9. Evaluation travels with the corpus¶
Each new core way ships with two golden prompts, one direct and one situational, and lint checks that they exist. The harness in experiments/content-corpus/ becomes the standing evaluation, run against the golden set and, when available, the real-prompt store.
10. Compatibility¶
No field becomes required. Configuration keys keep their meaning; the prefix form in ways: is new and optional. The sidecar is a new file that a binary without support ignores. With the judge flag off and no sidecar, an upgraded install behaves as today apart from masking, which can let an enabled way fire where a disabled one crowded it out.
Increments¶
- Masking at scan time and prefix toggles; top-5 logging with share and margin; event-log rotation; the three lint checks and the basename renames.
- Share, margin and band to the judge behind a flag; the A/B that decides the flag's default.
ways-mcplookup in-process,way_pulledand out-of-band reads.- Body sidecar build; body confirmation reads it after measurement.
- Per-turn decision records with retention counted in turns; tune and stats read them. Proposals from logs through
ways tuneandways suggest; the opt-in real-prompt store through the ways sensor. - Gated on evaluation: body score in ranking; a hubness penalty fitted at build against the probe set; admitting each chunk's top-ranked way in late interaction (ADR-700 §12). Competition within a subtree first is dropped: region scores from way scores and independent region vectors both failed to beat flat competition (ADR-700 §9, §11).
- When #668 lands: the daemon's search service holds the corpora and the sidecar.
Consequences¶
Positive¶
- Disabling unrelated domains narrows the competition, which by ADR-700 §6 raises accuracy and margins.
- The judge can see scores that separate right from wrong, once an A/B shows they help.
- Body confirmation stops embedding on every prompt and covers the whole body.
- Lookup on request gives the agent a path to ways injection missed, and records each miss as a label.
- Tuning moves from hand estimates to proposals drawn from logged use, still reviewed.
Negative¶
- Per-scan logging grows by the candidate list, held in check by rotation.
- New lint warnings can appear on existing user and project ways.
- The sidecar must stay complete or scans fall back to alias scores.
Neutral¶
- Files remain the only source of truth.
- ADR-702, if accepted, changes identity and edges without changing this record's mechanisms.
- ADR-188's move of tool-lane matching to PostToolUse leaves prompt and task surfaces as the main users of the body corpus.
Alternatives Considered¶
- Body text in place of the alias. Rejected by ADR-127 and ADR-700 §1.
- Maximum or reciprocal-rank fusion of alias and body. Rejected by ADR-700 §1: both lose to the alias alone.
- Dot product or Euclidean distance. Rejected: with unit vectors both rank identically to cosine (ADR-700 §5).
- Centred scores for the fire decision. Rejected: centring weakens the separation from unrelated prompts (ADR-700 §5).
- Raw cosine to the judge. Rejected: it is right about half the time in its middle range (ADR-700 §4).
- Excluding disabled ways at build time. Rejected: enabling one again would need a rebuild, and masking costs little.
- Masks computed at corpus build. Rejected: project toggles are only known at scan time.
- Automatic refit of calibration or bands from logs. Rejected: the fire decision would drift silently and differ between installs, and the logs are biased toward what the matcher already surfaced.
- The body corpus only in the daemon. Rejected: as a binary sidecar it is smaller than the alias JSON the hook already parses.
- Delete expired event lines, with a larger cap. Rejected: the event history is the learning corpus of §8, the size cap had already deleted six weeks of it (ADR-700 §8), and a deleted line cannot be recovered after a clock or rotation fault.
- Hold the graph in a database. Rejected: the corpus is small, authored and reviewed, and writes arrive through pull requests. ADR-702 records how its integrity is enforced on files.
- UUID identity,
parents:andrelated:in this record. Moved to ADR-702: nothing here depends on them, and at the measured move rate (12 renames since April, in four commits) they are not needed yet.
Note (2026-10-22): ADR-188 no longer moves tool-lane matching to PostToolUse¶
The Neutral consequences say ADR-188's move of tool-lane matching to PostToolUse leaves prompt and task surfaces as the main users of the body corpus. That move belonged to ADR-188's first draft, whose premise #528 overturned on 2026-09-29: PreToolUse context is delivered. ADR-188 was rewritten in place. Tool-lane ways fire on PreToolUse, and the Bash surface keeps a meaning-based lane whose matcher reads the call's verbs and description and whose candidates the relevance judge gates, with approved ways delivered through ADR-606's deferred pipeline. The matcher ADR-707 measured for that lane scores way aliases, not bodies, so the conclusion stands for it: prompt and task surfaces remain the main users of the body corpus.