# Memory Architecture

# Memory Architecture

Assistant memory and context-injection architecture. For the memory plugin's internal layering contract — the tier/directory map, the frozen-name registry, and the v1/v2 deletion runbooks — see `src/plugins/defaults/memory/AGENTS.md`.

## Tiers

An assistant runs exactly one memory tier, derived by `memoryTier()` in `src/config/memory-tier.ts`:

| Tier  | Selected when                                | Injection source                                 |
| ----- | -------------------------------------------- | ------------------------------------------------ |
| `off` | `memory.enabled === false`                   | none                                             |
| `v3`  | `memory.v3.live === true`                    | v3 lanes + sections (`v3/injector.ts`)           |
| `v2`  | `memory.v2.enabled === true` and v3 not live | v2 activation/router engine (`v2/`)              |
| `v1`  | otherwise                                    | PKB `<knowledge_base>` block (`v1/pkb/`, legacy) |

New workspaces are switched to `v3` at creation (workspace migration 105). `v1` and the `v2` injection engine are in retirement; v3 is the target state.

### Tier predicates

Tier decisions are made through named predicates in `src/config/memory-v3-gate.ts`, never by reading `memory.v2.*` / `memory.v3.*` directly — `memory.v2.enabled` defaults true and typically stays set on v3-live assistants, so a raw read misbehaves under v3. `memoryTier()` is fully derived from those predicates, so the telemetry buckets and the runtime gates can never disagree. The two that shape this document are `usesConceptPageMemory()` (the substrate is live) and `isMemoryV3Live()` (v3 is the injected source).

The full predicate table — what each one is true for and, importantly, exactly which machinery it gates — is in `src/plugins/defaults/memory/AGENTS.md`, which is authoritative. Consult it before gating new code.

The memory plugin is layered into tier directories (`substrate/`, `v1/`, `v2/`, `v3/`) that never import each other — multi-tier composition is confined to a frozen set of spine files, and both rules plus the raw-tier-key ban are enforced by `src/plugins/defaults/memory/__tests__/memory-tier-boundary-guard.test.ts`.

## The concept-page substrate (`substrate/`)

The durable core shared by v2 and v3 is the **concept-page substrate**: markdown articles under `memory/concepts/` (plus the aggregate views `essentials.md`, `threads.md`, `recent.md` and the intake `buffer.md`), their cached page index, and the concept-page Qdrant collection with dense + BM25 sparse vectors. It lives in `src/plugins/defaults/memory/substrate/`; it is memory-v3's foundation, and the v2 injection engine is a second (transitional) consumer.

Substrate activation is a single predicate, `usesConceptPageMemory()` in `src/config/memory-v3-gate.ts`: memory on AND (`memory.v3.live` OR `memory.v2.enabled`). Every substrate gate — the write path, consolidation scheduling, boot-time maintenance, capability seeding, the static `<info>` memory block, and the v1-machinery suppressions — keys on it. When the v2 engine is removed, the predicate collapses to `memory.enabled`.

### Write path

```mermaid
graph LR
    REM["remember tool /<br/>POST /v1/memory/remember"] --> BUF["memory/buffer.md<br/>+ archive/&lt;date&gt;.md"]
    SWEEP["idle sweep (opt-in)"] --> BUF
    BUF --> CONS["memory_v2_consolidate job<br/>(background agent run)"]
    CONS --> PAGES["concept pages<br/>memory/concepts/**"]
    CONS --> VIEWS["essentials / threads / recent"]
    CONS --> REEMBED["memory_v2_reembed →<br/>concept-page Qdrant collection"]
    CONS --> MAINT["memory_v3_maintain<br/>(v3-live follow-up)"]
    ING["POST /v1/memory/ingest<br/>(deterministic batch import)"] --> PAGES
    ING --> REEMBED
    ING --> MAINT
```

- `handleRemember` (`graph/tool-handlers.ts`) appends timestamped bullets to `memory/buffer.md` + the daily archive whenever memory is enabled. Facts may carry `[[slug]]` page hints that consolidation reads first when filing.

- The buffer entry format itself is owned by `buffer-format.ts` at the plugin root: the writer (`formatRememberEntry`) plus the single matcher every reader uses. A fact may span several lines, so the entry and the line are different units, and the readers below (consolidation's cutoff, the injected `<info>` Buffer cap, the Memory tab's pending nodes) all recognize entries through that one matcher rather than their own. An entry opens with a timestamped bullet at column 0 and its body is indented under it, which is what makes the format round-trip: the delimiter is the column-0 bullet shape, so nesting the body keeps fact content from imitating one. Entries written before that nesting existed still parse, since an unindented body line that is not itself entry-shaped is read as a continuation.

- **Consolidation** (`substrate/consolidation-job.ts`) is a background agent conversation that files buffer entries into concept pages and rewrites the aggregate views. The agent never writes the buffer: the job snapshots `buffer.md`, hands the run its pass's entries verbatim in the prompt, and after the run removes exactly those entries itself through `buffer-file.ts` (the module that also owns the append path, so the two writers of the buffer share one protocol). Removal happens only when the run's persisted messages hold a page-writing tool call with a non-error result, the same evidence bar the retrospective's cursor advance uses; a run that wrote nothing, failed, or timed out leaves the buffer intact for the next pass. Entries deferred past the per-run cap and entries appended while the run was in flight are therefore never lost. Scheduling (`maybeEnqueueGraphMaintenanceJobs` in `jobs-worker.ts`):

  - interval-based (`memory.v2.consolidation_interval_hours`, default 8h), skipped below `MIN_BUFFER_LINES_FOR_CONSOLIDATION` (10) **unless** the non-empty buffer has sat unwritten for a full interval (staleness override — a small buffer can never sit unconsolidated forever);
  - size-triggered at `consolidation_max_buffer_lines` (default 100);
  - nudged by the create-memory route after a user-authored save (deduped, failure-backoff-respecting);
  - manual "Run now" via `POST /v1/consolidation/run-now`. Failed runs enter an exponential backoff (transient vs billing curves).
  - The agent writes pages through the file tools, so nothing validates a page at write time. Two corpus-level defects are instead reported by the page index and fed back into the next pass's prompt as repair steps: pages the index could not parse (`PageIndex.parseFailures`) and structural references (`links:`, inline `[[wikilinks]]`, `edges:`) whose target page does not exist (`PageIndex.danglingLinks`). The read-side graph drops a dangling reference silently, so the job also counts them after each run (`danglingLinks` on the outcome, a warn line) without withholding the reindex follow-ups: the pages that were written still become retrievable. The `memory validate` CLI subcommand reports the same list.

- **Ingestion** (`substrate/ingest.ts`, exposed as `POST /v1/memory/ingest`; generated HTTP operation id `memory_ingest_post`, IPC method `memory_ingest`) is the second sanctioned writer of `memory/concepts/`: a deterministic batch import of fully-formed, page-shaped markdown (frontmatter + body) that bypasses the buffer entirely. The bypass is load-bearing, not a shortcut: buffer timestamps have minute precision, so a bulk append lands every entry on one shared minute stamp, the chunker's timestamp cutoff cannot split inside that minute, and its same-minute burst guard falls back to processing the whole buffer in a single oversized run. Ingest writes validated pages directly instead. Purely mechanical: each page is validated and reported individually (a `links:`/`[[wikilink]]`/`edges:` target that is neither on disk nor in the batch is a per-page warning, not a rejection), writes hold the consolidation lock so a batch cannot interleave with a consolidation pass, and a batch that wrote at least one page enqueues the same reindex follow-ups as consolidation (`memory_v2_reembed`, `memory_v3_maintain`). Consolidation remains the only LLM-driven writer.

### Ingestion tracks and provenance

Two user-facing flows feed the ingest route, chosen by the shape of the source material rather than its size:

- **Distilled import** (`skills/assistant-migration`): memory exported from another assistant is parsed, creator-reviewed, and shaped into concept pages. The input is already distilled, so everything approved becomes pages; assistant exports route here regardless of size because they need candidate parsing and creator review.
- **Large-corpus skim** (`skills/memory-corpus-ingest`): the raw dataset (recording archives, document dumps) is cold-stored under a workspace imports directory and never enters the memory corpus. Only a small map of roughly 10 to 50 skim pages is ingested, each pointing back at the raw files, and a drill-in retrieval skill authored alongside searches the cold store on demand.

Both stage pages on disk and write them through the `assistant memory ingest` CLI (`src/cli/commands/memory/memory-ingest.ts`: staging directory, JSON manifest, or stdin; dry-run; overwrite; 200-page batches). The one exception is a migration yielding only a handful of approved facts: assistant-migration saves those through the normal `remember` tool, so they enter via the buffer like any organic capture instead of the ingest route.

Ingested pages carry provenance frontmatter with distinct consumers:

- `source:` marks a page's origin, by convention `import:<provider>` (e.g. `import:chatgpt`), so imported pages stay distinguishable from consolidation-authored ones, which never carry the key.
- `origin_date:` (ISO 8601 date or datetime) declares when the page's content is from and drives `PageIndexEntry.freshAt`, the effective-recency signal: any finite parse is accepted (pre-epoch dates included; an offset-less datetime is read as UTC), and an absent or unparseable value falls back to file mtime. The v3 fresh lane ranks on `freshAt` and the card's `dated` stamp renders it, so a backdated import ranks by what its content is about rather than when it reached disk. `modifiedAt` stays raw mtime because the maintain job's re-embed delta diffs on it. Synthetic entries (skills, CLI commands) carry `freshAt: null` and are skipped by recency ranking.

### Read paths

- **v3 (live)**: per-turn lane selection over concept pages (dense/sparse retrieval via `substrate/sim.ts` over the concept-page collection, learned edges, entity/hot/fresh/core sets). The injection unit is the SECTION: the selector pool lists a page once per matched section, and `v3/injector.ts` renders each selected page's selected sections (its lead when it was selected with none) into a `<memory>` block, net-new sections only, deduped per `(page, section)` through `memory_v3_injected_sections` and bounded by a recency prune valve with no lane exemptions. Section bodies are backslash-escaped where a line would read as a block header, so page text never forges a section boundary, and a pruned section is dropped from any pointer that named it. Frozen section blocks stay on historical user messages so the provider prefix stays cacheable. Re-selected sections that are already resident are listed, paths only, in a per-turn `<memory_pointer>` block that stays on the user message it was sent with; a new pointer is added only on the new tail, so older messages are not rewritten. The static `<info>` block (`substrate/static-context.ts`: essentials/threads/recent/buffer) also injects whenever the substrate is active.
- **v2 (transitional)**: activation/router engine in `v2/` (`activation.ts`, `router.ts`, `reranker.ts`, `injection.ts`) selects concept pages per turn. Suppressed at assembly when v3 is live. Gated on `memory.v2.enabled` and deleted with it.
- **v1 (legacy)**: PKB retrieval over the v1 Qdrant collection. All v1 machinery (graph extraction, summarization, PKB indexing/filing, PKB injection) is suppressed while the substrate is active.

#### Live voice front-door memory

The live voice front door does not await current-turn memory retrieval. Its prompt hook skips legacy graph retrieval, and both v3 injectors skip orchestration for `voiceFrontDoor`. Frozen sections from prior turns and the static substrate context remain available. When the answer depends on a saved personal fact that is absent from that context, the front-door rule escalates instead of guessing.

The escalated leg normally runs the memory pipeline before the quality model. During screen sharing, the front door can emit `[ESCALATE_SCREEN]` for an annotation or immediate on-screen action answerable from the screen and current conversation. The bridge skips fresh retrieval only when that verdict and an active screen share are both present. This per-turn policy gates the legacy hook and both v3 injectors, including re-injection, and clears at turn release. Static context and historical memory blocks remain available. Requests needing saved facts or preferences absent from context, uncertain classifications, and ordinary `[ESCALATE]` verdicts retain retrieval. V3 retrieval routes on the latest visible caller message, ignoring the hidden continuation message used to start that leg. The selector uses low effort to keep retrieval latency bounded. This keeps current-turn memory work off the front-door TTFT path without maintaining speculative cross-leg state.

### Boot-time maintenance

`substrate/boot-maintenance.ts`, invoked from the memory plugin's startup path: skill + CLI-command capability seeding into the concept-page collection, BM25 corpus-stats rebuild, and collection schema reconcile/reembed. All gated on `usesConceptPageMemory()`.

## Memory graph (visualization)

`GET /v1/memory-graph` (`graph-topology/build-memory-graph.ts`) renders the substrate as a backend-agnostic node/edge graph for the web Memory tab: concept pages as nodes, authored links + learned co-selection associations as edges, and `memory/buffer.md` entries as `pending` nodes (`graph-topology/pending-buffer.ts`) so a just-saved fact appears before consolidation files it. Gated on `isV3TierActive()` — memory enabled and `memory.v3.live` — which is also the source of the cheap `graph_supported` bit on `GET /v1/memory/stats`, so the advertised capability and the actual build can never drift. The same predicate gates procedural-memory-as-skills, so both v3-tier features honor the Memory opt-out identically. `GET /v1/memory-graph-node` serves node detail, including `buffer:` ids for pending entries.

`GET /v1/memory/stats` also reports `tier` (`memoryTier()`: `off` / `v1` / `v2` / `v3`), which is what lets the web Memory tab explain an unavailable graph instead of stating a bare "not available": `off` is the user's own Memory opt-out and points at Settings, while `v1`/`v2` point at the v3 migration. `graph_supported` is exactly `tier === "v3"`, so the capability bit and its explanation are derived from one gate.

## Capture beyond `remember`

- **Retrospective** (`memory-retrospective-*.ts`): periodic per-conversation review pass that saves what wasn't captured in the moment (and, on v3-live assistants, authors procedural skills).
- **Sweep** (`substrate/sweep-job.ts`, `sweep_enabled` substrate tunable, default off): idle-debounced extraction of recent messages into the buffer.

## Config namespaces

Substrate tuning resolves from `memory.substrate.*` with fallback to the historical `memory.v2.*` keys: every substrate tunable (consolidation intervals/limits, embedding + retrieval weights, BM25 parameters, sweep toggle) is optional under `memory.substrate`, and `resolveSubstrateTuning` (`substrate/tuning.ts`) — the substrate's single config choke point — falls back per key to the `memory.v2` twin, which supplies the effective defaults and which the v2 injection engine also reads. v2 engine-only keys (activation weights, router, rerank) live only under `memory.v2.*`, and v3 lane tuning under `memory.v3.*`. The `memory.v2.enabled` flag gates only the v2 injection engine's turn-time selection; the substrate runs whenever `usesConceptPageMemory()` holds.

`memory.v3.poolLog.captureInput` (off by default) makes the v3 selector persist its exact per-turn input beside the always-written pool audit (`memory_v3_pool_inputs`, `memory_v3_pool_texts`), for offline selector evaluation and training data; the memory plugin `AGENTS.md` documents the tables.

`src/plugins/defaults/memory/AGENTS.md` carries the details that matter when you touch this: the three substrate keys whose names differ from their v2 twin, the three disjoint places the dense/sparse sum-to-1 invariant is checked, and why workspace migration 135 copies only explicitly-set NON-DEFAULT `memory.v2` tunables into `memory.substrate`.