mirror of
https://github.com/sortedcord/omnia.git
synced 2026-07-22 03:52:48 +05:30
refactor(memory,gui): Streamline memory archtiecture and terminology
- Short term memory/ recent events are collectively now Cognitive Buffer (Tier 1) - Long term memory/ memory ledgers are just Memory Ledger (Tier 2) - Dossiers stay Dossiers (Tier 3)
This commit is contained in:
10
README.md
10
README.md
@@ -91,7 +91,7 @@ Access the application locally at `http://localhost:3000`.
|
||||
|
||||
### The Actor Agent
|
||||
|
||||
Each entity takes turns through an **Actor Agent** that receives a strictly epistemically bounded prompt: its own attributes (public, plus private ones explicitly granted to itself), its subjective memory buffer, the entities co-present at its location, and the current moment. Nothing else. The actor responds with free narrative prose.
|
||||
Each entity takes turns through an **Actor Agent** that receives a strictly epistemically bounded prompt: its own attributes (public, plus private ones explicitly granted to itself), its **Cognitive Buffer**, the entities co-present at its location, and the current moment. Nothing else. The actor responds with free narrative prose.
|
||||
|
||||
Prose is decoded into typed intents:
|
||||
|
||||
@@ -121,8 +121,8 @@ Space is a graph: `world → region → location → point of interest`, connect
|
||||
|
||||
### Memory Tiers
|
||||
|
||||
- **Verbatim Buffer (implemented):** Per-character subjective event log. Every entry is stored from the owner's perspective actors resolved through the owner's alias map, outcomes attached — and recalled with naturalized time phrasing.
|
||||
- **Vector Archive (implemented):** Summarized, embedded memory entries for semantic retrieval, keeping verbatim quotes only for high-salience lines.
|
||||
- **Cognitive Buffer (implemented):** Per-character subjective event log. Every entry is stored from the owner's perspective actors resolved through the owner's alias map, outcomes attached — and recalled with naturalized time phrasing.
|
||||
- **Memory Ledger (implemented):** Summarized, embedded memory entries for semantic retrieval, keeping verbatim quotes only for high-salience lines.
|
||||
- **Dossier (planned):** Each observer's subjective beliefs about another character.
|
||||
|
||||
Memory is per-character on purpose: recall is testimony from a vantage point, which is what makes interrogating two witnesses interesting.
|
||||
@@ -156,14 +156,14 @@ The finish line for the first milestone is small on purpose. `v0` is almost on t
|
||||
- [x] Typed intent pipeline: `dialogue` / `action` / `monologue`, decoded from free prose.
|
||||
- [x] World Architect: LLM validation plus time-delta generation, end-to-end for single actions.
|
||||
- [x] Actor Agent with epistemically-bounded prompts (self, memory, co-located entities, subjective time).
|
||||
- [x] Verbatim memory buffer with per-observer subjective serialization and alias resolution.
|
||||
- [x] Verbatim Cognitive Buffer with per-observer subjective serialization and alias resolution.
|
||||
- [x] Spatial location graph (data model; perception is co-location only).
|
||||
- [x] Scenario loader (JSON → SQLite) and a playable CLI loop with human or LLM actors.
|
||||
|
||||
**[The `v0` Milestone:](https://github.com/sortedcord/omnia-consolidated/milestone/1)**
|
||||
|
||||
- [x] Two hand-authored NPCs live in one location, playable via CLI.
|
||||
- [x] Each has buffer and vector-archive memory and recalls something said a few turns earlier.
|
||||
- [x] Each has Cognitive Buffer and Memory Ledger memory and recalls something said a few turns earlier.
|
||||
- [x] One NPC knows a fact the other does not and, provably by testing, will not leak it.
|
||||
- [x] The Architect processes at least one non-trivial action per exchange with a visible state change.
|
||||
- [x] The whole thing persists to a SQLite file and reloads identically.
|
||||
|
||||
2
apps/gui/next-env.d.ts
vendored
2
apps/gui/next-env.d.ts
vendored
@@ -1,6 +1,6 @@
|
||||
/// <reference types="next" />
|
||||
/// <reference types="next/image-types/global" />
|
||||
import "./.next/types/routes.d.ts";
|
||||
import "./.next/dev/types/routes.d.ts";
|
||||
|
||||
// NOTE: This file should not be edited
|
||||
// see https://nextjs.org/docs/app/api-reference/config/typescript for more information.
|
||||
|
||||
@@ -177,13 +177,13 @@ export function ConfigView() {
|
||||
{
|
||||
key: "handoff",
|
||||
label: "Memory Handoff Engine",
|
||||
desc: "Promotes entities' working memories to the long-term Ledger via LLM summarization and pruning.",
|
||||
desc: "Promotes entities' Cognitive Buffer entries to the Memory Ledger via LLM summarization and pruning.",
|
||||
type: "generative",
|
||||
},
|
||||
{
|
||||
key: "embeddings",
|
||||
label: "Text Embeddings Generator",
|
||||
desc: "Generates vector embeddings for long-term memory retrieval.",
|
||||
desc: "Generates vector embeddings for Memory Ledger retrieval.",
|
||||
type: "embedding",
|
||||
},
|
||||
].map((task) => (
|
||||
|
||||
@@ -113,17 +113,17 @@ export function HandoffModal({ entry, onClose }: HandoffModalProps) {
|
||||
{chunks.length === 0 ? (
|
||||
<div className="border border-dotted border-border/30 p-8 text-center bg-card text-muted-foreground rounded">
|
||||
<p className="text-sm">
|
||||
No memories were promoted to the Ledger during this turn.
|
||||
No memories were promoted to the Memory Ledger during this
|
||||
turn.
|
||||
</p>
|
||||
<p className="text-xs mt-1">
|
||||
All working memory buffer entries were summarized or
|
||||
forgotten.
|
||||
All Cognitive Buffer entries were summarized or forgotten.
|
||||
</p>
|
||||
</div>
|
||||
) : (
|
||||
<div className="space-y-4">
|
||||
<h3 className="text-xs font-semibold uppercase tracking-wider text-muted-foreground font-mono">
|
||||
Ledger Additions
|
||||
Memory Ledger Additions
|
||||
</h3>
|
||||
{chunks.map((chunk: any, index: number) => (
|
||||
<div
|
||||
|
||||
@@ -167,7 +167,8 @@ export function InteractView({
|
||||
Handoff triggered for {entry.entityName}
|
||||
</AlertTitle>
|
||||
<AlertDescription>
|
||||
Memories were transferred from Buffer to Memory Ledger
|
||||
Memories were transferred from Cognitive Buffer to Memory
|
||||
Ledger
|
||||
</AlertDescription>
|
||||
</div>
|
||||
<AlertAction>
|
||||
|
||||
@@ -30,8 +30,8 @@ export function PromptModal({ entry, onClose }: PromptModalProps) {
|
||||
userContext: string,
|
||||
inputTokens: number,
|
||||
) => {
|
||||
const recentHeader = "=== RECENT EVENTS ===";
|
||||
const ledgerHeader = "=== YOUR MEMORIES ===";
|
||||
const recentHeader = "=== COGNITIVE BUFFER ===";
|
||||
const ledgerHeader = "=== MEMORY LEDGER ===";
|
||||
|
||||
const recentIdx = userContext.indexOf(recentHeader);
|
||||
let worldStr = userContext;
|
||||
@@ -54,14 +54,14 @@ export function PromptModal({ entry, onClose }: PromptModalProps) {
|
||||
{ label: "System Prompt", type: "system", content: systemPrompt },
|
||||
{ label: "World Info", type: "world", content: worldStr },
|
||||
{
|
||||
label: "Recent Events",
|
||||
label: "Cognitive Buffer",
|
||||
type: "events",
|
||||
content: recentStr || "(No recent events.)",
|
||||
content: recentStr || "(No cognitive buffer entries.)",
|
||||
},
|
||||
{
|
||||
label: "Long-Term Memories",
|
||||
label: "Memory Ledger",
|
||||
type: "memories",
|
||||
content: ledgerStr || "(No long-term memories.)",
|
||||
content: ledgerStr || "(No memory buffer entries.)",
|
||||
},
|
||||
];
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@ import type { SimSession } from "./types";
|
||||
|
||||
/**
|
||||
* Runs the HandoffEngine for every agent entity that has accumulated enough
|
||||
* buffer entries to warrant a handoff (compression to long-term memory).
|
||||
* buffer entries to warrant a handoff (compression to the Memory Ledger).
|
||||
*/
|
||||
export async function runHandoffResolution(session: SimSession): Promise<void> {
|
||||
const worldState = session.coreRepo.loadWorldState(session.worldInstanceId);
|
||||
@@ -49,7 +49,7 @@ export async function runHandoffResolution(session: SimSession): Promise<void> {
|
||||
turn: session.turn,
|
||||
entityId: entity.id,
|
||||
entityName,
|
||||
narrativeProse: `Handoff triggered for ${entityName}: memories were transferred from Buffer to Memory Ledger`,
|
||||
narrativeProse: `Handoff triggered for ${entityName}: memories were transferred from Cognitive Buffer to Memory Ledger`,
|
||||
intents: [],
|
||||
timestamp: worldState.clock.get().toISOString(),
|
||||
isHandoff: true,
|
||||
|
||||
@@ -33,17 +33,17 @@ export type ActorResponse = z.infer<typeof ActorResponseSchema>;
|
||||
*
|
||||
* The prompt is strictly epistemically bounded: the entity only sees what
|
||||
* it is allowed to see (public attributes + private attributes explicitly
|
||||
* ACL'd to it), its own recent memory buffer, and the entities co-located
|
||||
* ACL'd to it), its own Cognitive Buffer, and the entities co-located
|
||||
* with it. System UUIDs are surfaced as subjective aliases.
|
||||
*/
|
||||
export class ActorPromptBuilder {
|
||||
/**
|
||||
* @param bufferRepo Used to fetch the actor's recent memory. Optional —
|
||||
* @param bufferRepo Used to fetch the actor's Cognitive Buffer. Optional —
|
||||
* if absent, the memory section is omitted.
|
||||
* @param ledgerRepo Used to fetch long-term memories. Optional.
|
||||
* @param memoryLimit Maximum number of recent buffer entries to inject.
|
||||
* @param ledgerRepo Used to fetch Memory Ledger entries. Optional.
|
||||
* @param memoryLimit Maximum number of recent Cognitive Buffer entries to inject.
|
||||
* Defaults to 20.
|
||||
* @param ledgerLimit Maximum number of long-term memories to retrieve.
|
||||
* @param ledgerLimit Maximum number of Memory Ledger entries to retrieve.
|
||||
* Defaults to 5.
|
||||
*/
|
||||
constructor(
|
||||
@@ -111,13 +111,13 @@ Guidelines:
|
||||
}
|
||||
}
|
||||
|
||||
// --- Recent memory ---
|
||||
// --- Cognitive Buffer ---
|
||||
const memorySection = this.buildMemorySection(entity, recentEntries, now);
|
||||
if (memorySection) {
|
||||
sections.push(memorySection);
|
||||
}
|
||||
|
||||
// --- Recalled Long-Term memory ---
|
||||
// --- Recalled Memory Ledger ---
|
||||
const ledgerSection = this.buildLedgerSection(
|
||||
worldState,
|
||||
entity,
|
||||
@@ -139,7 +139,7 @@ Guidelines:
|
||||
if (!this.bufferRepo) return null;
|
||||
|
||||
if (entries.length === 0) {
|
||||
return `=== RECENT EVENTS ===\n(No recent events recorded.)`;
|
||||
return `=== COGNITIVE BUFFER ===\n(No recent events recorded.)`;
|
||||
}
|
||||
|
||||
const recent = entries.slice(-this.memoryLimit);
|
||||
@@ -159,7 +159,7 @@ Guidelines:
|
||||
groupedLines.push(` - ${serialized}`);
|
||||
}
|
||||
|
||||
return `=== RECENT EVENTS ===\n${groupedLines.join("\n")}`;
|
||||
return `=== COGNITIVE BUFFER ===\n${groupedLines.join("\n")}`;
|
||||
}
|
||||
|
||||
private buildLedgerSection(
|
||||
@@ -263,6 +263,6 @@ Guidelines:
|
||||
}
|
||||
}
|
||||
|
||||
return `=== YOUR MEMORIES ===\n${groupedLines.join("\n")}`;
|
||||
return `=== MEMORY LEDGER ===\n${groupedLines.join("\n")}`;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -4,7 +4,7 @@ import { WorldState, Entity } from "@omnia/core";
|
||||
import { BufferRepository, LedgerRepository } from "@omnia/memory";
|
||||
import { ActorPromptBuilder } from "../src/actor-prompt-builder";
|
||||
|
||||
describe("ActorPromptBuilder with Long-Term Memory Integration", () => {
|
||||
describe("ActorPromptBuilder with Memory Ledger Integration", () => {
|
||||
let db: Database.Database;
|
||||
let bufferRepo: BufferRepository;
|
||||
let ledgerRepo: LedgerRepository;
|
||||
@@ -31,7 +31,7 @@ describe("ActorPromptBuilder with Long-Term Memory Integration", () => {
|
||||
db.close();
|
||||
});
|
||||
|
||||
it("should inject both recent memory and recalled long-term memory with subjective aliases resolved", () => {
|
||||
it("should inject both Cognitive Buffer and recalled Memory Ledger entries with subjective aliases resolved", () => {
|
||||
const world = new WorldState(
|
||||
"world-123",
|
||||
new Date("2024-01-10T12:00:00.000Z"),
|
||||
@@ -60,7 +60,7 @@ describe("ActorPromptBuilder with Long-Term Memory Integration", () => {
|
||||
},
|
||||
});
|
||||
|
||||
// 2. Populate ledger repository (long-term memory)
|
||||
// 2. Populate ledger repository (Memory Ledger)
|
||||
ledgerRepo.save({
|
||||
id: "ledger1",
|
||||
ownerId: "alice",
|
||||
@@ -76,12 +76,12 @@ describe("ActorPromptBuilder with Long-Term Memory Integration", () => {
|
||||
const builder = new ActorPromptBuilder(bufferRepo, ledgerRepo, 20, 5);
|
||||
const { userContext } = builder.build(world, alice);
|
||||
|
||||
// Check recent memory exists
|
||||
expect(userContext).toContain("=== RECENT EVENTS ===");
|
||||
// Check Cognitive Buffer exists
|
||||
expect(userContext).toContain("=== COGNITIVE BUFFER ===");
|
||||
expect(userContext).toContain("Alice greets Bob");
|
||||
|
||||
// Check long-term memory exists
|
||||
expect(userContext).toContain("=== YOUR MEMORIES ===");
|
||||
// Check Memory Ledger exists
|
||||
expect(userContext).toContain("=== MEMORY LEDGER ===");
|
||||
// Bob should be resolved to Strider in the ledger content
|
||||
expect(userContext).toContain("alice met Strider at the tavern.");
|
||||
expect(userContext).toContain('Quote: "I am a ranger."');
|
||||
@@ -98,7 +98,7 @@ describe("ActorPromptBuilder with Long-Term Memory Integration", () => {
|
||||
const builder = new ActorPromptBuilder(bufferRepo, ledgerRepo, 20, 5);
|
||||
const { userContext } = builder.build(world, alice);
|
||||
|
||||
expect(userContext).toContain("=== RECENT EVENTS ===");
|
||||
expect(userContext).not.toContain("=== YOUR MEMORIES ===");
|
||||
expect(userContext).toContain("=== COGNITIVE BUFFER ===");
|
||||
expect(userContext).not.toContain("=== MEMORY LEDGER ===");
|
||||
});
|
||||
});
|
||||
|
||||
@@ -33,7 +33,7 @@ export function getMemorySectionLength(
|
||||
now: Date,
|
||||
): number {
|
||||
if (entries.length === 0) {
|
||||
return `=== RECENT EVENTS ===\n(No recent events recorded.)`.length;
|
||||
return `=== COGNITIVE BUFFER ===\n(No recent events recorded.)`.length;
|
||||
}
|
||||
|
||||
const groupedLines: string[] = [];
|
||||
@@ -52,7 +52,7 @@ export function getMemorySectionLength(
|
||||
groupedLines.push(` - ${serialized}`);
|
||||
}
|
||||
|
||||
return `=== RECENT EVENTS ===\n${groupedLines.join("\n")}`.length;
|
||||
return `=== COGNITIVE BUFFER ===\n${groupedLines.join("\n")}`.length;
|
||||
}
|
||||
|
||||
function checkSceneExit(entity: Entity, bufferEntries: BufferEntry[]): boolean {
|
||||
@@ -227,7 +227,7 @@ export class HandoffEngine {
|
||||
.join("\n---\n");
|
||||
|
||||
const systemPrompt = `
|
||||
You are the memory Handoff Engine. Your task is to process a list of recent working memory buffer entries for an entity and select which memories to promote to the long-term Ledger, and which to forget or summarize.
|
||||
You are the memory Handoff Engine. Your task is to process a list of Cognitive Buffer entries for an entity and select which memories to promote to the Memory Ledger, and which to forget or summarize.
|
||||
|
||||
Instructions:
|
||||
1. **Cluster** related consecutive buffer entries into high-level narrative beats or events (e.g. physical action and its outcome or trivial actions). Combine them into a single chunk.
|
||||
@@ -237,7 +237,7 @@ Instructions:
|
||||
4. **Determine Importance**: Assign an importance score from 1 (trivial, e.g. waking up) to 10 (life-altering, e.g. witnessing a crime).
|
||||
4. Discard small body movements like looking around, sighing, etc that do not contextually hold any meaning after it is done.
|
||||
5. **Involved Entities**: Identify all entity IDs involved in the memories in this chunk.
|
||||
6. **Retain in Buffer (Pinning)**: If a beat represents an unresolved high-stakes situation (e.g. a standing threat, an unanswered accusation, an ongoing chase or conflict), set "retainInBuffer" to true so it remains in the working memory buffer for immediate context. Otherwise, set it to false so it is safely pruned from the buffer.
|
||||
6. **Retain in Cognitive Buffer (Pinning)**: If a beat represents an unresolved high-stakes situation (e.g. a standing threat, an unanswered accusation, an ongoing chase or conflict), set "retainInBuffer" to true so it remains in the Cognitive Buffer for immediate context. Otherwise, set it to false so it is safely pruned from the Cognitive Buffer.
|
||||
7. **Exclude stage business**: Glances, sighs, ambient noticing, and irrelevant sensory details should be ignored and not included in any promoted chunk. They will be forgotten.
|
||||
8. **Forget by omission**: Any buffer entry ID that you do not include in any chunk's "sourceEntryIds" will be permanently deleted and forgotten.
|
||||
`.trim();
|
||||
|
||||
@@ -7,7 +7,7 @@ The Actor Agent is the system component that embodies a single entity and produc
|
||||
|
||||
## Design Principles
|
||||
|
||||
1. **Epistemic boundedness** — The actor only sees what its entity would perceive: public attributes of other entities, private attributes explicitly ACL'd to it, its own memory buffer, and co-located entities. It does not have system-level access to all world state.
|
||||
1. **Epistemic boundedness** — The actor only sees what its entity would perceive: public attributes of other entities, private attributes explicitly ACL'd to it, its own Cognitive Buffer, and co-located entities. It does not have system-level access to all world state.
|
||||
|
||||
2. **Proposal, not mutation** — The actor generates a _proposal_ (narrative prose). It never mutates world state, persists to the database, or writes to memory directly. Validation, execution, and persistence are the Architect's job.
|
||||
|
||||
@@ -38,7 +38,7 @@ Epistemically bounded, with these sections:
|
||||
| ---------------------------- | -------------------------------------------------------------------------------------------------- | -------------------------------------- |
|
||||
| Current moment | The subjective present time | `worldState.clock.get().toISOString()` |
|
||||
| The world as you perceive it | Self-visible attributes, co-located entities + their visible attributes, other presences elsewhere | `serializeSubjectiveWorldState()` |
|
||||
| Your recent memory | Recent `BufferEntry`s, alias-substituted, with relative time phrasing | `serializeSubjectiveBufferEntry()` |
|
||||
| Your cognitive buffer | Recent `BufferEntry`s, alias-substituted, with relative time phrasing | `serializeSubjectiveBufferEntry()` |
|
||||
|
||||
No system UUIDs, no private attributes the entity lacks ACL access to, and no objective-world-state dump are present.
|
||||
|
||||
|
||||
@@ -1,15 +1,15 @@
|
||||
---
|
||||
title: Memory Handoff Pipeline
|
||||
description: The Tier 1 (Working Buffer) to Tier 2 (Long-Term Ledger) memory promotion architecture
|
||||
description: The Cognitive Buffer to Memory Ledger promotion architecture
|
||||
sidebar:
|
||||
order: 6
|
||||
---
|
||||
|
||||
:::tip[Status: Fully Implemented]
|
||||
The handoff pipeline is fully integrated into the simulation step loop. The `HandoffEngine` and `checkHandoffTrigger` components process working memory entries, promoting them to the persistent episodic Ledger, while safely pruning the subjective working memory buffer.
|
||||
The handoff pipeline is fully integrated into the simulation step loop. The `HandoffEngine` and `checkHandoffTrigger` components process Cognitive Buffer entries, promoting them to the persistent Memory Ledger, while safely pruning the Cognitive Buffer.
|
||||
:::
|
||||
|
||||
The **Handoff** pipeline manages the promotion of subjective experiences from the short-term working memory (Tier 1 Buffer) to long-term episodic memory (Tier 2 Ledger). This process regulates context window utilization by deciding **when** promotion occurs, **how much** of the buffer is processed, and **what** details are preserved or pruned.
|
||||
The **Handoff** pipeline manages the promotion of subjective experiences from the **Cognitive Buffer** to the **Memory Ledger**. This process regulates context window utilization by deciding **when** promotion occurs, **how much** of the Cognitive Buffer is processed, and **what** details are preserved or pruned.
|
||||
|
||||
To maintain performance and prevent context bloat, the pipeline separates the process into three decoupled stages:
|
||||
|
||||
@@ -46,7 +46,7 @@ Voluntary triggers identify natural narrative boundaries where the entity is ina
|
||||
|
||||
Involuntary triggers enforce context constraints before building the next prompt to prevent token limit overflows:
|
||||
|
||||
- **Buffer Ceiling**: The serialized length of the working memory section (as generated by `getMemorySectionLength`) exceeds a configured fraction (default: 60%) of the provider's context window.
|
||||
- **Buffer Ceiling**: The serialized length of the Cognitive Buffer section (as generated by `getMemorySectionLength`) exceeds a configured fraction (default: 60%) of the provider's context window.
|
||||
- **Event Velocity**: The number of buffer entries exceeds a safe threshold (default: 20), indicating an event-heavy scene generating history faster than voluntary boundaries occur.
|
||||
|
||||
If the buffer is empty, trigger detection exits immediately with `"none"`.
|
||||
@@ -55,13 +55,13 @@ If the buffer is empty, trigger detection exits immediately with `"none"`.
|
||||
|
||||
## 2. Watermark Partitioning
|
||||
|
||||
To preserve short-term narrative continuity and avoid immediate memory decay, recent events are protected from handoff. The pipeline divides the active buffer into a protected **Watermark Tail** and a **Candidate Pool**.
|
||||
To preserve narrative continuity and avoid immediate memory decay, recent events are protected from handoff. The pipeline divides the active Cognitive Buffer into a protected **Watermark Tail** and a **Candidate Pool**.
|
||||
|
||||
The watermark boundary is computed as:
|
||||
|
||||
$$\text{Watermark} = \max(K \text{ entries}, \text{entries bucketed as fresh by } \textit{naturalizeTime})$$
|
||||
|
||||
- **Hard Floor ($K = 8$)**: The last 8 entries in the buffer are always preserved.
|
||||
- **Hard Floor ($K = 8$)**: The last 8 entries in the Cognitive Buffer are always preserved.
|
||||
- **Temporal Freshness**: Any entries that `naturalizeTime` classifies within the immediate temporal horizon (`"just now"`, `"moments ago"`, `"a few minutes ago"`, or `"several minutes ago"`) are also protected.
|
||||
|
||||
Entries older than the watermark boundary constitute the **Candidate Pool** and are eligible for handoff processing.
|
||||
@@ -100,8 +100,8 @@ The prompt instructs the model to apply the following cognitive operations:
|
||||
|
||||
When a chunk represents an unresolved high-stakes situation (e.g., an active conflict or standing threat), the LLM sets `retainInBuffer: true`.
|
||||
|
||||
- The summary is committed to the long-term Ledger as normal.
|
||||
- The source buffer entries are exempted from pruning, remaining in the working memory buffer with `pinned: true` until a future handoff pass determines the thread is resolved.
|
||||
- The summary is committed to the Memory Ledger as normal.
|
||||
- The source Cognitive Buffer entries are exempted from pruning, remaining in the Cognitive Buffer with `pinned: true` until a future handoff pass determines the thread is resolved.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,15 +1,15 @@
|
||||
---
|
||||
title: Tier 2 Memory (Long-Term Ledger)
|
||||
title: Memory Ledger
|
||||
description: Persistent episodic memory storage, semantic indexing, and retrieval mechanics
|
||||
---
|
||||
|
||||
**Tier 2 Memory (the Ledger)** represents an entity's long-term episodic memory. It archives historical summaries of past events, providing a persistent record of character experiences that can be retrieved and injected into LLM prompts as needed.
|
||||
**Memory Ledger** represents an entity's persistent episodic memory. It archives historical summaries of past events, providing a durable record of character experiences that can be retrieved and injected into LLM prompts as needed.
|
||||
|
||||
---
|
||||
|
||||
## 1. Data Model
|
||||
|
||||
A long-term memory is represented by a `LedgerEntry`. It includes metadata for deterministic database-level filtering, structured narrative content, and vector embeddings for semantic similarity scoring.
|
||||
A Memory Ledger entry is represented by a `LedgerEntry`. It includes metadata for deterministic database-level filtering, structured narrative content, and vector embeddings for semantic similarity scoring.
|
||||
|
||||
```ts
|
||||
interface LedgerEntry {
|
||||
@@ -30,7 +30,7 @@ interface LedgerEntry {
|
||||
|
||||
## 2. Storage Model
|
||||
|
||||
To support fast, low-latency queries across large historical datasets, Tier 2 memory is stored in standard SQLite tables. Vector embeddings are stored in raw binary format as `Float32Array` BLOBs.
|
||||
To support fast, low-latency queries across large historical datasets, Memory Ledger entries are stored in standard SQLite tables. Vector embeddings are stored in raw binary format as `Float32Array` BLOBs.
|
||||
|
||||
Standard secondary indices optimize query execution time to microseconds, eliminating the compilation and cross-platform installation overhead of native vector database extensions (such as `sqlite-vec` or `node-gyp` binaries).
|
||||
|
||||
@@ -64,7 +64,7 @@ CREATE INDEX IF NOT EXISTS idx_ledger_involved_entity ON ledger_involved_entitie
|
||||
|
||||
## 3. The Handoff Pipeline
|
||||
|
||||
Working memory (Tier 1 Buffer) entries are promoted to the Ledger through the automated [Handoff Pipeline](./handoff).
|
||||
**Cognitive Buffer** entries are promoted to the Memory Ledger through the automated [Handoff Pipeline](./handoff).
|
||||
|
||||
During handoff:
|
||||
|
||||
@@ -72,7 +72,7 @@ During handoff:
|
||||
2. Ambient stage business and redundant details are pruned.
|
||||
3. Chunks are synthesized into third-person summaries and assigned importance scores.
|
||||
4. Text embeddings are generated for the summary.
|
||||
5. The processed memories are committed to the SQLite store, and the short-term buffer is pruned.
|
||||
5. The processed memories are committed to the SQLite store, and the Cognitive Buffer is pruned.
|
||||
|
||||
---
|
||||
|
||||
@@ -116,13 +116,13 @@ Once the candidate pool is loaded, the ranking engine evaluates entries in appli
|
||||
|
||||
## 5. Active Focus & Attention Loop
|
||||
|
||||
In crowded settings (e.g., a room with many characters), retrieving long-term memories for all co-located entities would exhaust the prompt context window. To prevent this, retrieval uses an **Active Focus** selection strategy:
|
||||
In crowded settings (e.g., a room with many characters), retrieving Memory Ledger entries for all co-located entities would exhaust the prompt context window. To prevent this, retrieval uses an **Active Focus** selection strategy:
|
||||
|
||||
- **Active Focus Set**: The prompt builder scans the last 10 entries of the character's working memory buffer. Any entity targeted by, spoken to, or mentioned in these entries is added to the Active Focus set.
|
||||
- **Active Focus Set**: The prompt builder scans the last 10 entries of the character's Cognitive Buffer. Any entity targeted by, spoken to, or mentioned in these entries is added to the Active Focus set.
|
||||
- **Dynamic Capacity Limits**:
|
||||
- If the number of co-located characters is small ($\le 3$), long-term memory is retrieved for all of them.
|
||||
- If the environment is crowded ($> 3$), long-term retrieval is strictly restricted to the top 3 characters in the Active Focus set.
|
||||
- This creates a realistic attention loop: when a new character interacts with the actor, they enter the working buffer, triggering the retrieval of their long-term history on the subsequent turn.
|
||||
- If the number of co-located characters is small ($\le 3$), Memory Ledger entries are retrieved for all of them.
|
||||
- If the environment is crowded ($> 3$), Memory Ledger retrieval is strictly restricted to the top 3 characters in the Active Focus set.
|
||||
- This creates a realistic attention loop: when a new character interacts with the actor, they enter the Cognitive Buffer, triggering the retrieval of their Memory Ledger history on the subsequent turn.
|
||||
|
||||
---
|
||||
|
||||
@@ -130,15 +130,15 @@ In crowded settings (e.g., a room with many characters), retrieving long-term me
|
||||
|
||||
Retrieved ledger entries are formatted into the prompt chronologically and mapped to subjective aliases. Internal metrics (e.g., importance numbers and raw system IDs) are omitted to preserve immersion.
|
||||
|
||||
- Working memory entries are injected under `=== RECENT EVENTS ===` (narrated relative to the present moment).
|
||||
- Recalled ledger entries are injected under `=== YOUR MEMORIES ===` (presented as long-term recollections).
|
||||
- Cognitive Buffer entries are injected under `=== COGNITIVE BUFFER ===` (narrated relative to the present moment).
|
||||
- Recalled Memory Ledger entries are injected under `=== MEMORY LEDGER ===` (presented as episodic recollections).
|
||||
|
||||
```text
|
||||
=== RECENT EVENTS ===
|
||||
=== COGNITIVE BUFFER ===
|
||||
Moments ago
|
||||
- You spoke to Strider: "Hello there"
|
||||
|
||||
=== YOUR MEMORIES ===
|
||||
=== MEMORY LEDGER ===
|
||||
A couple days ago
|
||||
- You met a hooded figure named Strider at The Prancing Pony.
|
||||
Quote: "I can avoid being seen, if I wish, but to disappear entirely, that is a rare gift."
|
||||
|
||||
Reference in New Issue
Block a user