# DESIGN-0.18.0: Memory System

**Version:** 0.18.0
**Status:** Phase 1 Complete
**Depends on:** compaction (v0.15.0), knowledge bases (v0.14.0), persona-KB binding (v0.17.0), SQLite backend (v0.17.1)

---

## 1. Overview

Memory provides long-term fact persistence across conversations. Unlike
compaction (within-conversation context) or knowledge bases (static
documents), memory captures preferences, facts, and context that
accumulate over time from natural conversation.

**Key differentiator: Persona-scoped memory.** Each Persona builds
its own memory independently, preventing cross-context contamination.
A helpdesk Persona accumulates FAQ knowledge; a tutoring Persona
tracks per-student progress; a user's personal preferences stay
separate from any Persona context.

---

## 2. Scope Model

Three memory scopes with clear ownership:

```
┌─────────────────────────────────────────────────┐
│ user scope                                      │
│ owner_id = user_id                              │
│ "Jeff prefers Go over Python"                   │
│ "Deployment uses Kubernetes + Traefik"          │
│ Visible in ALL conversations for this user      │
└─────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────┐
│ persona scope                                   │
│ owner_id = persona_id                           │
│ "Common question: how to reset password"        │
│ "Policy: refunds within 30 days only"           │
│ Shared across ALL users of this Persona         │
└─────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────┐
│ persona_user scope                              │
│ owner_id = persona_id, user_id = user_id        │
│ "Student struggles with recursion"              │
│ "Customer prefers email over phone"             │
│ Per-user within a specific Persona              │
└─────────────────────────────────────────────────┘
```

**Scope resolution at recall time:**

When a Persona is active, all three scopes are queried and merged
with priority ordering: persona_user > persona > user. When no
Persona is active, only user scope is queried.

---

## 3. Data Model

### 3.1 Table: `memories`

| Column | Type | Description |
|--------|------|-------------|
| `id` | UUID / TEXT | Primary key |
| `scope` | TEXT | `user`, `persona`, `persona_user` |
| `owner_id` | UUID / TEXT | user_id (user scope) or persona_id (persona/persona_user) |
| `user_id` | UUID / TEXT | Only for persona_user scope — identifies which user |
| `key` | TEXT | Short label (2-6 words) |
| `value` | TEXT | The fact/detail |
| `source_channel_id` | UUID / TEXT | Provenance — which conversation |
| `confidence` | REAL | 0.0-1.0, LLM-provided |
| `status` | TEXT | active, pending_review, archived |
| `embedding` | vector(3072) / TEXT | For semantic recall (Phase 2) |
| `created_at` | TIMESTAMPTZ / TEXT | |
| `updated_at` | TIMESTAMPTZ / TEXT | |

### 3.2 Indexes

- **Primary lookup:** `(scope, owner_id, status)` — filtered to `status = 'active'`
- **Persona+user:** `(owner_id, user_id)` — filtered to `scope = 'persona_user'`
- **Dedup:** unique on `(scope, owner_id, COALESCE(user_id, nil_uuid), key)`
- **FTS:** GIN on `to_tsvector(key || ' ' || value)` (Postgres) / LIKE fallback (SQLite)
- **Provenance:** `(source_channel_id)` — "where did this memory come from?"

### 3.3 Model Struct

```go
type Memory struct {
    ID              string    `json:"id"`
    Scope           string    `json:"scope"`
    OwnerID         string    `json:"owner_id"`
    UserID          *string   `json:"user_id,omitempty"`
    Key             string    `json:"key"`
    Value           string    `json:"value"`
    SourceChannelID *string   `json:"source_channel_id,omitempty"`
    Confidence      float64   `json:"confidence"`
    Status          string    `json:"status"`
    CreatedAt       time.Time `json:"created_at"`
    UpdatedAt       time.Time `json:"updated_at"`
}
```

---

## 4. Store Interface

```go
type MemoryStore interface {
    Upsert(ctx, *Memory) error          // create or update (matched on scope+owner+user+key)
    GetByID(ctx, id) (*Memory, error)
    List(ctx, MemoryFilter) ([]Memory, error)
    Delete(ctx, id) error               // hard delete
    Archive(ctx, id) error              // soft delete (status → archived)
    Recall(ctx, userID, *personaID, query, limit) ([]Memory, error)  // scope-merged search
    CountByOwner(ctx, scope, ownerID) (int, error)
}
```

**Key design decisions:**

- `Upsert` uses the unique index for conflict resolution — same key = update value
- `Recall` does a UNION across all applicable scopes with priority ordering
- Both Postgres and SQLite implementations exist (same pattern as all other stores)
- SQLite falls back to LIKE for text search (no tsvector)

---

## 5. Tools

### 5.1 `memory_save`

The LLM calls this to explicitly store a fact. Scope is automatically
determined by whether a Persona is active:

- No Persona → `user` scope, `owner_id = user_id`
- Persona active → `persona_user` scope, `owner_id = persona_id`, `user_id = user_id`

Parameters: `key` (required), `value` (required), `confidence` (optional, default 1.0)

### 5.2 `memory_recall`

The LLM calls this to search stored memories. Results merge across
all applicable scopes.

Parameters: `query` (optional), `max_results` (optional, default 20)

### 5.3 Registration

Late registration pattern (same as notes, KB search, conversation search):

```go
// main.go
tools.RegisterMemoryTools(stores)
```

---

## 6. Memory Injection

At completion time, `loadConversation()` injects relevant memories
into the system prompt — same position as KB hints and compaction
summaries:

```
[admin system prompt]
[persona/channel system prompt]
[KB hint]
[memory injection]  ← NEW
[compaction summary OR message history]
```

`BuildMemoryHint()` calls `Recall()` with an empty query (returns
top memories by confidence/recency), formats as bullet points, and
caps at ~1500 tokens (~6000 chars) to preserve context budget.

---

## 7. Implementation Phases

### Phase 1: Foundation & Tools ✅ (this delivery)
- Migrations (Postgres + SQLite)
- Memory model struct
- MemoryStore interface + both implementations
- memory_save + memory_recall tools
- Memory injection in loadConversation
- 9 new files, ~5 edits to existing files

### Phase 2: Automatic Extraction & Embeddings
- Background job: post-conversation analysis extracts facts
- Configurable extraction prompt per Persona
- Extracted memories start as `pending_review`
- Embedding generation for semantic recall
- Recall upgraded to use vector similarity + keyword hybrid

### Phase 3: Review Pipeline & User Controls
- Admin review queue for pending memories
- Persona memory review for team admins
- User Settings → Memory: view, edit, delete
- Per-Persona memory toggle
- Retention policies
- Frontend memory management UI

---

## 8. Persona Memory Saves (Future: Phase 2)

Phase 1 only supports `user` and `persona_user` scopes via the tool
(the LLM saves about the user, scoped appropriately). Phase 2 adds
the ability for admins/team leads to configure Persona-scope memory
extraction — the Persona "learns" from conversations:

```
Persona system prompt addition (Phase 2):
"After each conversation, identify key facts, common questions,
 and useful patterns. Save these as persona-scope memories for
 future reference by all users."
```

This runs as a background job, not during the live conversation,
to avoid latency impact.

---

## 9. Privacy & Security

- User memories never cross team boundaries
- Persona memories respect group access controls
- persona_user memories are only visible to the specific user + admins
- Users can view/delete their own memories (Phase 3 UI)
- Admin can disable memory system globally or per-team
- No cross-persona memory leakage — scope isolation is enforced at the query level