web-official/public/docs/memory-evolving-and-proactive.en.md

Agent Memory Evolving & Proactive Interaction (Beta)

Beta Feature: QwenPaw's ReMeLight memory manager embeds ReMe as an in-process application. Auto Memory, Auto Resource, Auto Dream, search, and ReMe's low-level proactive topic reader are ReMe jobs. QwenPaw's /proactive command is a separate runtime feature that reads recent chat sessions and optional screen context.

QwenPaw stores memory as files under the agent workspace. Conversations are saved as JSONL source logs, useful conversation facts are written to daily Markdown notes, resources can be converted into daily notes, and Auto Dream periodically integrates reusable abstractions into digest memory.

---

Core Idea: A Self-Evolving Personal Knowledge Base

ReMe's goal is not to be a hidden vector store bolted onto a chat model. It is to grow a self-evolving personal knowledge base on the principle of Memory as File, File as Memory: every working or long-term memory node is a plain Markdown file you can open, read, edit, move, or delete, and at the same time an indexable, linkable node. Raw sources and derived system state use formats suited to their roles.

Because memory lives as files rather than opaque database rows, long-term memory gains properties a black box cannot offer:

Property	What it means in practice
Readable	Open the workspace and read daily notes and digest nodes like ordinary Markdown.
Editable	Correct, extend, move, or delete memory with plain file edits — no specialized client.
Traceable	Each long-term conclusion links back to its source through derived_from:: [[...]].
Portable	The workspace is an ordinary directory; back it up, sync it, or version it with git.
Collaborative	You judge and correct; the agent organizes, links, and retrieves — on the same files.

Memory Layers

The workspace organizes memory into four layers, from raw evidence to reusable knowledge:

raw input        → mem_session/ + resource/   original conversations and external material
working memory   → memory/                     daily notes: facts, decisions, resource readings
long-term memory → digest/                     reusable knowledge: personal / procedure / wiki
system state     → mem_metadata/               indexes, wikilink graph, catalogs (not hand-edited)

QwenPaw's directory names differ from ReMe's upstream defaults, but the layer roles are identical: mem_session/ ↔ ReMe session/, memory/ ↔ daily/, mem_metadata/ ↔ metadata/. resource/ and digest/ keep the same names.

How the Knowledge Base Evolves

The base grows through a continuous capture → index → consolidate → recall loop:

1. Capture — Auto Memory distills conversations into daily notes; Auto Resource turns files under resource/ into daily notes. The raw conversation is retained as evidence.
2. Index — A background job keeps memory/ and digest/ searchable through a BM25 keyword index, optional embeddings, and a wikilink graph.
3. Consolidate — Auto Dream reads recent daily notes and integrates them into long-term digest/ nodes. This is where memory actually evolves: instead of copying text, each extracted unit is merged into an existing node or creates a new one, and source and relationship wikilinks are woven in.
4. Recall — memory_search retrieves the most relevant fragments and expands along the wikilink graph; interest topics and QwenPaw's /proactive surface what is worth attention.

The digest layer is deliberately not append-only. When new material repeats, refines, or contradicts an existing node, Auto Dream corroborates, refines, or corrects it (see the integration actions below). Combined with the wikilink graph that keeps nodes connected and traceable, the knowledge base becomes denser and more accurate over time rather than merely larger.

---

Actual Flow

graph LR
    A[Conversation turns] --> B[MemoryMiddleware]
    B --> C[ReMe auto_memory job]
    C --> D[mem_session/dialog/*.jsonl]
    C --> E[memory/<date>/<note>.md]
    R[resource/<date>/*] --> S[resource_watch_loop]
    S --> T[ReMe auto_resource job]
    T --> E
    E --> U[ReMe auto_dream job]
    U --> V["digest/personal | procedure | wiki/*.md"]
    U --> W[memory/<date>/interests.yaml]

Capability	Code path	Trigger	Main output
Auto Memory	ReMeLightMemoryManager.auto_memory() -> ReMe auto_memory	MemoryMiddleware after every configured number of user turns, and before context compression when enabled	mem_session/dialog/<session_id>.jsonl, memory/<date>/<note>.md, memory/<date>.md
Auto Resource	ReMe resource_watch_loop -> auto_resource	Embedded ReMe background watcher for resource_dir	memory/<date>/<resource_note>.md
Auto Dream	ReMeLightMemoryManager.dream() -> ReMe auto_dream	/dream command or dream_cron scheduler	digest/*/*.md, memory/<date>/interests.yaml
ReMe proactive job	ReMe proactive	Direct ReMe job call only	Metadata/content from memory/<date>/interests.yaml
QwenPaw /proactive	src/pineagents/agents/memory/proactive	`/proactive [minutes	on	off]` idle loop	A proactive chat request sent through /api/console/chat

The important boundary is that memory/<date>/interests.yaml is produced by Auto Dream and can be read by ReMe's proactive job, but QwenPaw's current /proactive implementation does not call that job.

---

File Layout

The embedded ReMe config comes from src/pineagents/agents/memory/reme_config.py and the user-facing defaults come from ReMeLightMemoryConfig.

<workspace>/
├── mem_metadata/   # ReMe persistent state, indexes, catalogs
├── mem_session/    # Source conversation logs used by auto-memory
│   └── dialog/
│       └── <session_id>.jsonl
├── mem_agent/      # Internal ReMe memory-agent sessions
├── resource/       # External assets watched by Auto Resource
│   └── YYYY-MM-DD/
│       └── <resource>.<ext>
├── memory/         # Daily memory notes and day indexes
│   ├── YYYY-MM-DD.md
│   └── YYYY-MM-DD/
│       ├── <note>.md
│       └── interests.yaml
└── digest/         # Long-term digest memory
    ├── personal/
    ├── procedure/
    └── wiki/

Default directory names are configurable through metadata_dir, session_dir, mem_session_dir, resource_dir, daily_dir, and digest_dir.

---

Auto Memory

Auto Memory is invoked by MemoryMiddleware, not directly on every model call. The middleware:

• skips automation requests whose source is cron or heartbeat;
• optionally injects auto memory search context before model calls when auto_memory_search_config.enabled is true;
• collects user-turn markers after replies;
• flushes pending turns after auto_memory_interval user turns;
• also flushes before context compression when summarize_when_compact is true and compression is about to happen.

auto_memory_interval defaults to 5. None, 0, or a negative value disables periodic auto-memory.

When flushed, QwenPaw calls ReMe's auto_memory job with:

Field	Source
messages	Selected conversation messages for the pending user turns
session_id	Agent session id
memory_hint	Optional hint passed by caller

ReMe's AutoMemoryStep then:

1. validates that session_id is present and valid;
2. saves or appends sanitized source messages to mem_session/dialog/<session_id>.jsonl;
3. removes tool-result blocks and base64 data blocks from the saved source log;
4. chooses the note date from an explicit date, message timestamps, or the configured timezone's current date;
5. looks for an existing daily note whose frontmatter has the same session_id or source_conversation;
6. creates at most one note for a new session, or updates the existing note for that session;
7. ensures frontmatter contains session_id and source_conversation;
8. may rename the note from frontmatter name;
9. refreshes the day index at memory/<date>.md;
10. returns metadata including date, path, created, modified, n_messages, source_conversation, and index.

If the job succeeds but no note was changed, QwenPaw does not push an inbox event for auto_memory. Otherwise it pushes an inbox event titled Auto-memory result.

---

Auto Resource

QwenPaw configures a ReMe background job named resource_watch_loop. It watches resource_dir and dispatches change batches to auto_resource.

Watched suffixes are:

md, txt, json, jsonl, csv, yaml, html

Files can be placed directly in the resource_dir root, in which case QwenPaw's configured timezone determines the current date, or under resource_dir/YYYY-MM-DD/, in which case the path supplies the date. Additional subdirectories may follow the date directory. For added and modified resources, ReMe reads the content as UTF-8 text and asks the memory agent to create or update a daily note. Deleting a resource also deletes its corresponding source-linked note.

Binary files such as PDF, Word, Excel, and images are not parsed automatically. The yml suffix is not in the default allowlist either; convert these inputs to one of the supported text formats first.

Each change item may contain path or file_path and a change value such as added, modified, or deleted. The ReMe step interprets changed resource files into daily notes. QwenPaw pushes an Auto-resource result inbox event only when the job reports a real modification.

---

Auto Dream

Auto Dream is exposed in QwenPaw through:

• /dream [hint], handled by CommandHandler._process_dream();
• the scheduler configured by dream_cron when dream_cron_enabled is true, default 0 23 * * *; scheduled runs start after a random delay of 0–60 seconds to avoid simultaneous calls;
• ReMeLightMemoryManager.dream(date="", hint="").

QwenPaw runs the ReMe job named auto_dream with needs_llm=True, so the embedded ReMe app refreshes its LLM component from the active QwenPaw model before the job runs.

The embedded job configuration uses these defaults:

Parameter	Default	Meaning
date	""	Empty means today in the configured timezone
hint	""	Optional user/operator hint
scan_days	2	Scan target date and recent days
max_units	5	Maximum extracted reusable memory units
topic_count	3	Maximum final interest topics
topic_diversity_days	7	Avoid repeating topics from recent days

Auto Dream runs four ReMe steps:

Step	Actual behavior
dream_extract_step	Refreshes day indexes, compares daily files against the dream catalog, deletes missing catalog entries, and extracts reusable memory units plus topic candidates only from changed daily inputs.
dream_integrate_step	Integrates each extracted unit into one digest node. It uses node_search, read, frontmatter_read, write, edit, and frontmatter_update.
dream_topics_step	Selects and de-duplicates interest topics, writes memory/<date>/interests.yaml, and refreshes the day index.
dream_finish_step	Upserts successful changed paths, interest files, and day indexes into the dream catalog, persists the catalog, and returns a summary.

If there are no changed daily inputs, extract finishes with a no-change response. If an LLM is unavailable, extract or integrate fails because those steps require an LLM.

Digest nodes are stored by bucket:

Bucket	What belongs there
personal/	User, team, or project identity, preferences, conventions, constraints, and avoid-rules
procedure/	How-to workflows, runbooks, recipes, methods, and executable patterns
wiki/	Definitions, principles, observations, decisions as precedent, factual claims, and catch-all knowledge

Integration actions are CREATE, CORROBORATE, REFINE, or CORRECT. These four actions are what makes the knowledge base self-evolving: a unit that matches an existing node is not appended as a duplicate but merged into it — corroborated with an extra source, refined with new boundaries, or corrected when it conflicts.

Action	Meaning
CREATE	No equivalent abstraction exists yet; create a new digest node.
CORROBORATE	The same memory appeared again; append a source and strengthen the description.
REFINE	New material adds boundaries, steps, prerequisites, applicability, or detail.
CORRECT	New material corrects errors, omissions, or conflicts in the existing node.

Knowledge graph via wikilinks. ReMe's wikilink integration logic runs in dream_integrate_step. Before writing, it calls node_search to recall similar or related digest nodes, decides between the actions above, and then weaves two kinds of workspace-relative wikilinks into the node body:

• Source edges — derived_from:: [[memory/<date>/<note>.md]] keep every digest conclusion traceable back to the daily note or resource it came from.
• Relationship edges — relates_to:: [[digest/wiki/...]], depends_on:: [[digest/procedure/...]], and similar typed links connect a node to adjacent concepts, prerequisites, and procedures.

Updates are additive: existing wikilinks and derived_from entries are preserved, so the graph keeps growing without losing edges. memory_search later expands along these links, which is why recall can surface not just a matching fragment but the long-term nodes and sources it connects to.

When Auto Dream completes, QwenPaw pushes an inbox event titled Auto-dream result.

---

Interest Topics and ReMe Proactive Job

dream_topics_step writes:

memory/<date>/interests.yaml

The YAML payload contains:

Field	Meaning
date	Target date
topic_count	Requested maximum topic count
diversity_days	Recent-day duplicate avoidance window
topics	Selected topics with title, reason, evidence, keywords, and paths

ReMe also defines a proactive job implemented by proactive_step. That job only reads memory/<date>/interests.yaml. It accepts:

Parameter	Default	Meaning
date	""	Empty means today
include_content	true	Include raw YAML text in metadata

If the interests file is missing, the ReMe proactive job returns a normal skipped result.

---

QwenPaw /proactive

QwenPaw's current /proactive command is implemented under src/pineagents/agents/memory/proactive. It is separate from ReMe's proactive job.

Command behavior:

/proactive           # enable with default 30 minute idle threshold
/proactive on        # enable with default 30 minute idle threshold
/proactive 45        # enable with a 45 minute idle threshold
/proactive off       # cancel the background monitoring task

When enabled, QwenPaw stores an in-memory ProactiveConfig for the session and starts a background loop. The loop:

• wakes every 30 seconds;
• skips while the agent has active tasks;
• reads the latest chat update time;
• waits until the session has been idle for the configured number of minutes;
• avoids retrying more than once per 60 seconds;
• skips if the latest message is already an unanswered [PROACTIVE] message;
• runs the proactive responder.

The responder builds context from recent chat sessions, not from interests.yaml:

• reads chat metadata from workspace.chat_manager;
• keeps sessions updated within the last 7 days, or the latest 5 sessions when fewer than 5 match the date window;
• loads up to 100 recent text messages, capped at 50,000 characters;
• filters system messages, non-text blocks, and prior proactive helper requests;
• optionally analyzes a desktop screenshot when the active model supports multimodal input.

It then asks a temporary ProactiveAssistant agent to extract 1 to 3 likely tasks from that context, executes up to the first 3 task queries using tools, and sends a user-facing proactive request through:

POST <agent-api-base>/api/console/chat
session_id = proactive_mode:<active_agent_id>
text starts with "[Agent proactive_helper requesting]"

The final user-facing prompt instructs the agent response to begin with [PROACTIVE].

The command warning is accurate: proactive mode may read historical session memory and may take screenshots when multimodal screen analysis is available. The proactive agent uses tool protection bypass mode through its own temporary agent/tool setup.

---

Search and Indexing

The embedded ReMe app starts an index_update_loop background job. Search indexing watches:

Indexed directories	Suffixes
daily_dir, digest_dir	md

The QwenPaw memory_search tool runs ReMe's search job with query, limit, and min_score. The job is configured as hybrid workspace search with vector recall, BM25 keyword recall, RRF fusion, and wikilink expansion. The storage backend in QwenPaw's embedded ReMe config is local.

---

Current Status

This document describes the current code paths:

• ReMeLight is implemented by ReMeLightMemoryManager and embedded get_reme_app_config();
• Auto Memory is turn-count based and defaults to every 5 user turns;
• Auto Dream runs by /dream or dream_cron;
• ReMe writes interests.yaml, and ReMe has a low-level reader for it;
• QwenPaw /proactive currently uses recent chat/session/screen context rather than ReMe interest topics;
• Auto Memory, Auto Resource, and Auto Dream results may be delivered to the inbox when they produce reportable output.

The feature remains Beta, but the behavior above is the behavior represented by the current code.