MCP based ACP agent
crow-cli docs crow-cli-docs memory-tool-design.md
13 kB · 294 lines
Markdown
at main


title: Memory Tool Design #

Memory Tool Design #

Status: Design Phase #

The query_memory MCP tool is the primary interface agents have for introspecting their own conversation history. It's used for context recovery, debugging, and cross-session learning. This document collects pain points, design constraints, and proposals for improvement.


Current API #

query_memory is a single tool with progressive disclosure:

query_memory(
    query: str | None = None,       # Text search term
    session_id: str | None = None,  # Filter to session
    agent_idx: int | None = None,   # Filter to agent within session
    mode: ContentMode = "conversation",  # conversation | with_thinking | with_tools | full
    context: int = 0,               # Messages around each match (grep -C)
    after: str | None = None,       # ISO datetime lower bound
    before: str | None = None,      # ISO datetime upper bound
    limit: int = 50,                # Max results
    offset: int = 0,                # Pagination
) -> str  # Markdown output

Three modes of operation:

Mode Parameters Output
Discovery query only Table of matching messages across all sessions
Browse session_id only Transcript of messages in session
Deep dive session_id + query Search within session, optional context window

Database Schema #

agents (PK: agent_id)
├── agent_id       TEXT     -- "{session_id}-{idx}"
├── session_id     TEXT     -- Logical parent session
├── agent_idx      INTEGER  -- Agent number within session
├── cwd            TEXT     -- Working directory
├── prompt_id      TEXT     -- FK to prompts
├── prompt_args    JSON     -- Template variables
├── system_prompt  TEXT     -- Rendered system prompt
├── tool_definitions JSON   -- Available tools
├── request_params JSON     -- Model params
├── model_identifier TEXT   -- Model used
├── status         TEXT     -- "active", etc.
└── created_at     DATETIME

messages (PK: id, FK: agent_id)
├── data             JSON   -- Full message dict
├── role             TEXT   -- "system" | "user" | "assistant" | "tool"
├── prompt_tokens    INTEGER
├── completion_tokens INTEGER
├── total_tokens     INTEGER
└── created_at       DATETIME

Pain Points (from real usage) #

1. No way to discover sessions #

There is no list_sessions tool. To find what sessions exist, you need to already know a session_id or search for a term you hope is unique enough. The discovery mode returns messages, not sessions — so you get duplicate session_ids in the results and no sense of what each session was about.

Real scenario: "I worked on something last week, what was it?" → No way to answer this. You can't browse sessions chronologically, search by working directory, or see a title.

2. Session metadata is invisible to agents #

The agents table contains rich metadata: model_identifier, cwd, system_prompt, tool_definitions, created_at. None of this is surfaced by query_memory. An agent browsing a session has no idea what model was used, what tools were available, or what the working directory was.

3. No session titles or summaries #

Sessions are identified by coolname slugs (amusing-fragrant-sawfish-of-satiation). The first user prompt (which is effectively the session title) is buried in message #2 of the messages table. There's no denormalized title column and no tool to get one.

4. "Show me the last N messages" is awkward #

To see the most recent messages in a session, you either:

  • Browse the whole session and hope limit cuts off at the right place (it doesn't — it's a head, not a tail)
  • Guess an after timestamp
  • Use offset with a large value, hoping you know the approximate message count

There's no last_n, tail, or reverse chronological ordering.

5. Mode naming is confusing #

The ContentMode enum suggests mutual exclusivity but the semantics are additive:

conversation    = user + assistant
with_thinking   = user + assistant + reasoning_content  (no tools)
with_tools      = user + assistant + tool_results       (no thinking)
full            = everything

If you want thinking and tools, you need full. There's no with_thinking_and_tools. This forces agents to either get too much (full) or too little.

6. Tool results are truncated at 500 characters with no control #

if len(content) > 500:
    content = content[:500] + f"\n... [{len(content) - 500} chars truncated]"

Sometimes the tool result is the thing you're searching for. A failed build output, an error message, a JSON response — truncation destroys signal.

7. Context window is unpredictable #

context=N expands each match to include N messages before and after. With multiple matches that overlap, results can balloon. There's no way to cap the total output or control deduplication of overlapping context windows.

8. Search is substring-only #

_extract_searchable_text does a simple query.lower() in text.lower(). No fuzzy matching, no regex, no field-specific search. You can't search "only in tool calls" or "only in thinking content" or use a regex pattern.


Design Tension: Specific Tools vs Generic API #

There are two directions we can take this:

Direction A: More specific tools #

Add purpose-built tools for each use case:

list_sessions(cwd, limit, after, order="desc")
session_summary(session_id)
tail(session_id, last_n=10)
search_tools(session_id, tool_name)

Pros:

  • Each tool is simple and well-documented
  • Easy for agents to discover and use correctly
  • Output format is predictable
  • Low cognitive load per tool

Cons:

  • Tool sprawl — every new need is a new tool
  • Each tool needs its own docstring (token cost in system prompt)
  • Agents still can't compose queries ("sessions with >100 messages where tool X failed")
  • Maintenance burden on the tool surface

Direction B: Generic query API #

Expose the table structure and let agents compose queries:

query_db(
    table: str,
    columns: list[str],
    where: dict,
    order_by: str,
    limit: int,
)

Or go all the way to GraphQL.

Pros:

  • Composability — any query is possible
  • Single tool to learn
  • Agents with good reasoning can do complex things

Cons:

  • Agents are terrible at this. A real agent spent ~50k tokens trying to read a single session because it couldn't formulate the right query. It kept hitting the wall of "I don't know the schema" and "I don't know what columns exist" and "I don't know how to join these tables."
  • Schema exposure is a surface area problem
  • No guardrails against expensive queries
  • The output is raw data — agents still need to format/interpret it
  • GraphQL adds a runtime, schema definition, and learning curve

Proposed Middle Ground: Structured queries over pre-canned views #

Don't expose raw tables. Instead, expose a small set of views (pre-joined, pre-aggregated queries) with a simple filter/sort API:

query_memory(
    view: str = "messages",      # "messages" | "sessions" | "agents" | "tool_calls"
    filters: dict | None = None, # {session_id: "...", role: "assistant", ...}
    order: str = "asc",          # "asc" | "desc"
    limit: int = 50,
    include: list[str] | None = None,  # ["thinking", "tools", "metadata"]
)

The views are defined server-side:

  • sessions: One row per session. Columns: session_id, first_prompt, model, agent_count, message_count, total_tokens, created_at, last_activity, cwd, status.
  • agents: One row per agent. Columns: agent_id, session_id, agent_idx, model, cwd, created_at, status, message_count.
  • messages: Current behavior. One row per message.
  • tool_calls: One row per tool call. Columns: session_id, agent_idx, tool_name, arguments, result_summary, success, created_at.

Benefits:

  • Agents can discover what's available: "what views exist?" → documented list
  • Each view has a known schema → no guessing
  • order: "desc" solves the tail problem
  • include: ["thinking", "tools"] solves the mode problem
  • filters is simpler than SQL but more flexible than fixed parameters
  • No GraphQL runtime, no schema exposure
  • Server-side views can be optimized with indexes

Proposed Changes (prioritized) #

P0 — Must have before the tool is useful for self-introspection #

  1. view: "sessions" — List sessions with metadata, ordered by last activity (desc by default). This is the single most important missing capability.
  2. order: "desc" — Reverse chronological ordering for all views. The default should be desc for sessions (newest first), asc for messages (chronological transcript).
  3. include: list[str] replace mode: ContentMode — Additive inclusion of thinking, tools, metadata. No more guessing that you need full to get both thinking and tools.

P1 — Important for practical usage #

  1. last_n: int — Return the last N messages in a session. Simpler than offset arithmetic.
  2. tool_result_max_len: int — Configurable truncation, default 500, agent can override.
  3. view: "tool_calls" — Search tool calls by name, filter by success/failure.
  4. Session metadata in message output — When browsing a session, include a header with model, cwd, agent count, message count.

P2 — Nice to have #

  1. view: "agents" — List agents within sessions with their metadata.
  2. field: str — Restrict text search to a specific field (thinking, tool_name, tool_result, user_content).
  3. regex: bool — Enable regex matching for the query parameter.
  4. Context deduplication — When context produces overlapping windows, merge them.
  5. Token budget in output — Include approximate token cost of the returned data so agents can decide if they need more.

The Session Listing Problem #

Listing sessions is harder than it sounds because:

  1. Sessions can have multiple agents — A session is a logical group of agents. Each agent has its own message stream. Listing sessions means aggregating across agents.

  2. What's the "last activity"? — It's the max(created_at) across all messages for all agents in the session. This is an aggregation query, not a simple SELECT.

  3. What's the title? — The first user message (typically message id=2, after the system message). This requires a subquery or join.

  4. Inverse chronological order — Sessions should be listed newest-first by default, but messages within a session should be chronological. The default ordering depends on the view.

  5. Working directory grouping — Agents often work in the same cwd. It would be useful to group or filter by workspace, but cwd lives on the agent row, not the session row, and multiple agents in the same session could theoretically have different cwd values.

  6. Status — The status column on agents ("active", etc.) is per-agent, not per-session. A session is "active" if any agent in it is active.

The sessions view would need to be a proper SQL aggregation:

SELECT
    a.session_id,
    a.model_identifier,
    a.cwd,
    MIN(a.created_at) as created_at,
    MAX(m.created_at) as last_activity,
    COUNT(DISTINCT a.agent_id) as agent_count,
    COUNT(m.id) as message_count,
    SUM(m.total_tokens) as total_tokens,
    MAX(a.status) as status  -- "active" > other statuses
FROM agents a
LEFT JOIN messages m ON a.agent_id = m.agent_id
GROUP BY a.session_id
ORDER BY last_activity DESC

This is why a raw table query won't work — agents can't write this. But a pre-canned view can.


Backward Compatibility #

The current query_memory signature should remain functional. The view-based approach adds a view parameter that defaults to "messages", preserving current behavior. Deprecated parameters (mode) can coexist with new ones (include) during a transition period.


Feedback Collection #

This document is a living record. Update it as:

  • New pain points emerge from agent usage
  • Performance issues are discovered (large sessions, many agents)
  • New query patterns are identified
  • The implementation reveals constraints not visible in design

Known agent anti-patterns to watch for #

  • Agents browsing entire sessions to find a single fact (should use search)
  • Agents re-reading the same session multiple times across conversations (should summarize once)
  • Agents failing to paginate and hitting limits silently
  • Agents confusing session_id with agent_id