picoclaw/pkg/agent
Liu Yuan 15a70ac45c
feat(seahorse): implement short-term memory engine (LCM) (#2285)
* feat(seahorse): implement short-term memory engine of seahorse

Add pkg/seahorse/ module implementing a SQLite-backed DAG-based summary
hierarchy for context management, ported from lossless-claw's LCM design:

- types.go + short_constants.go: core types (Message, Summary, Conversation,
  ContextItem) and configuration constants (fanout, token targets, thresholds)
- migration.go: idempotent DB schema with FTS5 trigram tokenizer for CJK
- store.go: full SQLite CRUD (conversations, messages, summaries DAG,
  context_items with ordinal gap numbering, FTS5 search)
- short_engine.go: Engine lifecycle (NewEngine, Ingest, Assemble, Compact),
  session pattern filtering (ignore/stateless glob→regex compilation),
  per-session mutex via sync.Map
- short_assembler.go: budget-aware context assembly with fresh tail protection
  (32 messages), oldest-first eviction, summary XML formatting, RebuildContextItems
- short_compaction.go: leaf compaction (messages→summary) and condensed
  compaction (summaries→higher-level summary), 3-level LLM escalation,
  CompactUntilUnder for emergency overflow
- short_retrieval.go: lookupByID, FTS5/LIKE search, recursive expand with
  token cap
- context_seahorse.go: agent.ContextManager adapter, registered as "seahorse",
  provider↔seahorse message type conversion (ToolCalls, tool_result)

* fix(seahorse): correct 3 adapter bugs in context management

- TokenCount: use full message (Content+ToolCalls+Media) instead of Content-only
- Empty Content: rebuild Content from tool_result Parts when stored empty
- Duplicate summaries: summaries only in Summary field, not in History messages
- Grep: fix SearchResult.Snippet→Content for summaries
- Schema: fix FTS5 SQL uses VIRTUAL TABLE not TEMP TABLE
- TestFTS5SQLConstants: verify FTS5 SQL syntax correctness
- Test: fix flaky TestCompactLeaf

* fix(agent): ingest steering messages into seahorse SQLite

Steering messages were only persisted to session JSONL but not ingested
into seahorse SQLite, causing them to be missing from context assembly.

Added `ts.ingestMessage(turnCtx, al, pm)` call in the steering message
injection block alongside the existing JSONL persistence.

Test: TestSeahorseSteeringMessageIngested verifies steering messages
appear in seahorse SQLite DB after being processed.

* fix(seahorse): address 3 blocking bugs from code review

- Fix resequenceContextItemsTx scan error handling (store.go:850)
  Changed `return err` to `return scanErr` to properly propagate scan errors
  instead of returning nil (which silently corrupts data)

- Fix sql.NullString for INTEGER column (store.go:847)
  Changed `mid` from sql.NullString to sql.NullInt64 since message_id
  is INTEGER in schema. Removed unnecessary strconv.ParseInt call.

- Fix compactCondensed fallback deleting non-candidate items
  Added ReplaceContextItemsWithSummary method for per-item deletion
  when candidates are not contiguous in ordinal space.
  Optimized to use range deletion when candidates are consecutive.

* fix(seahorse): pass Budget to Compact for correct condensed threshold

Issue #4 from PR review: When Budget was not passed to seahorse.Compact,
it defaulted to `tokensBefore * 0.75`, making `tokensBefore > budget`
always true and causing condensed compaction to trigger unnecessarily.

Changes:
- context_seahorse.go: Forward Budget from CompactRequest to CompactInput
- loop.go: Pass Budget (ContextWindow) in all 3 Compact calls
- Add test verifying condensed is skipped when tokens < threshold
- Fix lint issues in store.go and store_test.go

* fix(seahorse): add mutex for assembler lazy initialization

Issue #5 from PR review: The check-then-create pattern for e.assembler
was a data race when multiple goroutines called Assemble() concurrently:
    if e.assembler == nil {
        e.assembler = &Assembler{...}
    }

Changes:
- Add assemblerMu sync.Mutex to Engine struct
- Add initAssemblerOnce() using double-checked locking (same pattern as initCompactionOnce)
- Add TestAssemblerLazyInitRace to verify thread-safety

* fix(seahorse): handle non-consecutive depths in selectShallowestCondensationCandidate

Issue #8 from PR review: the loop iterated depth 0, 1, 2... assuming
consecutive keys, but break when key was missing caused deeper depths
to never be checked.

Fix: collect all existing depth keys, sort, then iterate in order.

* fix(seahorse): wrap DeleteMessagesAfterID and appendContextItems in transactions

- DeleteMessagesAfterID: wrap all DELETE operations in a transaction for
  atomicity, remove redundant manual FTS delete (handled by trigger)
- appendContextItems: use transaction to fix read-then-write race condition
- Add GetMaxOrdinalTx and resolveItemTokenCountTx for transaction-scoped queries
- Remove unused resolveItemTokenCount function

Fixes PR review issues 6 and 7.

* fix(seahorse): derive readable content from Parts and cap CompactUntilUnder iterations

- Derive readable content from MessageParts in AddMessageWithParts so
  FTS5 indexing and summary formatting can access tool call information
- formatMessagesForSummary and truncateSummary now fall back to Parts
  when Content is empty, fixing blank summaries for Part-based messages
- Add MaxCompactIterations (20) to prevent CompactUntilUnder infinite
  loops; exceeded iterations are logged as warnings
2026-04-05 09:05:16 +08:00
..
context.go feat(web): support image messages in pico chat (#2299) 2026-04-03 14:15:20 +08:00
context_budget.go feat(seahorse): implement short-term memory engine (LCM) (#2285) 2026-04-05 09:05:16 +08:00
context_budget_test.go feat(seahorse): implement short-term memory engine (LCM) (#2285) 2026-04-05 09:05:16 +08:00
context_cache_test.go feat(web): support image messages in pico chat (#2299) 2026-04-03 14:15:20 +08:00
context_legacy.go feat(seahorse): implement short-term memory engine (LCM) (#2285) 2026-04-05 09:05:16 +08:00
context_manager.go feat(seahorse): implement short-term memory engine (LCM) (#2285) 2026-04-05 09:05:16 +08:00
context_manager_test.go feat: add ContextManager abstraction for pluggable context management (#2203) 2026-04-02 00:08:15 +08:00
context_seahorse.go feat(seahorse): implement short-term memory engine (LCM) (#2285) 2026-04-05 09:05:16 +08:00
context_seahorse_test.go feat(seahorse): implement short-term memory engine (LCM) (#2285) 2026-04-05 09:05:16 +08:00
context_test.go fix(provider): deduplicate tool results and merge consecutive tool_result blocks for Anthropic API (#1793) 2026-03-23 17:24:46 +08:00
definition.go merge: resolve conflicts between refactor/agent and main 2026-03-22 19:21:58 +08:00
definition_test.go merge: resolve conflicts between refactor/agent and main 2026-03-22 19:21:58 +08:00
eventbus.go merge: resolve conflicts between refactor/agent and main 2026-03-22 19:21:58 +08:00
eventbus_test.go feat: add ContextManager abstraction for pluggable context management (#2203) 2026-04-02 00:08:15 +08:00
events.go feat: add ContextManager abstraction for pluggable context management (#2203) 2026-04-02 00:08:15 +08:00
hook_mount.go merge: resolve conflicts between refactor/agent and main 2026-03-22 19:21:58 +08:00
hook_mount_test.go Merge branch 'main' into version 2026-03-23 10:54:08 +08:00
hook_process.go merge: resolve conflicts between refactor/agent and main 2026-03-22 19:21:58 +08:00
hook_process_test.go merge: resolve conflicts between refactor/agent and main 2026-03-22 19:21:58 +08:00
hooks.go merge: resolve conflicts between refactor/agent and main 2026-03-22 19:21:58 +08:00
hooks_test.go Merge branch 'main' into version 2026-03-23 10:54:08 +08:00
instance.go Feat/tool read_file by lines (#1981) 2026-04-02 18:49:08 +08:00
instance_test.go feat(model): llm rate limiting (#2198) 2026-04-02 19:26:26 +08:00
loop.go feat(seahorse): implement short-term memory engine (LCM) (#2285) 2026-04-05 09:05:16 +08:00
loop_mcp.go feat(mcp): store oversized text results as artifacts (#2308) 2026-04-04 01:30:36 +08:00
loop_mcp_test.go feat(mcp): per server deferred mode (#1654) 2026-03-19 17:03:17 +08:00
loop_media.go resolve conflicts 2026-03-22 23:36:25 +01:00
loop_test.go feat(channels): make Channel.Send return delivered message IDs (#2190) 2026-03-31 11:07:32 +08:00
memory.go refactor(modernize): apply safe modernize fixes 2026-02-27 16:35:07 +08:00
mock_provider_test.go feat(fmt): Fix formatting 2026-02-19 22:05:15 +02:00
model_resolution.go feat(model): llm rate limiting (#2198) 2026-04-02 19:26:26 +08:00
registry.go feat(session): integrate JSONL persistence into agent loop (#1170) 2026-03-10 15:14:09 +08:00
registry_test.go refactor Config to add Version and migratable 2026-03-12 13:52:55 +08:00
steering.go fix(agent): fix subturn panic result, hard abort rollback, and drain bus exit 2026-03-22 20:35:14 +08:00
steering_test.go Merge branch 'main' into version 2026-03-23 10:54:08 +08:00
subturn.go feat(seahorse): implement short-term memory engine (LCM) (#2285) 2026-04-05 09:05:16 +08:00
subturn_test.go Merge branch 'main' into version 2026-03-23 10:54:08 +08:00
thinking.go feat: add extended thinking support for Anthropic models (#1076) 2026-03-05 09:51:18 +08:00
thinking_test.go feat: add extended thinking support for Anthropic models (#1076) 2026-03-05 09:51:18 +08:00
turn.go feat: add ContextManager abstraction for pluggable context management (#2203) 2026-04-02 00:08:15 +08:00