21b5e6b0d4
- Introduced a new design document for research backend integration, outlining scope, architecture, and API specifications. - Added frontend API service functions for managing research agents, nodes, and reports. - Extended existing Go backend packages to support new research functionalities. feat: Implement PicoClaw context window improvements - Created a design document detailing seven context-window improvements to enhance performance and prevent errors. - Improvements include token estimation fixes, structured error handling, and proactive context management strategies. fix: Document known issues and resolutions - Added a known issues document detailing a critical compile error related to invalid Unicode escapes in Go, including the fix applied. docs: Add context management skill documentation - Introduced a new skill for context management, detailing usage scenarios, project mapping, state persistence, and integration with PicoClaw.
70 lines
4 KiB
Markdown
70 lines
4 KiB
Markdown
# PicoClaw Context Window Improvements — Design Document
|
||
|
||
## Date
|
||
2025-01-09
|
||
|
||
## Scope
|
||
|
||
Implement 7 context-window improvements to prevent "Context window exceeded" errors, based on research comparing PicoClaw's current approach with OpenCode's proven conservative strategy.
|
||
|
||
## Non-Goals
|
||
- No changes to provider adapters (Anthropic, OpenAI, etc.)
|
||
- No changes to the session store (JSONL) format
|
||
- No changes to the seahorse database schema
|
||
- No changes to tool execution behavior
|
||
|
||
## Architecture
|
||
|
||
### Improvement 1: Fix Token Estimation
|
||
- **File**: `pkg/tokenizer/estimator.go`
|
||
- **Change**: Remove `CharsPerToken` config (already at 4.0 chars/token). Add `CharsPerToken` config struct to `pkg/tokenizer` for future tuning.
|
||
- **Status**: Already partially implemented (4.0 chars/token). Need to add Config struct.
|
||
- **Risk**: Very low. The 4.0 heuristic is already the default.
|
||
|
||
### Improvement 2: Add ContextOverflowError Type
|
||
- **File**: Create `pkg/agent/errors.go`
|
||
- **Change**: Add structured `ContextOverflowError` with Model, ContextWindow, RequestedTokens, Reason fields. Update `pipeline_llm.go` to wrap/recognize this error.
|
||
- **Risk**: Low. New file; minimal existing dependency changes.
|
||
|
||
### Improvement 3: Token-Aware Fresh Tail Protection
|
||
- **File**: `pkg/seahorse/short_constants.go`, `pkg/seahorse/short_assembler.go`
|
||
- **Change**: Replace fixed `FreshTailCount` (32 messages) with token-budget-aware `CalculateFreshTailBudget(contextWindow)` (OpenCode's proven approach: 25% of usable context, bounded by 2000–8000 tokens, with minimum 2 turns).
|
||
- **Risk**: Medium. Changes core assembler logic. All seahorse tests will need updates for the new budget calculation.
|
||
|
||
### Improvement 4: Structured Summary Template
|
||
- **File**: `pkg/seahorse/short_compaction.go`
|
||
- **Change**: Add `SummaryTemplate` constant and `UseStructuredSummaries` toggle. Apply template in `generateLeafSummary` and `generateCondensedSummary`.
|
||
- **Risk**: Low. Template is additive; old behavior preserved as fallback.
|
||
|
||
### Improvement 5: Tool Output Pruning
|
||
- **File**: Create `pkg/utils/tool_pruner.go`
|
||
- **Change**: Add `PruneToolOutputs(messages, protectedTurns, pruneThreshold)` that walks backward and replaces old tool outputs with stubs.
|
||
- **Risk**: Medium. New utility; needs careful edge-case handling (orphaned tool results).
|
||
|
||
### Improvement 6: Proactive vs Reactive Compaction Separation
|
||
- **File**: `pkg/agent/context_manager.go`, `pkg/agent/pipeline_setup.go`, `pkg/agent/pipeline_llm.go`
|
||
- **Change**: Add `CompactReason` type with `proactive`, `retry`, `overflow` constants. Rename `ContextCompressReasonProactive` and `ContextCompressReasonRetry` to use the new type. Update all references.
|
||
- **Risk**: Medium. Refactors enum type used across pipeline.
|
||
|
||
### Improvement 7: Tool Output Truncation with Disk Saving
|
||
- **File**: Create `pkg/utils/truncate.go`
|
||
- **Change**: Add `TruncatedResult` struct and `TruncateToolOutput` function. Saves full output to disk, returns preview + hint.
|
||
- **Risk**: Low. New utility; no existing callers.
|
||
|
||
## Testing Strategy
|
||
- Unit tests for new utilities (`pkg/utils/tool_pruner_test.go`, `pkg/utils/truncate_test.go`)
|
||
- Update `pkg/seahorse/short_assembler_test.go` for token-budget fresh tail
|
||
- Update `pkg/agent/pipeline_llm_test.go` (or create one) for `ContextOverflowError` handling
|
||
- Ensure `make test` passes
|
||
|
||
## Rollout Notes
|
||
- The `FreshTailCount` constant in `short_constants.go` will be deprecated but kept for backward compatibility (exported but unused).
|
||
- `ContextCompressReasonProactive` and `ContextCompressReasonRetry` constants will be kept as deprecated aliases.
|
||
|
||
## Failure Mode Analysis
|
||
1. **Fresh tail budget too large**: Could reduce usable context. Mitigated by MinPreserveRecentTokens bound and 25% ratio.
|
||
2. **Structured summary too verbose**: Could increase token count instead of saving. Mitigated by keeping old format as fallback.
|
||
3. **Tool pruning breaks tool-call chains**: Could orphan tool results. Mitigated by protectedTurns parameter and turn-boundary safety.
|
||
|
||
## Approved
|
||
Proceed to implementation.
|