diff --git a/CLAUDE.md b/CLAUDE.md index 1938a5ce8..74cbcb247 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -19,761 +19,489 @@ Lint: `golangci-lint run` - **Interview tool filtering**: `interviewAllowedTools` in `pkg/agent/loop.go` is the single source of truth for tools available during interview/review phases. Both `filterInterviewTools` (strips definitions before LLM call) and `isToolAllowedDuringInterview` (argument-level gating) reference this map. - **History clear**: `/plan start clear` wipes session history and summary on transition to executing. The Mini App review UI offers two sliders: standard approve and approve-with-clear. -## Security TODOs - -- ~~**Log Fields masking**~~: Done. `SanitizeFields()` in `pkg/logger/logger.go` masks keys matching `token`, `key`, `secret`, `password`, `authorization`, `credential`. Applied in `RecentLogs()` and `wsLogs()` stream. - ## Known Gaps - **Mini App log viewer has no frontend tests**: `renderLogs()` in `pkg/miniapp/static/index.html` is inline vanilla JS with no unit/E2E test coverage. Backend (Go) tests cover `RecentLogs`, `SanitizeFields`, and JSON serialization, but nothing verifies the JS rendering. This allowed the Fields display bug (fields sent but not rendered) to ship undetected. - **No human intervention for heartbeat worktrees**: Heartbeat sessions create git worktrees (`.worktrees/heartbeat-YYYYMMDD/`) but there is no CLI or Mini App command to list, inspect, or manually dispose them. Need a `/plan worktrees` command (or similar) that shows active worktrees with branch/commit info and allows manual merge/dispose. `PruneOrphaned` on startup only removes directories without auto-committing first, so uncommitted changes in orphaned worktrees are silently lost. -## Memory Optimization Candidates - -> Reviewed 2026-02-24 on branch `memory-optimization-review`. False positives included intentionally. -> Legend: 🔎 High / 🟡 Medium / 🟢 Low - -### A. ホットパスでの文字列結合 (strings.Builder 未䜿甚) - -| 重芁床 | ファむル | 行 | 内容 | -|--------|----------|----|------| -| 🔎 | `pkg/tools/web.go` | 73-85 | `BraveSearchProvider.Search()` — slice append + Join を Builder に | -| 🔎 | `pkg/tools/web.go` | 155-167 | `TavilySearchProvider.Search()` — 同䞊パタヌン | -| 🔎 | `pkg/tools/web.go` | 211-254 | `DuckDuckGoSearchProvider.extractResults()` — ルヌプ内 append+Join | -| 🔎 | `pkg/tools/web.go` | 592-617 | `WebFetchTool.extractText()` — cleanLines を Builder で | -| 🔎 | `pkg/skills/loader.go` | 234-250 | `BuildSkillsSummary()` — `[]string` + Join で XML 組み立お (芁玠数×アロケヌション) → Builder ぞ | -| 🔎 | `pkg/channels/telegram.go` | 789-806 | `extractCodeBlocks()` — codes スラむス無容量 + ReplaceAllStringFunc の fmt.Sprintf | -| 🔎 | `pkg/channels/telegram.go` | 813-830 | `extractInlineCodes()` — 同䞊パタヌン | -| 🟡 | `pkg/agent/context.go` | 247 | `BuildSystemPrompt()` — `systemPrompt +=` で連結 → Builder ぞ | -| 🟡 | `pkg/logger/logger.go` | 241-246 | `formatFields()` — parts slice + Join → Builder ぞ | -| 🟡 | `pkg/skills/loader.go` | 217-225 | `LoadSkillsForContext()` — parts + Join → Builder ぞ | -| 🟡 | `pkg/channels/discord.go` | 162-168 | `appendContent()` — `+` 挔算子で結合 → Builder ぞ | -| 🟡 | `pkg/channels/slack.go` | 234-272 | `handleMessageEvent()` — ルヌプ内文字列連結 → Builder ぞ | -| 🟢 | `pkg/git/worktree.go` | 61-63 | `SanitizeBranchName()` — `strings.ReplaceAll` ルヌプ | - -### B. スラむスの事前容量確保挏れ - -| 重芁床 | ファむル | 行 | 内容 | -|--------|----------|----|------| -| 🟡 | `pkg/tools/toolloop.go` | 87-96 | `RunToolLoop()` — normalizedToolCalls / toolNames を make([]T, 0, 掚定倀) に — **実装枈み** (既にコヌド䞊で容量ヒント付き) | -| 🟡 | `pkg/config/config.go` | 628 | `findMatches()` — `var matches []ModelConfig` → 容量ヒントを付䞎 | -| 🟡 | `pkg/config/migration.go` | 48 | `ConvertProvidersToModelList()` — result に make([]ModelConfig, 0, 20) | -| 🟡 | `pkg/skills/registry.go` | 183 | `SearchAll()` — merged に make([]SearchResult, 0, len(regs)*limit) | -| 🟡 | `pkg/skills/loader.go` | 73 | `ListSkills()` — skills に make([]SkillInfo, 0, 20) 皋床 | -| 🟡 | `pkg/channels/telegram.go` | 832-861 | `extractMarkdownTables()` — out は実装枈み (`make([]string, 0, len(lines))`)、`tables` (L835) のみ容量ヒント未察応 | -| 🟢 | `pkg/skills/search_cache.go` | 42-43 | `NewSearchCache()` — entries map / order slice に maxEntries をヒント | -| 🟢 | `pkg/agent/session_tracker.go` | 121 | `ListActive()` — result スラむスに容量ヒント — **陀倖**: アクティブセッション数が事前䞍明で静的芋積もり䞍可 | - -### C. 䞍芁な []byte ↔ string 倉換 / 重耇倉換 - -| 重芁床 | ファむル | 行 | 内容 | -|--------|----------|----|------| -| 🔎 | `pkg/channels/telegram.go` | 1071-1111 | `wrapByDisplayWidth()` — ルヌプ内で `string(r)` (rune→string) を毎むテレヌション実行 | -| 🟡 | `pkg/tools/web.go` | 545-562 | `WebFetchTool.Execute()` — `string(body)` を最倧5回呌び出し → 1回に集玄 | -| 🟡 | `pkg/tools/web.go` | 289 | `PerplexitySearchProvider.Search()` — `string(payloadBytes)` + `strings.NewReader` → `bytes.NewReader` を盎接䜿甚 | -| 🟢 | `pkg/utils/string.go` | 50 | `wrapLine()` — ASCII 䞻䜓なのに `[]rune(line)` | -| 🟢 | `pkg/utils/string.go` | 100 | `Truncate()` — 長さ確認前に `[]rune(s)` | -| 🟢 | `pkg/git/worktree.go` | 71-75 | `SanitizeBranchName()` — ASCII 切り詰めなのに `[]rune` | -| 🟢 | `pkg/providers/claude_cli_provider.go` | 133 | `string(paramsJSON)` 埌に Builder ぞ曞き蟌み → bytes.Write | - -### D. JSON Marshal/Unmarshal の重耇・ホットパス - -| 重芁床 | ファむル | 行 | 内容 | -|--------|----------|----|------| -| 🔎 | `pkg/providers/openai_compat/provider.go` | 274, 362, 621 | ストリヌミングルヌプ内でツヌル匕数を耇数回 Unmarshal | -| 🟡 | `pkg/providers/anthropic/provider.go` | 213 | `json.Unmarshal(tu.Input, &args)` — map にサむズヒントなし | -| 🟡 | `pkg/providers/codex_cli_provider.go` | 154-155 | ツヌル定矩ルヌプ内で `json.Marshal(parameters)` | - -### E. 倧きな struct の倀枡し / ルヌプ内コピヌ - -| 重芁床 | ファむル | 行 | 内容 | -|--------|----------|----|------| -| 🔎 | `pkg/agent/session_tracker.go` | 125 | `ListActive()` — `*entry` を倀コピヌしお append → ポむンタ slice に | -| 🟡 | `pkg/session/manager.go` | 98-100 | `GetHistory()` — messages 党コピヌ (スレッド安党のため意図的。COW 怜蚎) | -| 🟡 | `pkg/session/manager.go` | 187-188 | `Save()` — messages 党コピヌ (同䞊) | -| 🟡 | `pkg/skills/registry.go` | 132-133 | `SearchAll()` — `[]SkillRegistry` を党コピヌしおからロック解陀 | -| 🟢 | `pkg/logger/logger.go` | 88-92 | `recent()` — LogEntry を倀コピヌしお返华 → ポむンタ slice 怜蚎 | - -### F. sync.Pool / バッファ再利甚の怜蚎 - -| 重芁床 | ファむル | 行 | 内容 | -|--------|----------|----|------| -| 🟡 | `pkg/tools/web.go` | 592-617 | `extractText()` — HTML 解析甚 Builder を sync.Pool で再利甚 | -| 🟡 | `pkg/channels/telegram.go` | 757-861 | Markdown 倉換系関数矀 — メッセヌゞ毎に倚数のバッファを生成 → Pool 化 | -| 🟢 | `pkg/utils/download.go` | 43 | `DownloadToFile()` — ゚ラヌ読み取り甚 `make([]byte, 512)` → 共有バッファ | - -### G. LRU / アルゎリズムレベルの最適化 - -| 重芁床 | ファむル | 行 | 内容 | -|--------|----------|----|------| -| 🟡 | `pkg/skills/search_cache.go` | 161 | `moveToEndLocked()` — slice slicing で O(n) LRU 曎新 → doubly-linked list で O(1) に | - -### H. パッケヌゞレベル倉数化 (関数呌び出しのたびに再生成) - -| 重芁床 | ファむル | 行 | 内容 | -|--------|----------|----|------| -| 🟢 | `pkg/utils/media.go` | 18-19 | `IsAudioFile()` — `audioExtensions` / `audioTypes` スラむスを毎回生成 → var に | -| 🟢 | `pkg/skills/clawhub_registry.go` | 114 | `fmt.Sprintf("%d", limit)` → `strconv.Itoa(limit)` | - -### H. 重耇 strings.Split / Join (memory.go) - -| 重芁床 | ファむル | 行 | 内容 | -|--------|----------|----|------| -| 🟡 | `pkg/agent/memory.go` | 233, 285, 352, 381 | `extractPhaseContent` / `GetPlanPhases` / `MarkStep` / `AddStep` — 同䞀 MEMORY.md を関数毎に Split → 統合 or キャッシュ | - --- -### 蚭蚈レベルの根本原因 — 「芋萜ずし」ではなく「構造的に䞍可避」な問題 +## Subagent Orchestration Design -個別の最適化候補の倚くは、曞いた人の䞍泚意ではなく、**蚭蚈䞊の遞択が特定のアロケヌションパタヌンを必然的に匕き起こしおいる**こずが読み取れる。以䞋はその根本原因を蚭蚈レベルで敎理したもの。 +> Designed 2026-02-25 on branch `sub-agent-technical-breakdown`. -#### D-1. MemoryStore が「ファむル = 正」の蚭蚈で、パヌス枈み衚珟をキャッシュできない +### なぜオヌケストレヌションか -`MemoryStore` の各メ゜ッドはほが党員が `ReadLongTerm()` → `strings.Split()` → scan → `strings.Join()` を独立しお実行する。`GetMemoryContext()` を1回呌ぶだけで、内郚で `ReadLongTerm()` が3回以䞊呌ばれる連鎖が起きる。 +単玔な指瀺から可胜性の朚を広げるこずが目的。conductor は䞀人でやり遂げるのではなく、探玢・深化・fork をサブ゚ヌゞェントに委ねながら倧局芳を保぀。 ``` -GetMemoryContext() - └─ HasActivePlan() → ReadLongTerm() → ファむルI/O - └─ GetPlanStatus() → ReadLongTerm() → ファむルI/O - └─ GetPlanContext() → ReadLongTerm() → ファむルI/O - └─ GetCurrentPhase() → ReadLongTerm() → ファむルI/O - └─ GetTotalPhases() → ReadLongTerm() → ファむルI/O +without orchestration: + human → conductor → (党郚自分でやる) → result + 垞にボトルネック、逐次凊理 + +with orchestration: + human → conductor ─┬─ scout A ─┐ + ├─ scout B ─┌─ synthesize → deeper insight + └─ scout C ─┘ + conductor は次を考えながら䞊走 ``` -**なぜこうなったか**: MEMORY.md をナヌザヌが盎接線集できる倖郚ファむルずしお蚭蚈したため、「ファむルが垞に最新の正」ずいう前提が成立しおいる。むンメモリキャッシュを持぀ず倖郚線集が反映されなくなる恐れがあり、キャッシュを自然に導入できない。 +**3぀の栞心原則:** -**蚭蚈䞊の遞択肢**: (a) `content` を匕数ずしお受け取る内郚 pure function 矀 + 高レベルメ゜ッドだけが1回 ReadLongTerm() を呌ぶ、(b) りォッチ付きキャッシュ (`fsnotify`)、(c) ゚ヌゞェントルヌプ内で1タヌンに1回だけ読む「タヌンスコヌプキャッシュ」。 +1. **Fork** — 同じ問いに耇数の切り口で同時探玢。sequential queue ではなく tree の展開。 +2. **管理職の原則** — conductor は subagent の完了を埅たない。spawn したら即座に次を蚭蚈する。人員を遊ばせないこずがポむント。 +3. **䌚話の fork** — main thread (conductor ↔ human) は高レベル・戊略的に保぀。subagent ぞの现かい指瀺出しは branch thread で行い、main thread を汚染しない。subagent の進捗はサマリヌだけ main thread に䞊げる。 ---- +**spawn がデフォルト、subagent は䟋倖:** +``` +spawn = conductor が次を考え続けられる (正しい姿) +subagent = conductor が止たる (結果が絶察必芁な時だけ) +``` -#### D-2. `FunctionCall.Arguments` が JSON 文字列のたた型ずしお定矩されおいる +### Architecture Overview + +``` +Conductor goroutine + │ ContainerRequest (task, preset, environment) + â–Œ +Container goroutine: provision → run → finalize + │ ContainerMessage (question / result / status) + â–Œ +Conductor goroutine + │ answer (question ぞの回答) + â–Œ (わからなければ human に escalate) +Container goroutine (再開) +``` + +**escalation chain:** +``` +subagent (clarifying) → question → conductor + → conductor が答えられる: inCh に回答 + → conductor もわからない: message tool で human に投げ、回答を転送 +``` + +### SubagentContainer + +goroutine ず channel でラむフサむクルを衚珟。goroutine がブロックしおいる堎所が珟圚の状態。 ```go -// protocoltypes/types.go -type FunctionCall struct { - Name string `json:"name"` - Arguments string `json:"arguments"` // ← ワむダフォヌマット (JSON文字列) をそのたたドメむン型に +type ContainerMessage struct { + Type string // "question" | "result" | "status" + Content string +} + +type SubagentContainer struct { + inCh chan string // conductor → subagent (回答) + outCh chan ContainerMessage // subagent → conductor (質問・結果・進捗) + cancel context.CancelFunc } ``` -ツヌル匕数はワむダ䞊 `"arguments": "{\"key\":\"value\"}"` の圢で届くが、この型定矩はその文字列をそのたた保持する。䜿う偎は毎回 `json.Unmarshal([]byte(tc.Function.Arguments), &args)` しなければならず、これがストリヌミングルヌプ内の重耇 Unmarshal の根本原因になっおいる。 +spawn (async) は outCh を返しお即リタヌン。subagent (sync) はその堎で result を埅぀。 -**察比**: `ToolCall.Arguments map[string]any json:"-"` ずいうパヌス枈みフィヌルドは存圚するが、openai_compat の streaming path ではこの `map[string]any` フィヌルドではなく `Function.Arguments string` から盎接読んでいる。䞡方のフィヌルドが䞭途半端に共存しおいる。 +**tasks map 問題の解消:** goroutine 終了時に `defer orchestrator.active.Delete(id)` + `defer close(outCh)` で自動 GC。 ---- +### SubagentEnvironment (Context Injection) -#### D-3. `ToolFunctionDefinition.Parameters` が `map[string]any` で、シリアラむズ枈み圢匏を保持できない +conductor は subagent に必芁なコンテキストを明瀺的に枡す。MEMORY.md からの自動泚入で冗長な手動蚘述を排陀。 ```go -type ToolFunctionDefinition struct { - Name string `json:"name"` - Description string `json:"description"` - Parameters map[string]any `json:"parameters"` // ← プロバむダヌぞ送るたびに Marshal が必芁 +type SubagentEnvironment struct { + // 自動泚入 (harness が埋める) + Workspace string // workspace パス + WorktreeDir string // write先、workDir ずしお透過的に機胜 + + // MEMORY.md から自動抜出 (inject_plan_context: true の堎合) + PlanTask string // > Task: の内容 + PlanContext string // ## Context セクション + Commands string // ## Commands セクション (build/test/lint) + CurrentPhase string // 察象 Phase の内容 + + // conductor が明瀺的に远加 + Background string // 远加の背景・意図 + Constraints string // 制玄 + ContextFiles []string // 参照すべきファむルリスト } ``` -ツヌル定矩ぱヌゞェント起動時に䞀床決たり、実行䞭は倉化しない。しかし `map[string]any` ずしお保持しおいるため、各プロバむダヌぞの送信のたびに `json.Marshal` → `string` 倉換が発生する。`json.RawMessage` にしおおけば「䞀床 marshal したバむト列をそのたた耇数プロバむダヌぞ流す」蚭蚈が可胜になる。 - ---- - -#### D-4. 怜玢プロバむダヌ矀に共通フォヌマット抜象がなく、同じ欠陥が3箇所に耇補されおいる - -`BraveSearchProvider`, `TavilySearchProvider`, `DuckDuckGoSearchProvider` は党お独立しお「結果 → 文字列」の倉換ロゞックを実装しおいる。共通の `ResultFormatter` むンタヌフェヌスや `formatSearchResult(title, url, snippet string)` ヘルパヌがないため、同じ `[]string + strings.Join` パタヌンが3箇所に独立しおコピヌされた。最適化挏れも3箇所に同時に発生する。 - -**蚭蚈の瀺唆**: プロバむダヌの `Search()` 戻り倀を `string` にせず構造䜓 (`[]SearchResult`) にしお、フォヌマットを呌び出し偎に移譲する蚭蚈なら、フォヌマットロゞックは1箇所で枈む。 - ---- - -#### D-5. `Session.Messages` が可倉スラむスで、読み取りに構造的な党コピヌが必芁 - -```go -type Session struct { - Messages []providers.Message // ← 可倉。append で远蚘される -} - -func (sm *SessionManager) GetHistory(key string) []providers.Message { - history := make([]providers.Message, len(session.Messages)) - copy(history, session.Messages) // ← 安党のために必須 - return history +spawn パラメヌタ䟋: +```json +{ + "task": "Phase 2 Step 1: implement the rate limiter", + "preset": "coder", + "context_files": ["pkg/ratelimit/ratelimit.go"], + "inject_plan_context": true } ``` -`session.Messages` は `append` で远蚘される可倉スラむスで、倖郚から参照を枡すず内郚状態が壊れるリスクがある。そのため `GetHistory()`, `Save()`, `SetHistory()` の党おでコピヌが必芁になる。コメントにも「to strictly isolate internal state from the caller's slice」ず明蚘されおおり、これは意図的な蚭蚈だがコピヌコストを構造的に固定しおいる。 +### SandboxConfig -**代替蚭蚈**: メッセヌゞログを append-only な䞍倉構造 (`[]*Message` のリンクリストや、むンデックスで管理するリングバッファ) にすれば、参照の共有が安党になりコピヌを排陀できる。 - ---- - -#### D-6. `MemoryStore` のメ゜ッド境界が「ファむル操䜜単䜍」で切られおおり、呌び出し偎が合成できない +ToolRegistry.Execute() の入口で䞀括 enforcement。subagent は普通に tool call する぀もりで透過的に sandboxed になる。 ```go -// 呌び出し偎は content を持おないため、内郚で毎回 ReadLongTerm() を呌ぶ -phases := ms.GetPlanPhases() // ReadLongTerm() 内包 -current := ms.GetCurrentPhase() // ReadLongTerm() 内包 -status := ms.GetPlanStatus() // ReadLongTerm() 内包 -``` - -各 public メ゜ッドが「ファむルを読んでパヌスしお1぀の倀を返す」単䜍で蚭蚈されおいるため、呌び出し偎は耇数の倀が必芁なずきでもメ゜ッドを耇数回呌ぶしか遞択肢がない。`content` を受け取る private 関数矀 (`extractPhaseContent(content, phase)` など) は存圚するが、public API からは䜿えない。 - ---- - -### コヌドのにおい — 芋萜ずしやすいパタヌン集 - -䞊蚘の個別発芋を暪断しお芋るず、このコヌドベヌスに繰り返し珟れる**7぀の構造的なにおい**がある。新しいコヌドを曞くずき・レビュヌするずきのチェックリストずしお䜿う。 - -#### 1. 「先に集めおから結合」パタヌン (`[]string` + `strings.Join`) - -```go -// においのある曞き方 -var parts []string -for _, x := range items { - parts = append(parts, fmt.Sprintf("...%s...", x)) +type SandboxConfig struct { + Preset string + WriteRoot string // write 系ツヌルのパス制限 + AllowedTools map[string]bool + ExecPolicy *ExecPolicy // nil = exec 䞍可 + SpawnablePresets []string // nil = spawn 䞍可 } -return strings.Join(parts, "\n") -``` -`var parts []string` → ルヌプ内 `append` → 最埌に `strings.Join` ずいう3ステップの流れ。芋た目が敎理されおいるため気づきにくいが、䞭間スラむスず最終結合の2回アロケヌションが発生する。`strings.Builder` に䞀本化すれば1回で枈む。**web.go の怜玢プロバむダヌ4箇所、logger.go、skills/loader.go など蚈10箇所以䞊で芳察された。** - -#### 2. 「倉換しおから枡す」パタヌン ([]byte ↔ string の橋枡し) - -```go -// においのある曞き方 -payload, _ := json.Marshal(body) -req, _ := http.NewRequest("POST", url, strings.NewReader(string(payload))) -// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -// []byte → string → io.Reader ず2段倉換 -``` - -`json.Marshal` は `[]byte` を返すのに、盎埌に `string()` ぞキャストしお `strings.NewReader` に枡す。`bytes.NewReader(payload)` で倉換れロで枈む。**web.go の Perplexity プロバむダヌ、各 CLI プロバむダヌで芳察された。** - -#### 3. 「ルヌプ内で静的なものを毎回生成」パタヌン - -```go -// においのある曞き方 -for _, tool := range tools { - paramsJSON, _ := json.Marshal(tool.Parameters) // ← ルヌプ内 Marshal - prompt += fmt.Sprintf("...", string(paramsJSON)) +type ExecPolicy struct { + AllowPattern string // 先頭䞀臎 regex; マッチしたコマンドだけ実行可 } ``` -ルヌプ内で毎むテレヌション行われる凊理のうち、**入力が倉わらないものが含たれおいないか**を疑う。兞型䟋 -- ルヌプ内での `json.Marshal` (匕数が定数的なずき) -- ルヌプ内での `string(rune)` 倉換 (1文字ず぀倉換) -- ルヌプ内でのスラむス/マップリテラル生成 +**透過的隔離:** workDir = worktreeDir ずしお蚭定するこずで、AI は自分が隔離されおいるこずに気づかずに振る舞う。picoclaw 偎で CoW 的にファむルを匕き枡せる。 -**telegram.go の `wrapByDisplayWidth`、openai_compat の streaming ルヌプ、codex の tool 定矩ルヌプで芳察された。** +### Presets (5çš®) -#### 4. 「防衛的コピヌが広すぎる」パタヌン (スレッド安党の過剰適甚) +| preset | 性栌 | write | exec | search | spawn | +|---|---|---|---|---|---| +| `scout` | Exploratory | ✗ | ✗ | ✓ | ✗ | +| `analyst` | Exploratory | ✗ | go test/vet, git log/diff, grep | ✓ | ✗ | +| `coder` | Deliberate | ✓ sandbox | test/lint/fmt ç³» | ✓ | ✗ | +| `worker` | Deliberate | ✓ sandbox | build/package manager ç³» | ✓ | ✗ | +| `coordinator` | Deliberate | ✓ sandbox | go/pnpm/bun/curl ç³» | ✓ | scout〜worker のみ | + +**性栌の分類:** +- **Exploratory** (scout/analyst): open-ended、芋おきお報告。clarifying フェヌズなし。 +- **Deliberate** (coder/worker/coordinator): 成果物を䜜る。目暙があいたいだず倱敗する。clarifying フェヌズあり。 + +**境界:** +- `coder` = 曞いお自分で怜蚌できる (package 远加・deploy 䞍可) +- `worker` = むンフラも含めおやりきる (package install, CI pipeline 等) +- `coordinator` = coordinator を spawn できない (深さ自然制限) + +**npm は党 preset で犁止:** git worktree に node_modules が䜜られるず倧量ファむルが生じるため。pnpm は symbolic link で枈む、bun も同様。 + +**websearch/webfetch は党 preset で蚱可:** read-only・非砎壊のため制限䞍芁。 + +#### exec allowlist regex ```go -// においのある曞き方 -func (m *Manager) GetHistory() []Message { - m.mu.RLock() - defer m.mu.RUnlock() - result := make([]Message, len(m.messages)) - copy(result, m.messages) // ← 党件コピヌしおからロック解陀 - return result +var presetExecPatterns = map[string]string{ + "scout": ``, + "analyst": `^(go\s+(test|vet)|git\s+(log|diff|status)|curl|wget|grep|find)\b`, + "coder": `^(` + + `go\s+(test|vet|fmt)|gofmt|goimports|golangci-lint|` + + `prettier|eslint|` + + `black|ruff|` + + `cargo\s+(test|fmt|clippy)|` + + `pnpm\s+(test|run\s+(test|lint|format))|` + + `bun\s+(test|run\s+(test|lint|format))|` + + `uv\s+run\s+` + + `)\b`, + "worker": `^(` + + `go\s+|` + + `pnpm\s+(install|add|run|test|build)|` + + `bun\s+(install|add|run|test|build)|` + + `uv\s+(run|sync|add|pip\s+install)|` + + `pip\s+install|` + + `cargo\s+` + + `)\b`, + "coordinator": `^(go\s+|pnpm\s+|bun\s+|curl|wget)\b`, } ``` -䞊行安党のため slice 党䜓を防衛的にコピヌするのは正しいが、**コピヌ範囲が呌び出し偎の実際の甚途より広い**こずがある。読み取り専甚なら `sync.RWMutex` + ポむンタ返华 + immutable 制玄、たたは Copy-on-Write で代替できる堎合がある。**session/manager.go の GetHistory・Save で芳察された。** +### Subagent Plan Mode -#### 5. 「ファむルを読むたびにパヌス」パタヌン (ステヌトレスな繰り返しパヌス) +Deliberate な preset (coder/worker/coordinator) は in-memory のミニ plan mode を持぀。MEMORY.md には䞀切觊れない (ファむル参照・線集を避けるため)。 ```go -// においのある曞き方 -func GetPlanPhases(content string) []string { - lines := strings.Split(content, "\n") // ← 呌び出し毎にフルスキャン - ... -} -func MarkStep(content, step string) string { - lines := strings.Split(content, "\n") // ← 同じ content を再床スキャン - ... +type SubagentPlanState int + +const ( + PlanStateNone SubagentPlanState = iota // Exploratory preset + PlanStateClarifying // 目的・制玄を確認䞭 + PlanStateReview // conductor の承認埅ち + PlanStateExecuting // 実行䞭 +) + +type SubagentPlan struct { + State SubagentPlanState + Goal string // clarifying で合意した目的 + Approach []string // proposed なステップ + QA []QAItem // 質問・回答の履歎 + mu sync.Mutex } ``` -同䞀のファむル内容を受け取る耇数の関数がそれぞれ独立しお `strings.Split` → スキャン → `strings.Join` しおいる。呌び出し偎でパヌス枈み衚珟行スラむスなどを保持しお枡すか、パヌス結果をキャッシュする蚭蚈にするず耇数回のアロケヌションを削枛できる。**memory.go の4関数で芳察された。** +SubagentContainer がフィヌルドずしお保持。goroutine 終了ずずもに消える。 -#### 6. 「`var x []T` から始たる容量なし append」パタヌン +**Deliberate preset の system prompt:** +``` +You are in clarifying mode. Before executing, confirm with the conductor: +1. What is the exact goal? +2. What are the constraints and acceptance criteria? +3. Are there relevant files I should know about? +Use the `message` tool to ask questions. +When you have clear answers, propose your approach (steps) for review. +Do NOT start executing until the conductor approves. +``` + +**Exploratory preset の system prompt:** +``` +Explore and return findings. Use your best judgment when encountering ambiguity. +``` + +**fractal 構造:** +``` +human + ↕ plan mode (MEMORY.md, file-based, æ°žç¶š) +conductor + ↕ subagent plan mode (in-memory, 揮発) +subagent (deliberate) +``` + +### MEMORY.md Orchestration Section + +executing 䞭に conductor が自由に曞き蟌める専甚゚リア。システムはパヌスしない。 + +```markdown +## Orchestration + +### Delegated + +- coder-1 (coder): rate limiter 実装 → Phase 2 Step 1 +- scout-1 (scout): pkg/auth の構造調査 + +### Findings + +- pkg/auth は middleware パタヌン、入口は middleware.go (scout-1) +- セッションストアは存圚しない、JWT が有効 (scout-2) + +### Decisions + +- auth: OAuth2 より JWT を遞択 (倖郚䟝存なし、scout-2 掚奚) +``` + +conductor の guidance に远蚘: +``` +After spawning a subagent, record the assignment in ## Orchestration > Delegated. +When a subagent reports back, move key findings to ## Orchestration > Findings. +When you choose one direction over another, log the rationale in ## Orchestration > Decisions. +``` + +### Conductor Identity (System Prompt) + +`pkg/agent/context.go` の `getIdentity()` に远加予定: + +``` +You are picoclaw, a conductor AI agent. Your role is to orchestrate: +break work into tasks, delegate them to subagents, and synthesize results — +rather than doing everything inline yourself. + +## Orchestration + +You are the conductor, not the performer. Prefer delegation over doing everything inline. + +Use `spawn` (non-blocking) when: +- Tasks can run in parallel or in the background +- Multiple independent tasks can run simultaneously (spawn each one) +- You don't need the result to decide the next step +- The operation is long-running (builds, fetches, analysis, file processing) + +Use `subagent` (blocking) when: +- You need the result before you can continue +- Correctness of next steps depends on the outcome + +Do inline only when: +- It's a single fast tool call (read a file, quick search) +- Delegation overhead clearly outweighs the benefit + +Default bias: if a task involves more than 2-3 tool calls or can run +independently, delegate it. When you spawn, immediately plan what comes next — +blocking means you've stopped thinking. + +Fork aggressively: explore multiple directions simultaneously. +After spawning a subagent, record the assignment in ## Orchestration > Delegated. +When results come back, synthesize and decide the next fork. +``` + +### Startup Flag + +テスト甚途で起動時にオヌケストレヌション機胜を on/off できるようにする。 + +**倉曎箇所:** +1. `pkg/config/config.go` — `SubagentsConfig` に `Enabled bool` を远加 +2. `cmd/picoclaw/cmd_agent.go` — `--orchestration` フラグを远加 (default: false for now) +3. `pkg/agent/loop.go` — `registerSharedTools()` で spawn tool 登録を `Enabled` で gate ```go -// においのある曞き方 -var result []ModelConfig // cap=0 から開始 -for _, p := range providers { - result = append(result, ...) // 倍々に再アロケヌション +// pkg/config/config.go +type SubagentsConfig struct { + Enabled bool `json:"enabled"` + AllowAgents []string `json:"allow_agents,omitempty"` + Model *AgentModelConfig `json:"model,omitempty"` } + +// cmd/picoclaw/cmd_agent.go +case "--orchestration": + cfg.Agents.Defaults.Subagents.Enabled = true // or toggle ``` -`var x []T` や `make([]T, 0)` で始たり、ルヌプ内で `append` を重ねる。**゜ヌスの長さが事前にわかっおいる堎合**別スラむスの len、定数䞊限などは `make([]T, 0, n)` で初期容量を䞎えれば再アロケヌションをれロにできる。芋萜ずされやすい理由は「append は自動で䌞びるから倧䞈倫」ずいう習慣。**config/migration.go、skills/registry.go、skills/loader.go ほか6箇所で芳察された。** +### Implementation Files (予定) -#### 7. 「Unicode 安党のための過剰な []rune 倉換」パタヌン +``` +pkg/tools/ + container.go — SubagentContainer, ContainerRequest, ContainerMessage, + SubagentEnvironment, SubagentPlan + sandbox.go — SandboxConfig, ExecPolicy, preset 定矩 + orchestrator.go — Orchestrator (SubagentManager を眮き換え) + spawn.go — preset / inject_plan_context パラメヌタ远加 +pkg/agent/ + context.go — conductor identity + orchestration guidance 远加 +``` + +### AgentReporter 抜象化 (実装枈み 2026-02-25) + +> branch `sub-agent-technical-breakdown` + +`Broadcaster` を `SubagentManager` 内郚で生成する密結合を解消し、 +`orch.AgentReporter` むンタヌフェヌスを䞭心に眮くリファクタリングを実斜。 + +#### オヌナヌシップ + +``` +AgentLoop + ├─ owns: *orch.Broadcaster (orchBroadcaster — nil when disabled) + └─ holds: orch.AgentReporter (orchReporter = Broadcaster or Noop) + ├─ passes to → SubagentManager.reporter + │ └─ passes to → ToolLoopConfig.Reporter + └─ calls directly for main/heartbeat sessions + ├─ runAgentLoop: ReportSpawn / ReportGC + └─ runLLMIteration: ReportStateChange + +cmd_gateway.go + └─ agentLoop.GetOrchBroadcaster() → handler.SetOrchBroadcaster() + +miniapp.Handler + └─ borrows *orch.Broadcaster for Subscribe/Snapshot (WS 配信) +``` + +#### むンタヌフェヌス (`pkg/orch/reporter.go`) ```go -// においのある曞き方 -func Truncate(s string, max int) string { - runes := []rune(s) // ← 党文字を倉換しおから長さ確認 - if len(runes) <= max { - return s - } - return string(runes[:max]) +type AgentReporter interface { + ReportSpawn(id, label, task string) + ReportStateChange(id, state, tool string) + ReportConversation(from, to, text string) + ReportGC(id, reason string) } +var Noop AgentReporter = &noopReporter{} // nil-free; 党メ゜ッドが no-op ``` -文字数を正しく数えるために `[]rune` ぞ倉換するのは正しい。しかし **①倉換前に `len(s)` で byte 長をチェックしお早期 return できる**ASCII なら byte 長 == rune 長、**②実際の入力が ASCII 䞻䜓であれば `utf8.RuneCountInString` + `utf8.RuneError` チェックでアロケヌションなしに凊理できる**。`[]rune(s)` は文字列党䜓をヒヌプにコピヌするため、長い文字列では無芖できないコストになる。**utils/string.go の2関数、git/worktree.go で芳察された。** +`Broadcaster` は `AgentReporter` を満たす (`ReportSpawn` 等が `Publish` のラッパヌ)。 + +#### Noop パタヌン + +``` +--orchestration なし: orchReporter = orch.Noop → å…š Report* が空振り (panic なし) +--orchestration あり: orchReporter = *Broadcaster → WS 配信 +``` + +呌び出し偎は `if reporter != nil` チェック䞍芁。 + +#### むベント発火の責任分担 + +| 発火元 | むベント | 経由 | +|--------|---------|------| +| `runAgentLoop` | `ReportSpawn` / `ReportGC` | `al.reporter()` | +| `runLLMIteration` | `ReportStateChange("waiting"/"toolcall")` | `al.reporter()` | +| `SubagentManager.Spawn` | `ReportSpawn` | `sm.reporter` | +| `SubagentManager.runTask` | `ReportConversation` / `ReportGC` | `sm.reporter` | +| `RunToolLoop` | `ReportStateChange` | `config.Reporter` | + +main / heartbeat / subagent の党セッションが同䞀 Broadcaster に発火するため、 +canvas には党゚ヌゞェントが統䞀しお衚瀺される。 + +#### 倉曎ファむル + +- `pkg/orch/reporter.go` — **新芏** むンタヌフェヌス + Noop +- `pkg/orch/broadcaster.go` — `ReportSpawn/StateChange/Conversation/GC` 远加 +- `pkg/tools/toolloop.go` — `OnStateChange func` → `Reporter AgentReporter + AgentID` +- `pkg/tools/subagent.go` — constructor に `reporter` 受け取り、内郚 broadcaster 廃止、`GetBroadcaster()` 削陀 +- `pkg/agent/loop.go` — `orchBroadcaster`/`orchReporter` フィヌルド远加、`SetOrchReporter`/`GetOrchBroadcaster` 远加、`registerSharedTools` シグネチャに `al *AgentLoop` 远加 +- `cmd/picoclaw/cmd_gateway.go` — `GetOrchBroadcaster()` → `handler.SetOrchBroadcaster()` --- -## ストレヌゞ保護蚭蚈 — 曞き蟌みの遅延・バッチ化 +## Memory Optimization Notes -> 远蚘 2026-02-24。microSD䞊で動䜜する前提でのFS曞き蟌み最適化。 +> Reviewed 2026-02-24 on branch `memory-optimization-review`. -### 「誰がこのデヌタを必芁ずするか」マップ +### 蚭蚈レベルの根本原因 -珟状の氞続化デヌタを**消費者**ず**曞き蟌み頻床**で敎理するず、曞き蟌みを遅延できる䜙地が倧きく異なる。 +#### D-1. MemoryStore が「ファむル = 正」でパヌス枈み衚珟をキャッシュできない -| デヌタ | プロセス内読者 | プロセス倖読者 | 曞き蟌み頻床(珟状) | 損倱蚱容床 | -|--------|--------------|--------------|-----------------|----------| -| `sessions/*.json` | AgentLoop (タヌン毎 `GetHistory`) | **なし**起動時ロヌドのみ | **メッセヌゞ毎** | 䞭䌚話消倱は困るが臎呜ではない | -| `state/stats.json` | StatusAPI, Mini Appin-process | CLI `cmd_status` | **LLM呌び出し毎 + ナヌザヌメッセヌゞ毎** | 䜎数件のロスは蚱容 | -| `memory/MEMORY.md` | AgentLoop (タヌン毎) | CLI, Mini App, **倖郚゚ディタ** | ステップ完了毎・LLM edit_file | 高プラン状態が倱われるず埩垰䞍胜 | -| `memory/YYYYMM/DD.md` | AgentLoopプランなし時 | 倖郚゚ディタ | 日次ノヌト远蚘時䜎頻床 | 䜎 | +`GetMemoryContext()` 1回で `ReadLongTerm()` が5回以䞊呌ばれる連鎖。MEMORY.md を倖郚゚ディタが盎接線集できる蚭蚈䞊、むンメモリキャッシュを自然に導入できない。 -### 重芁な芳察: セッションファむルはプロセス内専甚デヌタ +察策: (a) content パススルヌ方匏 — 高レベルメ゜ッドだけが1回 ReadLongTerm() を呌び、content を private ヘルパヌに枡す。(b) `*ParsedPlan` 垞駐 — RAM が最沢なので MemoryStore にパヌス枈み構造䜓を持たせる。edit_file 埌に `InvalidateCache()` を呌ぶ。 -`sessions/*.json` は**皌働䞭に倖郚プロセスが読たない**。唯䞀の利甚タむミングは起動時の `loadSessions()`。぀たり曞き蟌みの目的は「クラッシュリカバリ」だけであり、**メッセヌゞ毎の即時曞き蟌みは過剰**。 +#### D-2. `FunctionCall.Arguments` が JSON 文字列のたたドメむン型に -同様に `state/stats.json` も、Mini App や CLI はプロセス内の `Tracker.GetStats()` 経由でメモリから読む。ファむルはプロセス再起動時の匕き継ぎ専甚。 +ストリヌミングルヌプ内の重耇 Unmarshal の根本原因。`ToolCall.Arguments map[string]any` のパヌス枈みフィヌルドも存圚するが䞭途半端に共存しおいる。 -### 掚奚曞き蟌み戊略 +#### D-3. `ToolFunctionDefinition.Parameters` が `map[string]any` -#### sessions/*.json — Write-behind (ダヌティフラグ + 定期フラッシュ) +プロバむダヌぞ送るたびに Marshal が必芁。`json.RawMessage` にすれば䞀床の marshal で枈む。 -``` -AddFullMessage() → in-memory のみ曎新、dirty フラグ立お - ↓ - 定期タむマヌ (5分) or メッセヌゞ数閟倀 (20ä»¶) - たたはシャットダりンフック → Save() -``` +#### D-4. 怜玢プロバむダヌに共通フォヌマット抜象がない -- リカバリりィンドり: 最倧5分 or 20メッセヌゞ分 -- 曞き蟌み回数削枛率: 䌚話速床次第だが **10〜50倍** -- 実装: `SessionManager` に `dirtyKeys map[string]bool` + バックグラりンドフラッシャヌgoroutine +`[]string + strings.Join` パタヌンが3箇所に耇補。`Search()` 戻り倀を `string` でなく構造䜓にすれば1箇所で枈む。 -#### state/stats.json — 定期フラッシュのみ +#### D-5. `Session.Messages` が可倉スラむスで党コピヌが必芁 -``` -RecordUsage() / RecordPrompt() → in-memory のみ曎新 - ↓ - 定期タむマヌ (5分) → save() - + シャットダりンフック -``` +`GetHistory()` / `Save()` での防衛的コピヌは意図的蚭蚈。COW たたは append-only immutable 構造で解消できる。 -- 損倱リスク: 最倧5分分の統蚈カりント蚱容範囲 -- 曞き蟌み回数削枛率: **LLM呌び出し頻床 × 5分** = 数十〜数癟倍 +#### D-6. `MemoryStore` のメ゜ッド境界が「ファむル操䜜単䜍」 -#### memory/MEMORY.md — タヌンスコヌプキャッシュ (曞き蟌みは即時維持) +呌び出し偎は耇数の倀が必芁でも耇数回呌ぶしかない。D-1 の解決策 (ParsedPlan 垞駐) ず合わせお解消。 -曞き蟌みは珟状通り即時。読み取りの問題だけ解決する。 +### コヌドの匂い — チェックリスト -``` -゚ヌゞェントタヌン開始 → content := ReadLongTerm() を1回だけ - ↓ content を匕数ずしお党ヘルパヌに枡す - (HasActivePlan(content), GetPlanStatus(content), ...) -゚ヌゞェントタヌン終了 → content キャッシュ砎棄 -``` +新しいコヌドを曞くずき・レビュヌするずきの確認事項: -- 倖郚゚ディタずの敎合: タヌン境界でリフレッシュされるので1タヌン以内の倖郚線集のみ芋逃す蚱容範囲 -- LLM の edit_file 経由の曞き蟌み: ファむルシステムに即座に曞かれるため次タヌンで自動反映 -- 読み取り回数削枛: 1タヌンあたり `5回以䞊 → 1回` +1. **`[]string` + `strings.Join`** → `strings.Builder` に䞀本化 +2. **`[]byte → string → io.Reader`** → `bytes.NewReader(b)` を盎接䜿甚 +3. **ルヌプ内で静的なものを毎回生成** → ルヌプ倖で1回生成しおキャッシュ +4. **党件コピヌが呌び出し偎の甚途より広い** → COW たたは RWMutex + ポむンタ返华を怜蚎 +5. **同じ content を耇数関数が独立しお Split** → 呌び出し偎で1回 Split しお枡す +6. **`var x []T` から始たる容量なし append** → ゜ヌス長が既知なら `make([]T, 0, n)` +7. **`[]rune(s)` 倉換前に長さチェックなし** → `len(s) <= max` で ASCII fast path を先に -### microSD 寿呜ぞの圱響詊算 +### ストレヌゞ保護蚭蚈 (microSD 寿呜) -䞀般的な䌚話セッション1時間、60メッセヌゞ、10 LLM呌び出し/分の堎合: +| デヌタ | 珟状 | 掚奚戊略 | 削枛率 | +|---|---|---|---| +| `sessions/*.json` | メッセヌゞ毎曞き蟌み | write-behind (dirty flag + 5分タむマヌ) | 80% | +| `state/stats.json` | LLM呌び出し毎 | 定期フラッシュのみ (5分) | 98% | +| `memory/MEMORY.md` | 即時 (倉えない) | タヌンスコヌプキャッシュ (読み取りのみ最適化) | — | -| デヌタ | 珟状の曞き蟌み回数/時 | 改善埌 | 削枛率 | -|--------|-------------------|--------|-------| -| sessions/*.json | ~60回 (メッセヌゞ毎) | ~12回 (5分毎) | **80%æž›** | -| stats.json | ~660回 (LLM呌+prompt毎) | ~12回 (5分毎) | **98%æž›** | -| MEMORY.md | ステップ数分倉わらず | 同巊 | — | -| **合蚈** | **720+ 回/時** | **~24回/時** | **97%æž›** | - -### 実装䞊の泚意点 - -- **シャットダりンフック必須**: `SIGTERM` / `SIGINT` で dirty なデヌタを匷制フラッシュ。フラッシュ倱敗時はログに蚘録。 -- **クラッシュ埌のリカバリ**: dirty デヌタが倱われた堎合、セッション履歎は最埌のチェックポむント以降が消える。ナヌザヌぞの通知が必芁か怜蚎。 -- **フラッシュ䞭の競合**: フラッシュgoroutineず `Save()` の同時呌び出しを防ぐため、既存の mutex を流甚。 -- **MEMORY.md のタヌンキャッシュ**: `edit_file` ツヌルが MEMORY.md を曞き蟌んだ堎合、**同タヌン内のキャッシュを無効化**する仕組みが必芁`MemoryStore.InvalidateCache()` を edit_file のコヌルバックから呌ぶなど。そうしないず同タヌン内の埌続の `GetPlanStatus()` などが叀いキャッシュを読む。 - -### RAM が最沢な堎合の蚭蚈倉曎 - -察象デバむスは RAM 7GB / available 5GB 超䟋: `free -m` で available ~5260MB。 -この前提が䞊蚘の各戊略に䞎える圱響を敎理する。 - -#### 読み取りレむテンシの実態 - -`buff/cache` が 4.6GB 皋床を占めるずいうこずは、OS のペヌゞキャッシュが空き RAM をほが党お䜿い切っおいる状態。`ReadLongTerm()` の耇数回呌び出しは**実際にはディスクアクセスしおいない**2回目以降はペヌゞキャッシュヒット、マむクロ秒オヌダヌ。 - -読み取りの実コストは「ディスクI/O」ではなく「**syscall + 文字列 Split/Join のアロケヌション**」。タヌンスコヌプキャッシュの䞻な効果はレむテンシ削枛より**GC 圧力の軜枛**に倉わる。 - -#### 曞き蟌み寿呜はRAMに圱響されない - -曞き蟌みは `O_SYNC` ではなくおも `os.Rename` でアトミックに曞かれるが、カヌネルはラむトバックキャッシュを経由しお最終的に SD に曞く。ペヌゞキャッシュが曞き蟌みを吞収しおも**最終的な NAND ぞの曞き蟌み回数は倉わらない**。曞き蟌み削枛の優先床は倉わらず高い。 - -#### むンメモリ衚珟の垞駐が珟実的になる - -RAM が逌迫しおいない堎合、`MemoryStore` に `*ParsedPlan` をフィヌルドずしお持たせる蚭蚈D-1, D-6 の解決策のメモリコストは無芖できる。MEMORY.md が数KB〜数十KB であっおも、パヌス枈み構造䜓ずしお垞駐させお差し支えない。 - -```go -// 蚭蚈案: MemoryStore がパヌス枈み状態を保持 -type MemoryStore struct { - workspace string - memoryFile string - mu sync.RWMutex - cached *ParsedPlan // nil = 未ロヌド - cachedAt time.Time -} -// edit_file ツヌルが曞き蟌んだ埌に InvalidateCache() を呌ぶこずで -// 同タヌン内の再読み蟌みをトリガヌできる -``` - -これにより `GetMemoryContext()` 内の `ReadLongTerm()` 倚重呌び出し問題D-1ず、 -public メ゜ッドが `content` を隠す問題D-6が同時に解消される。 - -#### write-behind 窓をさらに広げられる - -RAM が十分にあるため、セッションデヌタを長時間むンメモリに保持するリスクがない。 -write-behind の戊略を「5分 or 20件」から**「グレヌスフルシャットダりン時のみ + 30分タむマヌ」**に緩和しおも、 -クラッシュ時の損倱最倧30分の䌚話ず実装の単玔さのトレヌドオフずしお蚱容できる可胜性がある。 -プロゞェクトの可甚性芁件に応じお刀断する。 - -#### 優先実装順の修正 - -RAM 制玄がない前提での掚奚順: - -1. **`stats.json` の write-behind** — 実装が最も単玔タむマヌ1本远加、曞き蟌み削枛率が最倧98% -2. **`sessions/*.json` の write-behind** — セッション単䜍の dirty フラグ + シャットダりンフック -3. **`MemoryStore` ぞの `*ParsedPlan` 垞駐** — D-1/D-6 を根本解決、読み取りアロケヌションをれロに -4. **タヌンスコヌプキャッシュ** — 3 が実装されれば自然に解決するため䞍芁になる可胜性あり +**実装ポむント:** +- `SessionManager` に `dirtyKeys map[string]bool` + バックグラりンドフラッシャヌ goroutine +- `stats.Tracker` に `Close()` メ゜ッド远加 (タむマヌ停止 + 最終 save) +- SIGTERM/SIGINT でシャットダりンフック必須 +- MEMORY.md の edit_file 曞き蟌み埌にタヌンキャッシュを無効化 (`InvalidateCache()`) --- -## 改修蚈画 — メモリ最適化の実装フェヌズ +## Session Management (Future) -> 䜜成 2026-02-24。レビュヌ結果 (A〜H + D-1〜D-6 + ストレヌゞ保護) を実装可胜な単䜍に分割。 -> 各フェヌズは `go build ./... && go test ./... && go vet ./...` が通る状態で完結する。 +珟状の蚭蚈は「正確性」は成熟しおいるが「ラむフサむクル」が欠萜しおいる。 -### フェヌズ 0: 機械的な眮き換え (䜎リスク・高カバレッゞ) +**近期:** +- `SessionManager.Delete(key)` + TTL ゚ビクション +- `sessionLocks sync.Map` (loop.go) の GC +- 起動時の `loadSessions()` を遅延ロヌド化 -**目的**: コヌド構造を倉えず、同じ関数内でパタヌンを眮き換えるだけの修正。レビュヌが容易で回垰リスクが最小。 +**䞭期:** +- チェックポむント / ロヌルバック (`Session.Messages` を append-only immutable に) +- 名前付きセッション (`/new-session`, `/switch-session`, `/list-sessions`) -#### 0-1. strings.Builder 眮き換え (カテゎリ A 残り) - -| ファむル | 関数 | 優先床 | -|----------|------|--------| -| `pkg/tools/web.go` | `BraveSearchProvider.Search()` L73-85 | 🔎 | -| `pkg/tools/web.go` | `TavilySearchProvider.Search()` L155-167 | 🔎 | -| `pkg/tools/web.go` | `DuckDuckGoSearchProvider.extractResults()` L211-254 | 🔎 | -| `pkg/tools/web.go` | `WebFetchTool.extractText()` L592-617 | 🔎 | -| `pkg/skills/loader.go` | `BuildSkillsSummary()` L234-250 | 🔎 | -| `pkg/agent/context.go` | `BuildSystemPrompt()` L247 — `+=` を Builder に | 🟡 | -| `pkg/logger/logger.go` | `formatFields()` L241-246 | 🟡 | -| `pkg/skills/loader.go` | `LoadSkillsForContext()` L217-225 | 🟡 | -| `pkg/channels/telegram.go` | `extractCodeBlocks()` L789-806 — codes 無容量 + ルヌプ内 `fmt.Sprintf` | 🔎 | -| `pkg/channels/telegram.go` | `extractInlineCodes()` L813-830 — 同䞊パタヌン | 🔎 | -| `pkg/channels/discord.go` | `appendContent()` L162-168 | 🟡 | -| `pkg/channels/slack.go` | `handleMessageEvent()` L234-272 | 🟡 | - -#### 0-2. スラむス事前容量 (カテゎリ B 残り) - -| ファむル | 倉曎 | -|----------|------| -| `pkg/skills/loader.go:73` | `make([]SkillInfo, 0)` → `make([]SkillInfo, 0, 20)` | -| `pkg/config/config.go:628` | `var matches` → `make([]ModelConfig, 0, 4)` | -| `pkg/config/migration.go:48` | `var result` → `make([]ModelConfig, 0, 20)` | -| `pkg/skills/registry.go:183` | `var merged` → `make([]SearchResult, 0, len(regs)*limit)` | -| `pkg/skills/search_cache.go:42-43` | map/slice に `maxEntries` ヒント | -| `pkg/channels/telegram.go:832-861` | `extractMarkdownTables()` — `tables` スラむスに容量ヒント | - -#### 0-3. byte/string 倉換の削枛 (カテゎリ C) - -| ファむル | 倉曎 | -|----------|------| -| `pkg/tools/web.go:289` | `strings.NewReader(string(payloadBytes))` → `bytes.NewReader(payloadBytes)` | -| `pkg/tools/web.go:545-562` | 耇数の `string(body)` → 1回だけ倉換しお倉数に保持 | -| `pkg/providers/claude_cli_provider.go:133` | `string(paramsJSON)` → `sb.Write(paramsJSON)` | -| `pkg/utils/string.go:100` | `Truncate()` — `len(s) <= max` で早期 return (ASCII fast path) | -| `pkg/utils/string.go:50` | `wrapLine()` — 同䞊 ASCII fast path | -| `pkg/git/worktree.go:71-75` | `[]rune` → byte 長チェックで早期 return | -| `pkg/channels/telegram.go:1071-1111` | `wrapByDisplayWidth()` — ルヌプ内 `string(r)` を `displayWidth` の匕数を `rune` に倉曎しお排陀 | - -#### 0-4. パッケヌゞ倉数化 (カテゎリ H) - -| ファむル | 倉曎 | -|----------|------| -| `pkg/utils/media.go:18-19` | `audioExtensions`/`audioTypes` を関数倖の `var` に | -| `pkg/skills/clawhub_registry.go:114` | `fmt.Sprintf("%d", limit)` → `strconv.Itoa(limit)` | -| `pkg/agent/memory.go:567, 663` | `regexp.MustCompile(...)` むンラむン → 既存パッケヌゞ倉数 `reTaskLine` (L469) に眮き換え | - -**コミット単䜍**: 0-1, 0-2, 0-3, 0-4 をそれぞれ個別コミット。 - ---- - -### フェヌズ 1: 倀枡し・コピヌの最適化 (カテゎリ E) - -**目的**: struct の䞍芁なコピヌを削枛。型シグネチャが倉わるため呌び出し偎の修正が必芁。 - -| ファむル | 倉曎 | 泚意 | -|----------|------|------| -| `pkg/logger/logger.go:88-92` | リングバッファ内郚型を `[]*LogEntry` に倉曎 + `visit(fn)` メ゜ッド远加。`RecentLogs()` を visit ベヌスに曞き換え (フィルタで匟く゚ントリのコピヌを排陀) | `push()` 毎に1ヒヌプアロケヌション増だがログI/Oパスなので蚱容 | - -**削陀した項目**: -- `session_tracker.go:125` — `Touch()` がロックなしにフィヌルドを盎接曎新しおおり、`*entry` 倀コピヌ (L125) が唯䞀の安党装眮。ポむンタ返华は安党䞊の退行。`SessionEntry` は ~80バむトの小さい struct でコピヌコストも無芖可胜。 -- `skills/registry.go:132-133` — `SkillRegistry` はむンタヌフェヌス型。`*SkillRegistry` は pointer-to-interface アンチパタヌン。コピヌも n × 16バむト (n=2〜5) で無芖可胜。 - -**コミット**: 1぀にたずめる。 - ---- - -### フェヌズ 2: JSON ホットパスの最適化 (カテゎリ D) - -**目的**: ストリヌミングルヌプ内の重耇 Marshal/Unmarshal を排陀。 - -#### 2-1. openai_compat streaming の Arguments 重耇 Unmarshal - -`pkg/providers/openai_compat/provider.go` L274, 362, 621 -— ストリヌム完了時に1回だけ Unmarshal するよう制埡フロヌを敎理。 - -#### 2-2. codex CLI の Parameters 重耇 Marshal - -`pkg/providers/codex_cli_provider.go:154-155` -— ツヌル定矩はルヌプ倖で1回 Marshal しおキャッシュ、たたはルヌプ内で `json.RawMessage` 盎接曞き蟌み。 - -※ `claude_cli_provider.go:133` の `string(paramsJSON)` → `sb.Write(paramsJSON)` は byte/string 倉換の問題であり Phase 0-3 で察応枈み。 - -**コミット**: 2-1, 2-2 を個別。 - ---- - -### フェヌズ 3: MemoryStore の読み取り最適化 (蚭蚈 D-1, D-6) - -**目的**: `GetMemoryContext()` 1回で `ReadLongTerm()` が 5回以䞊呌ばれる問題を解消。 - -#### 3-1. `GetMemoryContext()` を content パススルヌ方匏にリファクタ - -既存の `GetMemoryContext()` を盎接曞き盎す新関数は远加しない。 -内郚で `ReadLongTerm()` を1回だけ呌び、取埗した `content` を既存の private ヘルパヌ矀に枡す。 - -`HasActivePlan`, `GetPlanStatus`, `GetCurrentPhase`, `GetTotalPhases` はいずれもパッケヌゞ倉数 regex (`reActivePlan`, `reStatus`, `rePhase`, `rePhaseHeader`) を1〜2行で呌ぶだけなので、private 関数を新芏䜜成せずむンラむン化できる。`GetPlanPhases` のみ42行の耇雑なロゞックがあるため private variant (`getPlanPhasesFrom(content)`) を1぀远加。 - -修正察象は3関数: - -**`GetMemoryContext()` L725** — `HasActivePlan`/`GetPlanStatus` をむンラむン化 (`GetPlanPhases` は䜿わない): -```go -content := ms.ReadLongTerm() -if reActivePlan.MatchString(content) { - var status string - if m := reStatus.FindStringSubmatch(content); len(m) >= 2 { - status = strings.TrimSpace(m[1]) - } - switch status { ... } -} -``` - -**`FormatPlanDisplay()` L656** — 党メ゜ッドをむンラむン化 + `getPlanPhasesFrom`: -```go -content := ms.ReadLongTerm() -if !reActivePlan.MatchString(content) { return "No active plan." } -var status string -if m := reStatus.FindStringSubmatch(content); len(m) >= 2 { status = strings.TrimSpace(m[1]) } -var currentPhase int -if m := rePhase.FindStringSubmatch(content); len(m) >= 2 { currentPhase, _ = strconv.Atoi(m[1]) } -phases := getPlanPhasesFrom(content) // private 関数 (1぀だけ新蚭) -``` - -**`GetPlanContext()` L560** — `GetCurrentPhase`/`GetTotalPhases` をむンラむン化: -```go -content := ms.ReadLongTerm() -var currentPhase int -if m := rePhase.FindStringSubmatch(content); len(m) >= 2 { currentPhase, _ = strconv.Atoi(m[1]) } -// GetTotalPhases: rePhaseHeader.FindAllStringSubmatch(content, -1) → max loop -``` - -既存の public メ゜ッド (`HasActivePlan()`, `GetPlanStatus()` 等) は互換性のため残す単䜓テスト・CLI から個別に呌ばれる。 - -#### 3-2. Split 重耇の統合 (読み取りパスのみ) - -content パススルヌで解消されるのは `ReadLongTerm()` の倚重呌び出しのみ。 -`extractPhaseContent()`, `GetPlanPhases()` 等が個別に `strings.Split` する問題は残る。 - -察応: `GetPlanContext()` / `FormatPlanDisplay()` 内で1回 `strings.Split(content, "\n")` し、`[]string` (行スラむス) を受け取る内郚ヘルパヌを远加。既存の `content string` を受け取るヘルパヌは互換性のため残す。(`GetMemoryContext()` 自身は Split ヘルパヌを盎接呌ばないため察象倖。Split は呌び先の `GetPlanContext()` 等で発生する。) - -**効果範囲の限定**: この統合が効くのは読み取り専甚メ゜ッド (`GetPlanContext`, `FormatPlanDisplay`) のみ。ミュヌテヌション系 (`MarkStep`, `AddStep`) は `GetMemoryContext()` を経由せず盎接 `ReadLongTerm()` + `Split` + `WriteLongTerm()` を実行するため、この Phase では察象倖。ミュヌテヌション系の Split 統合には ParsedPlan むンメモリモデル (Phase 5) が必芁。 - -**コミット**: 1぀。 - ---- - -### フェヌズ 4: ストレヌゞ保護 — write-behind (蚭蚈セクション) - -**目的**: microSD 曞き蟌み回数を 97% 削枛。 - -#### 4-1. stats.json の write-behind - -- `RecordUsage()` L77 / `RecordPrompt()` L90 の `t.save()` 呌び出しを削陀 (カりンタ曎新はむンメモリのみに) -- 起動時に `time.NewTicker(5 * time.Minute)` → `t.save()` のタむマヌ goroutine 1本远加 -- `Close()` メ゜ッドを新芏远加: タむマヌ停止 + 最終 `t.save()` -- `Reset()` L111 の `t.save()` は意味的チェックポむントなので即時維持 -- dirty フラグは䞍芁 (タむマヌが無曎新時に save() しおも同内容の䞊曞きで無害) - -#### 4-2. sessions/*.json の write-behind - -`AddFullMessage()` はすでにむンメモリのみの操䜜。曞き蟌みは `loop.go` が `Save()` を明瀺的に呌ぶ5箇所で発生する。 - -| loop.go 行 | 文脈 | 頻床 | 方針 | -|------------|------|------|------| -| L996 | ゚ヌゞェントタヌン終了 | **毎タヌン** | dirty マヌク化 (䞻芁タヌゲット) | -| L299 | `/plan start clear` 履歎クリア | 䜎頻床 | 即時曞き蟌み維持 (意味的チェックポむント) | -| L836 | tool call sanitize | 䜎頻床 | 即時曞き蟌み維持 | -| L2339 | 匷制履歎圧瞮 | 䜎頻床 | 即時曞き蟌み維持 | -| L2568 | サマリヌ生成・トランケヌト | 䜎頻床 | 即時曞き蟌み維持 | - -- L996 の `Save()` を `MarkDirty()` に倉曎、バックグラりンドフラッシャヌ (5分タむマヌ) で遅延曞き蟌み -- 残り4箇所は意味的なチェックポむントなので `Save()` を即時維持 -- `SessionManager` に `dirtyKeys map[string]bool` + フラッシャヌ goroutine 远加 -- シャットダりンフックで党 dirty セッションをフラッシュ - -#### `AppendToday()` に぀いお - -`AppendToday()` (日次ノヌト远蚘) は write-behind の察象倖ずする。理由: -- 曞き蟌み頻床が䜎い日次ノヌト远蚘時のみ -- 曞き蟌み内容がナヌザヌの手動確認察象であり、即時反映が望たしい -- sessions/stats ず異なり、遅延のメリットが小さい - -**コミット**: 4-1, 4-2 を個別。 - ---- - -### フェヌズ 5: 発展的最適化 (任意) - -実装コストが高い or 効果が限定的なもの。必芁に応じお着手。 - -| 項目 | 内容 | 芋送り理由 | -|------|------|-----------| -| F: sync.Pool | web.go extractText, telegram.go Markdown 倉換 | 呌び出し頻床が䜎く Pool の効果が薄い可胜性 | -| G: LRU O(1) 化 | search_cache.go を doubly-linked list に | maxEntries=100 で O(n) でも十分高速 | -| D-2: FunctionCall.Arguments 型倉曎 | `string` → `json.RawMessage` | 党プロバむダヌに波及、砎壊的倉曎 | -| D-3: Parameters を RawMessage に | 同䞊 | 同䞊 | -| D-5: Session.Messages を immutable に | COW or linked list | セッション管理の根本再蚭蚈が必芁 | -| ParsedPlan むンメモリモデル | MemoryStore にパヌス枈み構造䜓を垞駐させ MarkStep/AddStep の Split 重耇を根本解消 (D-1/D-6 完党解決) | 蚭蚈倉曎が広範囲 | - ---- - -### セッション管理の発展的展望 - -> 珟状の蚭蚈は「正確性」は成熟しおいるが「ラむフサむクル」が欠萜しおいる。 -> 以䞋は実装コスト・実甚䟡倀の芳点で3段階に敎理した将来の発展方向。 - -#### 近期: 運甚䞊の成熟 - -**セッションラむフサむクルの明瀺化** - -珟圚 `Session.Created` / `Session.Updated` はフィヌルドに存圚するが䜿われおいない。`Delete()` API ず TTL 付き゚ビクションを加えるだけで「セッションを意識的に管理できる」状態になる。 - -```go -type Session struct { - // 既存フィヌルド ... - Name string // 「project-x」などの名前付け - Tags []string // タグによる分類 - Archived bool // アヌカむブ枈みフラグ - ParentKey string // 芪セッション (subagent chain の明瀺化) -} -``` - -- `SessionManager.Delete(key)` の远加 -- `sessionLocks sync.Map` (loop.go) の GC — 珟状ナニヌクキヌが増えるず未回収で膚れる -- 起動時の `loadSessions()` を遅延ロヌド化 (セッションファむル数が増えた堎合の起動時間察策) - -#### 䞭期: 䌚話の構造化 - -**チェックポむント / ロヌルバック** - -``` -[turn 1] → [turn 2] → [turn 3 : checkpoint A] → [turn 4] → [turn 5] - ↑ - 「turn 3 に戻る」= turn 4, 5 を捚おお再開 -``` - -LLM が方向を間違えた時点に戻るナヌスケヌスは個人利甚でも頻繁に発生する。 -`Session.Messages` を append-only immutable にする (D-5 COW) ず自然に぀ながる。 - -**名前付きセッション / 意図的な切り替え** - -珟圚セッションキヌは「どこから来たか」(チャンネル+ピア) で決たる。これを「䜕の文脈か」でも切り替えられるようにする: - -``` -/new-session "refactoring-auth" → 新しいセッションを明瀺的に開始 -/switch-session "refactoring-auth" → 過去の名前付きセッションに戻る -/list-sessions → セッション䞀芧 -``` - -`routing/session_key.go` の `BuildAgentPeerSessionKey()` はすでに柔軟な構造なので、セッション名を key の䞀郚ずしお持぀こずは蚭蚈䞊無理がない。 - -#### 長期: セッション間の関係 - -**サブ゚ヌゞェントセッションのグラフ化** - -珟圚、サブ゚ヌゞェントセッションは `IsSubagentSessionKey()` で刀定できるが、「どの芪セッションから生たれたか」ずいう芪子関係は key の呜名芏則に暗黙的に埋め蟌たれおいるだけ: - -``` -agent:main:main - └─ subagent:abc123:main ← 芪が main:main ずは構造的に管理されおいない - └─ subagent:def456:main -``` - -`Session.ParentKey` を远加しおセッションをグラフずしお持おるず、「このサブ゚ヌゞェントが䜕をやったか」を芪セッションから遡れるようになる。 - -**クロスセッション怜玢** - -``` -「以前 auth に぀いお話したずき䜕を決めたっけ」 -→ セッション暪断でキヌワヌド怜玢 → 関連タヌンを抜出しおプロンプトに泚入 -``` - -`MEMORY.md` は珟状「プラン専甚の氞続メモリ」だが、クロスセッション怜玢はその補完ずしお機胜する。 - -> **泚意: 遅延ロヌドずの競合** -> 近期の「遅延ロヌド化」を実装するず、起動時に党セッションがメモリにある前提が厩れる。 -> 䞡方を採甚する堎合は以䞋のいずれかを遞択する必芁がある: -> - **怜玢時フルスキャン**: 怜玢リク゚ストのたびに `sessions/` ディレクトリの党 JSON を読む (䜎頻床なら蚱容) -> - **バックグラりンドむンデックス**: 起動埌にゎルヌチンで党ファむルを非同期スキャンし、キヌワヌドむンデックスを構築・維持する - -#### 蚭蚈䞊の遞択肢 - -珟圚の構造は2぀の哲孊の䞭間に䜍眮しおいる: - -| 哲孊A: 履歎䞭心 | 哲孊B: 知識䞭心 | -|----------------|----------------| -| `Session.Messages` が唯䞀の真実 | 重芁な情報を `MEMORY.md` 等に蒞留 | -| 䌚話を「再生」しおコンテキスト再珟 | 構造化知識を「泚入」しおコンテキスト構築 | -| ロヌルバック・ブランチが自然な拡匵 | クロスセッション怜玢が自然な拡匵 | - -このコヌドベヌスはすでに `Session.Messages` (哲孊A) ず `MEMORY.md` (哲孊B) が共存しおおり、Phase 5 の `ParsedPlan むンメモリモデル` は哲孊B 方向ぞの垃石になる。 - ---- - -### 実装順サマリヌ - -``` -Phase 0 ──→ Phase 1 ──→ Phase 2 ──→ Phase 3 ──→ Phase 4 - 機械的 倀枡し JSON Memory Storage - 眮き換え 最適化 ホットパス 読み取り write-behind - (4 commits) (1 commit) (2 commits) (2 commits) (2 commits) -``` - -- Phase 0〜2: **アロケヌション削枛** (GC 圧力軜枛) -- Phase 3: **syscall + Split/Join 削枛** (CPU + アロケヌション) -- Phase 4: **ディスク曞き蟌み削枛** (microSD 寿呜保護) -- Phase 5: 必芁に応じお個別刀断 - ---- +**長期:** +- `Session.ParentKey` でサブ゚ヌゞェントセッションをグラフ化 +- クロスセッション怜玢 (MEMORY.md の補完ずしお) +蚭蚈の哲孊: `Session.Messages` (履歎䞭心) ず `MEMORY.md` (知識䞭心) が珟状共存しおいる。どちらを䞻軞にするかで発展方向が倉わる。 diff --git a/cmd/picoclaw/cmd_agent.go b/cmd/picoclaw/cmd_agent.go index 8658c9d32..5ce0c3b99 100644 --- a/cmd/picoclaw/cmd_agent.go +++ b/cmd/picoclaw/cmd_agent.go @@ -24,6 +24,7 @@ func agentCmd() { message := "" sessionKey := "cli:default" modelOverride := "" + orchestrationEnabled := false args := os.Args[2:] for i := 0; i < len(args); i++ { @@ -46,6 +47,8 @@ func agentCmd() { modelOverride = args[i+1] i++ } + case "--orchestration": + orchestrationEnabled = true } } @@ -59,6 +62,10 @@ func agentCmd() { cfg.Agents.Defaults.Model = modelOverride } + if orchestrationEnabled { + cfg.Agents.Defaults.Orchestration = true + } + provider, modelID, err := providers.CreateProvider(cfg) if err != nil { fmt.Printf("Error creating provider: %v\n", err) diff --git a/cmd/picoclaw/cmd_gateway.go b/cmd/picoclaw/cmd_gateway.go index b0a55dcce..2b9567af5 100644 --- a/cmd/picoclaw/cmd_gateway.go +++ b/cmd/picoclaw/cmd_gateway.go @@ -243,6 +243,9 @@ func gatewayCmd() { miniappNotifier = miniapp.NewStateNotifier() handler := miniapp.NewHandler(provider, sender, cfg.Channels.Telegram.Token, miniappNotifier, cfg.Channels.Telegram.AllowFrom, cfg.WorkspacePath()) agentLoop.OnStateChange = miniappNotifier.Notify + if b := agentLoop.GetOrchBroadcaster(); b != nil { + handler.SetOrchBroadcaster(b) + } handler.RegisterRoutes(healthServer.Mux()) // Register dev preview tool for all agents diff --git a/pkg/agent/instance.go b/pkg/agent/instance.go index a767bcb04..c84a70660 100644 --- a/pkg/agent/instance.go +++ b/pkg/agent/instance.go @@ -91,6 +91,15 @@ func NewAgentInstance( skillsFilter = agentCfg.Skills } + // Apply defaults.Orchestration: if the flag is set, ensure orchestration is enabled. + if defaults.Orchestration { + if subagents == nil { + subagents = &config.SubagentsConfig{Enabled: true} + } else { + subagents.Enabled = true + } + } + maxIter := defaults.MaxToolIterations if maxIter == 0 { maxIter = 20 diff --git a/pkg/agent/loop.go b/pkg/agent/loop.go index 75284b6dc..4649e71eb 100644 --- a/pkg/agent/loop.go +++ b/pkg/agent/loop.go @@ -25,6 +25,7 @@ import ( "github.com/sipeed/picoclaw/pkg/config" "github.com/sipeed/picoclaw/pkg/constants" "github.com/sipeed/picoclaw/pkg/logger" + "github.com/sipeed/picoclaw/pkg/orch" "github.com/sipeed/picoclaw/pkg/providers" "github.com/sipeed/picoclaw/pkg/providers/protocoltypes" "github.com/sipeed/picoclaw/pkg/routing" @@ -94,6 +95,8 @@ type AgentLoop struct { promptDirty atomic.Bool // true = rebuild needed on next GetSystemPrompt read OnStateChange func() // called on plan/session/skills mutations OnUserMessage func() // called when a real user message is processed + orchBroadcaster *orch.Broadcaster // nil when --orchestration not set + orchReporter orch.AgentReporter // always non-nil (Noop when disabled) } // processOptions configures how a message is processed @@ -114,9 +117,6 @@ type processOptions struct { func NewAgentLoop(cfg *config.Config, msgBus *bus.MessageBus, provider providers.LLMProvider, enableStats ...bool) *AgentLoop { registry := NewAgentRegistry(cfg, provider) - // Register shared tools to all agents - registerSharedTools(cfg, msgBus, registry, provider) - // Set up shared fallback chain cooldown := providers.NewCooldownTracker() fallbackChain := providers.NewFallbackChain(cooldown) @@ -136,17 +136,57 @@ func NewAgentLoop(cfg *config.Config, msgBus *bus.MessageBus, provider providers statsTracker = stats.NewTracker(defaultAgent.Workspace) } - return &AgentLoop{ - bus: msgBus, - cfg: cfg, - registry: registry, - state: stateManager, - stats: statsTracker, - summarizing: sync.Map{}, - fallback: fallbackChain, - providerCache: providerCache, - sessions: NewSessionTracker(), + // Determine if orchestration broadcaster is needed (any agent has subagents enabled). + var orchBroadcaster *orch.Broadcaster + var orchReporter orch.AgentReporter = orch.Noop + for _, id := range registry.ListAgentIDs() { + if a, ok := registry.GetAgent(id); ok && a.Subagents != nil && a.Subagents.Enabled { + orchBroadcaster = orch.NewBroadcaster() + orchReporter = orchBroadcaster + break + } } + + al := &AgentLoop{ + bus: msgBus, + cfg: cfg, + registry: registry, + state: stateManager, + stats: statsTracker, + summarizing: sync.Map{}, + fallback: fallbackChain, + providerCache: providerCache, + sessions: NewSessionTracker(), + orchBroadcaster: orchBroadcaster, + orchReporter: orchReporter, + } + + // Register shared tools to all agents (needs al for reporter injection). + registerSharedTools(cfg, msgBus, registry, provider, al) + + return al +} + +// reporter returns the active AgentReporter (never nil). +func (al *AgentLoop) reporter() orch.AgentReporter { + if al.orchReporter == nil { + return orch.Noop + } + return al.orchReporter +} + +// SetOrchReporter wires a Broadcaster as the active reporter. +// Called from cmd_gateway.go when --orchestration is set. +// --orchestration なし → 呌ばれない → reporter() は Noop を返す。 +func (al *AgentLoop) SetOrchReporter(b *orch.Broadcaster) { + al.orchBroadcaster = b + al.orchReporter = b +} + +// GetOrchBroadcaster returns the concrete Broadcaster for miniapp wiring. +// Returns nil when orchestration is disabled. +func (al *AgentLoop) GetOrchBroadcaster() *orch.Broadcaster { + return al.orchBroadcaster } func (al *AgentLoop) notifyStateChange() { @@ -162,6 +202,7 @@ func registerSharedTools( msgBus *bus.MessageBus, registry *AgentRegistry, provider providers.LLMProvider, + al *AgentLoop, ) { for _, agentID := range registry.ListAgentIDs() { agent, ok := registry.GetAgent(agentID) @@ -216,15 +257,17 @@ func registerSharedTools( agent.Tools.Register(tools.NewFindSkillsTool(registryMgr, searchCache)) agent.Tools.Register(tools.NewInstallSkillTool(registryMgr, agent.Workspace)) - // Spawn tool with allowlist checker - subagentManager := tools.NewSubagentManager(provider, agent.Model, agent.Workspace, msgBus) - subagentManager.SetLLMOptions(agent.MaxTokens, agent.Temperature) - spawnTool := tools.NewSpawnTool(subagentManager) - currentAgentID := agentID - spawnTool.SetAllowlistChecker(func(targetAgentID string) bool { - return registry.CanSpawnSubagent(currentAgentID, targetAgentID) - }) - agent.Tools.Register(spawnTool) + // Spawn tool — only registered when orchestration is explicitly enabled. + if agent.Subagents != nil && agent.Subagents.Enabled { + subagentManager := tools.NewSubagentManager(provider, agent.Model, agent.Workspace, msgBus, al.reporter()) + subagentManager.SetLLMOptions(agent.MaxTokens, agent.Temperature) + spawnTool := tools.NewSpawnTool(subagentManager) + currentAgentID := agentID + spawnTool.SetAllowlistChecker(func(targetAgentID string) bool { + return registry.CanSpawnSubagent(currentAgentID, targetAgentID) + }) + agent.Tools.Register(spawnTool) + } // Update context builder with the complete tools registry agent.ContextBuilder.SetToolsRegistry(agent.Tools) @@ -713,6 +756,10 @@ func (al *AgentLoop) runAgentLoop(ctx context.Context, agent *AgentInstance, opt } defer al.releaseSessionLock(opts.SessionKey) + // Report session lifecycle to canvas. + al.reporter().ReportSpawn(opts.SessionKey, opts.Channel, opts.UserMessage) + defer al.reporter().ReportGC(opts.SessionKey, "completed") + // -0. Create cancellable child context and register active task taskCtx, taskCancel := context.WithCancel(ctx) defer taskCancel() @@ -1785,6 +1832,9 @@ func (al *AgentLoop) runLLMIteration( return doCall(ctx, agent.Provider, primaryModel) } + // Report waiting state to canvas before each LLM call. + al.reporter().ReportStateChange(opts.SessionKey, "waiting", "") + // Retry loop for context/token errors maxRetries := 2 for retry := 0; retry <= maxRetries; retry++ { @@ -2134,6 +2184,9 @@ func (al *AgentLoop) runLLMIteration( } } + // Report toolcall state to canvas. + al.reporter().ReportStateChange(opts.SessionKey, "toolcall", tc.Name) + toolStart := time.Now() toolCtx := ctx if wt := agent.GetWorktree(opts.SessionKey); wt != nil { diff --git a/pkg/agent/loop_reporter_test.go b/pkg/agent/loop_reporter_test.go new file mode 100644 index 000000000..ef7ea0905 --- /dev/null +++ b/pkg/agent/loop_reporter_test.go @@ -0,0 +1,132 @@ +package agent + +import ( + "context" + "os" + "testing" + "time" + + "github.com/sipeed/picoclaw/pkg/bus" + "github.com/sipeed/picoclaw/pkg/config" + "github.com/sipeed/picoclaw/pkg/orch" +) + +// makeOrchTestLoop creates a minimal AgentLoop with a temp workspace and +// a real Broadcaster wired as the reporter. +// Returns the loop, the broadcaster, and a cleanup function. +func makeOrchTestLoop(t *testing.T) (*AgentLoop, *orch.Broadcaster) { + t.Helper() + tmpDir, err := os.MkdirTemp("", "agent-orch-test-*") + if err != nil { + t.Fatalf("MkdirTemp: %v", err) + } + t.Cleanup(func() { os.RemoveAll(tmpDir) }) + + cfg := &config.Config{ + Agents: config.AgentsConfig{ + Defaults: config.AgentDefaults{ + Workspace: tmpDir, + Model: "test-model", + MaxTokens: 512, + MaxToolIterations: 5, + }, + }, + } + al := NewAgentLoop(cfg, bus.NewMessageBus(), &mockProvider{}) + b := orch.NewBroadcaster() + al.SetOrchReporter(b) + return al, b +} + +// collectOrchEvents drains the subscriber channel until an agent_gc event +// arrives or the deadline is exceeded. +func collectOrchEvents(t *testing.T, ch <-chan orch.Event, timeout time.Duration) []orch.Event { + t.Helper() + var events []orch.Event + deadline := time.After(timeout) + for { + select { + case ev := <-ch: + events = append(events, ev) + if ev.Type == "agent_gc" { + return events + } + case <-deadline: + t.Fatalf("timed out waiting for agent_gc; events so far: %+v", events) + } + } +} + +// TestAgentLoop_ProcessDirect_EmitsSpawnWaitingGC verifies that a main +// session processed via ProcessDirect emits the full lifecycle: +// +// agent_spawn(sessionKey) → agent_state(waiting) → agent_gc(completed) +// +// and that the Broadcaster snapshot is empty after the call returns. +func TestAgentLoop_ProcessDirect_EmitsSpawnWaitingGC(t *testing.T) { + al, b := makeOrchTestLoop(t) + sub := b.Subscribe() + defer b.Unsubscribe(sub) + + const sessionKey = "orch-test-session" + _, err := al.ProcessDirect(context.Background(), "hello", sessionKey) + if err != nil { + t.Fatalf("ProcessDirect: %v", err) + } + + events := collectOrchEvents(t, sub.Ch, 5*time.Second) + + // First event: agent_spawn with correct ID. + if events[0].Type != "agent_spawn" || events[0].ID != sessionKey { + t.Errorf("first event must be agent_spawn(%s), got: %+v", sessionKey, events[0]) + } + + // At least one agent_state(waiting) for this session. + var hasWaiting bool + for _, ev := range events { + if ev.Type == "agent_state" && ev.ID == sessionKey && ev.State == "waiting" { + hasWaiting = true + break + } + } + if !hasWaiting { + t.Errorf("missing agent_state(waiting) for %s; events: %+v", sessionKey, events) + } + + // Last event: agent_gc(completed) for this session. + last := events[len(events)-1] + if last.Type != "agent_gc" || last.ID != sessionKey || last.Reason != "completed" { + t.Errorf("last event must be agent_gc(completed,%s), got: %+v", sessionKey, last) + } + + // Snapshot must be empty — session removed on GC. + if snap := b.Snapshot(); len(snap) != 0 { + t.Errorf("snapshot must be empty after GC, got: %v", snap) + } +} + +// TestAgentLoop_ProcessHeartbeat_EmitsSpawnAndGC verifies that heartbeat +// sessions appear on canvas with sessionKey = "heartbeat". +func TestAgentLoop_ProcessHeartbeat_EmitsSpawnAndGC(t *testing.T) { + al, b := makeOrchTestLoop(t) + sub := b.Subscribe() + defer b.Unsubscribe(sub) + + _, err := al.ProcessHeartbeat(context.Background(), "check system", "heartbeat-chan", "none") + if err != nil { + t.Fatalf("ProcessHeartbeat: %v", err) + } + + events := collectOrchEvents(t, sub.Ch, 5*time.Second) + + // ProcessHeartbeat always uses sessionKey = "heartbeat". + const want = "heartbeat" + if events[0].Type != "agent_spawn" || events[0].ID != want { + t.Errorf("first event must be agent_spawn(%s), got: %+v", want, events[0]) + } + + last := events[len(events)-1] + if last.Type != "agent_gc" || last.ID != want || last.Reason != "completed" { + t.Errorf("last event must be agent_gc(completed,%s), got: %+v", want, last) + } +} diff --git a/pkg/config/config.go b/pkg/config/config.go index 7ac337856..299f7334a 100644 --- a/pkg/config/config.go +++ b/pkg/config/config.go @@ -141,6 +141,7 @@ type AgentConfig struct { } type SubagentsConfig struct { + Enabled bool `json:"enabled,omitempty"` AllowAgents []string `json:"allow_agents,omitempty"` Model *AgentModelConfig `json:"model,omitempty"` } @@ -182,6 +183,7 @@ type AgentDefaults struct { Temperature *float64 `json:"temperature,omitempty" env:"PICOCLAW_AGENTS_DEFAULTS_TEMPERATURE"` MaxToolIterations int `json:"max_tool_iterations" env:"PICOCLAW_AGENTS_DEFAULTS_MAX_TOOL_ITERATIONS"` TaskReminderInterval int `json:"task_reminder_interval" env:"PICOCLAW_AGENTS_DEFAULTS_TASK_REMINDER_INTERVAL"` + Orchestration bool `json:"orchestration,omitempty" env:"PICOCLAW_AGENTS_DEFAULTS_ORCHESTRATION"` } type ChannelsConfig struct { diff --git a/pkg/miniapp/api.go b/pkg/miniapp/api.go new file mode 100644 index 000000000..697dafa79 --- /dev/null +++ b/pkg/miniapp/api.go @@ -0,0 +1,168 @@ +package miniapp + +import ( + "bytes" + "encoding/json" + "fmt" + "io" + "net/http" + "strings" + "time" +) + + +func (h *Handler) apiSkills(w http.ResponseWriter, r *http.Request) { + skillsList := h.provider.ListSkills() + writeJSON(w, skillsList) +} + + +func (h *Handler) apiPlan(w http.ResponseWriter, r *http.Request) { + info := h.provider.GetPlanInfo() + writeJSON(w, info) +} + + +func (h *Handler) apiSessions(w http.ResponseWriter, r *http.Request) { + sessions := h.provider.GetActiveSessions() + if sessions == nil { + sessions = []SessionInfo{} + } + writeJSON(w, sessions) +} + + +func (h *Handler) apiSession(w http.ResponseWriter, r *http.Request) { + s := h.provider.GetSessionStats() + if s == nil { + writeJSON(w, map[string]string{"status": "stats not enabled"}) + return + } + writeJSON(w, s) +} + + +func (h *Handler) apiContext(w http.ResponseWriter, r *http.Request) { + writeJSON(w, h.provider.GetContextInfo()) +} + + +func (h *Handler) apiPrompt(w http.ResponseWriter, r *http.Request) { + writeJSON(w, map[string]string{"prompt": h.provider.GetSystemPrompt()}) +} + + +func (h *Handler) apiGit(w http.ResponseWriter, r *http.Request) { + repo := r.URL.Query().Get("repo") + if repo == "" { + writeJSON(w, h.provider.GetGitRepos()) + } else { + writeJSON(w, h.provider.GetGitRepoDetail(repo)) + } +} + + +func (h *Handler) apiCommand(w http.ResponseWriter, r *http.Request) { + if r.Method != http.MethodPost { + http.Error(w, `{"error":"method not allowed"}`, http.StatusMethodNotAllowed) + return + } + + body, err := io.ReadAll(io.LimitReader(r.Body, 4096)) + if err != nil { + http.Error(w, `{"error":"bad request"}`, http.StatusBadRequest) + return + } + + var req struct { + Command string `json:"command"` + } + if err := json.Unmarshal(body, &req); err != nil || req.Command == "" { + http.Error(w, `{"error":"missing command"}`, http.StatusBadRequest) + return + } + + if !strings.HasPrefix(req.Command, "/") { + http.Error(w, `{"error":"command must start with /"}`, http.StatusBadRequest) + return + } + + // Extract user ID from initData to identify the sender + initData := r.URL.Query().Get("initData") + userID, chatID := extractUserFromInitData(initData) + if userID == "" { + http.Error(w, `{"error":"cannot identify user"}`, http.StatusBadRequest) + return + } + + h.sender.SendCommand(userID, chatID, req.Command) + writeJSON(w, map[string]string{"status": "ok"}) +} + + +func (h *Handler) apiEvents(w http.ResponseWriter, r *http.Request) { + flusher, ok := w.(http.Flusher) + if !ok { + http.Error(w, `{"error":"streaming not supported"}`, http.StatusInternalServerError) + return + } + rc := http.NewResponseController(w) + _ = rc.SetWriteDeadline(time.Time{}) + + w.Header().Set("Content-Type", "text/event-stream") + w.Header().Set("Cache-Control", "no-cache") + w.Header().Set("Connection", "keep-alive") + w.Header().Set("X-Accel-Buffering", "no") + + ch := h.notifier.Subscribe() + defer h.notifier.Unsubscribe(ch) + + var lastPlan, lastSession, lastSkills, lastDev, lastContext, lastPrompt []byte + + // Send initial state immediately + sendSSEIfChanged(w, flusher, "plan", h.provider.GetPlanInfo(), &lastPlan) + sendSSEIfChanged(w, flusher, "session", + map[string]any{"stats": h.provider.GetSessionStats(), "sessions": h.provider.GetActiveSessions()}, + &lastSession) + sendSSEIfChanged(w, flusher, "skills", h.provider.ListSkills(), &lastSkills) + sendSSEIfChanged(w, flusher, "dev", h.devStatus(), &lastDev) + sendSSEIfChanged(w, flusher, "context", h.provider.GetContextInfo(), &lastContext) + sendSSEIfChanged(w, flusher, "prompt", map[string]string{"prompt": h.provider.GetSystemPrompt()}, &lastPrompt) + + for { + select { + case <-r.Context().Done(): + return + case <-h.notifier.Done(): + return + case <-ch: + sendSSEIfChanged(w, flusher, "plan", h.provider.GetPlanInfo(), &lastPlan) + sendSSEIfChanged(w, flusher, "session", + map[string]any{"stats": h.provider.GetSessionStats(), "sessions": h.provider.GetActiveSessions()}, + &lastSession) + sendSSEIfChanged(w, flusher, "skills", h.provider.ListSkills(), &lastSkills) + sendSSEIfChanged(w, flusher, "dev", h.devStatus(), &lastDev) + sendSSEIfChanged(w, flusher, "context", h.provider.GetContextInfo(), &lastContext) + sendSSEIfChanged(w, flusher, "prompt", map[string]string{"prompt": h.provider.GetSystemPrompt()}, &lastPrompt) + } + } +} + + +func sendSSEIfChanged(w http.ResponseWriter, f http.Flusher, event string, v any, last *[]byte) { + data, _ := json.Marshal(v) + if !bytes.Equal(data, *last) { + fmt.Fprintf(w, "event: %s\ndata: %s\n\n", event, data) + f.Flush() + *last = data + } +} + + +func writeJSON(w http.ResponseWriter, v any) { + w.Header().Set("Content-Type", "application/json") + json.NewEncoder(w).Encode(v) +} + +// apiDevConsole receives console output from dev preview iframes. + diff --git a/pkg/miniapp/auth.go b/pkg/miniapp/auth.go new file mode 100644 index 000000000..cc2b3406f --- /dev/null +++ b/pkg/miniapp/auth.go @@ -0,0 +1,131 @@ +package miniapp + +import ( + "crypto/hmac" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "fmt" + "net/http" + "net/url" + "sort" + "strconv" + "strings" + "time" +) + +// initDataMaxAge is the maximum age of initData before it is considered expired. +const initDataMaxAge = 24 * time.Hour + +func (h *Handler) requireAuth(next http.HandlerFunc) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + initData := r.URL.Query().Get("initData") + if initData == "" { + http.Error(w, `{"error":"missing initData"}`, http.StatusUnauthorized) + return + } + if !ValidateInitData(initData, h.botToken) { + http.Error(w, `{"error":"invalid initData"}`, http.StatusUnauthorized) + return + } + if len(h.allowList) > 0 { + userID, _ := extractUserFromInitData(initData) + if userID == "" || !isAllowed(userID, h.allowList) { + http.Error(w, `{"error":"forbidden"}`, http.StatusForbidden) + return + } + } + next(w, r) + } +} + +// isAllowed checks whether userID matches any entry in the allow list. +// Logic mirrors BaseChannel.IsAllowed without importing channels package. +func isAllowed(userID string, allowList []string) bool { + if len(allowList) == 0 { + return true + } + for _, allowed := range allowList { + trimmed := strings.TrimPrefix(allowed, "@") + allowedID := trimmed + if idx := strings.Index(trimmed, "|"); idx > 0 { + allowedID = trimmed[:idx] + } + if userID == allowed || userID == trimmed || userID == allowedID { + return true + } + } + return false +} + +// extractUserFromInitData parses user.id from the initData query string. +// initData contains a "user" param with JSON like {"id":123456,...}. +func extractUserFromInitData(initData string) (userID, chatID string) { + values, err := url.ParseQuery(initData) + if err != nil { + return "", "" + } + userJSON := values.Get("user") + if userJSON == "" { + return "", "" + } + var user struct { + ID int64 `json:"id"` + } + if err := json.Unmarshal([]byte(userJSON), &user); err != nil || user.ID == 0 { + return "", "" + } + id := fmt.Sprintf("%d", user.ID) + // For Mini App commands, chatID = userID (private chat) + return id, id +} + +// ValidateInitData verifies the Telegram WebApp initData HMAC-SHA256 signature +// and checks that auth_date is not older than initDataMaxAge. +// See https://core.telegram.org/bots/webapps#validating-data-received-via-the-mini-app +func ValidateInitData(initData, botToken string) bool { + values, err := url.ParseQuery(initData) + if err != nil { + return false + } + + receivedHash := values.Get("hash") + if receivedHash == "" { + return false + } + + // Check auth_date freshness + if authDateStr := values.Get("auth_date"); authDateStr != "" { + authDate, err := strconv.ParseInt(authDateStr, 10, 64) + if err != nil { + return false + } + if time.Since(time.Unix(authDate, 0)) > initDataMaxAge { + return false + } + } + + // Build the data-check-string: sort all key=value pairs except "hash", + // join with newlines. + var pairs []string + for key := range values { + if key == "hash" { + continue + } + pairs = append(pairs, fmt.Sprintf("%s=%s", key, values.Get(key))) + } + sort.Strings(pairs) + dataCheckString := strings.Join(pairs, "\n") + + // secret_key = HMAC-SHA256("WebAppData", bot_token) + secretKeyMac := hmac.New(sha256.New, []byte("WebAppData")) + secretKeyMac.Write([]byte(botToken)) + secretKey := secretKeyMac.Sum(nil) + + // hash = HMAC-SHA256(secret_key, data_check_string) + hashMac := hmac.New(sha256.New, secretKey) + hashMac.Write([]byte(dataCheckString)) + computedHash := hex.EncodeToString(hashMac.Sum(nil)) + + return hmac.Equal([]byte(computedHash), []byte(receivedHash)) +} diff --git a/pkg/miniapp/dev.go b/pkg/miniapp/dev.go new file mode 100644 index 000000000..1dfbaa7fa --- /dev/null +++ b/pkg/miniapp/dev.go @@ -0,0 +1,477 @@ +package miniapp + +import ( + "bytes" + "encoding/json" + "fmt" + "io" + "net" + "net/http" + "net/http/httputil" + "net/url" + "sort" + "strconv" + "strings" + "time" + + "github.com/sipeed/picoclaw/pkg/logger" +) + + +// validateLocalhostURL parses and validates that a URL targets localhost. +func validateLocalhostURL(target string) (*url.URL, error) { + u, err := url.Parse(target) + if err != nil { + return nil, fmt.Errorf("invalid URL: %w", err) + } + host := u.Hostname() + if host != "localhost" && host != "127.0.0.1" && host != "::1" { + return nil, fmt.Errorf("only localhost targets are allowed, got %q", host) + } + return u, nil +} + +// RegisterDevTarget registers a new dev server target. Only localhost targets are allowed. + + +// RegisterDevTarget registers a new dev server target. Only localhost targets are allowed. +func (h *Handler) RegisterDevTarget(name, target string) (string, error) { + if _, err := validateLocalhostURL(target); err != nil { + return "", err + } + + h.devMu.Lock() + defer h.devMu.Unlock() + + h.devNextID++ + id := strconv.Itoa(h.devNextID) + + h.devTargets[id] = &DevTarget{ID: id, Name: name, Target: target} + if h.notifier != nil { + h.notifier.Notify() + } + return id, nil +} + +// UnregisterDevTarget removes a registered target. If it was active, the proxy is disabled. + + +// UnregisterDevTarget removes a registered target. If it was active, the proxy is disabled. +func (h *Handler) UnregisterDevTarget(id string) error { + h.devMu.Lock() + defer h.devMu.Unlock() + + if _, ok := h.devTargets[id]; !ok { + return fmt.Errorf("target %q not found", id) + } + delete(h.devTargets, id) + + if h.devActiveID == id { + h.devActiveID = "" + h.devTarget = nil + h.devProxy = nil + } + if h.notifier != nil { + h.notifier.Notify() + } + return nil +} + +// ActivateDevTarget sets the reverse proxy to the registered target with the given ID. + + +// ActivateDevTarget sets the reverse proxy to the registered target with the given ID. +func (h *Handler) ActivateDevTarget(id string) error { + h.devMu.Lock() + defer h.devMu.Unlock() + + dt, ok := h.devTargets[id] + if !ok { + return fmt.Errorf("target %q not found", id) + } + + u, err := url.Parse(dt.Target) + if err != nil { + return fmt.Errorf("invalid URL: %w", err) + } + + // Fix IPv6: resolve "localhost" to 127.0.0.1 to avoid connection refused on systems + // where localhost resolves to [::1] but the dev server only listens on IPv4. + if u.Hostname() == "localhost" { + u.Host = net.JoinHostPort("127.0.0.1", u.Port()) + } + + proxy := httputil.NewSingleHostReverseProxy(u) + proxy.ModifyResponse = func(resp *http.Response) error { + // Prevent browser/WebView from caching dev proxy responses (CSS, JS, etc.) + resp.Header.Set("Cache-Control", "no-cache, no-store, must-revalidate") + resp.Header.Del("ETag") + resp.Header.Del("Last-Modified") + + ct := resp.Header.Get("Content-Type") + if !strings.Contains(ct, "text/html") { + return nil + } + body, err := io.ReadAll(resp.Body) + if err != nil { + return err + } + resp.Body.Close() + modified := injectDevProxyScript(body) + resp.Body = io.NopCloser(bytes.NewReader(modified)) + resp.ContentLength = int64(len(modified)) + resp.Header.Set("Content-Length", strconv.Itoa(len(modified))) + resp.Header.Del("Content-Encoding") + return nil + } + proxy.ErrorHandler = func(w http.ResponseWriter, r *http.Request, err error) { + w.Header().Set("Content-Type", "text/html; charset=utf-8") + w.WriteHeader(http.StatusBadGateway) + fmt.Fprintf(w, ` +

Cannot connect

%s

Target: %s

`, + escapeHTMLString(err.Error()), escapeHTMLString(dt.Target)) + } + + h.devTarget = u + h.devProxy = proxy + h.devActiveID = id + if h.notifier != nil { + h.notifier.Notify() + } + return nil +} + +// DeactivateDevTarget disables the reverse proxy without removing registrations. + + +// DeactivateDevTarget disables the reverse proxy without removing registrations. +func (h *Handler) DeactivateDevTarget() error { + h.devMu.Lock() + defer h.devMu.Unlock() + + h.devActiveID = "" + h.devTarget = nil + h.devProxy = nil + if h.notifier != nil { + h.notifier.Notify() + } + return nil +} + +// GetDevTarget returns the current dev proxy target URL, or empty string if disabled. + + +// GetDevTarget returns the current dev proxy target URL, or empty string if disabled. +func (h *Handler) GetDevTarget() string { + h.devMu.RLock() + defer h.devMu.RUnlock() + if h.devTarget == nil { + return "" + } + return h.devTarget.String() +} + +// ListDevTargets returns all registered dev targets. + + +// ListDevTargets returns all registered dev targets. +func (h *Handler) ListDevTargets() []DevTarget { + h.devMu.RLock() + defer h.devMu.RUnlock() + + targets := make([]DevTarget, 0, len(h.devTargets)) + for _, dt := range h.devTargets { + targets = append(targets, *dt) + } + // Sort by ID for stable order + sort.Slice(targets, func(i, j int) bool { return targets[i].ID < targets[j].ID }) + return targets +} + +// devProxyScript is the JavaScript injected into HTML responses from the dev proxy. +// It rewrites fetch() and XMLHttpRequest.open() so that absolute paths like +// "/api/items" are prefixed with "/miniapp/dev", matching the reverse proxy mount. +// It also captures console.log/warn/error/info and forwards them to the server. + + +// devProxyScript is the JavaScript injected into HTML responses from the dev proxy. +// It rewrites fetch() and XMLHttpRequest.open() so that absolute paths like +// "/api/items" are prefixed with "/miniapp/dev", matching the reverse proxy mount. +// It also captures console.log/warn/error/info and forwards them to the server. +const devProxyScript = `` + +// injectDevProxyScript inserts the dev proxy rewrite script into an HTML document. +// Insertion priority: before , after , or prepend to document. + + +// injectDevProxyScript inserts the dev proxy rewrite script into an HTML document. +// Insertion priority: before , after , or prepend to document. +func injectDevProxyScript(html []byte) []byte { + script := []byte(devProxyScript) + + // Priority 1: before + if idx := bytes.Index(bytes.ToLower(html), []byte("")); idx >= 0 { + out := make([]byte, 0, len(html)+len(script)) + out = append(out, html[:idx]...) + out = append(out, script...) + out = append(out, html[idx:]...) + return out + } + + // Priority 2: after + lower := bytes.ToLower(html) + if idx := bytes.Index(lower, []byte("= 0 { + // Find the closing '>' of the tag + closeIdx := bytes.IndexByte(lower[idx:], '>') + if closeIdx >= 0 { + insertAt := idx + closeIdx + 1 + out := make([]byte, 0, len(html)+len(script)) + out = append(out, html[:insertAt]...) + out = append(out, script...) + out = append(out, html[insertAt:]...) + return out + } + } + + // Priority 3: prepend + out := make([]byte, 0, len(html)+len(script)) + out = append(out, script...) + out = append(out, html...) + return out +} + +// escapeHTMLString escapes HTML special characters in a string. + + +// escapeHTMLString escapes HTML special characters in a string. +func escapeHTMLString(s string) string { + s = strings.ReplaceAll(s, "&", "&") + s = strings.ReplaceAll(s, "<", "<") + s = strings.ReplaceAll(s, ">", ">") + s = strings.ReplaceAll(s, "\"", """) + return s +} + +// RegisterRoutes registers Mini App routes on the given mux. + + +func (h *Handler) apiDev(w http.ResponseWriter, r *http.Request) { + switch r.Method { + case http.MethodGet: + writeJSON(w, h.devStatus()) + case http.MethodPost: + body, err := io.ReadAll(io.LimitReader(r.Body, 4096)) + if err != nil { + http.Error(w, `{"error":"bad request"}`, http.StatusBadRequest) + return + } + var req struct { + Action string `json:"action"` + ID string `json:"id"` + } + if err := json.Unmarshal(body, &req); err != nil { + http.Error(w, `{"error":"invalid JSON"}`, http.StatusBadRequest) + return + } + switch req.Action { + case "activate": + if req.ID == "" { + writeJSON(w, map[string]any{"error": "id is required"}) + return + } + if err := h.ActivateDevTarget(req.ID); err != nil { + writeJSON(w, map[string]any{"error": err.Error()}) + return + } + case "deactivate": + if err := h.DeactivateDevTarget(); err != nil { + writeJSON(w, map[string]any{"error": err.Error()}) + return + } + case "unregister": + if req.ID == "" { + writeJSON(w, map[string]any{"error": "id is required"}) + return + } + if err := h.UnregisterDevTarget(req.ID); err != nil { + writeJSON(w, map[string]any{"error": err.Error()}) + return + } + default: + writeJSON(w, map[string]any{"error": "unknown action"}) + return + } + writeJSON(w, h.devStatus()) + default: + http.Error(w, `{"error":"method not allowed"}`, http.StatusMethodNotAllowed) + } +} + + +func (h *Handler) serveDevProxy(w http.ResponseWriter, r *http.Request) { + h.devMu.RLock() + proxy := h.devProxy + h.devMu.RUnlock() + + if proxy == nil { + http.Error(w, "dev proxy not configured", http.StatusServiceUnavailable) + return + } + + // Strip /miniapp/dev prefix so /miniapp/dev/foo → /foo + r.URL.Path = strings.TrimPrefix(r.URL.Path, "/miniapp/dev") + if r.URL.Path == "" { + r.URL.Path = "/" + } + proxy.ServeHTTP(w, r) +} + +// extractUserFromInitData parses user.id from the initData query string. +// initData contains a "user" param with JSON like {"id":123456,...}. + + +func (h *Handler) devStatus() map[string]any { + h.devMu.RLock() + defer h.devMu.RUnlock() + + active := h.devTarget != nil + target := "" + if h.devTarget != nil { + target = h.devTargets[h.devActiveID].Target // original URL before IPv6 rewrite + } + + targets := make([]DevTarget, 0, len(h.devTargets)) + for _, dt := range h.devTargets { + targets = append(targets, *dt) + } + sort.Slice(targets, func(i, j int) bool { return targets[i].ID < targets[j].ID }) + + return map[string]any{ + "active": active, + "active_id": h.devActiveID, + "target": target, + "targets": targets, + } +} + + +// apiDevConsole receives console output from dev preview iframes. +func (h *Handler) apiDevConsole(w http.ResponseWriter, r *http.Request) { + if r.Method != http.MethodPost { + http.Error(w, `{"error":"method not allowed"}`, http.StatusMethodNotAllowed) + return + } + + // Only accept console posts when dev proxy is active + if h.GetDevTarget() == "" { + http.Error(w, `{"error":"not available"}`, http.StatusNotFound) + return + } + + // Simple rate limit: max 10 requests per second + now := time.Now().Unix() + h.consoleMu.Lock() + if h.consoleReqSec != now { + h.consoleReqSec = now + h.consoleReqCount = 0 + } + h.consoleReqCount++ + over := h.consoleReqCount > 10 + h.consoleMu.Unlock() + if over { + http.Error(w, `{"error":"rate limit"}`, http.StatusTooManyRequests) + return + } + + body, err := io.ReadAll(io.LimitReader(r.Body, 32*1024)) + if err != nil { + http.Error(w, `{"error":"bad request"}`, http.StatusBadRequest) + return + } + + var entries []struct { + Level string `json:"level"` + Message string `json:"message"` + } + if err := json.Unmarshal(body, &entries); err != nil { + http.Error(w, `{"error":"invalid JSON"}`, http.StatusBadRequest) + return + } + + // Cap at 20 entries per batch + if len(entries) > 20 { + entries = entries[:20] + } + + for _, e := range entries { + msg := e.Message + if len(msg) > 1024 { + msg = msg[:1024] + } + switch e.Level { + case "warn": + logger.WarnC("dev-console", msg) + case "error": + logger.ErrorC("dev-console", msg) + default: + logger.InfoC("dev-console", msg) + } + } + + w.WriteHeader(http.StatusNoContent) +} + +// wsLogs serves a WebSocket endpoint that streams log entries in real time. + diff --git a/pkg/miniapp/logs.go b/pkg/miniapp/logs.go new file mode 100644 index 000000000..f0b6af368 --- /dev/null +++ b/pkg/miniapp/logs.go @@ -0,0 +1,143 @@ +package miniapp + +import ( + "archive/tar" + "compress/gzip" + "encoding/json" + "fmt" + "net/http" + "os" + "path/filepath" + "strings" + "time" + + "github.com/sipeed/picoclaw/pkg/logger" +) + + +// apiLogsSnapshot creates a tar.gz snapshot of the current log buffer. +func (h *Handler) apiLogsSnapshot(w http.ResponseWriter, r *http.Request) { + if r.Method != http.MethodPost { + http.Error(w, `{"error":"method not allowed"}`, http.StatusMethodNotAllowed) + return + } + + entries := logger.RecentLogs(logger.DEBUG, "", 300) + + snapshotDir := filepath.Join(h.workspace, "logs", "snapshots") + if err := os.MkdirAll(snapshotDir, 0o755); err != nil { + http.Error(w, `{"error":"cannot create snapshot dir"}`, http.StatusInternalServerError) + return + } + + id := time.Now().UTC().Format("20060102-150405") + filename := fmt.Sprintf("picoclaw-logs-%s.tar.gz", id) + snapshotPath := filepath.Join(snapshotDir, filename) + + // Create tar.gz + f, err := os.Create(snapshotPath) + if err != nil { + http.Error(w, `{"error":"cannot create snapshot file"}`, http.StatusInternalServerError) + return + } + + gw := gzip.NewWriter(f) + tw := tar.NewWriter(gw) + + prefix := fmt.Sprintf("picoclaw-logs-%s/", id) + + // logs.json + logsJSON, _ := json.MarshalIndent(entries, "", " ") + _ = tw.WriteHeader(&tar.Header{ + Name: prefix + "logs.json", + Size: int64(len(logsJSON)), + Mode: 0o644, + ModTime: time.Now(), + }) + _, _ = tw.Write(logsJSON) + + // metadata.json + hostname, _ := os.Hostname() + meta := map[string]any{ + "version": "1", + "hostname": hostname, + "timestamp": time.Now().UTC().Format(time.RFC3339), + "entry_count": len(entries), + } + metaJSON, _ := json.MarshalIndent(meta, "", " ") + _ = tw.WriteHeader(&tar.Header{ + Name: prefix + "metadata.json", + Size: int64(len(metaJSON)), + Mode: 0o644, + ModTime: time.Now(), + }) + _, _ = tw.Write(metaJSON) + + tw.Close() + gw.Close() + f.Close() + + // Cleanup old snapshots (>14 days) + go cleanOldSnapshots(snapshotDir, 14*24*time.Hour) + + downloadURL := fmt.Sprintf("/miniapp/api/logs/snapshot/%s", id) + writeJSON(w, map[string]string{"id": id, "download_url": downloadURL}) +} + +// apiLogsSnapshotDownload serves a snapshot tar.gz file. + + +// apiLogsSnapshotDownload serves a snapshot tar.gz file. +func (h *Handler) apiLogsSnapshotDownload(w http.ResponseWriter, r *http.Request) { + if r.Method != http.MethodGet { + http.Error(w, `{"error":"method not allowed"}`, http.StatusMethodNotAllowed) + return + } + + id := strings.TrimPrefix(r.URL.Path, "/miniapp/api/logs/snapshot/") + id = filepath.Base(id) // path traversal prevention + + if id == "" || id == "." || id == ".." { + http.Error(w, `{"error":"invalid id"}`, http.StatusBadRequest) + return + } + + filename := fmt.Sprintf("picoclaw-logs-%s.tar.gz", id) + snapshotPath := filepath.Join(h.workspace, "logs", "snapshots", filename) + + if _, err := os.Stat(snapshotPath); os.IsNotExist(err) { + http.Error(w, `{"error":"not found"}`, http.StatusNotFound) + return + } + + w.Header().Set("Content-Type", "application/gzip") + w.Header().Set("Content-Disposition", fmt.Sprintf(`attachment; filename="%s"`, filename)) + http.ServeFile(w, r, snapshotPath) +} + +// cleanOldSnapshots removes snapshot files older than maxAge. + + +// cleanOldSnapshots removes snapshot files older than maxAge. +func cleanOldSnapshots(dir string, maxAge time.Duration) { + entries, err := os.ReadDir(dir) + if err != nil { + return + } + cutoff := time.Now().Add(-maxAge) + for _, e := range entries { + if e.IsDir() { + continue + } + info, err := e.Info() + if err != nil { + continue + } + if info.ModTime().Before(cutoff) { + os.Remove(filepath.Join(dir, e.Name())) + } + } +} + +// initDataMaxAge is the maximum age of initData before it is considered expired. + diff --git a/pkg/miniapp/miniapp.go b/pkg/miniapp/miniapp.go index 6f1590d04..c2887d918 100644 --- a/pkg/miniapp/miniapp.go +++ b/pkg/miniapp/miniapp.go @@ -1,217 +1,27 @@ package miniapp import ( - "archive/tar" - "bytes" - "compress/gzip" - "crypto/hmac" - "crypto/sha256" "embed" - "encoding/hex" - "encoding/json" - "fmt" - "io" - "net" "net/http" "net/http/httputil" "net/url" - "os" - "path/filepath" - "sort" - "strconv" - "strings" "sync" - "time" - "github.com/gorilla/websocket" - "github.com/sipeed/picoclaw/pkg/logger" - "github.com/sipeed/picoclaw/pkg/skills" - "github.com/sipeed/picoclaw/pkg/stats" + "github.com/sipeed/picoclaw/pkg/orch" ) //go:embed static/index.html var staticFS embed.FS -// PlanPhase mirrors agent.PlanPhase for JSON serialization. -type PlanPhase struct { - Number int `json:"number"` - Title string `json:"title"` - Steps []PlanStep `json:"steps"` -} - -// PlanStep mirrors agent.PlanStep for JSON serialization. -type PlanStep struct { - Index int `json:"index"` - Description string `json:"description"` - Done bool `json:"done"` -} - -// PlanInfo represents the plan state exposed via the API. -type PlanInfo struct { - HasPlan bool `json:"has_plan"` - Status string `json:"status"` - CurrentPhase int `json:"current_phase"` - TotalPhases int `json:"total_phases"` - Display string `json:"display"` - Phases []PlanPhase `json:"phases"` - Memory string `json:"memory"` -} - -// SessionInfo represents an active session entry for the API response. -type SessionInfo struct { - SessionKey string `json:"session_key"` - Channel string `json:"channel"` - ChatID string `json:"chat_id"` - TouchDir string `json:"touch_dir"` - ProjectPath string `json:"project_path,omitempty"` - Purpose string `json:"purpose,omitempty"` - Branch string `json:"branch,omitempty"` - LastSeenAt string `json:"last_seen_at"` - AgeSec int `json:"age_sec"` -} - -// GitRepoSummary represents a lightweight repo entry for the list view. -type GitRepoSummary struct { - Name string `json:"name"` - Branch string `json:"branch"` -} - -// GitInfo represents the git repository state exposed via the API. -type GitInfo struct { - Name string `json:"name"` - Branch string `json:"branch"` - Commits []GitCommit `json:"commits"` - Modified []GitChange `json:"modified"` -} - -// GitCommit represents a single commit entry. -type GitCommit struct { - Hash string `json:"hash"` - Subject string `json:"subject"` - Author string `json:"author"` - Date string `json:"date"` -} - -// GitChange represents a modified/untracked file entry. -type GitChange struct { - Status string `json:"status"` - Path string `json:"path"` -} - -// BootstrapFileInfo describes a resolved bootstrap file for the context API. -type BootstrapFileInfo struct { - Name string `json:"name"` - Path string `json:"path"` - Scope string `json:"scope"` -} - -// ContextInfo describes the agent's directory context and bootstrap file resolution. -type ContextInfo struct { - WorkDir string `json:"work_dir"` - PlanWorkDir string `json:"plan_work_dir"` - Workspace string `json:"workspace"` - Bootstrap []BootstrapFileInfo `json:"bootstrap"` -} - -// DataProvider is the read-only interface to agent state for the Mini App API. -type DataProvider interface { - ListSkills() []skills.SkillInfo - GetPlanInfo() PlanInfo - GetSessionStats() *stats.Stats - GetActiveSessions() []SessionInfo - GetGitRepos() []GitRepoSummary - GetGitRepoDetail(name string) GitInfo - GetContextInfo() ContextInfo - GetSystemPrompt() string -} - -// CommandSender injects a command into the message bus on behalf of a user. -type CommandSender interface { - SendCommand(senderID, chatID, command string) -} - -// StateNotifier broadcasts state-change signals to SSE subscribers. -type StateNotifier struct { - mu sync.Mutex - subs map[chan struct{}]struct{} - done chan struct{} -} - -// NewStateNotifier creates a new StateNotifier. -func NewStateNotifier() *StateNotifier { - return &StateNotifier{ - subs: make(map[chan struct{}]struct{}), - done: make(chan struct{}), - } -} - -// Subscribe returns a channel that receives a signal on each state change. -func (n *StateNotifier) Subscribe() chan struct{} { - ch := make(chan struct{}, 1) - n.mu.Lock() - n.subs[ch] = struct{}{} - n.mu.Unlock() - return ch -} - -// Unsubscribe removes a subscriber channel. -func (n *StateNotifier) Unsubscribe(ch chan struct{}) { - n.mu.Lock() - delete(n.subs, ch) - n.mu.Unlock() -} - -// Close signals all SSE handlers to exit. -func (n *StateNotifier) Close() { - select { - case <-n.done: - default: - close(n.done) - } -} - -// Done returns a channel that is closed when the notifier is shut down. -func (n *StateNotifier) Done() <-chan struct{} { - return n.done -} - -// Notify sends a signal to all subscribers, coalescing rapid notifications. -func (n *StateNotifier) Notify() { - n.mu.Lock() - defer n.mu.Unlock() - for ch := range n.subs { - select { - case ch <- struct{}{}: - default: - } - } -} - -// DevTarget represents a registered dev server target. -type DevTarget struct { - ID string `json:"id"` - Name string `json:"name"` // display name (e.g. "frontend") - Target string `json:"target"` // URL (e.g. "http://localhost:3000") -} - -// DevTargetManager allows tools to register, activate, and deactivate dev proxy targets. -type DevTargetManager interface { - RegisterDevTarget(name, target string) (id string, err error) - UnregisterDevTarget(id string) error - ActivateDevTarget(id string) error - DeactivateDevTarget() error - GetDevTarget() string - ListDevTargets() []DevTarget -} - // Handler serves the Mini App HTML and API endpoints. type Handler struct { - provider DataProvider - sender CommandSender - botToken string - notifier *StateNotifier - allowList []string - workspace string + provider DataProvider + sender CommandSender + botToken string + notifier *StateNotifier + allowList []string + workspace string + orchBroadcaster *orch.Broadcaster devMu sync.RWMutex devTarget *url.URL @@ -228,35 +38,6 @@ type Handler struct { consoleReqSec int64 } -const maxWSClients = 4 - -const ( - wsPongWait = 60 * time.Second - wsPingPeriod = 54 * time.Second // must be less than wsPongWait -) - -type wsClient struct { - conn *websocket.Conn -} - -var wsUpgrader = websocket.Upgrader{ - CheckOrigin: func(r *http.Request) bool { - origin := r.Header.Get("Origin") - if origin == "" { - return true // non-browser clients (e.g. curl) - } - // Allow same-origin requests (e.g. Tailscale direct access) - if u, err := url.Parse(origin); err == nil && u.Host == r.Host { - return true - } - // Allow Telegram WebApp origins and localhost for dev - return strings.HasSuffix(origin, ".telegram.org") || - strings.HasSuffix(origin, ".t.me") || - strings.HasPrefix(origin, "http://localhost") || - strings.HasPrefix(origin, "http://127.0.0.1") - }, -} - // NewHandler creates a new Mini App handler. func NewHandler(provider DataProvider, sender CommandSender, botToken string, notifier *StateNotifier, allowList []string, workspace string) *Handler { return &Handler{ @@ -270,259 +51,10 @@ func NewHandler(provider DataProvider, sender CommandSender, botToken string, no } } -// validateLocalhostURL parses and validates that a URL targets localhost. -func validateLocalhostURL(target string) (*url.URL, error) { - u, err := url.Parse(target) - if err != nil { - return nil, fmt.Errorf("invalid URL: %w", err) - } - host := u.Hostname() - if host != "localhost" && host != "127.0.0.1" && host != "::1" { - return nil, fmt.Errorf("only localhost targets are allowed, got %q", host) - } - return u, nil -} - -// RegisterDevTarget registers a new dev server target. Only localhost targets are allowed. -func (h *Handler) RegisterDevTarget(name, target string) (string, error) { - if _, err := validateLocalhostURL(target); err != nil { - return "", err - } - - h.devMu.Lock() - defer h.devMu.Unlock() - - h.devNextID++ - id := strconv.Itoa(h.devNextID) - - h.devTargets[id] = &DevTarget{ID: id, Name: name, Target: target} - if h.notifier != nil { - h.notifier.Notify() - } - return id, nil -} - -// UnregisterDevTarget removes a registered target. If it was active, the proxy is disabled. -func (h *Handler) UnregisterDevTarget(id string) error { - h.devMu.Lock() - defer h.devMu.Unlock() - - if _, ok := h.devTargets[id]; !ok { - return fmt.Errorf("target %q not found", id) - } - delete(h.devTargets, id) - - if h.devActiveID == id { - h.devActiveID = "" - h.devTarget = nil - h.devProxy = nil - } - if h.notifier != nil { - h.notifier.Notify() - } - return nil -} - -// ActivateDevTarget sets the reverse proxy to the registered target with the given ID. -func (h *Handler) ActivateDevTarget(id string) error { - h.devMu.Lock() - defer h.devMu.Unlock() - - dt, ok := h.devTargets[id] - if !ok { - return fmt.Errorf("target %q not found", id) - } - - u, err := url.Parse(dt.Target) - if err != nil { - return fmt.Errorf("invalid URL: %w", err) - } - - // Fix IPv6: resolve "localhost" to 127.0.0.1 to avoid connection refused on systems - // where localhost resolves to [::1] but the dev server only listens on IPv4. - if u.Hostname() == "localhost" { - u.Host = net.JoinHostPort("127.0.0.1", u.Port()) - } - - proxy := httputil.NewSingleHostReverseProxy(u) - proxy.ModifyResponse = func(resp *http.Response) error { - // Prevent browser/WebView from caching dev proxy responses (CSS, JS, etc.) - resp.Header.Set("Cache-Control", "no-cache, no-store, must-revalidate") - resp.Header.Del("ETag") - resp.Header.Del("Last-Modified") - - ct := resp.Header.Get("Content-Type") - if !strings.Contains(ct, "text/html") { - return nil - } - body, err := io.ReadAll(resp.Body) - if err != nil { - return err - } - resp.Body.Close() - modified := injectDevProxyScript(body) - resp.Body = io.NopCloser(bytes.NewReader(modified)) - resp.ContentLength = int64(len(modified)) - resp.Header.Set("Content-Length", strconv.Itoa(len(modified))) - resp.Header.Del("Content-Encoding") - return nil - } - proxy.ErrorHandler = func(w http.ResponseWriter, r *http.Request, err error) { - w.Header().Set("Content-Type", "text/html; charset=utf-8") - w.WriteHeader(http.StatusBadGateway) - fmt.Fprintf(w, ` -

Cannot connect

%s

Target: %s

`, - escapeHTMLString(err.Error()), escapeHTMLString(dt.Target)) - } - - h.devTarget = u - h.devProxy = proxy - h.devActiveID = id - if h.notifier != nil { - h.notifier.Notify() - } - return nil -} - -// DeactivateDevTarget disables the reverse proxy without removing registrations. -func (h *Handler) DeactivateDevTarget() error { - h.devMu.Lock() - defer h.devMu.Unlock() - - h.devActiveID = "" - h.devTarget = nil - h.devProxy = nil - if h.notifier != nil { - h.notifier.Notify() - } - return nil -} - -// GetDevTarget returns the current dev proxy target URL, or empty string if disabled. -func (h *Handler) GetDevTarget() string { - h.devMu.RLock() - defer h.devMu.RUnlock() - if h.devTarget == nil { - return "" - } - return h.devTarget.String() -} - -// ListDevTargets returns all registered dev targets. -func (h *Handler) ListDevTargets() []DevTarget { - h.devMu.RLock() - defer h.devMu.RUnlock() - - targets := make([]DevTarget, 0, len(h.devTargets)) - for _, dt := range h.devTargets { - targets = append(targets, *dt) - } - // Sort by ID for stable order - sort.Slice(targets, func(i, j int) bool { return targets[i].ID < targets[j].ID }) - return targets -} - -// devProxyScript is the JavaScript injected into HTML responses from the dev proxy. -// It rewrites fetch() and XMLHttpRequest.open() so that absolute paths like -// "/api/items" are prefixed with "/miniapp/dev", matching the reverse proxy mount. -// It also captures console.log/warn/error/info and forwards them to the server. -const devProxyScript = `` - -// injectDevProxyScript inserts the dev proxy rewrite script into an HTML document. -// Insertion priority: before , after , or prepend to document. -func injectDevProxyScript(html []byte) []byte { - script := []byte(devProxyScript) - - // Priority 1: before - if idx := bytes.Index(bytes.ToLower(html), []byte("")); idx >= 0 { - out := make([]byte, 0, len(html)+len(script)) - out = append(out, html[:idx]...) - out = append(out, script...) - out = append(out, html[idx:]...) - return out - } - - // Priority 2: after - lower := bytes.ToLower(html) - if idx := bytes.Index(lower, []byte("= 0 { - // Find the closing '>' of the tag - closeIdx := bytes.IndexByte(lower[idx:], '>') - if closeIdx >= 0 { - insertAt := idx + closeIdx + 1 - out := make([]byte, 0, len(html)+len(script)) - out = append(out, html[:insertAt]...) - out = append(out, script...) - out = append(out, html[insertAt:]...) - return out - } - } - - // Priority 3: prepend - out := make([]byte, 0, len(html)+len(script)) - out = append(out, script...) - out = append(out, html...) - return out -} - -// escapeHTMLString escapes HTML special characters in a string. -func escapeHTMLString(s string) string { - s = strings.ReplaceAll(s, "&", "&") - s = strings.ReplaceAll(s, "<", "<") - s = strings.ReplaceAll(s, ">", ">") - s = strings.ReplaceAll(s, "\"", """) - return s +// SetOrchBroadcaster wires the orchestration broadcaster so the Mini App can +// push live agent state to the canvas UI via WebSocket. +func (h *Handler) SetOrchBroadcaster(b *orch.Broadcaster) { + h.orchBroadcaster = b } // RegisterRoutes registers Mini App routes on the given mux. @@ -541,6 +73,7 @@ func (h *Handler) RegisterRoutes(mux *http.ServeMux) { mux.HandleFunc("/miniapp/api/logs/ws", h.requireAuth(h.wsLogs)) mux.HandleFunc("/miniapp/api/logs/snapshot", h.requireAuth(h.apiLogsSnapshot)) mux.HandleFunc("/miniapp/api/logs/snapshot/", h.requireAuth(h.apiLogsSnapshotDownload)) + mux.HandleFunc("/miniapp/api/orchestration/ws", h.requireAuth(h.wsOrchestration)) mux.HandleFunc("/miniapp/dev/console", h.apiDevConsole) mux.HandleFunc("/miniapp/dev/", h.serveDevProxy) } @@ -554,647 +87,3 @@ func (h *Handler) serveIndex(w http.ResponseWriter, r *http.Request) { w.Header().Set("Content-Type", "text/html; charset=utf-8") w.Write(data) } - -func (h *Handler) requireAuth(next http.HandlerFunc) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - initData := r.URL.Query().Get("initData") - if initData == "" { - http.Error(w, `{"error":"missing initData"}`, http.StatusUnauthorized) - return - } - if !ValidateInitData(initData, h.botToken) { - http.Error(w, `{"error":"invalid initData"}`, http.StatusUnauthorized) - return - } - if len(h.allowList) > 0 { - userID, _ := extractUserFromInitData(initData) - if userID == "" || !isAllowed(userID, h.allowList) { - http.Error(w, `{"error":"forbidden"}`, http.StatusForbidden) - return - } - } - next(w, r) - } -} - -// isAllowed checks whether userID matches any entry in the allow list. -// Logic mirrors BaseChannel.IsAllowed without importing channels package. -func isAllowed(userID string, allowList []string) bool { - if len(allowList) == 0 { - return true - } - for _, allowed := range allowList { - trimmed := strings.TrimPrefix(allowed, "@") - allowedID := trimmed - if idx := strings.Index(trimmed, "|"); idx > 0 { - allowedID = trimmed[:idx] - } - if userID == allowed || userID == trimmed || userID == allowedID { - return true - } - } - return false -} - -func (h *Handler) apiSkills(w http.ResponseWriter, r *http.Request) { - skillsList := h.provider.ListSkills() - writeJSON(w, skillsList) -} - -func (h *Handler) apiPlan(w http.ResponseWriter, r *http.Request) { - info := h.provider.GetPlanInfo() - writeJSON(w, info) -} - -func (h *Handler) apiSessions(w http.ResponseWriter, r *http.Request) { - sessions := h.provider.GetActiveSessions() - if sessions == nil { - sessions = []SessionInfo{} - } - writeJSON(w, sessions) -} - -func (h *Handler) apiSession(w http.ResponseWriter, r *http.Request) { - s := h.provider.GetSessionStats() - if s == nil { - writeJSON(w, map[string]string{"status": "stats not enabled"}) - return - } - writeJSON(w, s) -} - -func (h *Handler) apiContext(w http.ResponseWriter, r *http.Request) { - writeJSON(w, h.provider.GetContextInfo()) -} - -func (h *Handler) apiPrompt(w http.ResponseWriter, r *http.Request) { - writeJSON(w, map[string]string{"prompt": h.provider.GetSystemPrompt()}) -} - -func (h *Handler) apiGit(w http.ResponseWriter, r *http.Request) { - repo := r.URL.Query().Get("repo") - if repo == "" { - writeJSON(w, h.provider.GetGitRepos()) - } else { - writeJSON(w, h.provider.GetGitRepoDetail(repo)) - } -} - -func (h *Handler) apiCommand(w http.ResponseWriter, r *http.Request) { - if r.Method != http.MethodPost { - http.Error(w, `{"error":"method not allowed"}`, http.StatusMethodNotAllowed) - return - } - - body, err := io.ReadAll(io.LimitReader(r.Body, 4096)) - if err != nil { - http.Error(w, `{"error":"bad request"}`, http.StatusBadRequest) - return - } - - var req struct { - Command string `json:"command"` - } - if err := json.Unmarshal(body, &req); err != nil || req.Command == "" { - http.Error(w, `{"error":"missing command"}`, http.StatusBadRequest) - return - } - - if !strings.HasPrefix(req.Command, "/") { - http.Error(w, `{"error":"command must start with /"}`, http.StatusBadRequest) - return - } - - // Extract user ID from initData to identify the sender - initData := r.URL.Query().Get("initData") - userID, chatID := extractUserFromInitData(initData) - if userID == "" { - http.Error(w, `{"error":"cannot identify user"}`, http.StatusBadRequest) - return - } - - h.sender.SendCommand(userID, chatID, req.Command) - writeJSON(w, map[string]string{"status": "ok"}) -} - -func (h *Handler) apiDev(w http.ResponseWriter, r *http.Request) { - switch r.Method { - case http.MethodGet: - writeJSON(w, h.devStatus()) - case http.MethodPost: - body, err := io.ReadAll(io.LimitReader(r.Body, 4096)) - if err != nil { - http.Error(w, `{"error":"bad request"}`, http.StatusBadRequest) - return - } - var req struct { - Action string `json:"action"` - ID string `json:"id"` - } - if err := json.Unmarshal(body, &req); err != nil { - http.Error(w, `{"error":"invalid JSON"}`, http.StatusBadRequest) - return - } - switch req.Action { - case "activate": - if req.ID == "" { - writeJSON(w, map[string]any{"error": "id is required"}) - return - } - if err := h.ActivateDevTarget(req.ID); err != nil { - writeJSON(w, map[string]any{"error": err.Error()}) - return - } - case "deactivate": - if err := h.DeactivateDevTarget(); err != nil { - writeJSON(w, map[string]any{"error": err.Error()}) - return - } - case "unregister": - if req.ID == "" { - writeJSON(w, map[string]any{"error": "id is required"}) - return - } - if err := h.UnregisterDevTarget(req.ID); err != nil { - writeJSON(w, map[string]any{"error": err.Error()}) - return - } - default: - writeJSON(w, map[string]any{"error": "unknown action"}) - return - } - writeJSON(w, h.devStatus()) - default: - http.Error(w, `{"error":"method not allowed"}`, http.StatusMethodNotAllowed) - } -} - -func (h *Handler) serveDevProxy(w http.ResponseWriter, r *http.Request) { - h.devMu.RLock() - proxy := h.devProxy - h.devMu.RUnlock() - - if proxy == nil { - http.Error(w, "dev proxy not configured", http.StatusServiceUnavailable) - return - } - - // Strip /miniapp/dev prefix so /miniapp/dev/foo → /foo - r.URL.Path = strings.TrimPrefix(r.URL.Path, "/miniapp/dev") - if r.URL.Path == "" { - r.URL.Path = "/" - } - proxy.ServeHTTP(w, r) -} - -// extractUserFromInitData parses user.id from the initData query string. -// initData contains a "user" param with JSON like {"id":123456,...}. -func extractUserFromInitData(initData string) (userID, chatID string) { - values, err := url.ParseQuery(initData) - if err != nil { - return "", "" - } - userJSON := values.Get("user") - if userJSON == "" { - return "", "" - } - var user struct { - ID int64 `json:"id"` - } - if err := json.Unmarshal([]byte(userJSON), &user); err != nil || user.ID == 0 { - return "", "" - } - id := fmt.Sprintf("%d", user.ID) - // For Mini App commands, chatID = userID (private chat) - return id, id -} - -func (h *Handler) apiEvents(w http.ResponseWriter, r *http.Request) { - flusher, ok := w.(http.Flusher) - if !ok { - http.Error(w, `{"error":"streaming not supported"}`, http.StatusInternalServerError) - return - } - rc := http.NewResponseController(w) - _ = rc.SetWriteDeadline(time.Time{}) - - w.Header().Set("Content-Type", "text/event-stream") - w.Header().Set("Cache-Control", "no-cache") - w.Header().Set("Connection", "keep-alive") - w.Header().Set("X-Accel-Buffering", "no") - - ch := h.notifier.Subscribe() - defer h.notifier.Unsubscribe(ch) - - var lastPlan, lastSession, lastSkills, lastDev, lastContext, lastPrompt []byte - - // Send initial state immediately - sendSSEIfChanged(w, flusher, "plan", h.provider.GetPlanInfo(), &lastPlan) - sendSSEIfChanged(w, flusher, "session", - map[string]any{"stats": h.provider.GetSessionStats(), "sessions": h.provider.GetActiveSessions()}, - &lastSession) - sendSSEIfChanged(w, flusher, "skills", h.provider.ListSkills(), &lastSkills) - sendSSEIfChanged(w, flusher, "dev", h.devStatus(), &lastDev) - sendSSEIfChanged(w, flusher, "context", h.provider.GetContextInfo(), &lastContext) - sendSSEIfChanged(w, flusher, "prompt", map[string]string{"prompt": h.provider.GetSystemPrompt()}, &lastPrompt) - - for { - select { - case <-r.Context().Done(): - return - case <-h.notifier.Done(): - return - case <-ch: - sendSSEIfChanged(w, flusher, "plan", h.provider.GetPlanInfo(), &lastPlan) - sendSSEIfChanged(w, flusher, "session", - map[string]any{"stats": h.provider.GetSessionStats(), "sessions": h.provider.GetActiveSessions()}, - &lastSession) - sendSSEIfChanged(w, flusher, "skills", h.provider.ListSkills(), &lastSkills) - sendSSEIfChanged(w, flusher, "dev", h.devStatus(), &lastDev) - sendSSEIfChanged(w, flusher, "context", h.provider.GetContextInfo(), &lastContext) - sendSSEIfChanged(w, flusher, "prompt", map[string]string{"prompt": h.provider.GetSystemPrompt()}, &lastPrompt) - } - } -} - -func (h *Handler) devStatus() map[string]any { - h.devMu.RLock() - defer h.devMu.RUnlock() - - active := h.devTarget != nil - target := "" - if h.devTarget != nil { - target = h.devTargets[h.devActiveID].Target // original URL before IPv6 rewrite - } - - targets := make([]DevTarget, 0, len(h.devTargets)) - for _, dt := range h.devTargets { - targets = append(targets, *dt) - } - sort.Slice(targets, func(i, j int) bool { return targets[i].ID < targets[j].ID }) - - return map[string]any{ - "active": active, - "active_id": h.devActiveID, - "target": target, - "targets": targets, - } -} - -func sendSSEIfChanged(w http.ResponseWriter, f http.Flusher, event string, v any, last *[]byte) { - data, _ := json.Marshal(v) - if !bytes.Equal(data, *last) { - fmt.Fprintf(w, "event: %s\ndata: %s\n\n", event, data) - f.Flush() - *last = data - } -} - -func writeJSON(w http.ResponseWriter, v any) { - w.Header().Set("Content-Type", "application/json") - json.NewEncoder(w).Encode(v) -} - -// apiDevConsole receives console output from dev preview iframes. -func (h *Handler) apiDevConsole(w http.ResponseWriter, r *http.Request) { - if r.Method != http.MethodPost { - http.Error(w, `{"error":"method not allowed"}`, http.StatusMethodNotAllowed) - return - } - - // Only accept console posts when dev proxy is active - if h.GetDevTarget() == "" { - http.Error(w, `{"error":"not available"}`, http.StatusNotFound) - return - } - - // Simple rate limit: max 10 requests per second - now := time.Now().Unix() - h.consoleMu.Lock() - if h.consoleReqSec != now { - h.consoleReqSec = now - h.consoleReqCount = 0 - } - h.consoleReqCount++ - over := h.consoleReqCount > 10 - h.consoleMu.Unlock() - if over { - http.Error(w, `{"error":"rate limit"}`, http.StatusTooManyRequests) - return - } - - body, err := io.ReadAll(io.LimitReader(r.Body, 32*1024)) - if err != nil { - http.Error(w, `{"error":"bad request"}`, http.StatusBadRequest) - return - } - - var entries []struct { - Level string `json:"level"` - Message string `json:"message"` - } - if err := json.Unmarshal(body, &entries); err != nil { - http.Error(w, `{"error":"invalid JSON"}`, http.StatusBadRequest) - return - } - - // Cap at 20 entries per batch - if len(entries) > 20 { - entries = entries[:20] - } - - for _, e := range entries { - msg := e.Message - if len(msg) > 1024 { - msg = msg[:1024] - } - switch e.Level { - case "warn": - logger.WarnC("dev-console", msg) - case "error": - logger.ErrorC("dev-console", msg) - default: - logger.InfoC("dev-console", msg) - } - } - - w.WriteHeader(http.StatusNoContent) -} - -// wsLogs serves a WebSocket endpoint that streams log entries in real time. -func (h *Handler) wsLogs(w http.ResponseWriter, r *http.Request) { - // Parse filter params - component := r.URL.Query().Get("component") - levelStr := r.URL.Query().Get("level") - minLevel := logger.INFO - if levelStr != "" { - minLevel = logger.ParseLevel(levelStr) - } - - // Clear HTTP server deadlines before WebSocket hijack - rc := http.NewResponseController(w) - _ = rc.SetWriteDeadline(time.Time{}) - _ = rc.SetReadDeadline(time.Time{}) - - conn, err := wsUpgrader.Upgrade(w, r, nil) - if err != nil { - return - } - - client := &wsClient{conn: conn} - - // Enforce max WS clients: evict oldest if full - h.wsClientsMu.Lock() - if len(h.wsClients) >= maxWSClients { - oldest := h.wsClients[0] - h.wsClients = h.wsClients[1:] - oldest.conn.Close() - } - h.wsClients = append(h.wsClients, client) - h.wsClientsMu.Unlock() - - defer func() { - h.wsClientsMu.Lock() - for i, c := range h.wsClients { - if c == client { - h.wsClients = append(h.wsClients[:i], h.wsClients[i+1:]...) - break - } - } - h.wsClientsMu.Unlock() - conn.Close() - }() - - // Build filter function - filter := func(e logger.LogEntry) bool { - if lvl := logger.ParseLevel(e.Level); lvl < minLevel { - return false - } - if component != "" && e.Component != component { - return false - } - return true - } - - sub := logger.Subscribe(filter) - defer logger.Unsubscribe(sub) - - // Configure ping/pong to detect dead connections - conn.SetReadDeadline(time.Now().Add(wsPongWait)) - conn.SetPongHandler(func(string) error { - conn.SetReadDeadline(time.Now().Add(wsPongWait)) - return nil - }) - - // Send initial data - initial := logger.RecentLogs(minLevel, component, 50) - if err := conn.WriteJSON(map[string]any{"type": "init", "entries": initial}); err != nil { - return - } - - // Close detection goroutine - done := make(chan struct{}) - go func() { - defer close(done) - for { - if _, _, err := conn.ReadMessage(); err != nil { - return - } - } - }() - - // Stream loop with periodic pings - ticker := time.NewTicker(wsPingPeriod) - defer ticker.Stop() - - for { - select { - case entry, ok := <-sub.Ch: - if !ok { - return - } - entry.Caller = "" // strip for security - entry.Fields = logger.SanitizeFields(entry.Fields) // mask sensitive values - if err := conn.WriteJSON(map[string]any{"type": "entry", "entry": entry}); err != nil { - return - } - case <-ticker.C: - if err := conn.WriteMessage(websocket.PingMessage, nil); err != nil { - return - } - case <-done: - return - } - } -} - -// apiLogsSnapshot creates a tar.gz snapshot of the current log buffer. -func (h *Handler) apiLogsSnapshot(w http.ResponseWriter, r *http.Request) { - if r.Method != http.MethodPost { - http.Error(w, `{"error":"method not allowed"}`, http.StatusMethodNotAllowed) - return - } - - entries := logger.RecentLogs(logger.DEBUG, "", 300) - - snapshotDir := filepath.Join(h.workspace, "logs", "snapshots") - if err := os.MkdirAll(snapshotDir, 0o755); err != nil { - http.Error(w, `{"error":"cannot create snapshot dir"}`, http.StatusInternalServerError) - return - } - - id := time.Now().UTC().Format("20060102-150405") - filename := fmt.Sprintf("picoclaw-logs-%s.tar.gz", id) - snapshotPath := filepath.Join(snapshotDir, filename) - - // Create tar.gz - f, err := os.Create(snapshotPath) - if err != nil { - http.Error(w, `{"error":"cannot create snapshot file"}`, http.StatusInternalServerError) - return - } - - gw := gzip.NewWriter(f) - tw := tar.NewWriter(gw) - - prefix := fmt.Sprintf("picoclaw-logs-%s/", id) - - // logs.json - logsJSON, _ := json.MarshalIndent(entries, "", " ") - _ = tw.WriteHeader(&tar.Header{ - Name: prefix + "logs.json", - Size: int64(len(logsJSON)), - Mode: 0o644, - ModTime: time.Now(), - }) - _, _ = tw.Write(logsJSON) - - // metadata.json - hostname, _ := os.Hostname() - meta := map[string]any{ - "version": "1", - "hostname": hostname, - "timestamp": time.Now().UTC().Format(time.RFC3339), - "entry_count": len(entries), - } - metaJSON, _ := json.MarshalIndent(meta, "", " ") - _ = tw.WriteHeader(&tar.Header{ - Name: prefix + "metadata.json", - Size: int64(len(metaJSON)), - Mode: 0o644, - ModTime: time.Now(), - }) - _, _ = tw.Write(metaJSON) - - tw.Close() - gw.Close() - f.Close() - - // Cleanup old snapshots (>14 days) - go cleanOldSnapshots(snapshotDir, 14*24*time.Hour) - - downloadURL := fmt.Sprintf("/miniapp/api/logs/snapshot/%s", id) - writeJSON(w, map[string]string{"id": id, "download_url": downloadURL}) -} - -// apiLogsSnapshotDownload serves a snapshot tar.gz file. -func (h *Handler) apiLogsSnapshotDownload(w http.ResponseWriter, r *http.Request) { - if r.Method != http.MethodGet { - http.Error(w, `{"error":"method not allowed"}`, http.StatusMethodNotAllowed) - return - } - - id := strings.TrimPrefix(r.URL.Path, "/miniapp/api/logs/snapshot/") - id = filepath.Base(id) // path traversal prevention - - if id == "" || id == "." || id == ".." { - http.Error(w, `{"error":"invalid id"}`, http.StatusBadRequest) - return - } - - filename := fmt.Sprintf("picoclaw-logs-%s.tar.gz", id) - snapshotPath := filepath.Join(h.workspace, "logs", "snapshots", filename) - - if _, err := os.Stat(snapshotPath); os.IsNotExist(err) { - http.Error(w, `{"error":"not found"}`, http.StatusNotFound) - return - } - - w.Header().Set("Content-Type", "application/gzip") - w.Header().Set("Content-Disposition", fmt.Sprintf(`attachment; filename="%s"`, filename)) - http.ServeFile(w, r, snapshotPath) -} - -// cleanOldSnapshots removes snapshot files older than maxAge. -func cleanOldSnapshots(dir string, maxAge time.Duration) { - entries, err := os.ReadDir(dir) - if err != nil { - return - } - cutoff := time.Now().Add(-maxAge) - for _, e := range entries { - if e.IsDir() { - continue - } - info, err := e.Info() - if err != nil { - continue - } - if info.ModTime().Before(cutoff) { - os.Remove(filepath.Join(dir, e.Name())) - } - } -} - -// initDataMaxAge is the maximum age of initData before it is considered expired. -const initDataMaxAge = 24 * time.Hour - -// ValidateInitData verifies the Telegram WebApp initData HMAC-SHA256 signature -// and checks that auth_date is not older than initDataMaxAge. -// See https://core.telegram.org/bots/webapps#validating-data-received-via-the-mini-app -func ValidateInitData(initData, botToken string) bool { - values, err := url.ParseQuery(initData) - if err != nil { - return false - } - - receivedHash := values.Get("hash") - if receivedHash == "" { - return false - } - - // Check auth_date freshness - if authDateStr := values.Get("auth_date"); authDateStr != "" { - authDate, err := strconv.ParseInt(authDateStr, 10, 64) - if err != nil { - return false - } - if time.Since(time.Unix(authDate, 0)) > initDataMaxAge { - return false - } - } - - // Build the data-check-string: sort all key=value pairs except "hash", - // join with newlines. - var pairs []string - for key := range values { - if key == "hash" { - continue - } - pairs = append(pairs, fmt.Sprintf("%s=%s", key, values.Get(key))) - } - sort.Strings(pairs) - dataCheckString := strings.Join(pairs, "\n") - - // secret_key = HMAC-SHA256("WebAppData", bot_token) - secretKeyMac := hmac.New(sha256.New, []byte("WebAppData")) - secretKeyMac.Write([]byte(botToken)) - secretKey := secretKeyMac.Sum(nil) - - // hash = HMAC-SHA256(secret_key, data_check_string) - hashMac := hmac.New(sha256.New, secretKey) - hashMac.Write([]byte(dataCheckString)) - computedHash := hex.EncodeToString(hashMac.Sum(nil)) - - return hmac.Equal([]byte(computedHash), []byte(receivedHash)) -} diff --git a/pkg/miniapp/static/map-preview.html b/pkg/miniapp/static/map-preview.html new file mode 100644 index 000000000..5cadcae5d --- /dev/null +++ b/pkg/miniapp/static/map-preview.html @@ -0,0 +1,458 @@ + + + + + + Orchestration Room + + + + + + +
+ + +
+
+
👑
+
CNDR
+
+
+
+
👩‍💌
+
SEC
+
+
+
+ + +
+ +
+ + +
+
+
🔍
+
SCOUT
+
+
+
+
📊
+
ANLY
+
+
+
+
💻
+
CODE
+
+
+
+
🔧
+
WRKR
+
+
+
+
🎯
+
CORD
+
+
+
+ +
+ +
+ demo: idle + ⬡ fast = toolcall + ⬡ slow = llm wait +
+ + + + diff --git a/pkg/miniapp/static/map.js b/pkg/miniapp/static/map.js new file mode 100644 index 000000000..74b56e433 --- /dev/null +++ b/pkg/miniapp/static/map.js @@ -0,0 +1,204 @@ +// map.js — Orchestration Room +// +// External asset: drop map.png (320×320px) next to index.html to replace +// the procedural fallback. Character positions (MAP_POSITIONS) are defined +// in canvas-pixel coordinates and remain valid regardless of which rendering +// path is used — just make sure your map.png matches them. +// +// Usage: +// loadMapAsset(function() { drawMap(ctx); }); // call once on init +// drawMap(ctx); // call each frame + +// ─── Character home positions (px, canvas 320×320) ───────────────────────── +// +// ┌──────────────────────────────┐ +// │ [conductor desk] │ y ≈ 20–50 +// │ 👑(160,58) 👩‍💌(108,58) │ +// │ [carpet] │ +// │ [WS1] [WS2] [WS3] │ y ≈ 80–100 +// │ 🔍40 💻144 📊248 │ y = 106 +// │ [meeting area] │ y ≈ 130–192 +// │ [WS4] [WS5] │ y ≈ 200–220 +// │ 🔧40 🎯144 │ y = 222 +// │ 🚪(160,308) │ door +// └──────────────────────────────┘ + +var MAP_POSITIONS = { + door: { x: 160, y: 314 }, // entry / exit point + conductor: { x: 160, y: 58 }, + secretary: { x: 108, y: 58 }, + meeting: { x: 160, y: 161 }, // neutral zone for conversations + stations: [ + { x: 40, y: 106 }, // S0 scout + { x: 144, y: 106 }, // S1 analyst + { x: 248, y: 106 }, // S2 coder + { x: 40, y: 222 }, // S3 worker + { x: 144, y: 222 }, // S4 coordinator + ], +}; + +// ─── Asset loading ────────────────────────────────────────────────────────── + +var _mapImage = null; + +// Call once before first draw. cb() is invoked when ready (image or fallback). +function loadMapAsset(cb) { + var img = new Image(); + img.onload = function() { _mapImage = img; cb(); }; + img.onerror = function() { cb(); }; // no map.png → use fallback + img.src = './map.png'; +} + +// ─── Public draw entry point ──────────────────────────────────────────────── + +function drawMap(ctx) { + ctx.imageSmoothingEnabled = false; + if (_mapImage) { + ctx.drawImage(_mapImage, 0, 0, 320, 320); + } else { + _drawMapFallback(ctx); + } +} + +// ─── Procedural fallback ──────────────────────────────────────────────────── + +var _C = { + wallDark: '#0c1018', + wallHighlight: '#252d3f', + floorA: '#171b2c', + floorB: '#1b2033', + carpetBase: '#1a2050', + carpetBorder: '#2a3480', + deskBack: '#2c3e6b', + deskTop: '#3a50a0', + deskEdge: '#4a6ac0', + deskShadow: '#1a2448', + monitorFrame: '#070b14', + monitorBlue: '#1040a0', + monitorGlow: '#4488ff', + wsBase: '#162818', + wsTop: '#1e3822', + wsEdge: '#2a5030', + termGlow: '#00dd55', + rugFill: '#1c2248', + rugBorder: '#283070', + doorMid: '#8a5818', + doorLight: '#a06820', + doorGold: '#c8940a', +}; + +function _r(ctx, color, x, y, w, h, alpha) { + ctx.globalAlpha = alpha === undefined ? 1 : alpha; + ctx.fillStyle = color; + ctx.fillRect(x, y, w, h); + ctx.globalAlpha = 1; +} + +function _b(ctx, color, x, y, w, h) { + ctx.strokeStyle = color; + ctx.lineWidth = 1; + ctx.strokeRect(x + 0.5, y + 0.5, w - 1, h - 1); +} + +function _dot(ctx, color, x, y) { + ctx.fillStyle = color; + ctx.fillRect(x, y, 2, 2); +} + +function _workstation(ctx, x, y) { + _r(ctx, _C.wsBase, x, y, 48, 20); + _r(ctx, _C.wsTop, x, y, 48, 8); + _r(ctx, _C.wsEdge, x, y, 2, 20); + _r(ctx, _C.wsEdge, x+46, y, 2, 20); + _r(ctx, _C.wsEdge, x, y, 48, 2); + // terminal screen + _r(ctx, _C.monitorFrame, x+16, y+2, 16, 12); + _r(ctx, '#041008', x+17, y+3, 14, 10); + _r(ctx, '#003315', x+18, y+4, 12, 8); + _r(ctx, _C.termGlow, x+20, y+6, 8, 3); + _dot(ctx, '#00ff88', x+22, y+6); +} + +function _drawMapFallback(ctx) { + var T = 16; + + // floor tiles + for (var ty = 0; ty < 20; ty++) { + for (var tx = 0; tx < 20; tx++) { + ctx.fillStyle = (tx + ty) % 2 === 0 ? _C.floorA : _C.floorB; + ctx.fillRect(tx * T, ty * T, T, T); + } + } + + // conductor carpet + _r(ctx, _C.carpetBase, 16, 16, 288, 50); + _b(ctx, _C.carpetBorder, 18, 18, 284, 46); + + // conductor desk + _r(ctx, _C.deskBack, 96, 20, 128, 30); + _r(ctx, _C.deskTop, 96, 20, 128, 12); + _r(ctx, _C.deskEdge, 96, 20, 128, 2); + _r(ctx, _C.deskEdge, 96, 20, 2, 30); + _r(ctx, _C.deskEdge, 222, 20, 2, 30); + _r(ctx, _C.deskShadow,96,48, 128, 4); + // monitor + _r(ctx, _C.monitorFrame, 138, 22, 44, 14); + _r(ctx, _C.monitorBlue, 140, 23, 40, 12); + _r(ctx, _C.monitorGlow, 156, 26, 8, 6); + _r(ctx, '#6699ff', 158, 27, 4, 3); + + // workstations + _workstation(ctx, 16, 80); // S0 + _workstation(ctx, 128, 80); // S1 (x+24 = 152 ≈ 144 center) + _workstation(ctx, 224, 80); // S2 + _workstation(ctx, 16, 200); // S3 + _workstation(ctx, 128, 200); // S4 + + // meeting rug + _r(ctx, _C.rugFill, 64, 130, 192, 62, 0.55); + _b(ctx, _C.rugBorder, 66, 132, 188, 58); + _b(ctx, '#202860', 70, 136, 180, 50); + + // bulletin board (left wall) + _r(ctx, '#2c1a06', 18, 148, 36, 44); + _r(ctx, '#3a2508', 20, 150, 32, 40); + _r(ctx, '#cc9900', 22, 153, 12, 8); + _r(ctx, '#dd8800', 22, 164, 10, 6); + _r(ctx, '#bb7700', 34, 155, 13, 8); + _r(ctx, '#ccaa00', 33, 165, 11, 6); + _dot(ctx, '#ff4444', 28, 153); + _dot(ctx, '#44aaff', 41, 158); + _dot(ctx, '#44ff88', 27, 165); + + // server rack (right wall) + _r(ctx, '#111122', 285, 80, 18, 112); + _r(ctx, '#181830', 287, 82, 14, 108); + for (var i = 0; i < 10; i++) { + var ry = 85 + i * 10; + _r(ctx, '#0a0a12', 288, ry, 12, 8); + var lc = ['#00ff44','#0044ff','#ff3300','#111111'][i % 4]; + _r(ctx, lc, 296, ry + 2, 3, 4); + } + + // walls (drawn last to cover any overruns) + _r(ctx, _C.wallDark, 0, 0, 320, 16); + _r(ctx, _C.wallHighlight, 0, 14, 320, 2); + _r(ctx, _C.wallDark, 0, 0, 16, 320); + _r(ctx, _C.wallHighlight,14, 0, 2, 320); + _r(ctx, _C.wallDark, 304, 0, 16, 320); + _r(ctx, _C.wallHighlight,304, 0, 2, 320); + _r(ctx, _C.wallDark, 0, 304, 144, 16); + _r(ctx, _C.wallDark, 176, 304, 144, 16); + _r(ctx, _C.wallHighlight, 0, 304, 144, 2); + _r(ctx, _C.wallHighlight,176, 304, 144, 2); + + // door + _r(ctx, '#0a0808', 144, 292, 32, 12); // outside (dark) + _r(ctx, _C.doorMid, 144, 280, 32, 24); + _r(ctx, _C.doorLight, 144, 280, 32, 3); + _r(ctx, _C.doorLight, 144, 280, 3, 24); + _r(ctx, _C.doorLight, 173, 280, 3, 24); + _r(ctx, '#4a2408', 146, 284, 12, 16); // door panels + _r(ctx, '#4a2408', 162, 284, 12, 16); + _r(ctx, _C.doorGold, 170, 291, 5, 5); // handle +} diff --git a/pkg/miniapp/types.go b/pkg/miniapp/types.go new file mode 100644 index 000000000..2e2fd1016 --- /dev/null +++ b/pkg/miniapp/types.go @@ -0,0 +1,180 @@ +package miniapp + +import ( + "sync" + + "github.com/sipeed/picoclaw/pkg/skills" + "github.com/sipeed/picoclaw/pkg/stats" +) + +// PlanPhase mirrors agent.PlanPhase for JSON serialization. +type PlanPhase struct { + Number int `json:"number"` + Title string `json:"title"` + Steps []PlanStep `json:"steps"` +} + +// PlanStep mirrors agent.PlanStep for JSON serialization. +type PlanStep struct { + Index int `json:"index"` + Description string `json:"description"` + Done bool `json:"done"` +} + +// PlanInfo represents the plan state exposed via the API. +type PlanInfo struct { + HasPlan bool `json:"has_plan"` + Status string `json:"status"` + CurrentPhase int `json:"current_phase"` + TotalPhases int `json:"total_phases"` + Display string `json:"display"` + Phases []PlanPhase `json:"phases"` + Memory string `json:"memory"` +} + +// SessionInfo represents an active session entry for the API response. +type SessionInfo struct { + SessionKey string `json:"session_key"` + Channel string `json:"channel"` + ChatID string `json:"chat_id"` + TouchDir string `json:"touch_dir"` + ProjectPath string `json:"project_path,omitempty"` + Purpose string `json:"purpose,omitempty"` + Branch string `json:"branch,omitempty"` + LastSeenAt string `json:"last_seen_at"` + AgeSec int `json:"age_sec"` +} + +// GitRepoSummary represents a lightweight repo entry for the list view. +type GitRepoSummary struct { + Name string `json:"name"` + Branch string `json:"branch"` +} + +// GitInfo represents the git repository state exposed via the API. +type GitInfo struct { + Name string `json:"name"` + Branch string `json:"branch"` + Commits []GitCommit `json:"commits"` + Modified []GitChange `json:"modified"` +} + +// GitCommit represents a single commit entry. +type GitCommit struct { + Hash string `json:"hash"` + Subject string `json:"subject"` + Author string `json:"author"` + Date string `json:"date"` +} + +// GitChange represents a modified/untracked file entry. +type GitChange struct { + Status string `json:"status"` + Path string `json:"path"` +} + +// BootstrapFileInfo describes a resolved bootstrap file for the context API. +type BootstrapFileInfo struct { + Name string `json:"name"` + Path string `json:"path"` + Scope string `json:"scope"` +} + +// ContextInfo describes the agent's directory context and bootstrap file resolution. +type ContextInfo struct { + WorkDir string `json:"work_dir"` + PlanWorkDir string `json:"plan_work_dir"` + Workspace string `json:"workspace"` + Bootstrap []BootstrapFileInfo `json:"bootstrap"` +} + +// DataProvider is the read-only interface to agent state for the Mini App API. +type DataProvider interface { + ListSkills() []skills.SkillInfo + GetPlanInfo() PlanInfo + GetSessionStats() *stats.Stats + GetActiveSessions() []SessionInfo + GetGitRepos() []GitRepoSummary + GetGitRepoDetail(name string) GitInfo + GetContextInfo() ContextInfo + GetSystemPrompt() string +} + +// CommandSender injects a command into the message bus on behalf of a user. +type CommandSender interface { + SendCommand(senderID, chatID, command string) +} + +// DevTarget represents a registered dev server target. +type DevTarget struct { + ID string `json:"id"` + Name string `json:"name"` // display name (e.g. "frontend") + Target string `json:"target"` // URL (e.g. "http://localhost:3000") +} + +// DevTargetManager allows tools to register, activate, and deactivate dev proxy targets. +type DevTargetManager interface { + RegisterDevTarget(name, target string) (id string, err error) + UnregisterDevTarget(id string) error + ActivateDevTarget(id string) error + DeactivateDevTarget() error + GetDevTarget() string + ListDevTargets() []DevTarget +} + +// StateNotifier broadcasts state-change signals to SSE subscribers. +type StateNotifier struct { + mu sync.Mutex + subs map[chan struct{}]struct{} + done chan struct{} +} + +// NewStateNotifier creates a new StateNotifier. +func NewStateNotifier() *StateNotifier { + return &StateNotifier{ + subs: make(map[chan struct{}]struct{}), + done: make(chan struct{}), + } +} + +// Subscribe returns a channel that receives a signal on each state change. +func (n *StateNotifier) Subscribe() chan struct{} { + ch := make(chan struct{}, 1) + n.mu.Lock() + n.subs[ch] = struct{}{} + n.mu.Unlock() + return ch +} + +// Unsubscribe removes a subscriber channel. +func (n *StateNotifier) Unsubscribe(ch chan struct{}) { + n.mu.Lock() + delete(n.subs, ch) + n.mu.Unlock() +} + +// Close signals all SSE handlers to exit. +func (n *StateNotifier) Close() { + select { + case <-n.done: + default: + close(n.done) + } +} + +// Done returns a channel that is closed when the notifier is shut down. +func (n *StateNotifier) Done() <-chan struct{} { + return n.done +} + +// Notify sends a signal to all subscribers, coalescing rapid notifications. +func (n *StateNotifier) Notify() { + n.mu.Lock() + defer n.mu.Unlock() + for ch := range n.subs { + select { + case ch <- struct{}{}: + default: + } + } +} diff --git a/pkg/miniapp/ws.go b/pkg/miniapp/ws.go new file mode 100644 index 000000000..d754c6b7c --- /dev/null +++ b/pkg/miniapp/ws.go @@ -0,0 +1,224 @@ +package miniapp + +import ( + "net/http" + "net/url" + "strings" + "time" + + "github.com/gorilla/websocket" + "github.com/sipeed/picoclaw/pkg/logger" +) + + +const maxWSClients = 4 + +const ( + wsPongWait = 60 * time.Second + wsPingPeriod = 54 * time.Second // must be less than wsPongWait +) + +type wsClient struct { + conn *websocket.Conn +} + + +var wsUpgrader = websocket.Upgrader{ + CheckOrigin: func(r *http.Request) bool { + origin := r.Header.Get("Origin") + if origin == "" { + return true // non-browser clients (e.g. curl) + } + // Allow same-origin requests (e.g. Tailscale direct access) + if u, err := url.Parse(origin); err == nil && u.Host == r.Host { + return true + } + // Allow Telegram WebApp origins and localhost for dev + return strings.HasSuffix(origin, ".telegram.org") || + strings.HasSuffix(origin, ".t.me") || + strings.HasPrefix(origin, "http://localhost") || + strings.HasPrefix(origin, "http://127.0.0.1") + }, +} + +// NewHandler creates a new Mini App handler. + + +// wsLogs serves a WebSocket endpoint that streams log entries in real time. +func (h *Handler) wsLogs(w http.ResponseWriter, r *http.Request) { + // Parse filter params + component := r.URL.Query().Get("component") + levelStr := r.URL.Query().Get("level") + minLevel := logger.INFO + if levelStr != "" { + minLevel = logger.ParseLevel(levelStr) + } + + // Clear HTTP server deadlines before WebSocket hijack + rc := http.NewResponseController(w) + _ = rc.SetWriteDeadline(time.Time{}) + _ = rc.SetReadDeadline(time.Time{}) + + conn, err := wsUpgrader.Upgrade(w, r, nil) + if err != nil { + return + } + + client := &wsClient{conn: conn} + + // Enforce max WS clients: evict oldest if full + h.wsClientsMu.Lock() + if len(h.wsClients) >= maxWSClients { + oldest := h.wsClients[0] + h.wsClients = h.wsClients[1:] + oldest.conn.Close() + } + h.wsClients = append(h.wsClients, client) + h.wsClientsMu.Unlock() + + defer func() { + h.wsClientsMu.Lock() + for i, c := range h.wsClients { + if c == client { + h.wsClients = append(h.wsClients[:i], h.wsClients[i+1:]...) + break + } + } + h.wsClientsMu.Unlock() + conn.Close() + }() + + // Build filter function + filter := func(e logger.LogEntry) bool { + if lvl := logger.ParseLevel(e.Level); lvl < minLevel { + return false + } + if component != "" && e.Component != component { + return false + } + return true + } + + sub := logger.Subscribe(filter) + defer logger.Unsubscribe(sub) + + // Configure ping/pong to detect dead connections + conn.SetReadDeadline(time.Now().Add(wsPongWait)) + conn.SetPongHandler(func(string) error { + conn.SetReadDeadline(time.Now().Add(wsPongWait)) + return nil + }) + + // Send initial data + initial := logger.RecentLogs(minLevel, component, 50) + if err := conn.WriteJSON(map[string]any{"type": "init", "entries": initial}); err != nil { + return + } + + // Close detection goroutine + done := make(chan struct{}) + go func() { + defer close(done) + for { + if _, _, err := conn.ReadMessage(); err != nil { + return + } + } + }() + + // Stream loop with periodic pings + ticker := time.NewTicker(wsPingPeriod) + defer ticker.Stop() + + for { + select { + case entry, ok := <-sub.Ch: + if !ok { + return + } + entry.Caller = "" // strip for security + entry.Fields = logger.SanitizeFields(entry.Fields) // mask sensitive values + if err := conn.WriteJSON(map[string]any{"type": "entry", "entry": entry}); err != nil { + return + } + case <-ticker.C: + if err := conn.WriteMessage(websocket.PingMessage, nil); err != nil { + return + } + case <-done: + return + } + } +} + +// apiLogsSnapshot creates a tar.gz snapshot of the current log buffer. + +// wsOrchestration streams live orchestration events (agent spawn/state/gc and +// conductor<->agent conversations) to the canvas UI. +// +// Protocol: +// +// {"type":"init","agents":[...orch.AgentInfo]} -- sent once on connect +// {"type":"event","event":{...orch.Event}} -- pushed on each state change +func (h *Handler) wsOrchestration(w http.ResponseWriter, r *http.Request) { + if h.orchBroadcaster == nil { + http.Error(w, `{"error":"orchestration not enabled"}`, http.StatusServiceUnavailable) + return + } + + rc := http.NewResponseController(w) + _ = rc.SetWriteDeadline(time.Time{}) + _ = rc.SetReadDeadline(time.Time{}) + + conn, err := wsUpgrader.Upgrade(w, r, nil) + if err != nil { + return + } + defer conn.Close() + + sub := h.orchBroadcaster.Subscribe() + defer h.orchBroadcaster.Unsubscribe(sub) + + // Send current agent snapshot so the canvas can populate immediately + snapshot := h.orchBroadcaster.Snapshot() + if err := conn.WriteJSON(map[string]any{"type": "init", "agents": snapshot}); err != nil { + return + } + + conn.SetReadDeadline(time.Now().Add(wsPongWait)) + conn.SetPongHandler(func(string) error { + conn.SetReadDeadline(time.Now().Add(wsPongWait)) + return nil + }) + + done := make(chan struct{}) + go func() { + defer close(done) + for { + if _, _, err := conn.ReadMessage(); err != nil { + return + } + } + }() + + ticker := time.NewTicker(wsPingPeriod) + defer ticker.Stop() + + for { + select { + case ev, ok := <-sub.Ch: + if !ok { + return + } + if err := conn.WriteJSON(map[string]any{"type": "event", "event": ev}); err != nil { + return + } + case <-ticker.C: + if err := conn.WriteMessage(websocket.PingMessage, nil); err != nil { + return + } + case <-done: + return + } + } +} diff --git a/pkg/orch/broadcaster.go b/pkg/orch/broadcaster.go new file mode 100644 index 000000000..70fff1881 --- /dev/null +++ b/pkg/orch/broadcaster.go @@ -0,0 +1,143 @@ +// Package orch provides the orchestration event broadcaster used by the +// subagent system and the Mini App WebSocket UI. +package orch + +import ( + "sync" + "time" +) + +// Event is a single orchestration event pushed over WebSocket to the UI. +// type values: "agent_spawn" | "agent_state" | "conversation" | "agent_gc" +type Event struct { + Type string `json:"type"` + ID string `json:"id,omitempty"` + Label string `json:"label,omitempty"` + Task string `json:"task,omitempty"` + State string `json:"state,omitempty"` // waiting | toolcall | idle + Tool string `json:"tool,omitempty"` // tool name during toolcall + From string `json:"from,omitempty"` + To string `json:"to,omitempty"` + Text string `json:"text,omitempty"` + Reason string `json:"reason,omitempty"` // agent_gc: completed | failed | cancelled + Created int64 `json:"created,omitempty"` +} + +// AgentInfo is the live snapshot of one active agent. +// Kept inside Broadcaster so new WS connections can get current state. +type AgentInfo struct { + ID string `json:"id"` + Label string `json:"label"` + Task string `json:"task"` + State string `json:"state"` + Tool string `json:"tool,omitempty"` + Created int64 `json:"created"` +} + +// Subscriber is a single WebSocket client subscription. +type Subscriber struct { + Ch chan Event +} + +// Broadcaster distributes orchestration events to all connected WS clients. +// It also maintains a live agent snapshot for initial-state delivery on connect. +// +// Publish is non-blocking: events are dropped if a subscriber's buffer is full +// (same pattern as pkg/logger). +type Broadcaster struct { + mu sync.Mutex + subs map[*Subscriber]struct{} + agents map[string]*AgentInfo // live agents, keyed by task ID +} + +func NewBroadcaster() *Broadcaster { + return &Broadcaster{ + subs: make(map[*Subscriber]struct{}), + agents: make(map[string]*AgentInfo), + } +} + +func (b *Broadcaster) Subscribe() *Subscriber { + sub := &Subscriber{Ch: make(chan Event, 32)} + b.mu.Lock() + b.subs[sub] = struct{}{} + b.mu.Unlock() + return sub +} + +func (b *Broadcaster) Unsubscribe(sub *Subscriber) { + b.mu.Lock() + delete(b.subs, sub) + b.mu.Unlock() +} + +// Snapshot returns the current set of active agents. +// Called once on new WS connection to send initial state. +func (b *Broadcaster) Snapshot() []AgentInfo { + b.mu.Lock() + defer b.mu.Unlock() + out := make([]AgentInfo, 0, len(b.agents)) + for _, a := range b.agents { + out = append(out, *a) + } + return out +} + +// ReportSpawn implements AgentReporter. +func (b *Broadcaster) ReportSpawn(id, label, task string) { + b.Publish(Event{Type: "agent_spawn", ID: id, Label: label, Task: task}) +} + +// ReportStateChange implements AgentReporter. +func (b *Broadcaster) ReportStateChange(id, state, tool string) { + b.Publish(Event{Type: "agent_state", ID: id, State: state, Tool: tool}) +} + +// ReportConversation implements AgentReporter. +func (b *Broadcaster) ReportConversation(from, to, text string) { + b.Publish(Event{Type: "conversation", From: from, To: to, Text: text}) +} + +// ReportGC implements AgentReporter. +func (b *Broadcaster) ReportGC(id, reason string) { + b.Publish(Event{Type: "agent_gc", ID: id, Reason: reason}) +} + +// Publish updates internal agent state and fans out to all subscribers. +func (b *Broadcaster) Publish(ev Event) { + if ev.Created == 0 { + ev.Created = time.Now().UnixMilli() + } + + b.mu.Lock() + switch ev.Type { + case "agent_spawn": + b.agents[ev.ID] = &AgentInfo{ + ID: ev.ID, + Label: ev.Label, + Task: ev.Task, + State: "idle", + Created: ev.Created, + } + case "agent_state": + if a, ok := b.agents[ev.ID]; ok { + a.State = ev.State + a.Tool = ev.Tool + } + case "agent_gc": + delete(b.agents, ev.ID) + } + // snapshot subs while holding lock, then release before sending + subs := make([]*Subscriber, 0, len(b.subs)) + for sub := range b.subs { + subs = append(subs, sub) + } + b.mu.Unlock() + + for _, sub := range subs { + select { + case sub.Ch <- ev: + default: // subscriber slow — drop (non-blocking) + } + } +} diff --git a/pkg/orch/broadcaster_test.go b/pkg/orch/broadcaster_test.go new file mode 100644 index 000000000..54ae8af02 --- /dev/null +++ b/pkg/orch/broadcaster_test.go @@ -0,0 +1,140 @@ +package orch + +import ( + "testing" + "time" +) + +func TestBroadcasterSpawnAndGC(t *testing.T) { + b := NewBroadcaster() + sub := b.Subscribe() + defer b.Unsubscribe(sub) + + b.Publish(Event{Type: "agent_spawn", ID: "t1", Label: "scout", Task: "do something"}) + + select { + case ev := <-sub.Ch: + if ev.Type != "agent_spawn" || ev.ID != "t1" { + t.Fatalf("expected agent_spawn for t1, got %+v", ev) + } + case <-time.After(100 * time.Millisecond): + t.Fatal("timeout waiting for agent_spawn event") + } + + snap := b.Snapshot() + if len(snap) != 1 || snap[0].ID != "t1" { + t.Fatalf("expected 1 agent in snapshot, got %v", snap) + } + + b.Publish(Event{Type: "agent_gc", ID: "t1", Reason: "completed"}) + + select { + case ev := <-sub.Ch: + if ev.Type != "agent_gc" || ev.Reason != "completed" { + t.Fatalf("expected agent_gc/completed, got %+v", ev) + } + case <-time.After(100 * time.Millisecond): + t.Fatal("timeout waiting for agent_gc event") + } + + if len(b.Snapshot()) != 0 { + t.Fatal("snapshot should be empty after agent_gc") + } +} + +func TestBroadcasterAgentState(t *testing.T) { + b := NewBroadcaster() + sub := b.Subscribe() + defer b.Unsubscribe(sub) + + b.Publish(Event{Type: "agent_spawn", ID: "t1"}) + <-sub.Ch // consume spawn + + b.Publish(Event{Type: "agent_state", ID: "t1", State: "toolcall", Tool: "bash"}) + + select { + case ev := <-sub.Ch: + if ev.State != "toolcall" || ev.Tool != "bash" { + t.Fatalf("unexpected state event: %+v", ev) + } + case <-time.After(100 * time.Millisecond): + t.Fatal("timeout waiting for agent_state event") + } + + snap := b.Snapshot() + if len(snap) == 0 || snap[0].State != "toolcall" || snap[0].Tool != "bash" { + t.Fatalf("snapshot state not updated: %v", snap) + } +} + +func TestBroadcasterNonBlocking(t *testing.T) { + b := NewBroadcaster() + sub := b.Subscribe() // do NOT read from sub.Ch + defer b.Unsubscribe(sub) + + // Fill buffer beyond capacity (cap=32) — must not block or deadlock + done := make(chan struct{}) + go func() { + for i := 0; i < 50; i++ { + b.Publish(Event{Type: "agent_state", ID: "t1", State: "waiting"}) + } + close(done) + }() + + select { + case <-done: + case <-time.After(500 * time.Millisecond): + t.Fatal("Publish blocked on slow subscriber") + } +} + +func TestBroadcasterMultipleSubscribers(t *testing.T) { + b := NewBroadcaster() + s1 := b.Subscribe() + s2 := b.Subscribe() + defer b.Unsubscribe(s1) + defer b.Unsubscribe(s2) + + b.Publish(Event{Type: "agent_spawn", ID: "t1", Label: "worker"}) + + for _, sub := range []*Subscriber{s1, s2} { + select { + case ev := <-sub.Ch: + if ev.Type != "agent_spawn" { + t.Fatalf("expected agent_spawn, got %s", ev.Type) + } + case <-time.After(100 * time.Millisecond): + t.Fatal("timeout: not all subscribers received event") + } + } +} + +func TestBroadcasterUnsubscribe(t *testing.T) { + b := NewBroadcaster() + sub := b.Subscribe() + b.Unsubscribe(sub) + + b.Publish(Event{Type: "agent_spawn", ID: "t1"}) + + select { + case ev := <-sub.Ch: + t.Fatalf("received event after unsubscribe: %+v", ev) + case <-time.After(50 * time.Millisecond): + // correct: nothing delivered after unsubscribe + } +} + +func TestBroadcasterTimestampAutoSet(t *testing.T) { + b := NewBroadcaster() + sub := b.Subscribe() + defer b.Unsubscribe(sub) + + before := time.Now().UnixMilli() + b.Publish(Event{Type: "agent_spawn", ID: "t1"}) // Created == 0 + after := time.Now().UnixMilli() + + ev := <-sub.Ch + if ev.Created < before || ev.Created > after { + t.Fatalf("Created timestamp %d not in [%d, %d]", ev.Created, before, after) + } +} diff --git a/pkg/orch/reporter.go b/pkg/orch/reporter.go new file mode 100644 index 000000000..b8229314d --- /dev/null +++ b/pkg/orch/reporter.go @@ -0,0 +1,21 @@ +package orch + +// AgentReporter is the interface for reporting agent lifecycle events. +// Both Broadcaster (real events) and noopReporter (disabled) implement this. +type AgentReporter interface { + ReportSpawn(id, label, task string) + ReportStateChange(id, state, tool string) + ReportConversation(from, to, text string) + ReportGC(id, reason string) +} + +type noopReporter struct{} + +func (n *noopReporter) ReportSpawn(id, label, task string) {} +func (n *noopReporter) ReportStateChange(id, state, tool string) {} +func (n *noopReporter) ReportConversation(from, to, text string) {} +func (n *noopReporter) ReportGC(id, reason string) {} + +// Noop is the AgentReporter to use when orchestration is disabled. +// Allows nil-free code in callers. +var Noop AgentReporter = &noopReporter{} diff --git a/pkg/orch/reporter_test.go b/pkg/orch/reporter_test.go new file mode 100644 index 000000000..120f03c26 --- /dev/null +++ b/pkg/orch/reporter_test.go @@ -0,0 +1,102 @@ +package orch + +import "testing" + +// Compile-time: Broadcaster must satisfy AgentReporter. +var _ AgentReporter = (*Broadcaster)(nil) + +// TestNoop_AllMethods_NoPanic verifies that orch.Noop can be called for all +// four methods without panic. This is the nil-safe baseline for disabled +// orchestration mode. +func TestNoop_AllMethods_NoPanic(t *testing.T) { + Noop.ReportSpawn("id", "label", "task") + Noop.ReportStateChange("id", "waiting", "") + Noop.ReportStateChange("id", "toolcall", "bash") + Noop.ReportConversation("conductor", "sub-1", "do something") + Noop.ReportGC("id", "completed") +} + +// TestBroadcaster_ReportSpawn_MapsToAgentSpawnEvent verifies that ReportSpawn +// publishes an Event with Type="agent_spawn" and the correct ID/Label/Task +// fields, and that the agent appears in the Snapshot immediately. +func TestBroadcaster_ReportSpawn_MapsToAgentSpawnEvent(t *testing.T) { + b := NewBroadcaster() + sub := b.Subscribe() + defer b.Unsubscribe(sub) + + b.ReportSpawn("agent-1", "scout", "find all TODOs") + + ev := <-sub.Ch + if ev.Type != "agent_spawn" { + t.Fatalf("want agent_spawn, got %q", ev.Type) + } + if ev.ID != "agent-1" || ev.Label != "scout" || ev.Task != "find all TODOs" { + t.Fatalf("field mismatch: %+v", ev) + } + snap := b.Snapshot() + if len(snap) != 1 || snap[0].ID != "agent-1" || snap[0].Label != "scout" { + t.Fatalf("snapshot not updated correctly: %v", snap) + } +} + +// TestBroadcaster_ReportStateChange_MapsToAgentStateEvent verifies that +// ReportStateChange publishes agent_state and updates the live snapshot. +func TestBroadcaster_ReportStateChange_MapsToAgentStateEvent(t *testing.T) { + b := NewBroadcaster() + sub := b.Subscribe() + defer b.Unsubscribe(sub) + + b.ReportSpawn("agent-1", "coder", "implement it") + <-sub.Ch // consume spawn + + b.ReportStateChange("agent-1", "toolcall", "bash") + ev := <-sub.Ch + if ev.Type != "agent_state" || ev.State != "toolcall" || ev.Tool != "bash" { + t.Fatalf("unexpected event: %+v", ev) + } + snap := b.Snapshot() + if snap[0].State != "toolcall" || snap[0].Tool != "bash" { + t.Fatalf("snapshot state not updated: %v", snap) + } +} + +// TestBroadcaster_ReportConversation_MapsToConversationEvent verifies that +// ReportConversation publishes a conversation event with correct From/To/Text +// fields and does NOT modify the agent snapshot (conversation is not a state +// change of any agent). +func TestBroadcaster_ReportConversation_MapsToConversationEvent(t *testing.T) { + b := NewBroadcaster() + sub := b.Subscribe() + defer b.Unsubscribe(sub) + + b.ReportConversation("conductor", "sub-1", "please do the task") + + ev := <-sub.Ch + if ev.Type != "conversation" || ev.From != "conductor" || ev.To != "sub-1" || ev.Text != "please do the task" { + t.Fatalf("unexpected event: %+v", ev) + } + if len(b.Snapshot()) != 0 { + t.Fatal("conversation event must not modify agent snapshot") + } +} + +// TestBroadcaster_ReportGC_RemovesAgentFromSnapshot verifies that ReportGC +// publishes agent_gc with the correct Reason and removes the agent from the +// live snapshot so new WS connections no longer see it. +func TestBroadcaster_ReportGC_RemovesAgentFromSnapshot(t *testing.T) { + b := NewBroadcaster() + sub := b.Subscribe() + defer b.Unsubscribe(sub) + + b.ReportSpawn("agent-1", "scout", "task") + <-sub.Ch // consume spawn + + b.ReportGC("agent-1", "completed") + ev := <-sub.Ch + if ev.Type != "agent_gc" || ev.ID != "agent-1" || ev.Reason != "completed" { + t.Fatalf("unexpected event: %+v", ev) + } + if len(b.Snapshot()) != 0 { + t.Fatal("agent must be removed from snapshot after ReportGC") + } +} diff --git a/pkg/tools/subagent.go b/pkg/tools/subagent.go index 91ebff636..3956cf0fd 100644 --- a/pkg/tools/subagent.go +++ b/pkg/tools/subagent.go @@ -7,6 +7,7 @@ import ( "time" "github.com/sipeed/picoclaw/pkg/bus" + "github.com/sipeed/picoclaw/pkg/orch" "github.com/sipeed/picoclaw/pkg/providers" ) @@ -36,13 +37,18 @@ type SubagentManager struct { hasMaxTokens bool hasTemperature bool nextID int + reporter orch.AgentReporter } func NewSubagentManager( provider providers.LLMProvider, defaultModel, workspace string, bus *bus.MessageBus, + reporter orch.AgentReporter, ) *SubagentManager { + if reporter == nil { + reporter = orch.Noop + } return &SubagentManager{ tasks: make(map[string]*SubagentTask), provider: provider, @@ -52,6 +58,7 @@ func NewSubagentManager( tools: NewToolRegistry(), maxIterations: 10, nextID: 1, + reporter: reporter, } } @@ -103,6 +110,8 @@ func (sm *SubagentManager) Spawn( } sm.tasks[taskID] = subagentTask + sm.reporter.ReportSpawn(taskID, label, task) + // Start task in background with context cancellation support go sm.runTask(ctx, subagentTask, callback) @@ -114,7 +123,6 @@ func (sm *SubagentManager) Spawn( func (sm *SubagentManager) runTask(ctx context.Context, task *SubagentTask, callback AsyncCallback) { task.Status = "running" - task.Created = time.Now().UnixMilli() // Build system prompt for subagent systemPrompt := `You are a subagent. Complete the given task independently and report the result. @@ -164,12 +172,17 @@ After completing the task, provide a clear summary of what was done.` } } + // Notify conductor that the subagent is starting + sm.reporter.ReportConversation("conductor", task.ID, task.Task) + loopResult, err := RunToolLoop(ctx, ToolLoopConfig{ Provider: sm.provider, Model: sm.defaultModel, Tools: tools, MaxIterations: maxIter, LLMOptions: llmOptions, + Reporter: sm.reporter, + AgentID: task.ID, }, messages, task.OriginChannel, task.OriginChatID) sm.mu.Lock() @@ -186,10 +199,13 @@ After completing the task, provide a clear summary of what was done.` task.Status = "failed" task.Result = fmt.Sprintf("Error: %v", err) // Check if it was cancelled + gcReason := "failed" if ctx.Err() != nil { task.Status = "cancelled" task.Result = "Task cancelled during execution" + gcReason = "cancelled" } + sm.reporter.ReportGC(task.ID, gcReason) result = &ToolResult{ ForLLM: task.Result, ForUser: "", @@ -201,6 +217,9 @@ After completing the task, provide a clear summary of what was done.` } else { task.Status = "completed" task.Result = loopResult.Content + // Notify conductor of the result + sm.reporter.ReportConversation(task.ID, "conductor", loopResult.Content) + sm.reporter.ReportGC(task.ID, "completed") result = &ToolResult{ ForLLM: fmt.Sprintf( "Subagent '%s' completed (iterations: %d): %s", diff --git a/pkg/tools/subagent_reporter_test.go b/pkg/tools/subagent_reporter_test.go new file mode 100644 index 000000000..aaf3bd7a4 --- /dev/null +++ b/pkg/tools/subagent_reporter_test.go @@ -0,0 +1,243 @@ +package tools + +import ( + "context" + "sync/atomic" + "testing" + "time" + + "github.com/sipeed/picoclaw/pkg/orch" + "github.com/sipeed/picoclaw/pkg/providers" +) + +// blockingProvider blocks inside Chat until the context is cancelled. +// The ready channel is closed the moment Chat is entered, so callers can +// synchronise before cancelling the context. +type blockingProvider struct { + ready chan struct{} +} + +func newBlockingProvider() *blockingProvider { + return &blockingProvider{ready: make(chan struct{})} +} + +func (p *blockingProvider) Chat(ctx context.Context, _ []providers.Message, _ []providers.ToolDefinition, _ string, _ map[string]any) (*providers.LLMResponse, error) { + close(p.ready) // signal: we are now blocking + <-ctx.Done() + return nil, ctx.Err() +} + +func (p *blockingProvider) GetDefaultModel() string { return "test" } + +// TestSubagentManager_Spawn_EmitsLifecycleEvents verifies that Spawn() fires +// the correct sequence of orchestration events through a real Broadcaster: +// +// agent_spawn → conversation(conductor→sub) → agent_state(waiting) → +// conversation(sub→conductor) → agent_gc(completed) +// +// It also verifies that the snapshot is empty after ReportGC and that the +// completion callback is invoked. +func TestSubagentManager_Spawn_EmitsLifecycleEvents(t *testing.T) { + b := orch.NewBroadcaster() + sub := b.Subscribe() + defer b.Unsubscribe(sub) + + provider := &MockLLMProvider{} + mgr := NewSubagentManager(provider, "test-model", "/tmp/test", nil, b) + + var callbackCalled int32 + cb := AsyncCallback(func(_ context.Context, _ *ToolResult) { + atomic.StoreInt32(&callbackCalled, 1) + }) + + _, err := mgr.Spawn( + context.Background(), + "say hello", "hello-task", "", "cli", "direct", + cb, + ) + if err != nil { + t.Fatalf("Spawn() error: %v", err) + } + + // Collect events until agent_gc or timeout. + var events []orch.Event + deadline := time.After(3 * time.Second) +loop: + for { + select { + case ev := <-sub.Ch: + events = append(events, ev) + if ev.Type == "agent_gc" { + break loop + } + case <-deadline: + t.Fatalf("timed out waiting for agent_gc; events so far: %+v", events) + } + } + + // 1. First event must be agent_spawn with the correct label. + if len(events) == 0 || events[0].Type != "agent_spawn" { + t.Fatalf("first event must be agent_spawn, got: %+v", events) + } + if events[0].Label != "hello-task" { + t.Errorf("agent_spawn label = %q, want %q", events[0].Label, "hello-task") + } + spawnedID := events[0].ID + + // 2. There must be a conversation from conductor → subagent. + var hasConvToSub bool + for _, ev := range events { + if ev.Type == "conversation" && ev.From == "conductor" && ev.To == spawnedID { + hasConvToSub = true + break + } + } + if !hasConvToSub { + t.Errorf("missing conversation(conductor → %s); events: %+v", spawnedID, events) + } + + // 3. There must be at least one agent_state(waiting) for the subagent. + var hasWaiting bool + for _, ev := range events { + if ev.Type == "agent_state" && ev.ID == spawnedID && ev.State == "waiting" { + hasWaiting = true + break + } + } + if !hasWaiting { + t.Errorf("missing agent_state(waiting) for %s; events: %+v", spawnedID, events) + } + + // 4. Last event must be agent_gc with reason "completed". + last := events[len(events)-1] + if last.Type != "agent_gc" || last.ID != spawnedID || last.Reason != "completed" { + t.Errorf("last event must be agent_gc(completed), got: %+v", last) + } + + // 5. Snapshot must be empty after GC (agent removed from live map). + if snap := b.Snapshot(); len(snap) != 0 { + t.Errorf("snapshot must be empty after agent_gc, got: %v", snap) + } + + // 6. Callback must be called. The callback fires in the same goroutine + // as ReportGC (after the deferred unlock), so we poll briefly. + for i := 0; i < 100; i++ { + if atomic.LoadInt32(&callbackCalled) == 1 { + break + } + time.Sleep(10 * time.Millisecond) + } + if atomic.LoadInt32(&callbackCalled) != 1 { + t.Error("completion callback was not called after agent_gc") + } +} + +// TestSubagentManager_Spawn_SnapshotLiveDuringExecution verifies that the +// Broadcaster snapshot contains the agent between agent_spawn and agent_gc. +// Because Publish() updates the agent map before dispatching to subscribers, +// the snapshot is guaranteed to be non-empty as soon as agent_spawn is +// received on the channel. +func TestSubagentManager_Spawn_SnapshotLiveDuringExecution(t *testing.T) { + b := orch.NewBroadcaster() + sub := b.Subscribe() + defer b.Unsubscribe(sub) + + provider := &MockLLMProvider{} + mgr := NewSubagentManager(provider, "test-model", "/tmp/test", nil, b) + + _, err := mgr.Spawn( + context.Background(), + "any task", "live-test", "", "cli", "direct", + nil, + ) + if err != nil { + t.Fatalf("Spawn() error: %v", err) + } + + // Wait for agent_spawn, then immediately check snapshot. + deadline := time.After(2 * time.Second) + for { + select { + case ev := <-sub.Ch: + if ev.Type == "agent_spawn" { + snap := b.Snapshot() + if len(snap) == 0 { + t.Error("snapshot must contain the spawned agent after agent_spawn event") + } + return // test complete; background goroutine drains safely + } + case <-deadline: + t.Fatal("timed out waiting for agent_spawn event") + } + } +} + +// TestSubagentManager_Spawn_CancelledDuringExecution verifies that when the +// context is cancelled while a subagent's LLM call is in progress, the +// Broadcaster receives agent_gc with reason="cancelled" and the agent is +// removed from the snapshot. +// +// Synchronisation: +// 1. blockingProvider.ready is closed when Chat() is entered (goroutine is +// now blocked inside the LLM call). +// 2. Only then is the context cancelled, so there is no race between spawn +// and cancellation. +func TestSubagentManager_Spawn_CancelledDuringExecution(t *testing.T) { + b := orch.NewBroadcaster() + sub := b.Subscribe() + defer b.Unsubscribe(sub) + + bp := newBlockingProvider() + mgr := NewSubagentManager(bp, "test-model", "/tmp/test", nil, b) + + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + + _, err := mgr.Spawn(ctx, "long task", "cancel-me", "", "cli", "direct", nil) + if err != nil { + t.Fatalf("Spawn() error: %v", err) + } + + // Wait until the subagent goroutine is inside Chat (blocking on ctx). + select { + case <-bp.ready: + case <-time.After(3 * time.Second): + t.Fatal("timed out waiting for blockingProvider to enter Chat") + } + + // Now cancel — the LLM call unblocks with ctx.Err(). + cancel() + + // Collect events until agent_gc. + var events []orch.Event + deadline := time.After(3 * time.Second) +loop: + for { + select { + case ev := <-sub.Ch: + events = append(events, ev) + if ev.Type == "agent_gc" { + break loop + } + case <-deadline: + t.Fatalf("timed out waiting for agent_gc; events so far: %+v", events) + } + } + + // Locate agent_gc and verify reason = "cancelled". + var gcEv orch.Event + for _, ev := range events { + if ev.Type == "agent_gc" { + gcEv = ev + break + } + } + if gcEv.Reason != "cancelled" { + t.Errorf("agent_gc reason = %q, want %q; events: %+v", gcEv.Reason, "cancelled", events) + } + + // Snapshot must be empty after the GC event. + if snap := b.Snapshot(); len(snap) != 0 { + t.Errorf("snapshot must be empty after agent_gc(cancelled), got: %v", snap) + } +} diff --git a/pkg/tools/subagent_tool_test.go b/pkg/tools/subagent_tool_test.go index 59bfdffae..30838d843 100644 --- a/pkg/tools/subagent_tool_test.go +++ b/pkg/tools/subagent_tool_test.go @@ -6,6 +6,7 @@ import ( "testing" "github.com/sipeed/picoclaw/pkg/bus" + "github.com/sipeed/picoclaw/pkg/orch" "github.com/sipeed/picoclaw/pkg/providers" ) @@ -47,7 +48,7 @@ func (m *MockLLMProvider) GetContextWindow() int { func TestSubagentManager_SetLLMOptions_AppliesToRunToolLoop(t *testing.T) { provider := &MockLLMProvider{} - manager := NewSubagentManager(provider, "test-model", "/tmp/test", nil) + manager := NewSubagentManager(provider, "test-model", "/tmp/test", nil, orch.Noop) manager.SetLLMOptions(2048, 0.6) tool := NewSubagentTool(manager) tool.SetContext("cli", "direct") @@ -74,7 +75,7 @@ func TestSubagentManager_SetLLMOptions_AppliesToRunToolLoop(t *testing.T) { // TestSubagentTool_Name verifies tool name func TestSubagentTool_Name(t *testing.T) { provider := &MockLLMProvider{} - manager := NewSubagentManager(provider, "test-model", "/tmp/test", nil) + manager := NewSubagentManager(provider, "test-model", "/tmp/test", nil, orch.Noop) tool := NewSubagentTool(manager) if tool.Name() != "subagent" { @@ -85,7 +86,7 @@ func TestSubagentTool_Name(t *testing.T) { // TestSubagentTool_Description verifies tool description func TestSubagentTool_Description(t *testing.T) { provider := &MockLLMProvider{} - manager := NewSubagentManager(provider, "test-model", "/tmp/test", nil) + manager := NewSubagentManager(provider, "test-model", "/tmp/test", nil, orch.Noop) tool := NewSubagentTool(manager) desc := tool.Description() @@ -100,7 +101,7 @@ func TestSubagentTool_Description(t *testing.T) { // TestSubagentTool_Parameters verifies tool parameters schema func TestSubagentTool_Parameters(t *testing.T) { provider := &MockLLMProvider{} - manager := NewSubagentManager(provider, "test-model", "/tmp/test", nil) + manager := NewSubagentManager(provider, "test-model", "/tmp/test", nil, orch.Noop) tool := NewSubagentTool(manager) params := tool.Parameters() @@ -150,7 +151,7 @@ func TestSubagentTool_Parameters(t *testing.T) { // TestSubagentTool_SetContext verifies context setting func TestSubagentTool_SetContext(t *testing.T) { provider := &MockLLMProvider{} - manager := NewSubagentManager(provider, "test-model", "/tmp/test", nil) + manager := NewSubagentManager(provider, "test-model", "/tmp/test", nil, orch.Noop) tool := NewSubagentTool(manager) tool.SetContext("test-channel", "test-chat") @@ -164,7 +165,7 @@ func TestSubagentTool_SetContext(t *testing.T) { func TestSubagentTool_Execute_Success(t *testing.T) { provider := &MockLLMProvider{} msgBus := bus.NewMessageBus() - manager := NewSubagentManager(provider, "test-model", "/tmp/test", msgBus) + manager := NewSubagentManager(provider, "test-model", "/tmp/test", msgBus, orch.Noop) tool := NewSubagentTool(manager) tool.SetContext("telegram", "chat-123") @@ -220,7 +221,7 @@ func TestSubagentTool_Execute_Success(t *testing.T) { func TestSubagentTool_Execute_NoLabel(t *testing.T) { provider := &MockLLMProvider{} msgBus := bus.NewMessageBus() - manager := NewSubagentManager(provider, "test-model", "/tmp/test", msgBus) + manager := NewSubagentManager(provider, "test-model", "/tmp/test", msgBus, orch.Noop) tool := NewSubagentTool(manager) ctx := context.Background() @@ -243,7 +244,7 @@ func TestSubagentTool_Execute_NoLabel(t *testing.T) { // TestSubagentTool_Execute_MissingTask tests error handling for missing task func TestSubagentTool_Execute_MissingTask(t *testing.T) { provider := &MockLLMProvider{} - manager := NewSubagentManager(provider, "test-model", "/tmp/test", nil) + manager := NewSubagentManager(provider, "test-model", "/tmp/test", nil, orch.Noop) tool := NewSubagentTool(manager) ctx := context.Background() @@ -294,7 +295,7 @@ func TestSubagentTool_Execute_NilManager(t *testing.T) { func TestSubagentTool_Execute_ContextPassing(t *testing.T) { provider := &MockLLMProvider{} msgBus := bus.NewMessageBus() - manager := NewSubagentManager(provider, "test-model", "/tmp/test", msgBus) + manager := NewSubagentManager(provider, "test-model", "/tmp/test", msgBus, orch.Noop) tool := NewSubagentTool(manager) // Set context @@ -323,7 +324,7 @@ func TestSubagentTool_ForUserTruncation(t *testing.T) { // Create a mock provider that returns very long content provider := &MockLLMProvider{} msgBus := bus.NewMessageBus() - manager := NewSubagentManager(provider, "test-model", "/tmp/test", msgBus) + manager := NewSubagentManager(provider, "test-model", "/tmp/test", msgBus, orch.Noop) tool := NewSubagentTool(manager) ctx := context.Background() diff --git a/pkg/tools/toolloop.go b/pkg/tools/toolloop.go index cdfe0d6ce..d8793caef 100644 --- a/pkg/tools/toolloop.go +++ b/pkg/tools/toolloop.go @@ -12,6 +12,7 @@ import ( "fmt" "github.com/sipeed/picoclaw/pkg/logger" + "github.com/sipeed/picoclaw/pkg/orch" "github.com/sipeed/picoclaw/pkg/providers" "github.com/sipeed/picoclaw/pkg/utils" ) @@ -23,6 +24,12 @@ type ToolLoopConfig struct { Tools *ToolRegistry MaxIterations int LLMOptions map[string]any + // Reporter and AgentID replace the old OnStateChange func. + // Reporter is called with ReportStateChange("waiting","") before each LLM + // call and ReportStateChange("toolcall", toolName) when each tool starts. + // Pass nil or orch.Noop to disable. nil is treated as orch.Noop internally. + Reporter orch.AgentReporter + AgentID string } // ToolLoopResult contains the result of running the tool loop. @@ -39,6 +46,11 @@ func RunToolLoop( messages []providers.Message, channel, chatID string, ) (*ToolLoopResult, error) { + reporter := config.Reporter + if reporter == nil { + reporter = orch.Noop + } + iteration := 0 var finalContent string @@ -62,7 +74,8 @@ func RunToolLoop( if llmOpts == nil { llmOpts = map[string]any{} } - // 3. Call LLM + // 3. Call LLM (hook: waiting for response) + reporter.ReportStateChange(config.AgentID, "waiting", "") response, err := config.Provider.Chat(ctx, messages, providerToolDefs, config.Model, llmOpts) if err != nil { logger.ErrorCF("toolloop", "LLM call failed", @@ -121,7 +134,7 @@ func RunToolLoop( } messages = append(messages, assistantMsg) - // 7. Execute tool calls + // 7. Execute tool calls (hook: toolcall per tool) for _, tc := range normalizedToolCalls { argsJSON, _ := json.Marshal(tc.Arguments) argsPreview := utils.Truncate(string(argsJSON), 200) @@ -130,6 +143,7 @@ func RunToolLoop( "tool": tc.Name, "iteration": iteration, }) + reporter.ReportStateChange(config.AgentID, "toolcall", tc.Name) // Execute tool (no async callback for subagents - they run independently) var toolResult *ToolResult diff --git a/pkg/tools/toolloop_reporter_test.go b/pkg/tools/toolloop_reporter_test.go new file mode 100644 index 000000000..38284b722 --- /dev/null +++ b/pkg/tools/toolloop_reporter_test.go @@ -0,0 +1,170 @@ +package tools + +import ( + "context" + "sync" + "testing" + + "github.com/sipeed/picoclaw/pkg/orch" + "github.com/sipeed/picoclaw/pkg/providers" +) + +// reporterSpy records every ReportStateChange call in order. +// Spawn/Conversation/GC are not needed for toolloop tests. +type reporterSpy struct { + mu sync.Mutex + calls []spyCall +} + +type spyCall struct { + state string + tool string +} + +func (r *reporterSpy) ReportSpawn(id, label, task string) {} +func (r *reporterSpy) ReportConversation(from, to, text string) {} +func (r *reporterSpy) ReportGC(id, reason string) {} +func (r *reporterSpy) ReportStateChange(id, state, tool string) { + r.mu.Lock() + r.calls = append(r.calls, spyCall{state, tool}) + r.mu.Unlock() +} +func (r *reporterSpy) snapshot() []spyCall { + r.mu.Lock() + defer r.mu.Unlock() + out := make([]spyCall, len(r.calls)) + copy(out, r.calls) + return out +} + +// sequenceMockProvider returns a tool call on the first Chat() call and a +// plain text response on all subsequent calls. Used to exercise the +// waiting → toolcall → waiting event sequence in RunToolLoop. +type sequenceMockProvider struct { + mu sync.Mutex + callCount int +} + +func (m *sequenceMockProvider) Chat( + _ context.Context, + _ []providers.Message, + _ []providers.ToolDefinition, + _ string, + _ map[string]any, +) (*providers.LLMResponse, error) { + m.mu.Lock() + m.callCount++ + n := m.callCount + m.mu.Unlock() + if n == 1 { + return &providers.LLMResponse{ + ToolCalls: []providers.ToolCall{ + {ID: "tc-1", Name: "echo_tool", Arguments: map[string]any{"msg": "hi"}}, + }, + }, nil + } + return &providers.LLMResponse{Content: "done"}, nil +} +func (m *sequenceMockProvider) GetDefaultModel() string { return "test" } +func (m *sequenceMockProvider) SupportsTools() bool { return true } +func (m *sequenceMockProvider) GetContextWindow() int { return 4096 } + +// echoTool is a minimal Tool stub registered as "echo_tool". +type echoTool struct{} + +func (t *echoTool) Name() string { return "echo_tool" } +func (t *echoTool) Description() string { return "echo" } +func (t *echoTool) Parameters() map[string]any { + return map[string]any{"type": "object", "properties": map[string]any{}} +} +func (t *echoTool) Execute(_ context.Context, _ map[string]any) *ToolResult { + return &ToolResult{ForLLM: "echoed"} +} + +// TestToolLoop_NilReporter_FallsBackToNoop ensures that passing nil as +// Reporter does not panic — the loop must substitute orch.Noop internally. +func TestToolLoop_NilReporter_FallsBackToNoop(t *testing.T) { + _, err := RunToolLoop(context.Background(), ToolLoopConfig{ + Provider: &MockLLMProvider{}, + Model: "test", + MaxIterations: 1, + Reporter: nil, // must not panic + }, []providers.Message{{Role: "user", Content: "hi"}}, "cli", "direct") + if err != nil { + t.Fatalf("unexpected error with nil reporter: %v", err) + } +} + +// TestToolLoop_Reporter_WaitingBeforeLLM verifies that ReportStateChange is +// called with state="waiting" before the first LLM call. The mock provider +// returns a direct text answer (no tool calls), so exactly one waiting event +// is expected. +func TestToolLoop_Reporter_WaitingBeforeLLM(t *testing.T) { + rep := &reporterSpy{} + _, err := RunToolLoop(context.Background(), ToolLoopConfig{ + Provider: &MockLLMProvider{}, + Model: "test", + MaxIterations: 1, + Reporter: rep, + AgentID: "sess-1", + }, []providers.Message{{Role: "user", Content: "hi"}}, "cli", "direct") + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + calls := rep.snapshot() + if len(calls) == 0 { + t.Fatal("expected at least one ReportStateChange call") + } + if calls[0].state != "waiting" { + t.Fatalf("first call must be state=waiting, got %+v", calls[0]) + } +} + +// TestToolLoop_Reporter_ToolcallOrderedAfterWaiting verifies the canonical +// two-iteration sequence: +// +// waiting (before 1st LLM call) +// toolcall(echo_tool) (before tool execution) +// waiting (before 2nd LLM call) +// +// The sequenceMockProvider returns a tool call on iteration 1 and a text +// response on iteration 2, driving exactly this path. +func TestToolLoop_Reporter_ToolcallOrderedAfterWaiting(t *testing.T) { + rep := &reporterSpy{} + reg := NewToolRegistry() + reg.Register(&echoTool{}) + + _, err := RunToolLoop(context.Background(), ToolLoopConfig{ + Provider: &sequenceMockProvider{}, + Model: "test", + Tools: reg, + MaxIterations: 5, + Reporter: rep, + AgentID: "sess-1", + }, []providers.Message{{Role: "user", Content: "do it"}}, "cli", "direct") + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + + calls := rep.snapshot() + if len(calls) < 3 { + t.Fatalf("expected at least 3 calls, got %d: %+v", len(calls), calls) + } + if calls[0].state != "waiting" { + t.Fatalf("calls[0] must be waiting, got %+v", calls[0]) + } + if calls[1].state != "toolcall" || calls[1].tool != "echo_tool" { + t.Fatalf("calls[1] must be toolcall(echo_tool), got %+v", calls[1]) + } + if calls[2].state != "waiting" { + t.Fatalf("calls[2] must be waiting (2nd LLM iteration), got %+v", calls[2]) + } +} + +// TestToolLoop_Reporter_NoopImplementsInterface is a compile-time check that +// orch.Noop satisfies the orch.AgentReporter interface accepted by +// ToolLoopConfig.Reporter. If Noop ever stops implementing the interface the +// build will fail here before any test runs. +func TestToolLoop_Reporter_NoopImplementsInterface(t *testing.T) { + var _ orch.AgentReporter = orch.Noop +}