diff --git a/README.md b/README.md index 6714ac6eb..88d2355ad 100644 --- a/README.md +++ b/README.md @@ -82,6 +82,87 @@ PicoClaw +## 🧠 Our Thinking + +PicoClaw isn't just small — it **thinks differently**. While most AI agent frameworks treat the LLM as a black box that swallows everything (all tools, all history, all context), PicoClaw introduces a structured **Runtime Loop** that makes every token count. + +### Three-Phase Runtime Loop + +Every user interaction flows through three distinct phases: + +``` +┌──────────────┐ ┌──────────────┐ ┌──────────────┐ +│ Phase 1 │ → │ Phase 2 │ → │ Phase 3 │ +│ Analyse │ │ ExecuteLLM │ │ Reflect │ +│ (Orchestrate) │ (Execute) │ │ (Learn) │ +└──────────────┘ └──────────────┘ └──────────────┘ +``` + +- **Phase 1 — Analyse**: A lightweight LLM call (can use a cheaper/faster model) that understands intent, assigns semantic tags, and prepares the optimal context. No tools, no history bloat — just pure comprehension. +- **Phase 2 — ExecuteLLM**: The main LLM iterates with only the tools and context that Phase 1 deemed relevant. Fewer tools = less confusion = better decisions. +- **Phase 3 — Reflect**: Synchronous scoring + context update (< 2ms) before the user sees the reply, followed by async persistence. The agent learns from every interaction without adding latency. + +### Turn Scoring & Instant Memory + +Every Turn gets a **score** based on tool activity, intent weight, content density, and explicit user markers. This score drives a novel context selection algorithm: + +``` +Instant Memory = { high-score Turns } // always kept, unconditionally + ∪ { tag-matched Turns, score > 0 } // relevant old Turns "resurrected" + ∪ { recent M Turns } // continuity baseline + → sorted by time, truncated to capacity +``` + +**No Turn is ever deleted.** Low-score Turns are simply excluded from context — but when a future Phase 1 produces matching tags, they can be recalled. It's like human memory: you don't forget things, you just can't always access them until something triggers the recall. + +### Tag-Gated Tool Loading + +Instead of feeding 20+ tool definitions to the LLM every request (wasting ~3,000 tokens per iteration), PicoClaw categorizes tools: + +| Layer | Tools | Loading | +|-------|-------|---------| +| **always-on** | `read_file`, `write_file`, `shell`, etc. | Always present | +| **tag-gated** | MCP servers, Skills, `web_search`, `cron` | Only when Phase 1 tags match | + +As MCP servers and Skills grow, the savings compound exponentially — and the LLM makes better decisions with a focused toolset. + +### KV Cache-Friendly Message Ordering + +We obsess over **prefix stability** to maximize KV cache hits across providers (Gemini implicit cache, Anthropic `cache_control`, OpenAI prompt caching): + +``` +[system_prompt] ← always cached ✅ +[long_term_memory by tags] ← stable when same tags ✅ +[high-score Turns, ASC by ID] ← fixed positions, append-only ✅ +[tag-matched Turns] ← may vary +[recent Turns] ← rolling window +[current user message] ← always new +``` + +High-score Turns are **pinned** right after long-term memory in ascending order. New Turns only append — they never shift existing content. This maximizes the cache-hit prefix length, and the cost difference is dramatic at scale. + +### Three-Layer Memory Hierarchy + +| Layer | Lifecycle | Source | Consumer | +|-------|-----------|--------|----------| +| **Instant Memory** | Assembled per-Turn | Turn Store (score + tag filter) | Phase 2 (ExecuteLLM) | +| **Active Context** | Rolling update per `channel:chatID` | Phase 3 sync update | Phase 1 (Analyse) | +| **Long-Term Memory** | Persistent | MemoryDigest batch worker | Phase 1 (via tag retrieval) | + +Each layer serves a different phase. No overlap, no waste. Active Context (current files, recent errors) helps Phase 1 understand terse messages like "fix it". Instant Memory gives Phase 2 the right historical context. Long-term memory accumulates wisdom across sessions. + +### Why This Matters + +Traditional agent loops dump everything into one LLM call and hope for the best. PicoClaw's approach means: + +- 📉 **~60% fewer wasted tokens** from irrelevant tools and stale context +- 🎯 **Higher decision quality** — focused toolsets reduce LLM confusion +- ⚡ **Sub-2ms overhead** for scoring and context updates (zero perceived latency) +- 🧲 **Associative recall** — old conversations resurface when relevant, like human memory +- 💰 **Multi-model cost optimization** — use cheap models for analysis, strong models for execution + +> *"The best token is the one you never send."* + ## 🦾 Demonstration ### 🛠️ Standard Assistant Workflows diff --git a/README.zh.md b/README.zh.md index d3a49ee8d..824e7f521 100644 --- a/README.zh.md +++ b/README.zh.md @@ -80,6 +80,87 @@ PicoClaw +## 🧠 我们的思考 + +PicoClaw 不只是小——它的**思维方式**和其他 AI Agent 框架根本不同。大多数框架把 LLM 当黑箱,把所有工具、所有历史、所有上下文一股脑塞进去。PicoClaw 引入了结构化的 **Runtime Loop**,让每一个 token 都物尽其用。 + +### 三阶段 Runtime Loop + +每次用户交互都经过三个清晰的阶段: + +``` +┌──────────────┐ ┌──────────────┐ ┌──────────────┐ +│ Phase 1 │ → │ Phase 2 │ → │ Phase 3 │ +│ 分析 Analyse│ │ 执行 ExecuteLLM│ │ 反思 Reflect │ +│ (编排器) │ │ (执行器) │ │ (学习器) │ +└──────────────┘ └──────────────┘ └──────────────┘ +``` + +- **Phase 1 — 分析**:轻量 LLM 调用(可用便宜/快速模型),理解用户意图、分配语义标签、准备最优上下文。无工具、无历史膨胀——纯粹的理解力。 +- **Phase 2 — 执行**:主 LLM 只携带 Phase 1 认为相关的工具和上下文进行迭代。工具越少 = 干扰越少 = 决策越好。 +- **Phase 3 — 反思**:同步打分 + 上下文更新(< 2ms),在用户看到回复之前完成,然后异步持久化。Agent 从每次交互中学习,且不增加延迟。 + +### Turn 打分 & 瞬时记忆 + +每个 Turn 都会根据工具活动、意图权重、内容密度和用户显式标记获得一个**分数**。这个分数驱动一个新颖的上下文选择算法: + +``` +瞬时记忆 = { 高分 Turn } // 无条件保留 + ∪ { tag 匹配的 Turn, score > 0 } // 旧 Turn 被"唤醒" + ∪ { 最近 M 个 Turn } // 连贯性保底 + → 按时间排序,截断到容量上限 +``` + +**Turn 永不删除。** 低分 Turn 只是被排除在上下文之外——但当未来 Phase 1 产生匹配的 tags 时,它们可以被重新召回。就像人类记忆:你不会忘记东西,只是在某个触发点之前无法访问它们。 + +### Tag 驱动的工具加载 + +传统框架每次请求都把 20+ 个工具定义喂给 LLM(每次迭代浪费 ~3,000 tokens)。PicoClaw 把工具分为两层: + +| 层级 | 工具 | 加载方式 | +|------|------|---------| +| **always-on** | `read_file`、`write_file`、`shell` 等 | 始终加载 | +| **tag-gated** | MCP servers、Skills、`web_search`、`cron` | 仅当 Phase 1 的 tags 匹配时加载 | + +随着 MCP Servers 和 Skills 的增多,节省量呈指数级增长——而且 LLM 在精简的工具集下做出更好的决策。 + +### KV Cache 友好的消息排列 + +我们执着于**前缀稳定性**,以最大化跨 Provider 的 KV Cache 命中率(Gemini 隐式缓存、Anthropic `cache_control`、OpenAI prompt caching): + +``` +[system_prompt] ← 始终缓存 ✅ +[long_term_memory by tags] ← 相同 tags 时稳定 ✅ +[高分 Turn, 按 ID 升序] ← 位置固定,只增不移 ✅ +[tag 匹配的 Turn] ← 可能变化 +[最近的 Turn] ← 滚动窗口 +[当前用户消息] ← 总是新内容 +``` + +高分 Turn 被**固定**在长期记忆之后,按升序排列。新 Turn 只追加到末尾,不改变已有内容的位置。这最大化了缓存命中的前缀长度,在规模化场景下成本差异显著。 + +### 三层记忆体系 + +| 层次 | 生命周期 | 数据来源 | 消费者 | +|------|---------|---------|--------| +| **瞬时记忆** | 每轮动态组装 | Turn 库(score + tag 筛选)| Phase 2(ExecuteLLM)| +| **活跃上下文** | 跨 Turn 滚动更新,per `channel:chatID` | Phase 3 同步维护 | Phase 1(Analyse)| +| **长期记忆** | 持久化 | MemoryDigest 定时批量提炼 | Phase 1(通过 tag 检索)| + +每层服务不同的阶段。没有重叠,没有浪费。活跃上下文(当前文件、近期错误)帮助 Phase 1 理解简短消息如"修一下"。瞬时记忆为 Phase 2 提供正确的历史上下文。长期记忆跨会话积累智慧。 + +### 为什么这很重要 + +传统的 Agent 循环把所有东西塞进一个 LLM 调用,然后听天由命。PicoClaw 的方式意味着: + +- 📉 **减少 ~60% 的浪费 token** — 排除无关工具和陈旧上下文 +- 🎯 **更高的决策质量** — 精简工具集减少 LLM 的"选择困难症" +- ⚡ **< 2ms 的额外开销** — 打分和上下文更新零感知延迟 +- 🧲 **联想式召回** — 旧对话在相关时自动浮现,就像人类记忆 +- 💰 **多模型成本优化** — 分析用便宜模型,执行用强模型 + +> *"最好的 token 是你永远不需要发送的那个。"* + ## 🦾 演示 ### 🛠️ 标准助手工作流