chore: add codebase map with 7 structured analysis documents
Documents cover: tech stack (Go 1.25.9 + React 19), integrations (18 messaging channels, MCP, Docker), architecture (event-driven agent gateway), code structure (cmd/pkg/web/workspace layout), conventions/testing patterns, and code concerns (security, performance, complexity, technical debt). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
srcrs
parent
748ac58dd1
commit
46500b28f4
7 changed files with 1438 additions and 0 deletions
148
.planning/codebase/ARCHITECTURE.md
Normal file
148
.planning/codebase/ARCHITECTURE.md
Normal file
|
|
@ -0,0 +1,148 @@
|
|||
# Architecture
|
||||
|
||||
**Analysis Date:** 2026-04-10
|
||||
|
||||
## Pattern Overview
|
||||
|
||||
**Overall:** Event-driven agent gateway with message bus architecture
|
||||
|
||||
**Key Characteristics:**
|
||||
- Multi-agent support with per-instance workspace, session, and tool registry
|
||||
- Message bus (`pkg/bus`) decouples channels from agent processing loops
|
||||
- Channel manager (`pkg/channels`) abstracts 17+ messaging platforms behind unified interfaces
|
||||
- Hot-reloadable gateway with graceful shutdown and provider fallback chains
|
||||
- Embedded web launcher that serves both the dashboard UI and manages the gateway process
|
||||
|
||||
## Layers
|
||||
|
||||
**Gateway Layer (`pkg/gateway/`):**
|
||||
- Purpose: Top-level runtime orchestrator — starts agent loops, channels, services
|
||||
- Location: `pkg/gateway/gateway.go`
|
||||
- Contains: Service lifecycle, config loading, signal handling
|
||||
- Depends on: `pkg/agent`, `pkg/bus`, `pkg/channels`, `pkg/cron`, `pkg/health`
|
||||
- Used by: CLI entry point (`cmd/picoclaw`), Web launcher (`web/backend/`)
|
||||
|
||||
**Agent Layer (`pkg/agent/`):**
|
||||
- Purpose: Core AI agent loop — LLM interaction, tool execution, context management
|
||||
- Location: `pkg/agent/loop.go`, `pkg/agent/instance.go`, `pkg/agent/turn.go`
|
||||
- Contains: AgentLoop, AgentInstance, AgentRegistry, EventBus, HookManager
|
||||
- Depends on: `pkg/bus`, `pkg/providers`, `pkg/tools`, `pkg/session`, `pkg/memory`
|
||||
- Used by: Gateway service
|
||||
|
||||
**Bus Layer (`pkg/bus/`):**
|
||||
- Purpose: Asynchronous message routing between channels and agents
|
||||
- Location: `pkg/bus/bus.go`, `pkg/bus/types.go`
|
||||
- Contains: MessageBus with inbound, outbound, media, audio, voice channels
|
||||
- Depends on: `pkg/logger`
|
||||
- Used by: All agent and channel code
|
||||
|
||||
**Channel Layer (`pkg/channels/`):**
|
||||
- Purpose: Platform-agnostic messaging interface with per-platform adapters
|
||||
- Location: `pkg/channels/` with subpackages for each platform
|
||||
- Contains: Channel interface, Manager, dynamic mux, per-platform implementations
|
||||
- Depends on: `pkg/bus`, `pkg/config`, `pkg/health`
|
||||
- Used by: Gateway service
|
||||
|
||||
**Provider Layer (`pkg/providers/`):**
|
||||
- Purpose: LLM provider abstraction with fallback, routing, and rate limiting
|
||||
- Location: `pkg/providers/` with subpackages for Anthropic, OpenAI, Bedrock, etc.
|
||||
- Contains: LLMProvider interface, factory, fallback chain, model router
|
||||
- Depends on: `pkg/logger`, `pkg/config`
|
||||
- Used by: Agent instances
|
||||
|
||||
**Tool Layer (`pkg/tools/`):**
|
||||
- Purpose: Tool registry and built-in tool implementations
|
||||
- Location: `pkg/tools/`
|
||||
- Contains: ToolRegistry, shell, filesystem, web, MCP, spawn, SPI/I2C hardware tools
|
||||
- Depends on: `pkg/logger`, `pkg/providers`
|
||||
- Used by: Agent instances during turn execution
|
||||
|
||||
**Session Layer (`pkg/session/`):**
|
||||
- Purpose: Session persistence and management with JSONL backend
|
||||
- Location: `pkg/session/manager.go`, `pkg/session/session_store.go`
|
||||
- Contains: SessionManager, SessionStore interface, JSONL backend
|
||||
- Depends on: `pkg/providers`
|
||||
- Used by: Agent instances for conversation history
|
||||
|
||||
## Data Flow
|
||||
|
||||
**Message Processing Flow:**
|
||||
|
||||
1. External message arrives on a channel (e.g., Telegram webhook, Discord event, Pico WebSocket)
|
||||
2. Channel adapter converts platform message to `bus.InboundMessage` and publishes to the bus
|
||||
3. `AgentLoop` receives the inbound message from the bus
|
||||
4. `EventBus` fires pre-processing hooks; `HookManager` executes registered hooks
|
||||
5. `ContextBuilder` assembles the prompt (system, history, skills, tools)
|
||||
6. Agent routes to the correct `AgentInstance` via `AgentRegistry` (with optional model routing)
|
||||
7. LLM call via `Provider` (with fallback chain on failure)
|
||||
8. Tool calls are dispatched via `ToolRegistry.Execute()` in a tool loop
|
||||
9. Final content is published as `bus.OutboundMessage`
|
||||
10. `ChannelManager` routes the response back through the originating channel
|
||||
11. Channel sends response to the user on the platform
|
||||
|
||||
**State Management:**
|
||||
- Session state persisted as JSONL files in `~/.picoclaw/sessions/`
|
||||
- Agent context managed in-memory with `ContextBuilder` + `ContextManager`
|
||||
- Long-term memory via `pkg/memory` package (JSONL-based store)
|
||||
- Config loaded from `~/.picoclaw/config.json` with hot-reload support
|
||||
|
||||
## Key Abstractions
|
||||
|
||||
**Channel Interface:**
|
||||
- Purpose: Abstracts 17+ messaging platforms behind a common interface
|
||||
- Examples: `pkg/channels/telegram/`, `pkg/channels/discord/`, `pkg/channels/pico/`
|
||||
- Pattern: Capability-based interfaces (`TypingCapable`, `StreamingCapable`, `MessageEditor`, `ReactionCapable`, `PlaceholderCapable`, `CommandRegistrarCapable`)
|
||||
|
||||
**LLMProvider Interface:**
|
||||
- Purpose: Unified LLM interaction contract
|
||||
- Examples: `pkg/providers/anthropic/`, `pkg/providers/openai_compat/`, `pkg/providers/bedrock/`
|
||||
- Pattern: Factory-based provider creation with per-candidate credentials
|
||||
|
||||
**Tool Interface:**
|
||||
- Purpose: Pluggable tool execution with TTL-based registration
|
||||
- Examples: `pkg/tools/shell.go`, `pkg/tools/filesystem.go`, `pkg/tools/mcp_tool.go`
|
||||
- Pattern: `Tool` interface with `Name()`, `Description()`, `Parameters()`, `Execute()`
|
||||
|
||||
**ContextManager:**
|
||||
- Purpose: Manages conversation context window with budget-based truncation
|
||||
- Examples: `pkg/agent/context_budget.go`, `pkg/agent/context_seahorse.go`, `pkg/agent/context_legacy.go`
|
||||
- Pattern: Strategy pattern with multiple implementations for different context strategies
|
||||
|
||||
## Entry Points
|
||||
|
||||
**CLI Agent (`cmd/picoclaw/main.go`):**
|
||||
- Location: `cmd/picoclaw/main.go`
|
||||
- Triggers: Command-line invocation
|
||||
- Responsibilities: Subcommands for agent, auth, cron, gateway, skills, model, migrate, status, version
|
||||
|
||||
**Web Launcher (`web/backend/main.go`):**
|
||||
- Location: `web/backend/main.go`
|
||||
- Triggers: Direct execution or system tray
|
||||
- Responsibilities: Embedded HTTP server (default port 18800), dashboard auth, gateway auto-start, system tray
|
||||
|
||||
**Gateway Service (`pkg/gateway/gateway.go`):**
|
||||
- Location: `pkg/gateway/gateway.go`
|
||||
- Triggers: `gateway` subcommand or launcher auto-start
|
||||
- Responsibilities: Full runtime — agent loops, channel workers, cron, heartbeat, health server
|
||||
|
||||
## Error Handling
|
||||
|
||||
**Strategy:** Structured logging with provider fallback chains
|
||||
|
||||
**Patterns:**
|
||||
- `providers.FallbackChain` — automatic failover to backup models on error
|
||||
- `routing.Router` — intelligent model selection based on message complexity
|
||||
- `channels.Manager` — per-channel rate limiting with exponential backoff
|
||||
- `providers.ErrorClassifier` — categorizes errors (rate limit, auth, context, etc.) for appropriate retry behavior
|
||||
|
||||
## Cross-Cutting Concerns
|
||||
|
||||
**Logging:** `pkg/logger` — structured JSON logging with console/file modes, component-tagged output (`InfoCF`, `ErrorCF`, etc.)
|
||||
**Validation:** `pkg/tools/validate.go` — tool parameter validation with JSON schema
|
||||
**Authentication:** Dashboard auth (`web/backend/dashboardauth/`), OAuth flows (`pkg/auth/oauth.go`), PKCE for external platforms
|
||||
**Configuration:** JSON/YAML config with environment variable override support (`pkg/config/`)
|
||||
**Security:** Sensitive data filtering before LLM calls, credential storage isolation
|
||||
|
||||
---
|
||||
|
||||
*Architecture analysis: 2026-04-10*
|
||||
251
.planning/codebase/CONCERNS.md
Normal file
251
.planning/codebase/CONCERNS.md
Normal file
|
|
@ -0,0 +1,251 @@
|
|||
# Codebase Concerns
|
||||
|
||||
**Analysis Date:** 2026-04-10
|
||||
|
||||
---
|
||||
|
||||
## Security
|
||||
|
||||
### 1. Hardcoded OAuth Client Credentials (Obfuscated but Reversible)
|
||||
|
||||
**Files:** `pkg/auth/oauth.go:47-50`
|
||||
|
||||
The Google Antigravity OAuth client ID and secret are base64-encoded inline strings, not environment variables. These are public OAuth client credentials (used by OpenCode/pi-ai) rather than secrets, so the risk is low. However, the pattern of embedding credentials in source code is a concern if extended to actual secrets.
|
||||
|
||||
### 2. Self-Update Endpoint with No Binary Signature Verification
|
||||
|
||||
**Files:** `pkg/updater/updater.go:33-80`, `pkg/updater/updater.go:611-646`
|
||||
|
||||
The self-update mechanism downloads release archives from GitHub and extracts them. SHA256 checksum verification exists for release downloads, but there is no code signing verification of the binary itself. The `minio/selfupdate` library handles binary replacement.
|
||||
|
||||
- **Mitigation present:** SHA256 checksum verification on release downloads
|
||||
- **Remaining risk:** No cryptographic signature verification before applying the update
|
||||
- **Fix approach:** Add signature verification (minisign is already an indirect dependency)
|
||||
|
||||
### 3. WebSocket Proxy Token Validation via Custom Header
|
||||
|
||||
**Files:** `web/backend/api/pico.go:57-100`
|
||||
|
||||
The Pico WebSocket proxy validates tokens via a custom header. The token is compared against a cached config value. If the token changes in config while the gateway is running, there is a brief window where the cached token may be stale, allowing old tokens to work or rejecting valid ones.
|
||||
|
||||
### 4. Login Rate Limiting is In-Memory Only
|
||||
|
||||
**Files:** `web/backend/api/auth_login_limiter.go:17-40`
|
||||
|
||||
The dashboard login rate limiter uses in-memory maps keyed by IP address. Rate limits reset on process restart, and in distributed deployments each instance has independent limits.
|
||||
|
||||
### 5. No CSRF Protection on API Endpoints
|
||||
|
||||
**Files:** `web/backend/api/router.go:52-95`
|
||||
|
||||
The API routes use `http.ServeMux` directly with no CSRF middleware. If the server is run with `-public` flag, any website could make cross-origin requests unless CORS is properly configured.
|
||||
|
||||
---
|
||||
|
||||
## Performance
|
||||
|
||||
### 1. AgentLoop is a God File (3685 lines)
|
||||
|
||||
**File:** `pkg/agent/loop.go` (3685 lines)
|
||||
|
||||
Largest non-test file. Contains message routing (lines ~444-580), turn execution with tool loop (lines ~1800-2700), provider hot-reloading with goroutine isolation (lines ~982-1077), tool registration for all agents (lines ~165-442), and media resolution.
|
||||
|
||||
**Fix approach:** Extract sub-components into separate files within `pkg/agent/`.
|
||||
|
||||
### 2. Seahorse Store is Large (1542 lines)
|
||||
|
||||
**File:** `pkg/seahorse/store.go` (1542 lines)
|
||||
|
||||
All database operations in a single file with 44+ `ExecContext`/`QueryContext` calls. Uses parameterized queries (safe from injection), but size makes auditing difficult.
|
||||
|
||||
### 3. JSONL Store maxLineSize = 10 MB
|
||||
|
||||
**File:** `pkg/memory/jsonl.go:32`
|
||||
|
||||
A single tool result can be up to 10 MB. Messages are never physically deleted from JSONL files -- only logically skipped via metadata offset.
|
||||
|
||||
### 4. Media Cleanup Disabled
|
||||
|
||||
**File:** `pkg/agent/loop.go:487-498`
|
||||
|
||||
```go
|
||||
// TODO: Re-enable media cleanup after inbound media is properly consumed by the agent.
|
||||
// Currently disabled because files are deleted before the LLM can access their content.
|
||||
```
|
||||
|
||||
Media files are never cleaned up, consuming disk space over time.
|
||||
|
||||
---
|
||||
|
||||
## Complexity
|
||||
|
||||
### 1. Complex Goroutine Lifecycle Management
|
||||
|
||||
Key unbounded goroutine spawns:
|
||||
- `pkg/agent/loop.go:1001` -- Registry creation in goroutine with recover
|
||||
- `pkg/seahorse/short_compaction.go:80` -- Async condensed compaction per conversation
|
||||
- `web/backend/api/gateway.go:758-782` -- Multiple goroutines for gateway process management
|
||||
- `pkg/agent/loop.go:477` -- `drainBusToSteering` goroutine per message
|
||||
|
||||
**Risk:** Goroutine leaks during error paths or rapid config reloads.
|
||||
|
||||
### 2. Global Mutable State in Gateway Package
|
||||
|
||||
**File:** `web/backend/api/gateway.go:29-43`
|
||||
|
||||
Package-level mutable singleton holding process state, config signatures, and cached tokens. Protected by `sync.Mutex` but creates tight coupling between gateway lifecycle and API handler.
|
||||
|
||||
### 3. Multiple Mutexes in Channel Implementations
|
||||
|
||||
- `pkg/channels/wecom/wecom.go` -- 6 separate mutexes
|
||||
- `pkg/channels/onebot/onebot.go` -- 3 mutexes
|
||||
- `pkg/channels/manager.go` -- 1 RWMutex plus per-channel operations
|
||||
|
||||
Increases deadlock risk if lock ordering is not consistent.
|
||||
|
||||
---
|
||||
|
||||
## Technical Debt
|
||||
|
||||
### 1. config_old.go -- Legacy Config Migration Code
|
||||
|
||||
**File:** `pkg/config/config_old.go` (1001 lines)
|
||||
|
||||
Contains V0 config structs for backward compatibility migration. Will grow as config schema evolves.
|
||||
|
||||
### 2. Media Cleanup Commented Out (TODO)
|
||||
|
||||
**File:** `pkg/agent/loop.go:487-498`
|
||||
|
||||
Known feature regression -- media cleanup disabled due to timing issue.
|
||||
|
||||
### 3. Logger TimeFormat Not Configurable (TODO)
|
||||
|
||||
**File:** `pkg/logger/logger.go:55`
|
||||
|
||||
### 4. MCP Tool Artifact Lifecycle Not Managed (TODO)
|
||||
|
||||
**File:** `pkg/tools/mcp_tool.go:365`
|
||||
|
||||
### 5. GitHub Copilot Provider Incomplete (TODO)
|
||||
|
||||
**File:** `pkg/providers/github_copilot_provider.go:29`
|
||||
|
||||
Only supports HTTP mode, not stdio.
|
||||
|
||||
### 6. ASR Model Restriction Incomplete (TODO)
|
||||
|
||||
**File:** `pkg/audio/asr/asr.go:36`
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
### 1. Swallowed Errors in Config Reload
|
||||
|
||||
**File:** `web/backend/api/gateway.go:55-60`
|
||||
|
||||
```go
|
||||
func refreshPicoTokensLocked(configPath string) {
|
||||
cfg, err := config.LoadConfig(h.configPath)
|
||||
if err != nil {
|
||||
return // Error silently swallowed
|
||||
}
|
||||
```
|
||||
|
||||
If config reload fails, the token is not updated and no error is logged.
|
||||
|
||||
### 2. Ignored `LastInsertId` Error
|
||||
|
||||
**File:** `pkg/seahorse/store.go:57`
|
||||
|
||||
### 3. Panics in Package Initialization
|
||||
|
||||
- `pkg/agent/context_seahorse.go:267`
|
||||
- `pkg/logger/panic_unix.go:16-19`
|
||||
- `web/backend/main.go:126-140`
|
||||
|
||||
---
|
||||
|
||||
## Scalability
|
||||
|
||||
### 1. Single-Process Architecture
|
||||
|
||||
Web backend, agent loop, channel connections, MCP servers, and cron jobs all share one process.
|
||||
|
||||
### 2. In-Memory Rate Limiting
|
||||
|
||||
**File:** `pkg/providers/ratelimiter.go:13`
|
||||
|
||||
All rate limiting is in-memory; state lost on restart.
|
||||
|
||||
### 3. JSONL File-per-Session Storage
|
||||
|
||||
**File:** `pkg/memory/jsonl.go:46-55`
|
||||
|
||||
Each session creates two files. Lock sharding (`numLockShards = 64`) mitigates contention but not file count growth.
|
||||
|
||||
---
|
||||
|
||||
## Dependencies
|
||||
|
||||
### 1. Large Dependency Surface for Channel Integrations
|
||||
|
||||
15+ messaging platform SDKs compiled in regardless of usage.
|
||||
|
||||
**Fix approach:** Consider build tags to compile only needed channels.
|
||||
|
||||
### 2. WebRTC Dependency for Discord Voice
|
||||
|
||||
**Files:** `pkg/channels/discord/voice.go`
|
||||
|
||||
Pion WebRTC stack (~10 transitive deps) used only for Discord voice.
|
||||
|
||||
---
|
||||
|
||||
## Maintainability
|
||||
|
||||
### 1. Large Test Files
|
||||
|
||||
- `pkg/agent/loop_test.go` -- 3367 lines
|
||||
- `pkg/agent/subturn_test.go` -- 2067 lines
|
||||
- `pkg/agent/steering_test.go` -- 1591 lines
|
||||
- `pkg/config/config_test.go` -- 1976 lines
|
||||
|
||||
### 2. Duplicate Test Patterns
|
||||
|
||||
Agent test files contain repeated mock structures (`mockProvider`, `mockChannel`, `turnState`) defined inline rather than shared.
|
||||
|
||||
---
|
||||
|
||||
## Data Integrity
|
||||
|
||||
### 1. Race Condition Window in Conversation Creation
|
||||
|
||||
**File:** `pkg/seahorse/store.go:35-62`
|
||||
|
||||
Classic TOCTOU pattern handled via unique violation detection and retry. Correct for SQLite but fragile.
|
||||
|
||||
### 2. No WAL Mode Configuration for SQLite
|
||||
|
||||
Under concurrent load, this could cause "database is locked" errors. Compaction goroutines (`runCondensedLoop`) can write concurrently with ingestion.
|
||||
|
||||
---
|
||||
|
||||
## Missing Critical Features
|
||||
|
||||
### 1. No Audit Logging
|
||||
|
||||
No structured audit log for security-sensitive operations.
|
||||
|
||||
### 2. No Health/Metrics Endpoint
|
||||
|
||||
`pkg/health/server.go` provides basic health checking but no Prometheus-compatible metrics.
|
||||
|
||||
### 3. No Graceful Shutdown for All Components
|
||||
|
||||
`AgentLoop.Run()` returns on context cancellation, but sub-goroutines (compaction, media, drains) may outlive the main loop.
|
||||
|
||||
---
|
||||
|
||||
*Concerns analysis: 2026-04-10*
|
||||
229
.planning/codebase/CONVENTIONS.md
Normal file
229
.planning/codebase/CONVENTIONS.md
Normal file
|
|
@ -0,0 +1,229 @@
|
|||
# Coding Conventions
|
||||
|
||||
**Analysis Date:** 2026-04-10
|
||||
|
||||
## Naming Patterns
|
||||
|
||||
**Packages:**
|
||||
- Lowercase, single-word names (idiomatic Go). Examples: `logger`, `config`, `session`, `providers`, `seahorse`, `tokenizer`, `memory`, `isolation`, `credential`.
|
||||
- Located under `pkg/` for shared libraries and `cmd/picoclaw/internal/` for CLI-internal packages.
|
||||
- Internal packages live in `cmd/picoclaw/internal/<feature>/`, each feature gets its own sub-package: `auth`, `agent`, `gateway`, `cron`, `skills`, `onboard`, `migrate`, `model`, `status`, `version`.
|
||||
|
||||
**Functions:**
|
||||
- Public: `PascalCase` -- `NewPicoclawCommand`, `SetLevelFromString`, `GetOrCreateConversation`, `RegisterLauncherAuthRoutes`.
|
||||
- Private: `camelCase` -- `newBenchStore`, `logMessage`, `appendFields`, `getCallerSkip`, `openTestDB`.
|
||||
- Constructor prefix: `New` for public (`NewAntigravityProvider`, `NewSubagentManager`), `new` for private (`newBenchStore`, `newTestCompactionEngineWithStore`).
|
||||
- Cobra command constructors: `NewXxxCommand()` in `cmd/picoclaw/internal/<pkg>/` (e.g., `NewAuthCommand()`, `NewAgentCommand()`). Tests call `newXxxCommand()` (lowercase) when the constructor is internal.
|
||||
|
||||
**Variables:**
|
||||
- `camelCase` for locals and package-level variables: `currentLevel`, `logFile`, `rrCounter`, `consoleWriter`.
|
||||
- Constants use `PascalCase` or `UPPER_SNAKE_CASE`: `CurrentVersion`, `Component`, `DEBUG`, `INFO`, `WARN`, `ERROR`, `FATAL`.
|
||||
- Environment variable constants: `EnvHome`, `EnvConfig` in `pkg/config/`. Actual env var names: `PICOCLAW_HOME`, `PICOCLAW_CONFIG`, `PICOCLAW_LOG_FILE`, `TZ`, `ZONEINFO`.
|
||||
|
||||
**Types/Interfaces:**
|
||||
- `PascalCase` structs: `Config`, `Store`, `SessionManager`, `AntigravityProvider`, `JSONLBackend`.
|
||||
- Interface names: `LLMProvider`, `SessionStore`, `CompleteFn` (function type alias), `Tool` (implicit).
|
||||
- Type aliases for external types: `type LogLevel = zerolog.Level` in `pkg/logger/logger.go`.
|
||||
- Config sub-types use `XxxConfig` suffix: `IsolationConfig`, `AgentsConfig`, `SessionConfig`, `ChannelsConfig`.
|
||||
|
||||
## Code Formatting
|
||||
|
||||
**Tooling:** `golangci-lint` v2 with formatters configured in `.golangci.yaml`.
|
||||
|
||||
**Formatters enabled:**
|
||||
- `gci` -- import grouping: `standard` -> `default` -> `localmodule` with custom order.
|
||||
- `gofmt` -- with rewrite rules: `interface{}` -> `any`, `a[b:len(a)]` -> `a[b:]`.
|
||||
- `gofumpt` -- strict formatting.
|
||||
- `goimports` -- auto import management.
|
||||
- `golines` -- max line length 120.
|
||||
|
||||
**Commands:**
|
||||
```bash
|
||||
make fmt # runs golangci-lint fmt
|
||||
make lint # runs golangci-lint run --build-tags=goolm,stdjson
|
||||
make fix # runs golangci-lint run --fix --build-tags=goolm,stdjson
|
||||
```
|
||||
|
||||
**Lint settings:**
|
||||
- `default: all` with 35+ linters disabled (see `.golangci.yaml` lines 5-63).
|
||||
- Line length: 120 chars (`.golangci.yaml` line 85).
|
||||
- `funlen`: 120 lines / 40 statements.
|
||||
- `gocognit`: min-complexity 25.
|
||||
- `gocyclo`: min-complexity 20.
|
||||
- `lll` exclusions for `//go:generate` lines.
|
||||
- Test files excluded from `funlen`, `maintidx`, `gocognit`, `gocyclo` (`.golangci.yaml` lines 147-148).
|
||||
- `testpackage` is disabled -- tests can use the same package name (not `_test`).
|
||||
|
||||
## Import Organization
|
||||
|
||||
**Order (gci config):**
|
||||
1. Standard library
|
||||
2. Third-party packages
|
||||
3. Local module (`github.com/sipeed/picoclaw/...`)
|
||||
|
||||
**Example from `cmd/picoclaw/main.go`:**
|
||||
```go
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
"github.com/spf13/cobra"
|
||||
|
||||
"github.com/sipeed/picoclaw/cmd/picoclaw/internal"
|
||||
"github.com/sipeed/picoclaw/cmd/picoclaw/internal/agent"
|
||||
// ... more internal imports
|
||||
"github.com/sipeed/picoclaw/pkg/config"
|
||||
"github.com/sipeed/picoclaw/pkg/updater"
|
||||
)
|
||||
```
|
||||
|
||||
**Path aliases:** None used. Full import paths everywhere: `github.com/sipeed/picoclaw/pkg/...`.
|
||||
|
||||
## Error Handling
|
||||
|
||||
**Patterns:**
|
||||
- `fmt.Errorf("context: %w", err)` for error wrapping with `fmt` and `%w`. Example from `pkg/seahorse/schema.go`:
|
||||
```go
|
||||
return fmt.Errorf("FTS5 check: %w", err)
|
||||
```
|
||||
- Direct `return nil, fmt.Errorf("antigravity auth: %w", err)` in provider code (`pkg/providers/antigravity_provider.go`).
|
||||
- Config errors use descriptive messages with `fmt.Errorf("failed to create log directory: %w", err)` (`pkg/logger/logger.go`).
|
||||
- Tests use `t.Fatalf("operation: %v", err)` for fatal errors and `t.Errorf("description: %v", err)` for non-fatal.
|
||||
- Some code logs errors rather than returning them (fire-and-forget pattern in `pkg/session/jsonl_backend.go`):
|
||||
```go
|
||||
if err := b.store.AddMessage(...); err != nil {
|
||||
log.Printf("session: add message: %v", err)
|
||||
}
|
||||
```
|
||||
- Sentinel errors not widely used. Most errors are constructed inline with `fmt.Errorf`.
|
||||
|
||||
## Logging
|
||||
|
||||
**Framework:** `github.com/rs/zerolog` wrapped by a custom logger at `pkg/logger/logger.go`.
|
||||
|
||||
**API pattern:** Suffix convention for logging variants:
|
||||
- `Debug(message)` -- plain message, auto-detected component
|
||||
- `DebugC(component, message)` -- explicit component
|
||||
- `Debugf(message, args...)` -- sprintf-style
|
||||
- `DebugF(message, fields)` -- structured fields as `map[string]any`
|
||||
- `DebugCF(component, message, fields)` -- component + fields
|
||||
|
||||
**Same pattern for all levels:** `Info/InfoC/Infof/InfoF/InfoCF`, `Warn/WarnC/Warnf/WarnF/WarnCF`, `Error/ErrorC/Errorf/ErrorF/ErrorCF`, `Fatal/FatalC/Fatalf/FatalF/FatalCF`.
|
||||
|
||||
**Log format (TTY):**
|
||||
```
|
||||
15:04:05 WARN component caller message
|
||||
```
|
||||
Component shown in yellow (`\x1b[33m`). Time-only timestamps.
|
||||
|
||||
**Log format (non-TTY):** JSON (zerolog default).
|
||||
|
||||
**Configuration from env:** `PICOCLAW_LOG_FILE` -- if set, enables file logging and disables console. Supports `~/` expansion.
|
||||
|
||||
**Global logger:** Package-level singleton with `sync.RWMutex` for thread safety. Not passed as dependency -- imported directly.
|
||||
|
||||
## Configuration Patterns
|
||||
|
||||
**Main config:** `pkg/config/config.go` -- JSON-based with `json` struct tags.
|
||||
|
||||
**Loading:**
|
||||
- Config file path via `PICOCLAW_CONFIG` env var or `PICOCLAW_HOME` env var (defaults to `~/.picoclaw`).
|
||||
- Environment variable overrides via `github.com/caarlos0/env/v11`.
|
||||
- Schema versioning (`CurrentVersion = 2`) with migration support.
|
||||
- Config struct uses `json:"-"` on most fields (only `channels`, `model_list`, and build info serialize).
|
||||
|
||||
**Version injection via ldflags:**
|
||||
```
|
||||
-X github.com/sipeed/picoclaw/pkg/config.Version=...
|
||||
-X github.com/sipeed/picoclaw/pkg/config.GitCommit=...
|
||||
-X github.com/sipeed/picoclaw/pkg/config.BuildTime=...
|
||||
-X github.com/sipeed/picoclaw/pkg/config.GoVersion=...
|
||||
```
|
||||
Accessed via `config.GetVersion()`, `config.GetGitCommit()`, etc.
|
||||
|
||||
## Interface Design
|
||||
|
||||
**Style:** Small, focused interfaces. Examples:
|
||||
- `LLMProvider` -- `Chat(ctx, messages, tools, model, options) (*LLMResponse, error)` + `GetDefaultModel()`, `SupportsTools()`, `GetContextWindow()`.
|
||||
- `SessionStore` -- `AddMessage`, `AddFullMessage`, `GetHistory`, `GetSummary`, `SetSummary`, `SetHistory`, `TruncateHistory`, `Save`, `Close`.
|
||||
- `memory.Store` -- file-based session persistence.
|
||||
|
||||
**Function type aliases** used for callbacks: `type CompleteFn func(ctx, prompt, opts) (string, error)` in seahorse compaction.
|
||||
|
||||
**Interface implementations** are created via constructor functions: `NewAntigravityProvider()`, `NewJSONLBackend(store)`, `NewSubagentManager(provider, model, workspace)`.
|
||||
|
||||
## Struct Organization
|
||||
|
||||
**Pattern:**
|
||||
- Exported structs with public fields for configuration (JSON tags).
|
||||
- Unexported fields for internal state: `sensitiveCache *SensitiveDataCache`, `store memory.Store`.
|
||||
- Methods on pointer receivers: `func (c *Config) FilterSensitiveData(...)`.
|
||||
- Small config sub-structs composed into main `Config`.
|
||||
|
||||
**Example from `pkg/config/config.go`:**
|
||||
```go
|
||||
type Config struct {
|
||||
Version int `json:"version"`
|
||||
Isolation IsolationConfig `json:"isolation,omitempty"`
|
||||
Agents AgentsConfig `json:"agents"`
|
||||
// ... many more sub-configs
|
||||
sensitiveCache *SensitiveDataCache // unexported, computed cache
|
||||
}
|
||||
```
|
||||
|
||||
## Git Commit Message Style
|
||||
|
||||
**Format:** Conventional Commits. Examples from recent commits:
|
||||
```
|
||||
fix(chat): keep tool-call summary and assistant output in sync (#2449)
|
||||
fix(seahorse): sanitize user input for FTS5 MATCH queries (#2436)
|
||||
fix(launcher): align react and react-dom versions (#2467)
|
||||
build(deps): bump github.com/modelcontextprotocol/go-sdk (#2455)
|
||||
feat(launcher): standard HTTP login/setup/logout flow for dashboard...
|
||||
style(lint): satisfy gci and golines for review fixes
|
||||
fix(agent): gate pico interim publish for internal turns
|
||||
```
|
||||
|
||||
**Rules (from CONTRIBUTING.md):**
|
||||
- English language, imperative mood: "Add retry logic" not "Added retry logic".
|
||||
- Reference issues: `Fix session leak (#123)`.
|
||||
- One logical change per commit.
|
||||
- Squash minor cleanups/typos into a single commit.
|
||||
- Follow https://www.conventionalcommits.org/zh-hans/v1.0.0/
|
||||
- Squash merge is the default strategy.
|
||||
|
||||
## Branch Naming
|
||||
|
||||
**Pattern:** `type/description` -- examples from CONTRIBUTING.md:
|
||||
- `fix/telegram-timeout`
|
||||
- `feat/ollama-provider`
|
||||
- `docs/contributing-guide`
|
||||
|
||||
**Long-lived branches:** `main` (active development), `release/x.y` (stable releases).
|
||||
|
||||
## Documentation Style
|
||||
|
||||
**Package comments:** Minimal. Only the main entry point (`cmd/picoclaw/main.go`) has a header comment block with project description and license.
|
||||
|
||||
**Function comments:** Godoc style when present, mostly on public API:
|
||||
```go
|
||||
// ParseLevel converts a case-insensitive level name to a LogLevel.
|
||||
// Returns the level and true if valid, or (INFO, false) if unrecognized.
|
||||
func ParseLevel(s string) (LogLevel, bool) { ... }
|
||||
|
||||
// NewAntigravityProvider creates a new Antigravity provider using stored auth credentials.
|
||||
func NewAntigravityProvider() *AntigravityProvider { ... }
|
||||
```
|
||||
|
||||
**Internal code:** Comments explain non-obvious logic, especially in `pkg/seahorse/` (schema, compaction) and complex SQL. Bug-fix tests include detailed BUG comments explaining the issue (see `pkg/seahorse/store_test.go` lines 497-619).
|
||||
|
||||
**nolint directives:** Used sparingly with justification:
|
||||
```go
|
||||
//nolint:zerologlint
|
||||
func getEvent(logger zerolog.Logger, level LogLevel) *zerolog.Event { ... }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*Convention analysis: 2026-04-10*
|
||||
212
.planning/codebase/INTEGRATIONS.md
Normal file
212
.planning/codebase/INTEGRATIONS.md
Normal file
|
|
@ -0,0 +1,212 @@
|
|||
# External Integrations
|
||||
|
||||
**Analysis Date:** 2026-04-10
|
||||
|
||||
## AI Model Providers
|
||||
|
||||
PicoClaw supports a protocol-based model routing system (`pkg/providers/factory_provider.go`). Models are specified as `protocol/model-id` (e.g., `openai/gpt-4o`, `anthropic/claude-sonnet-4.6`). Default protocol is `openai` when no prefix.
|
||||
|
||||
**Native protocol providers (non-OpenAI-compatible):**
|
||||
- **Anthropic** (`pkg/providers/anthropic/provider.go`) - Claude models, OAuth or API key auth
|
||||
- **Anthropic Messages** (`pkg/providers/anthropic_messages/provider.go`) - Messages API variant
|
||||
- **Claude CLI** (`pkg/providers/claude_cli_provider.go`) - Claude Code CLI via stdio
|
||||
- **Codex CLI** (`pkg/providers/codex_cli_provider.go`) - OpenAI Codex CLI via stdio
|
||||
- **AWS Bedrock** (`pkg/providers/bedrock/provider_bedrock.go`) - AWS SDK v2, IAM or credentials
|
||||
- **Azure OpenAI** (`pkg/providers/azure/provider.go`) - Azure-specific endpoint handling
|
||||
- **GitHub Copilot** (`pkg/providers/github_copilot_provider.go`) - Copilot SDK integration
|
||||
- **Antigravity** (`pkg/providers/antigravity_provider.go`) - Custom provider
|
||||
|
||||
**OpenAI-compatible protocols** (`pkg/providers/factory_provider.go`, lines 25-61):
|
||||
- `openai` (api.openai.com/v1)
|
||||
- `openrouter` (openrouter.ai/api/v1)
|
||||
- `groq` (api.groq.com/openai/v1)
|
||||
- `deepseek` (api.deepseek.com/v1)
|
||||
- `ollama` (localhost:11434/v1, no API key required)
|
||||
- `lmstudio` (localhost:1234/v1, no API key required)
|
||||
- `vllm` (localhost:8000/v1, no API key required)
|
||||
- `gemini` (generativelanguage.googleapis.com/v1beta)
|
||||
- `litellm` (localhost:4000/v1)
|
||||
- `qwen` / `qwen-intl` / `qwen-us` (Dashscope, Alibaba Cloud)
|
||||
- `moonshot`, `mistral`, `cerebras`, `nvidia`, `volcengine`, `modelscope`, `novita`, `minimax`, `longcat`, `avian`, `zhipu`, `venice`, `vivgrid`
|
||||
- Coding-specific: `coding-plan`, `alibaba-coding`, `coding-plan-anthropic`
|
||||
|
||||
## Messaging Channels (Chat Platforms)
|
||||
|
||||
Registered via blank imports in `pkg/gateway/gateway.go` (lines 21-37). Each channel lives in `pkg/channels/<name>/`.
|
||||
|
||||
| Channel | Package | SDK | Auth | Protocol |
|
||||
|---------|---------|-----|------|----------|
|
||||
| Telegram | `pkg/channels/telegram/` | telego v1.8.0 | Bot token | Polling |
|
||||
| Discord | `pkg/channels/discord/` | discordgo (fork) v0.29.0 | Bot token | WebSocket |
|
||||
| Feishu/Lark | `pkg/channels/feishu/` | oapi-sdk-go v3.5.3 | App ID + secret | Webhook |
|
||||
| Slack | `pkg/channels/slack/` | slack-go v0.17.3 | Bot token + App token | WebSocket (Socket Mode) |
|
||||
| DingTalk | `pkg/channels/dingtalk/` | dingtalk-stream-sdk v0.9.1 | Client ID + secret | Stream |
|
||||
| QQ | `pkg/channels/qq/` | botgo v0.2.1 | App ID + secret | WebSocket |
|
||||
| WhatsApp | `pkg/channels/whatsapp/` | Custom bridge | Bridge URL | HTTP bridge |
|
||||
| WhatsApp Native | `pkg/channels/whatsapp_native/` | whatsmeow | Direct login | WebSocket |
|
||||
| WeCom | `pkg/channels/wecom/` | Custom | Bot ID + secret | WebSocket |
|
||||
| Weixin | `pkg/channels/weixin/` | Custom | Token | HTTP webhook |
|
||||
| Matrix | `pkg/channels/matrix/` | mautrix v0.26.4 | Homeserver + access token | Matrix API |
|
||||
| IRC | `pkg/channels/irc/` | irc-go v0.6.0 | Nick/password/SASL | IRC |
|
||||
| LINE | `pkg/channels/line/` | REST client | Channel secret + token | Webhook |
|
||||
| OneBot | `pkg/channels/onebot/` | WebSocket client | Access token | WebSocket |
|
||||
| VK | `pkg/channels/vk/` | vksdk v3.3.1 | Group token | Long polling |
|
||||
| Teams | `pkg/channels/teams_webhook/` | go-teams-notify v2.14.0 | Webhook URL | HTTP POST |
|
||||
| MaixCam | `pkg/channels/maixcam/` | HTTP client | None | HTTP |
|
||||
| Pico | `pkg/channels/pico/` | Custom WebSocket | Token | WebSocket (built-in web channel) |
|
||||
| PicoClient | `pkg/channels/pico/` | Custom WebSocket client | Token | WebSocket (connect to remote Pico) |
|
||||
|
||||
## MCP (Model Context Protocol) Integration
|
||||
|
||||
**Manager:** `pkg/mcp/manager.go`
|
||||
|
||||
- Uses `github.com/modelcontextprotocol/go-sdk v1.5.0`
|
||||
- Supports **stdio** transport (spawn external process) and **Streamable HTTP** transport
|
||||
- Reads env files for MCP server configuration (`loadEnvFile()` line 49)
|
||||
- Custom HTTP headers supported via `headerTransport` wrapper
|
||||
- MCP tools are wrapped as native PicoClaw tools via `pkg/tools/mcp_tool.go`
|
||||
- Isolated command transport for sandboxed MCP servers (`pkg/mcp/isolated_command_transport.go`)
|
||||
|
||||
**MCP server configuration** is defined in `ToolsConfig.MCPServers` with fields: `Command`, `Args`, `Env`, `URL`, `Headers`, `EnvFile`.
|
||||
|
||||
## Built-in Tools
|
||||
|
||||
Defined in `pkg/tools/`:
|
||||
|
||||
- **Shell execution** (`shell.go`) - Run shell commands with PTY support, timeout, allow/deny patterns
|
||||
- **File operations** (`filesystem.go`) - Read, write, edit, list files (workspace-restricted)
|
||||
- **Edit** (`edit.go`) - Search-and-replace file editing
|
||||
- **Web search** (`search_tool.go`) - Multiple backends: Brave, Tavily, DuckDuckGo, Perplexity, SearXNG, GLM Search, Baidu Search
|
||||
- **Web fetch** (`web.go`) - Fetch and extract web page content
|
||||
- **Subagent** (`subagent.go`) - Spawn sub-agents for delegated tasks
|
||||
- **Cron** (`cron.go`) - Schedule and manage cron jobs
|
||||
- **Message tools** (`message.go`) - Send messages, reactions
|
||||
- **File send** (`send_file.go`) - Send files to chat
|
||||
- **Image loading** (`load_image.go`) - Load and process images
|
||||
- **SPI/I2C** (`spi.go`, `i2c.go`) - Hardware bus access (Linux-only, for embedded devices)
|
||||
- **Skills** (`skills_install.go`, `skills_search.go`) - Install and search skills marketplace
|
||||
- **TTS send** (`tts_send.go`) - Text-to-speech output
|
||||
- **Spawn** (`spawn.go`) - Process spawning with status tracking
|
||||
|
||||
## Audio Integrations (ASR/TTS)
|
||||
|
||||
**Speech-to-Text (ASR)** (`pkg/audio/asr/`):
|
||||
- **OpenAI-compatible Whisper** (`whisper_transcriber.go`) - Via any OpenAI-compatible provider
|
||||
- **ElevenLabs** (`elevenlabs_transcriber.go`) - ElevenLabs STT API
|
||||
- **Audio model transcriber** (`audio_model_transcriber.go`) - Generic audio-capable LLM
|
||||
|
||||
**Text-to-Speech (TTS)** (`pkg/audio/tts/`):
|
||||
- **OpenAI-compatible TTS** (`openai_tts.go`) - Any OpenAI-compatible endpoint
|
||||
- **Mimo TTS** (`mimo_tts.go`) - Xiaomi Mimo TTS service
|
||||
|
||||
**Audio codecs:**
|
||||
- OGG Opus encoding (`pkg/audio/ogg.go`)
|
||||
- Sentence segmentation (`pkg/audio/sentence.go`)
|
||||
|
||||
## Third-Party Library Integrations
|
||||
|
||||
**Data processing:**
|
||||
- `tidwall/gjson/sjson/pretty/match` - JSON manipulation
|
||||
- `segmentio/encoding` - High-performance JSON
|
||||
- `bytedance/sonic` - Fast JSON serialization
|
||||
- `vmihailenco/msgpack/v5` - MessagePack serialization
|
||||
|
||||
**Security:**
|
||||
- `cloudflare/circl` - Cryptographic primitives
|
||||
- `aead.dev/minisign` - Minisign signature verification (for release updates)
|
||||
- `filippo.io/edwards25519` - Ed25519 curves
|
||||
|
||||
**Observability:**
|
||||
- `opentelemetry.io/otel` (v1.35.0) - OpenTelemetry tracing (auto-instrumentation)
|
||||
- `zerolog` - Structured logging with sensitive data filtering
|
||||
|
||||
## Docker/Containerization
|
||||
|
||||
**Images:**
|
||||
- `docker/Dockerfile` - Minimal Alpine image (`alpine:3.23`), Go 1.25 builder, health check on `:18790/health`
|
||||
- `docker/Dockerfile.full` - Full image with Node.js 24 + `uv`/`uvx` (Python) for MCP tool support
|
||||
- `docker/Dockerfile.goreleaser` - Release build variant
|
||||
- `docker/Dockerfile.goreleaser.launcher` - Launcher-specific variant
|
||||
- `docker/Dockerfile.heavy` - Heavy variant
|
||||
- `docker/entrypoint.sh` - First-run entrypoint
|
||||
|
||||
**Docker Compose** (`docker/docker-compose.yml`):
|
||||
- `picoclaw-agent` - One-shot agent mode (profile: `agent`)
|
||||
- `picoclaw-gateway` - Long-running bot (profile: `gateway`)
|
||||
- `picoclaw-launcher` - Web console (profile: `launcher`, ports 18800/18790)
|
||||
|
||||
**Docker Compose Full** (`docker/docker-compose.full.yml`):
|
||||
- Same services but with full MCP tool support (Node.js runtime)
|
||||
|
||||
**Image registry:** `docker.io/sipeed/picoclaw:latest` / `:full` / `:launcher`
|
||||
|
||||
## CI/CD Configuration
|
||||
|
||||
**GitHub Actions** (`.github/workflows/`):
|
||||
|
||||
| Workflow | File | Purpose |
|
||||
|----------|------|---------|
|
||||
| Build | `build.yml` | On push to `main` - `make build-all` on ubuntu-latest |
|
||||
| PR | `pr.yml` | On pull requests - build + test |
|
||||
| Release | `release.yml` | On tags - multi-platform release artifacts |
|
||||
| Nightly | `nightly.yml` | Scheduled nightly builds |
|
||||
| Docker | `docker-build.yml` | Docker image build and push |
|
||||
| macOS DMG | `create_dmg.yml` | macOS application bundle |
|
||||
| TOS upload | `upload-tos.yml` | Terms of service upload |
|
||||
|
||||
**Dependabot** (`.github/dependabot.yml`) - Automated dependency updates
|
||||
|
||||
## Plugin/Extension Systems
|
||||
|
||||
**Skills** - Marketplace-style extension system:
|
||||
- Skills installed to `~/.picoclaw/workspace/skills/`
|
||||
- Built-in skills in `skills/` directory at project root
|
||||
- CLI commands: `picoclaw skills install`, `skills list`, `skills search`, `skills remove`, `skills list-builtin`, `skills show`
|
||||
- Tools: `pkg/tools/skills_install.go`, `pkg/tools/skills_search.go`
|
||||
- Skills package: `pkg/skills/`
|
||||
|
||||
**MCP Servers** - External tool providers:
|
||||
- Configured via `config.json` under `tools.mcp_servers`
|
||||
- Spawned as child processes (stdio) or connected via HTTP (Streamable HTTP)
|
||||
- Tools auto-discovered and registered as native tools
|
||||
|
||||
**Process Hooks** (`HooksConfig` in `pkg/config/config.go`):
|
||||
- Observer/interceptor hooks on agent events
|
||||
- Built-in hooks (e.g., content moderation)
|
||||
- Custom process hooks (spawn external process, observe/intercept events)
|
||||
|
||||
**Channel registry** (`pkg/channels/registry.go`):
|
||||
- Channels auto-register via `init()` functions in their packages
|
||||
- New channels added by creating a package under `pkg/channels/<name>/` with an `init.go` that calls `Register()`
|
||||
|
||||
**Provider factory** (`pkg/providers/factory_provider.go`):
|
||||
- New OpenAI-compatible providers added by adding a protocol entry to `protocolMetaByName` map
|
||||
- No code change needed for most new providers - just config
|
||||
|
||||
## Environment Variables
|
||||
|
||||
**Critical configuration:**
|
||||
- `PICOCLAW_CHANNELS_<NAME>_TOKEN` - Channel authentication tokens
|
||||
- `PICOCLAW_AGENTS_DEFAULTS_PROVIDER` / `MODEL_NAME` - Default model
|
||||
- `PICOCLAW_TOOLS_WEB_BRAVE_API_KEYS` / `TAVILY_API_KEYS` - Search API keys
|
||||
- `PICOCLAW_HOME` - Override home directory (default: `~/.picoclaw`)
|
||||
- `TZ` / `ZONEINFO` - Timezone configuration
|
||||
|
||||
**Secrets:** Stored in `~/.picoclaw/config.json` (with `SecureString` masking) and `~/.picoclaw/auth.json` (OAuth tokens). Neither file should be committed.
|
||||
|
||||
## Webhooks & Callbacks
|
||||
|
||||
**Incoming:**
|
||||
- Feishu webhook endpoint (channel-specific)
|
||||
- LINE webhook (`webhook_host:webhook_port/webhook_path`)
|
||||
- Weixin webhook
|
||||
- LINE webhook for callback events
|
||||
- Teams webhook (output-only, no incoming)
|
||||
|
||||
**Outgoing:**
|
||||
- Teams webhook notifications (`pkg/channels/teams_webhook/`)
|
||||
- Web search API calls (Brave, Tavily, Perplexity, etc.)
|
||||
|
||||
---
|
||||
|
||||
*Integration audit: 2026-04-10*
|
||||
164
.planning/codebase/STACK.md
Normal file
164
.planning/codebase/STACK.md
Normal file
|
|
@ -0,0 +1,164 @@
|
|||
# Technology Stack
|
||||
|
||||
**Analysis Date:** 2026-04-10
|
||||
|
||||
## Languages
|
||||
|
||||
**Primary:**
|
||||
- **Go 1.25.9** (per `go.mod` line 3) - Backend runtime, all core logic
|
||||
- **TypeScript ~5.9.3** - Web frontend (`web/frontend/`)
|
||||
|
||||
**Secondary:**
|
||||
- **JavaScript/Node.js 24** - Full Docker image runtime for MCP tools (`docker/Dockerfile.full`)
|
||||
|
||||
## Runtime
|
||||
|
||||
**Environment:**
|
||||
- Go 1.25.9 (compiled binary, CGO_ENABLED=0 by default; CGO_ENABLED=1 on macOS for systray)
|
||||
- Node.js ^20.19.0 || ^22.13.0 || >=24 (frontend via `web/frontend/package.json` engines field)
|
||||
|
||||
**Package Managers:**
|
||||
- **Go modules** - Lockfile: `go.sum` present
|
||||
- **pnpm** - Lockfile: `web/frontend/pnpm-lock.yaml` present (inferred from pnpm-lock.yaml)
|
||||
|
||||
## Frameworks
|
||||
|
||||
**Core (Go):**
|
||||
- **cobra v1.10.2** (`cmd/picoclaw/main.go`) - CLI framework, root command with subcommands: `onboard`, `agent`, `auth`, `gateway`, `status`, `cron`, `migrate`, `skills`, `model`, `version`, `update`
|
||||
- **modelcontextprotocol/go-sdk v1.5.0** (`pkg/mcp/manager.go`) - MCP (Model Context Protocol) client for external tool servers
|
||||
- **zerolog v1.35.0** (`pkg/logger/`) - Structured logging
|
||||
- **modernc.org/sqlite v1.48.2** - SQLite database (pure Go, no CGO)
|
||||
|
||||
**Frontend (web/frontend/):**
|
||||
- **React 19.2.5** - UI framework
|
||||
- **Vite 8.0.8** - Build tool and dev server
|
||||
- **TailwindCSS 4.2.2** - Styling with `@tailwindcss/vite` plugin
|
||||
- **Radix UI 1.4.3** - Headless component primitives
|
||||
- **TanStack Router 1.167.0 + React Query 5.97.0** - Routing and data fetching
|
||||
- **Jotai 2.19.1** - Atomic state management
|
||||
- **i18next 26.0.3** - Internationalization (en/zh)
|
||||
|
||||
**Backend Web (web/backend/):**
|
||||
- **net/http** (stdlib) - HTTP server for launcher dashboard
|
||||
|
||||
**Testing:**
|
||||
- **testify v1.11.1** - Assertion library and mocking (`github.com/stretchr/testify`)
|
||||
|
||||
**Build/Dev:**
|
||||
- **golangci-lint** - Linting (configured via `.golangci.yaml`)
|
||||
- **Make** - Build orchestration (`Makefile`, 399 lines)
|
||||
|
||||
## Key Dependencies
|
||||
|
||||
**AI Model Providers (SDK clients):**
|
||||
- `anthropic-sdk-go v1.26.0` - Anthropic Claude API
|
||||
- `openai-go/v3 v3.22.0` - OpenAI API
|
||||
- `aws-sdk-go-v2 + bedrockruntime v1.50.4` - AWS Bedrock
|
||||
- `github/copilot-sdk/go v0.2.0` - GitHub Copilot CLI
|
||||
|
||||
**Messaging/Chat SDKs (Channels):**
|
||||
- `slack-go/slack v0.17.3` - Slack
|
||||
- `bwmarrin/discordgo v0.29.0` (replaced with `yeongaori/discordgo-fork`) - Discord
|
||||
- `mymmrac/telego v1.8.0` - Telegram
|
||||
- `larksuite/oapi-sdk-go/v3 v3.5.3` - Feishu/Lark
|
||||
- `open-dingtalk/dingtalk-stream-sdk-go v0.9.1` - DingTalk
|
||||
- `tencent-connect/botgo v0.2.1` - QQ
|
||||
- `ergochat/irc-go v0.6.0` - IRC
|
||||
- `SevereCloud/vksdk/v3 v3.3.1` - VK
|
||||
- `atc0005/go-teams-notify/v2 v2.14.0` - Microsoft Teams
|
||||
- `maunium.net/go/mautrix v0.26.4` - Matrix
|
||||
- `go.mau.fi/whatsmeow` - WhatsApp (native)
|
||||
- `gorilla/websocket v1.5.3` - WebSocket (Pico channel)
|
||||
|
||||
**Audio (ASR/TTS):**
|
||||
- `pion/webrtc/v3 v3.3.6` + `pion/rtp v1.10.1` - WebRTC (Discord voice, audio streaming)
|
||||
- ElevenLabs transcriber (`pkg/audio/asr/elevenlabs_transcriber.go`)
|
||||
- OpenAI-compatible TTS (`pkg/audio/tts/openai_tts.go`)
|
||||
- Mimo TTS (`pkg/audio/tts/mimo_tts.go`)
|
||||
|
||||
**Web/HTTP:**
|
||||
- `valyala/fasthttp v1.69.0` - High-performance HTTP (Feishu channel)
|
||||
- `go-resty/resty/v2 v2.17.1` - HTTP client
|
||||
- `klauspost/compress v1.18.4` - Compression
|
||||
|
||||
**Terminal UI (Launcher TUI):**
|
||||
- `gdamore/tcell/v2 v2.13.8` - Terminal cell library
|
||||
- `rivo/tview v0.42.0` - Terminal UI widgets
|
||||
- `ergochat/readline v0.1.3` - Readline support
|
||||
|
||||
**Utilities:**
|
||||
- `spf13/cobra v1.10.2` - CLI framework
|
||||
- `caarlos0/env/v11 v11.4.0` - Environment variable parsing with struct tags
|
||||
- `BurntSushi/toml v1.6.0` - TOML parsing
|
||||
- `adhocore/gronx v1.19.6` - Cron expression parsing
|
||||
- `google/uuid v1.6.0` - UUID generation
|
||||
- `h2non/filetype v1.1.3` - File type detection
|
||||
- `mdP/qrterminal/v3 v3.2.1` - QR code terminal output
|
||||
- `minio/selfupdate v0.6.0` - Binary self-updates
|
||||
- `creack/pty v1.1.24` - PTY allocation (shell tool)
|
||||
- `gomarkdown/markdown` - Markdown parsing
|
||||
|
||||
## Database/Storage
|
||||
|
||||
**Primary:**
|
||||
- **SQLite** via `modernc.org/sqlite v1.48.2` (pure Go, no CGO dependency) - Used by dashboard auth store (`web/backend/dashboardauth/sql.go`)
|
||||
- **JSONL files** (`pkg/memory/jsonl.go`) - Session memory storage, append-only JSONL format with per-session metadata files
|
||||
- **JSON files** - Config (`config.json`), auth store (`auth.json`), state (`state/state.json`)
|
||||
|
||||
**No external database server required** - all storage is file-based.
|
||||
|
||||
## Authentication Mechanisms
|
||||
|
||||
1. **API Key auth** - Per-model `api_keys` in config (supports multiple keys for failover, `SecureString` wrapper)
|
||||
2. **OAuth/PKCE** (`pkg/auth/`) - Anthropic and OpenAI OAuth login flows with token refresh
|
||||
3. **Platform-specific tokens** - Each channel has its own token/secret (Telegram bot token, Discord token, Feishu app secret, etc.)
|
||||
4. **Dashboard auth** - Launcher token-based (`PICOCLAW_LAUNCHER_TOKEN` env var) for web console
|
||||
5. **WeCom/Weixin** - Custom OAuth flows (`pkg/auth/wecom.go`, `pkg/auth/weixin.go`)
|
||||
|
||||
## Configuration
|
||||
|
||||
**Method:**
|
||||
- JSON config file (`~/.picoclaw/config.json`)
|
||||
- Environment variables with `env:` struct tags (e.g., `PICOCLAW_CHANNELS_TELEGRAM_TOKEN`)
|
||||
- `pkg/config/` handles loading, validation, and secure string masking
|
||||
|
||||
**Key directories:**
|
||||
- `~/.picoclaw/` - Home directory (config, auth, workspace)
|
||||
- `~/.picoclaw/workspace/` - Workspace (state, skills, session memory)
|
||||
- `~/.picoclaw/logs/` - Log files
|
||||
|
||||
## Build System
|
||||
|
||||
**Makefile targets** (key ones):
|
||||
- `make build` - Build for current platform (runs `go generate` first)
|
||||
- `make build-all` - Cross-compile for 10+ platforms
|
||||
- `make build-launcher` - Build web console binary
|
||||
- `make build-launcher-tui` - Build terminal UI binary
|
||||
- `make build-whatsapp-native` - Build with native WhatsApp support (larger binary)
|
||||
- `make build-linux-arm` / `build-linux-arm64` / `build-linux-mipsle` / `build-pi-zero` - Embedded targets
|
||||
- `make test` - Run all tests
|
||||
- `make lint` / `make fmt` / `make vet` - Code quality
|
||||
- `make docker-build` / `docker-build-full` - Docker images
|
||||
- `make install` / `make uninstall` - Local install to `~/.local/bin`
|
||||
|
||||
**Build tags:** `goolm,stdjson` (default); `whatsapp_native` for native WhatsApp
|
||||
|
||||
**Supported platforms:** linux/amd64, linux/arm, linux/arm64, linux/loong64, linux/riscv64, linux/mipsle, darwin/arm64, windows/amd64, netbsd/amd64, netbsd/arm64
|
||||
|
||||
## Platform Requirements
|
||||
|
||||
**Development:**
|
||||
- Go 1.25.9+
|
||||
- Node.js 20+ (for frontend builds)
|
||||
- pnpm (for frontend dependencies)
|
||||
- golangci-lint (for linting)
|
||||
- Make
|
||||
|
||||
**Production:**
|
||||
- Single static binary (no runtime dependencies when CGO_ENABLED=0)
|
||||
- Alpine 3.23 (minimal Docker image)
|
||||
- Node.js 24 (full Docker image for MCP tool support)
|
||||
|
||||
---
|
||||
|
||||
*Stack analysis: 2026-04-10*
|
||||
265
.planning/codebase/STRUCTURE.md
Normal file
265
.planning/codebase/STRUCTURE.md
Normal file
|
|
@ -0,0 +1,265 @@
|
|||
# Codebase Structure
|
||||
|
||||
**Analysis Date:** 2026-04-10
|
||||
|
||||
## Directory Layout
|
||||
|
||||
```
|
||||
picoclaw/
|
||||
├── cmd/ # CLI entry points
|
||||
│ ├── picoclaw/ # Main CLI agent
|
||||
│ │ ├── main.go # CLI root command
|
||||
│ │ └── internal/ # CLI subcommand implementations
|
||||
│ │ ├── agent/ # `picoclaw agent` command
|
||||
│ │ ├── auth/ # `picoclaw auth` command
|
||||
│ │ ├── cron/ # `picoclaw cron` subcommands
|
||||
│ │ ├── gateway/ # `picoclaw gateway` command
|
||||
│ │ ├── migrate/ # `picoclaw migrate` command
|
||||
│ │ ├── model/ # `picoclaw model` command
|
||||
│ │ ├── onboard/ # `picoclaw onboard` command
|
||||
│ │ ├── skills/ # `picoclaw skills` subcommands
|
||||
│ │ ├── status/ # `picoclaw status` command
|
||||
│ │ └── version/ # `picoclaw version` command
|
||||
│ ├── picoclaw-launcher-tui/ # Terminal UI launcher
|
||||
│ │ ├── main.go
|
||||
│ │ ├── ui/ # TUI screen components
|
||||
│ │ └── config/ # TUI configuration
|
||||
│ └── membench/ # Memory/performance benchmarking
|
||||
├── pkg/ # Shared libraries (all reusable packages)
|
||||
│ ├── agent/ # Agent loop, instances, registry
|
||||
│ ├── audio/ # Audio processing (ASR/TTS)
|
||||
│ │ ├── asr/ # Speech recognition (Whisper, ElevenLabs, etc.)
|
||||
│ │ └── tts/ # Text-to-speech (OpenAI, Mimo, etc.)
|
||||
│ ├── auth/ # OAuth, PKCE, token management
|
||||
│ ├── bus/ # Message bus (core async messaging)
|
||||
│ ├── channels/ # Messaging platform adapters
|
||||
│ │ ├── dingtalk/ # DingTalk (钉钉) integration
|
||||
│ │ ├── discord/ # Discord bot + voice
|
||||
│ │ ├── feishu/ # Feishu/Lark integration
|
||||
│ │ ├── irc/ # IRC client
|
||||
│ │ ├── line/ # LINE messaging
|
||||
│ │ ├── maixcam/ # MaixCam device channel
|
||||
│ │ ├── matrix/ # Matrix protocol
|
||||
│ │ ├── onebot/ # OneBot protocol (QQ)
|
||||
│ │ ├── pico/ # Built-in WebSocket channel (for web UI)
|
||||
│ │ ├── qq/ # QQ Bot
|
||||
│ │ ├── slack/ # Slack integration
|
||||
│ │ ├── teams_webhook/ # Microsoft Teams webhook
|
||||
│ │ ├── telegram/ # Telegram bot
|
||||
│ │ ├── vk/ # VK (ВКонтакте)
|
||||
│ │ ├── wecom/ # WeChat Work (企业微信)
|
||||
│ │ ├── weixin/ # WeChat (微信)
|
||||
│ │ ├── whatsapp/ # WhatsApp (web-based)
|
||||
│ │ └── whatsapp_native/ # WhatsApp native integration
|
||||
│ ├── commands/ # Command definition and registry
|
||||
│ ├── config/ # Configuration loading, validation, migration
|
||||
│ ├── constants/ # Shared constants
|
||||
│ ├── credential/ # Secure credential storage
|
||||
│ ├── cron/ # Cron job scheduler
|
||||
│ ├── devices/ # Device management (hardware I/O)
|
||||
│ ├── fileutil/ # File system utilities
|
||||
│ ├── gateway/ # Gateway runtime orchestrator
|
||||
│ ├── health/ # Health check server
|
||||
│ ├── heartbeat/ # Heartbeat service
|
||||
│ ├── identity/ # User identity management
|
||||
│ ├── isolation/ # Process isolation/sandboxing
|
||||
│ ├── logger/ # Structured logging
|
||||
│ ├── mcp/ # MCP (Model Context Protocol) support
|
||||
│ ├── media/ # Media file storage
|
||||
│ ├── memory/ # Long-term conversation memory (JSONL)
|
||||
│ ├── migrate/ # Config/data migration
|
||||
│ ├── pid/ # PID file management
|
||||
│ ├── providers/ # LLM provider implementations
|
||||
│ │ ├── anthropic/ # Anthropic API (Claude)
|
||||
│ │ ├── anthropic_messages/ # Anthropic Messages API
|
||||
│ │ ├── azure/ # Azure OpenAI
|
||||
│ │ ├── bedrock/ # AWS Bedrock
|
||||
│ │ ├── openai_compat/ # OpenAI-compatible providers
|
||||
│ │ └── common/ # Shared provider utilities
|
||||
│ ├── routing/ # Model routing (smart model selection)
|
||||
│ ├── seahorse/ # Context compression engine (FTS5-based)
|
||||
│ ├── session/ # Session management (JSONL backend)
|
||||
│ ├── skills/ # Skill system (agent capabilities)
|
||||
│ ├── state/ # State management
|
||||
│ ├── tokenizer/ # Token counting/estimation
|
||||
│ ├── tools/ # Tool implementations
|
||||
│ ├── updater/ # Self-update mechanism
|
||||
│ └── utils/ # General utilities
|
||||
├── web/
|
||||
│ ├── backend/ # Web launcher backend (Go)
|
||||
│ │ ├── main.go # Launcher entry point
|
||||
│ │ ├── api/ # REST API handlers
|
||||
│ │ │ ├── router.go # Route registration
|
||||
│ │ │ ├── channels.go # Channel CRUD
|
||||
│ │ │ ├── config.go # Config management
|
||||
│ │ │ ├── gateway.go # Gateway start/stop/logs
|
||||
│ │ │ ├── models.go # Model list management
|
||||
│ │ │ ├── oauth.go # OAuth flow handlers
|
||||
│ │ │ ├── pico.go # WebSocket chat endpoint
|
||||
│ │ │ ├── session.go # Session history API
|
||||
│ │ │ ├── skills.go # Skills management
|
||||
│ │ │ └── tools.go # Tool actions
|
||||
│ │ ├── dashboardauth/ # Dashboard authentication
|
||||
│ │ ├── launcherconfig/ # Launcher-specific config
|
||||
│ │ ├── middleware/ # HTTP middleware (auth, access control)
|
||||
│ │ ├── model/ # Status models
|
||||
│ │ └── utils/ # Backend utilities
|
||||
│ └── frontend/ # Web dashboard UI (React + TypeScript)
|
||||
│ ├── src/
|
||||
│ │ ├── api/ # API client layer
|
||||
│ │ ├── components/ # Reusable UI components
|
||||
│ │ │ ├── agent/ # Agent-related components
|
||||
│ │ │ │ ├── hub/ # Agent Hub marketplace
|
||||
│ │ │ │ ├── skills/ # Skills display
|
||||
│ │ │ │ └── tools/ # Tool configuration
|
||||
│ │ │ ├── channels/ # Channel management
|
||||
│ │ │ ├── chat/ # Chat UI components
|
||||
│ │ │ ├── config/ # Configuration forms
|
||||
│ │ │ ├── credentials/ # Credential management
|
||||
│ │ │ ├── logs/ # Log viewer
|
||||
│ │ │ ├── models/ # Model selector
|
||||
│ │ │ ├── tour/ # Onboarding tour
|
||||
│ │ │ └── ui/ # Base UI components (shadcn)
|
||||
│ │ ├── features/ # Feature modules
|
||||
│ │ │ └── chat/ # Chat feature (controller, state, protocol)
|
||||
│ │ ├── hooks/ # React custom hooks
|
||||
│ │ ├── i18n/ # Internationalization
|
||||
│ │ │ └── locales/ # Locale JSON files
|
||||
│ │ ├── lib/ # Utility libraries
|
||||
│ │ ├── routes/ # TanStack Router routes
|
||||
│ │ │ ├── agent/ # Agent management page
|
||||
│ │ │ └── channels/ # Channels management page
|
||||
│ │ └── store/ # Jotai stores (state management)
|
||||
│ └── public/ # Static assets
|
||||
├── workspace/
|
||||
│ ├── memory/ # Agent memory templates
|
||||
│ └── skills/ # Built-in skill definitions
|
||||
│ ├── agent-browser/ # Browser automation skill
|
||||
│ ├── github/ # GitHub integration skill
|
||||
│ ├── hardware/ # Hardware control skill
|
||||
│ ├── skill-creator/ # Skill creation helper
|
||||
│ ├── summarize/ # Conversation summarization
|
||||
│ ├── tmux/ # tmux session management
|
||||
│ └── weather/ # Weather lookup
|
||||
├── docs/ # Documentation (multi-language)
|
||||
│ ├── design/ # Design documents
|
||||
│ ├── zh/ # Chinese docs
|
||||
│ ├── ja/ # Japanese docs
|
||||
│ ├── pt-br/ # Portuguese (BR) docs
|
||||
│ └── vi/ # Vietnamese docs
|
||||
└── config/ # Example configuration files
|
||||
```
|
||||
|
||||
## Directory Purposes
|
||||
|
||||
**`cmd/`:** CLI binary entry points. Each subdirectory produces a separate binary.
|
||||
|
||||
**`pkg/`:** Shared Go packages. All business logic lives here. Packages are designed for reuse across binaries.
|
||||
|
||||
**`cmd/picoclaw/internal/`:** CLI subcommand implementations. Thin wrappers around `pkg/` packages with Cobra integration.
|
||||
|
||||
**`web/backend/`:** Desktop launcher backend. Embeds frontend assets and provides HTTP API + gateway management.
|
||||
|
||||
**`web/frontend/`:** React + TypeScript dashboard. Built with Vite, TanStack Router, shadcn/ui. Output embedded into Go binary.
|
||||
|
||||
**`workspace/`:** Runtime workspace. Memory templates, skill definitions, agent-specific data. Copied to `~/.picoclaw/` on first run.
|
||||
|
||||
## Key File Locations
|
||||
|
||||
**Entry Points:**
|
||||
- `cmd/picoclaw/main.go`: CLI root (Cobra-based subcommands)
|
||||
- `web/backend/main.go`: Desktop launcher (HTTP server + system tray)
|
||||
- `pkg/gateway/gateway.go`: Core gateway runtime (agent loops, channels, services)
|
||||
- `cmd/picoclaw-launcher-tui/main.go`: Terminal UI launcher
|
||||
|
||||
**Configuration:**
|
||||
- `pkg/config/config.go`: Config loading and environment variable integration
|
||||
- `pkg/config/config_struct.go`: Config type definitions
|
||||
- `web/backend/launcherconfig/config.go`: Launcher-specific settings
|
||||
|
||||
**Core Logic:**
|
||||
- `pkg/agent/loop.go`: Main agent event loop
|
||||
- `pkg/agent/instance.go`: Agent instance with provider, tools, sessions
|
||||
- `pkg/agent/turn.go`: Turn execution (LLM call + tool loop)
|
||||
- `pkg/agent/registry.go`: Multi-agent management
|
||||
- `pkg/bus/bus.go`: Message bus (inbound/outbound/media/audio/voice)
|
||||
- `pkg/channels/manager.go`: Channel lifecycle and message routing
|
||||
- `pkg/channels/interfaces.go`: Capability interfaces (streaming, typing, reactions)
|
||||
- `pkg/tools/registry.go`: Tool registration and execution
|
||||
|
||||
**Routing & API:**
|
||||
- `web/backend/api/router.go`: All API route registration
|
||||
- `web/frontend/src/routes/`: TanStack Router route definitions
|
||||
- `web/frontend/src/routeTree.gen.ts`: Auto-generated route tree
|
||||
|
||||
**Testing:**
|
||||
- Co-located `*_test.go` files alongside source files throughout `pkg/` and `cmd/`
|
||||
|
||||
## Naming Conventions
|
||||
|
||||
**Files:**
|
||||
- Go: snake_case for test files (`context_budget_test.go`), CamelCase for implementation files (`context_budget.go`)
|
||||
- TypeScript: kebab-case for components (`channel-config-fields.ts`), camelCase for hooks (`use-chat-models.ts`)
|
||||
- Routes: kebab-case (`launcher-login.tsx`)
|
||||
|
||||
**Directories:**
|
||||
- Go packages: lowercase, single word where possible (`agent`, `channels`, `tools`)
|
||||
- Frontend features: kebab-case (`features/chat/`)
|
||||
- Component groups: kebab-case (`components/agent/hub/`)
|
||||
|
||||
**Functions:**
|
||||
- Go: PascalCase for exported, camelCase for unexported
|
||||
- Structured logging: `InfoCF`, `ErrorCF`, `DebugCF` (component + field variants)
|
||||
|
||||
## Where to Add New Code
|
||||
|
||||
**New Messaging Channel:**
|
||||
- Implementation: `pkg/channels/<channel_name>/` (with `init.go` for self-registration)
|
||||
- Register: Import with blank identifier in `pkg/gateway/gateway.go`
|
||||
- Capability interfaces: `pkg/channels/interfaces.go`
|
||||
|
||||
**New LLM Provider:**
|
||||
- Implementation: `pkg/providers/<provider_name>/`
|
||||
- Factory: Register in `pkg/providers/factory.go`
|
||||
- Test: `<provider_name>_test.go`
|
||||
|
||||
**New Tool:**
|
||||
- Implementation: `pkg/tools/<tool_name>.go`
|
||||
- Register: Add to tool list in `pkg/gateway/gateway.go` or agent loop setup
|
||||
|
||||
**New Skill:**
|
||||
- Definition: `workspace/skills/<skill_name>/` (SKILL.md + references)
|
||||
|
||||
**New CLI Subcommand:**
|
||||
- Implementation: `cmd/picoclaw/internal/<subcommand>/`
|
||||
- Register: Add to `cmd/picoclaw/main.go` command list
|
||||
|
||||
**New API Endpoint:**
|
||||
- Handler: `web/backend/api/<resource>.go`
|
||||
- Route: Register in `web/backend/api/router.go`
|
||||
|
||||
**New Frontend Page:**
|
||||
- Route: `web/frontend/src/routes/<page_name>.tsx`
|
||||
- API client: `web/frontend/src/api/<resource>.ts`
|
||||
- Hook: `web/frontend/src/hooks/use-<feature>.ts`
|
||||
|
||||
## Special Directories
|
||||
|
||||
**`web/backend/dist/`:**
|
||||
- Purpose: Compiled frontend assets (embedded into Go binary)
|
||||
- Generated: Yes (by `npm run build:backend`)
|
||||
- Committed: Yes (for self-contained Go builds)
|
||||
|
||||
**`workspace/`:**
|
||||
- Purpose: Default skill and memory templates copied to user home on first run
|
||||
- Generated: No (hand-authored)
|
||||
- Committed: Yes
|
||||
|
||||
**`docs/`:**
|
||||
- Purpose: Project documentation in multiple languages
|
||||
- Languages: English (root), zh, ja, pt-br, vi, my
|
||||
- Committed: Yes
|
||||
|
||||
---
|
||||
|
||||
*Structure analysis: 2026-04-10*
|
||||
169
.planning/codebase/TESTING.md
Normal file
169
.planning/codebase/TESTING.md
Normal file
|
|
@ -0,0 +1,169 @@
|
|||
# Testing Patterns
|
||||
|
||||
**Analysis Date:** 2026-04-10
|
||||
|
||||
## Test Framework
|
||||
|
||||
**Runner:** Go 标准 `testing` 包。
|
||||
|
||||
**Assertion libraries:**
|
||||
- `github.com/stretchr/testify/assert` -- 非致命断言
|
||||
- `github.com/stretchr/testify/require` -- 致命断言
|
||||
|
||||
**无外部测试框架** -- 纯 `testing` + `testify`。
|
||||
|
||||
## 运行测试
|
||||
|
||||
```bash
|
||||
make test # 运行所有测试
|
||||
make check # deps + fmt + vet + test
|
||||
go test -run TestName -v ./pkg/session/ # 运行单个测试
|
||||
go test -bench=. -benchmem -run='^$' ./... # 仅运行 benchmark
|
||||
cd web && make test # Web backend 测试
|
||||
go test -tags goolm,stdjson ./... # CI 测试命令
|
||||
```
|
||||
|
||||
**必需的 Build tags:** `goolm,stdjson`
|
||||
|
||||
## 测试文件组织
|
||||
|
||||
- **位置:** 与源码同目录同包（非 `_test` 包），`.golangci.yaml` 第25行禁用了 `testpackage`
|
||||
- **示例:** `pkg/seahorse/store_test.go` (`package seahorse`), `cmd/picoclaw/internal/auth/command_test.go` (`package auth`)
|
||||
- **命名:** `*_test.go` 后缀
|
||||
- **数量:** 约 240 个测试文件
|
||||
|
||||
## 测试结构模式
|
||||
|
||||
**表驱动测试** (标准模式):
|
||||
```go
|
||||
func TestShouldEnableLauncherFileLogging(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
enableConsole bool
|
||||
debug bool
|
||||
want bool
|
||||
}{
|
||||
{name: "gui mode", enableConsole: false, debug: false, want: true},
|
||||
{name: "console mode", enableConsole: true, debug: false, want: false},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
if got := fn(tt.enableConsole, tt.debug); got != tt.want {
|
||||
t.Fatalf("... = %t, want %t", got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Cobra 命令测试** (`cmd/picoclaw/internal/*/command_test.go`):
|
||||
```go
|
||||
func TestNewAuthCommand(t *testing.T) {
|
||||
cmd := NewAuthCommand()
|
||||
require.NotNil(t, cmd)
|
||||
assert.Equal(t, "auth", cmd.Use)
|
||||
allowedCommands := []string{"login", "logout", "status", "models", "weixin", "wecom"}
|
||||
subcommands := cmd.Commands()
|
||||
assert.Len(t, subcommands, len(allowedCommands))
|
||||
for _, subcmd := range subcommands {
|
||||
found := slices.Contains(allowedCommands, subcmd.Name())
|
||||
assert.True(t, found, "unexpected subcommand %q", subcmd.Name())
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**HTTP 处理器测试** (`web/backend/api/*_test.go`):
|
||||
```go
|
||||
mux := http.NewServeMux()
|
||||
RegisterLauncherAuthRoutes(mux, LauncherAuthRouteOpts{DashboardToken: tok, SessionCookie: sess})
|
||||
t.Run("status_unauthenticated", func(t *testing.T) {
|
||||
rec := httptest.NewRecorder()
|
||||
mux.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/api/auth/status", nil))
|
||||
if rec.Code != http.StatusOK { t.Fatalf("status code = %d", rec.Code) }
|
||||
})
|
||||
```
|
||||
|
||||
**数据库测试** (`pkg/seahorse/store_test.go`):
|
||||
```go
|
||||
func openTestStore(t *testing.T) *Store {
|
||||
t.Helper()
|
||||
db := openTestDB(t)
|
||||
if err := runSchema(db); err != nil { t.Fatalf("migration: %v", err) }
|
||||
return &Store{db: db}
|
||||
}
|
||||
```
|
||||
|
||||
## Mock 策略
|
||||
|
||||
**手写 mock 结构体**，无 gomock/mockgen/testify/mock。
|
||||
|
||||
**`pkg/tools/subagent_tool_test.go`:**
|
||||
```go
|
||||
type MockLLMProvider struct { lastOptions map[string]any }
|
||||
func (m *MockLLMProvider) Chat(ctx, messages, tools, model, options) (*providers.LLMResponse, error) {
|
||||
m.lastOptions = options
|
||||
return &providers.LLMResponse{Content: "Task completed"}, nil
|
||||
}
|
||||
```
|
||||
|
||||
**回调函数 mock** (`pkg/seahorse/short_compaction_test.go`):
|
||||
```go
|
||||
var mockCompleteFn CompleteFn = func(ctx, prompt, opts) (string, error) {
|
||||
return "Mock summary of the conversation segment.", nil
|
||||
}
|
||||
```
|
||||
|
||||
**Mock 对象:** LLM providers, CompleteFn, 时间 (`l.now = func() time.Time { return t0 }`)
|
||||
**不 Mock 对象:** 数据库 (使用 in-memory SQLite), HTTP handlers (`httptest`), 文件系统 (`t.TempDir()`)
|
||||
|
||||
## Fixtures 和测试数据
|
||||
|
||||
- 内存数据库: `openTestDB(t)` + `runSchema(db)`, 每个测试独立
|
||||
- 环境变量: `t.Setenv()` 自动清理
|
||||
- Testdata: `pkg/channels/telegram/testdata/md2_all_formats.txt`
|
||||
- 测试常量: 硬编码在测试文件中
|
||||
|
||||
## Benchmark 测试
|
||||
|
||||
**位置:** `cmd/membench/` 和 `pkg/seahorse/short_bench_test.go`
|
||||
|
||||
**模式:**
|
||||
```go
|
||||
func newBenchStore(b *testing.B) (*Store, func()) {
|
||||
b.Helper()
|
||||
db, err := sql.Open("sqlite", ":memory:")
|
||||
if err != nil { b.Fatalf("open test db: %v", err) }
|
||||
if err := runSchema(db); err != nil { db.Close(); b.Fatalf("migration: %v", err) }
|
||||
return &Store{db: db}, func() { db.Close() }
|
||||
}
|
||||
func BenchmarkIngest_SingleMessage(b *testing.B) {
|
||||
s, cleanup := newBenchStore(b); defer cleanup()
|
||||
b.ResetTimer()
|
||||
for i := 0; i < b.N; i++ { s.AddMessage(ctx, convID, "user", "Test", 15) }
|
||||
}
|
||||
```
|
||||
|
||||
**运行:** `go test -bench=. -benchmem -run='^$' ./pkg/seahorse/`
|
||||
|
||||
## CI 配置
|
||||
|
||||
**PR workflow** (`.github/workflows/pr.yml`):
|
||||
- Lint: `golangci-lint-action@v9` (v2.10.1), `--build-tags=goolm,stdjson`
|
||||
- 安全检查: `govulncheck`
|
||||
- 测试: `go test -tags goolm,stdjson ./...`
|
||||
- 所有任务在 `ubuntu-latest` 上运行
|
||||
|
||||
**无覆盖率报告** -- CI 中没有 codecov 或 `-coverprofile`
|
||||
|
||||
## 通用模式
|
||||
|
||||
- `t.Helper()` 在测试工具函数中
|
||||
- `t.TempDir()` 用于临时目录
|
||||
- `t.Fatalf()` 致命错误, `t.Errorf()` 断言失败
|
||||
- OS 特定跳过: `t.Skip("user environment variables only apply on Linux")`
|
||||
- 并发测试: 直接操作互斥锁 (`manager.mu.Lock()`)
|
||||
- 错误测试: 检查 `err == nil` 或 `result.IsError`
|
||||
|
||||
---
|
||||
|
||||
*Testing analysis: 2026-04-10*
|
||||
Loading…
Add table
Reference in a new issue