diff --git a/.env.example b/.env.example index e0a07236e..66010b1f5 100644 --- a/.env.example +++ b/.env.example @@ -5,6 +5,7 @@ # ANTHROPIC_API_KEY=sk-ant-xxx # OPENAI_API_KEY=sk-xxx # GEMINI_API_KEY=xxx +# MODELSCOPE_API_KEY=xxx # CLAUDE_CODE_OAUTH=xxx # ── Chat Channel ────────────────────────── # TELEGRAM_BOT_TOKEN=123456:ABC... diff --git a/.goreleaser.yaml b/.goreleaser.yaml index 622cf054b..a73f87f30 100644 --- a/.goreleaser.yaml +++ b/.goreleaser.yaml @@ -27,6 +27,7 @@ builds: - windows - darwin - freebsd + - netbsd goarch: - amd64 - arm64 @@ -44,6 +45,12 @@ builds: ignore: - goos: windows goarch: arm + - goos: netbsd + goarch: s390x + - goos: netbsd + goarch: mips64 + - goos: netbsd + goarch: arm - id: picoclaw-launcher binary: picoclaw-launcher @@ -58,6 +65,7 @@ builds: - windows - darwin - freebsd + - netbsd goarch: - amd64 - arm64 @@ -75,6 +83,12 @@ builds: ignore: - goos: windows goarch: arm + - goos: netbsd + goarch: s390x + - goos: netbsd + goarch: mips64 + - goos: netbsd + goarch: arm - id: picoclaw-launcher-tui binary: picoclaw-launcher-tui @@ -89,6 +103,7 @@ builds: - windows - darwin - freebsd + - netbsd goarch: - amd64 - arm64 @@ -106,6 +121,12 @@ builds: ignore: - goos: windows goarch: arm + - goos: netbsd + goarch: s390x + - goos: netbsd + goarch: mips64 + - goos: netbsd + goarch: arm dockers_v2: - id: picoclaw diff --git a/Makefile b/Makefile index 98642703f..2f673d3b9 100644 --- a/Makefile +++ b/Makefile @@ -181,6 +181,8 @@ build-all: generate GOOS=linux GOARCH=arm GOARM=7 $(GO) build $(LDFLAGS) -o $(BUILD_DIR)/$(BINARY_NAME)-linux-armv7 ./$(CMD_DIR) GOOS=darwin GOARCH=arm64 $(GO) build $(LDFLAGS) -o $(BUILD_DIR)/$(BINARY_NAME)-darwin-arm64 ./$(CMD_DIR) GOOS=windows GOARCH=amd64 $(GO) build $(LDFLAGS) -o $(BUILD_DIR)/$(BINARY_NAME)-windows-amd64.exe ./$(CMD_DIR) + GOOS=netbsd GOARCH=amd64 $(GO) build $(LDFLAGS) -o $(BUILD_DIR)/$(BINARY_NAME)-netbsd-amd64 ./$(CMD_DIR) + GOOS=netbsd GOARCH=arm64 $(GO) build $(LDFLAGS) -o $(BUILD_DIR)/$(BINARY_NAME)-netbsd-arm64 ./$(CMD_DIR) @echo "All builds complete" ## install: Install picoclaw to system and copy builtin skills diff --git a/README.fr.md b/README.fr.md index 82e587f75..d5fe873bf 100644 --- a/README.fr.md +++ b/README.fr.md @@ -4,14 +4,18 @@

PicoClaw : Assistant IA Ultra-Efficace en Go

Matériel à 10$ · 10 Mo de RAM · Démarrage en 1s · 皮皮虾，我们走！

-

Go Hardware License
Website + Docs + Wiki +
Twitter + + Discord

[中文](README.zh.md) | [日本語](README.ja.md) | [Português](README.pt-br.md) | [Tiếng Việt](README.vi.md) | [English](README.md) | **Français** @@ -222,7 +226,8 @@ picoclaw onboard { "model_name": "ark-code-latest", "model": "volcengine/ark-code-latest", - "api_key": "sk-your-api-key" + "api_key": "sk-your-api-key", + "api_base":"https://ark.cn-beijing.volces.com/api/coding/v3" }, { "model_name": "gpt-5.4", @@ -985,6 +990,7 @@ Cette conception permet également le **support multi-agent** avec une sélectio | **ShengsuanYun** | `shengsuanyun/` | `https://router.shengsuanyun.com/api/v1` | OpenAI | - | | **BytePlus** | `byteplus/` | `https://ark.ap-southeast.bytepluses.com/api/v3` | OpenAI | [Obtenir Clé](https://www.byteplus.com/) | | **LongCat** | `longcat/` | `https://api.longcat.chat/openai` | OpenAI | [Obtenir une clé](https://longcat.chat/platform) | +| **ModelScope (魔搭)**| `modelscope/` | `https://api-inference.modelscope.cn/v1` | OpenAI | [Obtenir un Token](https://modelscope.cn/my/tokens) | | **Antigravity** | `antigravity/` | Google Cloud | Custom | OAuth uniquement | | **GitHub Copilot** | `github-copilot/` | `localhost:4321` | gRPC | - | @@ -1223,6 +1229,7 @@ Cela se produit lorsqu'une autre instance du bot est en cours d'exécution. Assu | **Zhipu** | 200K tokens/mois | Convient aux utilisateurs chinois | | **Brave Search** | 2000 requêtes/mois | Fonctionnalité de recherche web | | **Groq** | Offre gratuite dispo | Inférence ultra-rapide (Llama, Mixtral) | +| **ModelScope** | 2000 requêtes/jour | Inférence gratuite (Qwen, GLM, DeepSeek, etc.) | --- diff --git a/README.ja.md b/README.ja.md index ad4a86505..7fff46d13 100644 --- a/README.ja.md +++ b/README.ja.md @@ -5,12 +5,19 @@

$10 ハードウェア · 10MB RAM · 1秒起動 · 行くぜ、シャコ！

- -

-Go -Hardware -License -

+

+ Go + Hardware + License +
+ Website + Docs + Wiki +
+ Twitter + + Discord +

[中文](README.zh.md) | **日本語** | [Português](README.pt-br.md) | [Tiếng Việt](README.vi.md) | [Français](README.fr.md) | [English](README.md) @@ -184,7 +191,8 @@ picoclaw onboard { "model_name": "ark-code-latest", "model": "volcengine/ark-code-latest", - "api_key": "sk-your-api-key" + "api_key": "sk-your-api-key", + "api_base":"https://ark.cn-beijing.volces.com/api/coding/v3" }, { "model_name": "gpt-5.4", @@ -926,6 +934,7 @@ HEARTBEAT_OK 応答 ユーザーが直接結果を受け取る | **ShengsuanYun** | `shengsuanyun/` | `https://router.shengsuanyun.com/api/v1` | OpenAI | - | | **BytePlus** | `byteplus/` | `https://ark.ap-southeast.bytepluses.com/api/v3` | OpenAI | [キーを取得](https://www.byteplus.com) | | **LongCat** | `longcat/` | `https://api.longcat.chat/openai` | OpenAI | [キーを取得](https://longcat.chat/platform) | +| **ModelScope (魔搭)**| `modelscope/` | `https://api-inference.modelscope.cn/v1` | OpenAI | [トークンを取得](https://modelscope.cn/my/tokens) | | **Antigravity** | `antigravity/` | Google Cloud | カスタム | OAuthのみ | | **GitHub Copilot** | `github-copilot/` | `localhost:4321` | gRPC | - | @@ -1146,6 +1155,7 @@ Web 検索を有効にするには： | **Tavily** | 月 1000 クエリ | AI エージェント検索最適化 | | **Groq** | 無料枠あり | 高速推論（Llama, Mixtral） | | **Cerebras** | 無料枠あり | 高速推論（Llama, Qwen など） | +| **ModelScope** | 1 日 2000 リクエスト | 無料推論（Qwen, GLM, DeepSeek など） | --- diff --git a/README.md b/README.md index b816b7f52..e64daf0e4 100644 --- a/README.md +++ b/README.md @@ -4,15 +4,16 @@

PicoClaw: Ultra-Efficient AI Assistant in Go

$10 Hardware · 10MB RAM · 1s Boot · 皮皮虾，我们走！

-

Go Hardware License
Website - Twitter + Docs + Wiki
+ Twitter Discord

@@ -252,7 +253,8 @@ picoclaw onboard { "model_name": "ark-code-latest", "model": "volcengine/ark-code-latest", - "api_key": "sk-your-api-key" + "api_key": "sk-your-api-key", + "api_base":"https://ark.cn-beijing.volces.com/api/coding/v3" }, { "model_name": "gpt-5.4", @@ -1039,6 +1041,7 @@ This design also enables **multi-agent support** with flexible provider selectio | **BytePlus** | `byteplus/` | `https://ark.ap-southeast.bytepluses.com/api/v3` | OpenAI | [Get Key](https://www.byteplus.com) | | **Vivgrid** | `vivgrid/` | `https://api.vivgrid.com/v1` | OpenAI | [Get Key](https://vivgrid.com) | | **LongCat** | `longcat/` | `https://api.longcat.chat/openai` | OpenAI | [Get Key](https://longcat.chat/platform) | +| **ModelScope (魔搭)**| `modelscope/` | `https://api-inference.modelscope.cn/v1` | OpenAI | [Get Token](https://modelscope.cn/my/tokens) | | **Antigravity** | `antigravity/` | Google Cloud | Custom | OAuth only | | **GitHub Copilot** | `github-copilot/` | `localhost:4321` | gRPC | - | @@ -1130,6 +1133,26 @@ This design also enables **multi-agent support** with flexible provider selectio > Run `picoclaw auth login --provider anthropic` to paste your API token. +**Anthropic Messages API (native format)** + +For direct Anthropic API access or custom endpoints that only support Anthropic's native message format: + +```json +{ + "model_name": "claude-opus-4-6", + "model": "anthropic-messages/claude-opus-4-6", + "api_key": "sk-ant-your-key", + "api_base": "https://api.anthropic.com" +} +``` + +> Use `anthropic-messages` protocol when: +> - Using third-party proxies that only support Anthropic's native `/v1/messages` endpoint (not OpenAI-compatible `/v1/chat/completions`) +> - Connecting to services like MiniMax, Synthetic that require Anthropic's native message format +> - The existing `anthropic` protocol returns 404 errors (indicating the endpoint doesn't support OpenAI-compatible format) +> +> **Note:** The `anthropic` protocol uses OpenAI-compatible format (`/v1/chat/completions`), while `anthropic-messages` uses Anthropic's native format (`/v1/messages`). Choose based on your endpoint's supported format. + **Ollama (local)** ```json @@ -1526,6 +1549,7 @@ This happens when another instance of the bot is running. Make sure only one `pi | **Groq** | Free tier available | Fast inference (Llama, Mixtral) | | **Cerebras** | Free tier available | Fast inference (Llama, Qwen, etc.) | | **LongCat** | Up to 5M tokens/day | Fast inference (free tier) | +| **ModelScope** | 2000 requests/day | Free inference (Qwen, GLM, DeepSeek, etc.) | --- diff --git a/README.pt-br.md b/README.pt-br.md index 58c69f4d9..3fe24d7ea 100644 --- a/README.pt-br.md +++ b/README.pt-br.md @@ -4,14 +4,18 @@

PicoClaw: Assistente de IA Ultra-Eficiente em Go

Hardware de $10 · 10MB de RAM · Boot em 1s · 皮皮虾，我们走！

-

Go Hardware License
Website + Docs + Wiki +
Twitter + + Discord

[中文](README.zh.md) | [日本語](README.ja.md) | **Português** | [Tiếng Việt](README.vi.md) | [Français](README.fr.md) | [English](README.md) @@ -223,7 +227,8 @@ picoclaw onboard { "model_name": "ark-code-latest", "model": "volcengine/ark-code-latest", - "api_key": "sk-your-api-key" + "api_key": "sk-your-api-key", + "api_base":"https://ark.cn-beijing.volces.com/api/coding/v3" }, { "model_name": "gpt-5.4", @@ -981,6 +986,7 @@ Este design também possibilita o **suporte multi-agent** com seleção flexíve | **ShengsuanYun** | `shengsuanyun/` | `https://router.shengsuanyun.com/api/v1` | OpenAI | - | | **BytePlus** | `byteplus/` | `https://ark.ap-southeast.bytepluses.com/api/v3` | OpenAI | [Obter Chave](https://www.byteplus.com) | | **LongCat** | `longcat/` | `https://api.longcat.chat/openai` | OpenAI | [Obter Chave](https://longcat.chat/platform) | +| **ModelScope (魔搭)**| `modelscope/` | `https://api-inference.modelscope.cn/v1` | OpenAI | [Obter Token](https://modelscope.cn/my/tokens) | | **Antigravity** | `antigravity/` | Google Cloud | Custom | Apenas OAuth | | **GitHub Copilot** | `github-copilot/` | `localhost:4321` | gRPC | - | @@ -1220,6 +1226,7 @@ Isso acontece quando outra instância do bot está em execução. Certifique-se | **Brave Search** | 2000 consultas/mês | Funcionalidade de busca web | | **Groq** | Plano gratuito disponível | Inferência ultra-rápida (Llama, Mixtral) | | **Cerebras** | Plano gratuito disponível | Inferência ultra-rápida (Llama 3.3 70B) | +| **ModelScope** | 2000 requisições/dia | Inferência gratuita (Qwen, GLM, DeepSeek, etc.) | --- diff --git a/README.vi.md b/README.vi.md index ac8efb900..3ee0209f6 100644 --- a/README.vi.md +++ b/README.vi.md @@ -4,14 +4,18 @@

PicoClaw: Trợ lý AI Siêu Nhẹ viết bằng Go

Phần cứng $10 · RAM 10MB · Khởi động 1 giây · Nào, xuất phát!

-

Go Hardware License
Website + Docs + Wiki +
Twitter + + Discord

[中文](README.zh.md) | [日本語](README.ja.md) | [Português](README.pt-br.md) | **Tiếng Việt** | [Français](README.fr.md) | [English](README.md) @@ -203,7 +207,8 @@ picoclaw onboard { "model_name": "ark-code-latest", "model": "volcengine/ark-code-latest", - "api_key": "sk-your-api-key" + "api_key": "sk-your-api-key", + "api_base":"https://ark.cn-beijing.volces.com/api/coding/v3" }, { "model_name": "gpt-5.4", @@ -950,6 +955,7 @@ Thiết kế này cũng cho phép **hỗ trợ đa tác nhân** với lựa ch | **ShengsuanYun** | `shengsuanyun/` | `https://router.shengsuanyun.com/api/v1` | OpenAI | - | | **BytePlus** | `byteplus/` | `https://ark.ap-southeast.bytepluses.com/api/v3` | OpenAI | [Lấy Khóa](https://www.byteplus.com) | | **LongCat** | `longcat/` | `https://api.longcat.chat/openai` | OpenAI | [Lấy Key](https://longcat.chat/platform) | +| **ModelScope (魔搭)**| `modelscope/` | `https://api-inference.modelscope.cn/v1` | OpenAI | [Lấy Token](https://modelscope.cn/my/tokens) | | **Antigravity** | `antigravity/` | Google Cloud | Tùy chỉnh | Chỉ OAuth | | **GitHub Copilot** | `github-copilot/` | `localhost:4321` | gRPC | - | @@ -1188,6 +1194,7 @@ Một số nhà cung cấp (như Zhipu) có bộ lọc nội dung nghiêm ngặt | **Zhipu** | 200K tokens/tháng | Phù hợp cho người dùng Trung Quốc | | **Brave Search** | 2000 truy vấn/tháng | Chức năng tìm kiếm web | | **Groq** | Có gói miễn phí | Suy luận siêu nhanh (Llama, Mixtral) | +| **ModelScope** | 2000 yêu cầu/ngày | Suy luận miễn phí (Qwen, GLM, DeepSeek, v.v.) | --- diff --git a/README.zh.md b/README.zh.md index f720cf202..66d7c5f7c 100644 --- a/README.zh.md +++ b/README.zh.md @@ -4,14 +4,18 @@

PicoClaw: 基于Go语言的超高效 AI 助手

10$硬件 · 10MB内存 · 1秒启动 · 皮皮虾，我们走！

-

Go Hardware License
Website + Docs + Wiki +
Twitter + + Discord

**中文** | [日本語](README.ja.md) | [Português](README.pt-br.md) | [Tiếng Việt](README.vi.md) | [Français](README.fr.md) | [English](README.md) @@ -234,7 +238,8 @@ picoclaw onboard { "model_name": "ark-code-latest", "model": "volcengine/ark-code-latest", - "api_key": "sk-your-api-key" + "api_key": "sk-your-api-key", + "api_base":"https://ark.cn-beijing.volces.com/api/coding/v3" }, { "model_name": "gpt-5.4", @@ -522,6 +527,7 @@ Agent 读取 HEARTBEAT.md | **神算云** | `shengsuanyun/` | `https://router.shengsuanyun.com/api/v1` | OpenAI | - | | **BytePlus** | `byteplus/` | `https://ark.ap-southeast.bytepluses.com/api/v3` | OpenAI | [获取密钥](https://www.byteplus.com) | | **LongCat** | `longcat/` | `https://api.longcat.chat/openai` | OpenAI | [获取密钥](https://longcat.chat/platform) | +| **ModelScope (魔搭)**| `modelscope/` | `https://api-inference.modelscope.cn/v1` | OpenAI | [获取 Token](https://modelscope.cn/my/tokens) | | **Antigravity** | `antigravity/` | Google Cloud | 自定义 | 仅 OAuth | | **GitHub Copilot** | `github-copilot/` | `localhost:4321` | gRPC | - | @@ -613,6 +619,26 @@ Agent 读取 HEARTBEAT.md > 运行 `picoclaw auth login --provider anthropic` 来设置 OAuth 凭证。 +**Anthropic Messages API（原生格式）** + +用于直接访问 Anthropic API 或仅支持 Anthropic 原生消息格式的自定义端点： + +```json +{ + "model_name": "claude-opus-4-6", + "model": "anthropic-messages/claude-opus-4-6", + "api_key": "sk-ant-your-key", + "api_base": "https://api.anthropic.com" +} +``` + +> 使用 `anthropic-messages` 协议的场景： +> - 使用仅支持 Anthropic 原生 `/v1/messages` 端点的第三方代理（不支持 OpenAI 兼容的 `/v1/chat/completions`） +> - 连接到 MiniMax、Synthetic 等需要 Anthropic 原生消息格式的服务 +> - 现有的 `anthropic` 协议返回 404 错误（说明端点不支持 OpenAI 兼容格式） +> +> **注意：** `anthropic` 协议使用 OpenAI 兼容格式（`/v1/chat/completions`），而 `anthropic-messages` 使用 Anthropic 原生格式（`/v1/messages`）。请根据端点支持的格式选择。 + **Ollama (本地)** ```json @@ -901,6 +927,7 @@ Discord: [https://discord.gg/V4sAZ9XWpN](https://discord.gg/V4sAZ9XWpN) | **Tavily** | 1000 次查询/月 | AI Agent 搜索优化 | | **Groq** | 提供免费层级 | 极速推理 (Llama, Mixtral) | | **LongCat** | 最多 5M tokens/天 | 推理速度快 (免费额度) | +| **ModelScope (魔搭)** | 2000 次请求/天 | 免费推理 (Qwen, GLM, DeepSeek 等) | --- diff --git a/cmd/picoclaw/internal/agent/helpers.go b/cmd/picoclaw/internal/agent/helpers.go index a995945d2..c3ddbb77f 100644 --- a/cmd/picoclaw/internal/agent/helpers.go +++ b/cmd/picoclaw/internal/agent/helpers.go @@ -9,7 +9,7 @@ import ( "path/filepath" "strings" - "github.com/chzyer/readline" + "github.com/ergochat/readline" "github.com/sipeed/picoclaw/cmd/picoclaw/internal" "github.com/sipeed/picoclaw/pkg/agent" diff --git a/cmd/picoclaw/internal/gateway/helpers.go b/cmd/picoclaw/internal/gateway/helpers.go index c9fce8a23..101ca16f5 100644 --- a/cmd/picoclaw/internal/gateway/helpers.go +++ b/cmd/picoclaw/internal/gateway/helpers.go @@ -3,7 +3,6 @@ package gateway import ( "context" "fmt" - "log" "net" "net/http" "os" @@ -12,6 +11,7 @@ import ( "path/filepath" "strconv" "strings" + "sync" "time" "github.com/sipeed/picoclaw/cmd/picoclaw/internal" @@ -50,12 +50,31 @@ import ( "github.com/sipeed/picoclaw/pkg/voice" ) +// Timeout constants for service operations +const ( + serviceRestartTimeout = 30 * time.Second + serviceShutdownTimeout = 30 * time.Second + providerReloadTimeout = 30 * time.Second + gracefulShutdownTimeout = 15 * time.Second +) + +// gatewayServices holds references to all running services +type gatewayServices struct { + CronService *cron.CronService + HeartbeatService *heartbeat.HeartbeatService + MediaStore media.MediaStore + ChannelManager *channels.Manager + DeviceService *devices.Service + HealthServer *health.Server +} + func gatewayCmd(debug bool, orchestration bool, enableStats bool) error { if debug { logger.SetLevel(logger.DEBUG) fmt.Println("🔍 Debug mode enabled") } + configPath := internal.GetConfigPath() cfg, err := internal.LoadConfig() if err != nil { return fmt.Errorf("error loading config: %w", err) @@ -100,9 +119,55 @@ func gatewayCmd(debug bool, orchestration bool, enableStats bool) error { "skills_available": skillsInfo["available"], }) + // Setup and start all services + services, err := setupAndStartServices(cfg, agentLoop, msgBus) + if err != nil { + return err + } + + fmt.Printf("✓ Gateway started on %s:%d\n", cfg.Gateway.Host, cfg.Gateway.Port) + fmt.Println("Press Ctrl+C to stop") + + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + + go agentLoop.Run(ctx) + + // Setup config file watcher for hot reload + configReloadChan, stopWatch := setupConfigWatcherPolling(configPath, debug) + defer stopWatch() + + sigChan := make(chan os.Signal, 1) + signal.Notify(sigChan, os.Interrupt) + + // Main event loop - wait for signals or config changes + for { + select { + case <-sigChan: + logger.Info("Shutting down...") + shutdownGateway(services, agentLoop, provider, true) + return nil + + case newCfg := <-configReloadChan: + err := handleConfigReload(ctx, agentLoop, newCfg, &provider, services, msgBus) + if err != nil { + logger.Errorf("Config reload failed: %v", err) + } + } + } +} + +// setupAndStartServices initializes and starts all services +func setupAndStartServices( + cfg *config.Config, + agentLoop *agent.AgentLoop, + msgBus *bus.MessageBus, +) (*gatewayServices, error) { + services := &gatewayServices{} + // Setup cron tool and service execTimeout := time.Duration(cfg.Tools.Cron.ExecTimeoutMinutes) * time.Minute - cronService := setupCronTool( + services.CronService = setupCronTool( agentLoop, msgBus, cfg.WorkspacePath(), @@ -110,25 +175,31 @@ func gatewayCmd(debug bool, orchestration bool, enableStats bool) error { execTimeout, cfg, ) + if err := services.CronService.Start(); err != nil { + return nil, fmt.Errorf("error starting cron service: %w", err) + } + fmt.Println("✓ Cron service started") - heartbeatService := heartbeat.NewHeartbeatService( + // Setup heartbeat service + services.HeartbeatService = heartbeat.NewHeartbeatService( cfg.WorkspacePath(), cfg.Heartbeat.Interval, cfg.Heartbeat.Enabled, ) - heartbeatService.SetHeartbeatThreadID(cfg.Channels.Telegram.HeartbeatThreadID) - heartbeatService.SetBus(msgBus) - agentLoop.SetHeartbeatThreadUpdater(heartbeatService.SetHeartbeatThreadID) + services.HeartbeatService.SetHeartbeatThreadID(cfg.Channels.Telegram.HeartbeatThreadID) + services.HeartbeatService.SetBus(msgBus) + agentLoop.SetHeartbeatThreadUpdater(services.HeartbeatService.SetHeartbeatThreadID) agentLoop.SetConfigSaver(func(c *config.Config) error { return config.SaveConfig(internal.GetConfigPath(), c) }) - heartbeatService.SetHandler(func(prompt, channel, chatID string) *tools.ToolResult { + services.HeartbeatService.SetHandler(func(prompt, channel, chatID string) *tools.ToolResult { // Use cli:direct as fallback if no valid channel if channel == "" || chatID == "" { channel, chatID = "cli", "direct" } // Use ProcessHeartbeat - no session history, each heartbeat is independent var response string + var err error response, err = agentLoop.ProcessHeartbeat(context.Background(), prompt, channel, chatID) if err != nil { return tools.ErrorResult(fmt.Sprintf("Heartbeat error: %v", err)) @@ -140,27 +211,39 @@ func gatewayCmd(debug bool, orchestration bool, enableStats bool) error { } return tools.SilentResult(response) }) + if err := services.HeartbeatService.Start(); err != nil { + return nil, fmt.Errorf("error starting heartbeat service: %w", err) + } + fmt.Println("✓ Heartbeat service started") // Reset heartbeat suppression when a real user message arrives - agentLoop.OnUserMessage = heartbeatService.ResetSuppression + agentLoop.OnUserMessage = services.HeartbeatService.ResetSuppression // Create media store for file lifecycle management with TTL cleanup - mediaStore := media.NewFileMediaStoreWithCleanup(media.MediaCleanerConfig{ + services.MediaStore = media.NewFileMediaStoreWithCleanup(media.MediaCleanerConfig{ Enabled: cfg.Tools.MediaCleanup.Enabled, MaxAge: time.Duration(cfg.Tools.MediaCleanup.MaxAge) * time.Minute, Interval: time.Duration(cfg.Tools.MediaCleanup.Interval) * time.Minute, }) - mediaStore.Start() + // Start the media store if it's a FileMediaStore with cleanup + if fms, ok := services.MediaStore.(*media.FileMediaStore); ok { + fms.Start() + } - channelManager, err := channels.NewManager(cfg, msgBus, mediaStore) + // Create channel manager + var err error + services.ChannelManager, err = channels.NewManager(cfg, msgBus, services.MediaStore) if err != nil { - mediaStore.Stop() - return fmt.Errorf("error creating channel manager: %w", err) + // Stop the media store if it's a FileMediaStore with cleanup + if fms, ok := services.MediaStore.(*media.FileMediaStore); ok { + fms.Stop() + } + return nil, fmt.Errorf("error creating channel manager: %w", err) } // Inject channel manager and media store into agent loop - agentLoop.SetChannelManager(channelManager) - agentLoop.SetMediaStore(mediaStore) + agentLoop.SetChannelManager(services.ChannelManager) + agentLoop.SetMediaStore(services.MediaStore) // Wire up voice transcription if a supported provider is configured. if transcriber := voice.DetectTranscriber(cfg); transcriber != nil { @@ -168,49 +251,20 @@ func gatewayCmd(debug bool, orchestration bool, enableStats bool) error { logger.InfoCF("voice", "Transcription enabled (agent-level)", map[string]any{"provider": transcriber.Name()}) } - enabledChannels := channelManager.GetEnabledChannels() + enabledChannels := services.ChannelManager.GetEnabledChannels() if len(enabledChannels) > 0 { fmt.Printf("✓ Channels enabled: %s\n", enabledChannels) } else { fmt.Println("⚠ Warning: No channels enabled") } - fmt.Printf("✓ Gateway started on %s:%d\n", cfg.Gateway.Host, cfg.Gateway.Port) - fmt.Println("Press Ctrl+C to stop") - - ctx, cancel := context.WithCancel(context.Background()) - defer cancel() - - if err := cronService.Start(); err != nil { - fmt.Printf("Error starting cron service: %v\n", err) - } - fmt.Println("✓ Cron service started") - - if err := heartbeatService.Start(); err != nil { - fmt.Printf("Error starting heartbeat service: %v\n", err) - } - fmt.Println("✓ Heartbeat service started") - - stateManager := state.NewManager(cfg.WorkspacePath()) - deviceService := devices.NewService(devices.Config{ - Enabled: cfg.Devices.Enabled, - MonitorUSB: cfg.Devices.MonitorUSB, - }, stateManager) - deviceService.SetBus(msgBus) - if err := deviceService.Start(ctx); err != nil { - fmt.Printf("Error starting device service: %v\n", err) - } else if cfg.Devices.Enabled { - fmt.Println("✓ Device event service started") - } - // Setup shared HTTP server with health endpoints and webhook handlers - healthServer := health.NewServer(cfg.Gateway.Host, cfg.Gateway.Port) addr := fmt.Sprintf("%s:%d", cfg.Gateway.Host, cfg.Gateway.Port) - channelManager.SetupHTTPServer(addr, healthServer) + services.HealthServer = health.NewServer(cfg.Gateway.Host, cfg.Gateway.Port) + services.ChannelManager.SetupHTTPServer(addr, services.HealthServer) - if err := channelManager.StartAll(ctx); err != nil { - fmt.Printf("Error starting channels: %v\n", err) - return err + if err := services.ChannelManager.StartAll(context.Background()); err != nil { + return nil, fmt.Errorf("error starting channels: %w", err) } // Mini App setup: register routes and determine TLS mode @@ -259,7 +313,7 @@ func gatewayCmd(debug bool, orchestration bool, enableStats bool) error { if b := agentLoop.GetOrchBroadcaster(); b != nil { handler.SetOrchBroadcaster(b) } - handler.RegisterRoutes(healthServer.Mux()) + handler.RegisterRoutes(services.HealthServer.Mux()) // Register dev preview tool for all agents devPreviewTool := tools.NewDevPreviewTool(handler) @@ -272,9 +326,9 @@ func gatewayCmd(debug bool, orchestration bool, enableStats bool) error { go func() { var serverErr error if useTLS { - serverErr = healthServer.StartTLS(tlsCert, tlsKey) + serverErr = services.HealthServer.StartTLS(tlsCert, tlsKey) } else { - serverErr = healthServer.Start() + serverErr = services.HealthServer.Start() } if serverErr != nil && serverErr != http.ErrServerClosed { logger.ErrorCF("health", "Health server error", map[string]any{"error": serverErr.Error()}) @@ -293,40 +347,370 @@ func gatewayCmd(debug bool, orchestration bool, enableStats bool) error { ) } - go agentLoop.Run(ctx) - - sigChan := make(chan os.Signal, 1) - signal.Notify(sigChan, os.Interrupt) - <-sigChan - - fmt.Println("\nShutting down...") - if cp, ok := provider.(providers.StatefulProvider); ok { - cp.Close() + // Setup state manager and device service + stateManager := state.NewManager(cfg.WorkspacePath()) + services.DeviceService = devices.NewService(devices.Config{ + Enabled: cfg.Devices.Enabled, + MonitorUSB: cfg.Devices.MonitorUSB, + }, stateManager) + services.DeviceService.SetBus(msgBus) + if err := services.DeviceService.Start(context.Background()); err != nil { + logger.ErrorCF("device", "Error starting device service", map[string]any{"error": err.Error()}) + } else if cfg.Devices.Enabled { + fmt.Println("✓ Device event service started") } - cancel() - if miniappNotifier != nil { - miniappNotifier.Close() - } - msgBus.Close() + return services, nil +} - // Use a fresh context with timeout for graceful shutdown, - // since the original ctx is already canceled. - shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), 15*time.Second) +// stopAndCleanupServices stops all services and cleans up resources +func stopAndCleanupServices( + services *gatewayServices, + shutdownTimeout time.Duration, +) { + shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), shutdownTimeout) defer shutdownCancel() - channelManager.StopAll(shutdownCtx) - healthServer.Stop(shutdownCtx) - deviceService.Stop() - heartbeatService.Stop() - cronService.Stop() - mediaStore.Stop() + if services.ChannelManager != nil { + services.ChannelManager.StopAll(shutdownCtx) + } + if services.HealthServer != nil { + services.HealthServer.Stop(shutdownCtx) + } + if services.DeviceService != nil { + services.DeviceService.Stop() + } + if services.HeartbeatService != nil { + services.HeartbeatService.Stop() + } + if services.CronService != nil { + services.CronService.Stop() + } + if services.MediaStore != nil { + // Stop the media store if it's a FileMediaStore with cleanup + if fms, ok := services.MediaStore.(*media.FileMediaStore); ok { + fms.Stop() + } + } +} + +// shutdownGateway performs a complete gateway shutdown +func shutdownGateway( + services *gatewayServices, + agentLoop *agent.AgentLoop, + provider providers.LLMProvider, + fullShutdown bool, +) { + if cp, ok := provider.(providers.StatefulProvider); ok && fullShutdown { + cp.Close() + } + + stopAndCleanupServices(services, gracefulShutdownTimeout) + agentLoop.Stop() agentLoop.Close() - fmt.Println("✓ Gateway stopped") + + logger.Info("✓ Gateway stopped") +} + +// handleConfigReload handles config file reload by stopping all services, +// reloading the provider and config, and restarting services with the new config. +func handleConfigReload( + ctx context.Context, + al *agent.AgentLoop, + newCfg *config.Config, + providerRef *providers.LLMProvider, + services *gatewayServices, + msgBus *bus.MessageBus, +) error { + logger.Info("🔄 Config file changed, reloading...") + + newModel := newCfg.Agents.Defaults.ModelName + if newModel == "" { + newModel = newCfg.Agents.Defaults.Model + } + + logger.Infof(" New model is '%s', recreating provider...", newModel) + + // Stop all services before reloading + logger.Info(" Stopping all services...") + stopAndCleanupServices(services, serviceShutdownTimeout) + + // Create new provider from updated config first to ensure validity + // This will use the correct API key and settings from newCfg.ModelList + newProvider, newModelID, err := providers.CreateProvider(newCfg) + if err != nil { + logger.Errorf(" ⚠ Error creating new provider: %v", err) + logger.Warn(" Attempting to restart services with old provider and config...") + // Try to restart services with old configuration + if restartErr := restartServices(al, services, msgBus); restartErr != nil { + logger.Errorf(" ⚠ Failed to restart services: %v", restartErr) + } + return fmt.Errorf("error creating new provider: %w", err) + } + + if newModelID != "" { + newCfg.Agents.Defaults.ModelName = newModelID + } + + // Use the atomic reload method on AgentLoop to safely swap provider and config. + // This handles locking internally to prevent races with in-flight LLM calls + // and concurrent reads of registry/config while the swap occurs. + reloadCtx, reloadCancel := context.WithTimeout(context.Background(), providerReloadTimeout) + defer reloadCancel() + + if err := al.ReloadProviderAndConfig(reloadCtx, newProvider, newCfg); err != nil { + logger.Errorf(" ⚠ Error reloading agent loop: %v", err) + // Close the newly created provider since it wasn't adopted + if cp, ok := newProvider.(providers.StatefulProvider); ok { + cp.Close() + } + logger.Warn(" Attempting to restart services with old provider and config...") + if restartErr := restartServices(al, services, msgBus); restartErr != nil { + logger.Errorf(" ⚠ Failed to restart services: %v", restartErr) + } + return fmt.Errorf("error reloading agent loop: %w", err) + } + + // Update local provider reference only after successful atomic reload + *providerRef = newProvider + + // Restart all services with new config + logger.Info(" Restarting all services with new configuration...") + if err := restartServices(al, services, msgBus); err != nil { + logger.Errorf(" ⚠ Error restarting services: %v", err) + return fmt.Errorf("error restarting services: %w", err) + } + + logger.Info(" ✓ Provider, configuration, and services reloaded successfully (thread-safe)") + return nil +} + +// restartServices restarts all services after a config reload +func restartServices( + al *agent.AgentLoop, + services *gatewayServices, + msgBus *bus.MessageBus, +) error { + // Create an independent context with timeout for service restart + // This prevents cancellation from the main loop context during reload + ctx, cancel := context.WithTimeout(context.Background(), serviceRestartTimeout) + defer cancel() + + // Get current config from agent loop (which has been updated if this is a reload) + cfg := al.GetConfig() + + // Re-create and start cron service with new config + execTimeout := time.Duration(cfg.Tools.Cron.ExecTimeoutMinutes) * time.Minute + services.CronService = setupCronTool( + al, + msgBus, + cfg.WorkspacePath(), + cfg.Agents.Defaults.RestrictToWorkspace, + execTimeout, + cfg, + ) + if err := services.CronService.Start(); err != nil { + return fmt.Errorf("error restarting cron service: %w", err) + } + fmt.Println(" ✓ Cron service restarted") + + // Re-create and start heartbeat service with new config + services.HeartbeatService = heartbeat.NewHeartbeatService( + cfg.WorkspacePath(), + cfg.Heartbeat.Interval, + cfg.Heartbeat.Enabled, + ) + services.HeartbeatService.SetBus(msgBus) + services.HeartbeatService.SetHandler(func(prompt, channel, chatID string) *tools.ToolResult { + if channel == "" || chatID == "" { + channel, chatID = "cli", "direct" + } + var response string + var err error + response, err = al.ProcessHeartbeat(context.Background(), prompt, channel, chatID) + if err != nil { + return tools.ErrorResult(fmt.Sprintf("Heartbeat error: %v", err)) + } + if response == "HEARTBEAT_OK" { + return tools.SilentResult("Heartbeat OK") + } + return tools.SilentResult(response) + }) + if err := services.HeartbeatService.Start(); err != nil { + return fmt.Errorf("error restarting heartbeat service: %w", err) + } + fmt.Println(" ✓ Heartbeat service restarted") + + // Stop the old media store before creating a new one + if fms, ok := services.MediaStore.(*media.FileMediaStore); ok { + fms.Stop() + } + + // Re-create media store with new config + services.MediaStore = media.NewFileMediaStoreWithCleanup(media.MediaCleanerConfig{ + Enabled: cfg.Tools.MediaCleanup.Enabled, + MaxAge: time.Duration(cfg.Tools.MediaCleanup.MaxAge) * time.Minute, + Interval: time.Duration(cfg.Tools.MediaCleanup.Interval) * time.Minute, + }) + // Start the media store if it's a FileMediaStore with cleanup + if fms, ok := services.MediaStore.(*media.FileMediaStore); ok { + fms.Start() + } + al.SetMediaStore(services.MediaStore) + + // Re-create channel manager with new config + var err error + services.ChannelManager, err = channels.NewManager(cfg, msgBus, services.MediaStore) + if err != nil { + // Stop the media store if it's a FileMediaStore with cleanup + if fms, ok := services.MediaStore.(*media.FileMediaStore); ok { + fms.Stop() + } + return fmt.Errorf("error recreating channel manager: %w", err) + } + al.SetChannelManager(services.ChannelManager) + + enabledChannels := services.ChannelManager.GetEnabledChannels() + if len(enabledChannels) > 0 { + fmt.Printf(" ✓ Channels enabled: %s\n", enabledChannels) + } else { + fmt.Println(" ⚠ Warning: No channels enabled") + } + + // Setup HTTP server with new config + addr := fmt.Sprintf("%s:%d", cfg.Gateway.Host, cfg.Gateway.Port) + services.HealthServer = health.NewServer(cfg.Gateway.Host, cfg.Gateway.Port) + services.ChannelManager.SetupHTTPServer(addr, services.HealthServer) + + if err := services.ChannelManager.StartAll(ctx); err != nil { + return fmt.Errorf("error restarting channels: %w", err) + } + fmt.Printf( + " ✓ Channels restarted, health endpoints at http://%s:%d/health and ready\n", + cfg.Gateway.Host, + cfg.Gateway.Port, + ) + + // Re-create device service with new config + stateManager := state.NewManager(cfg.WorkspacePath()) + services.DeviceService = devices.NewService(devices.Config{ + Enabled: cfg.Devices.Enabled, + MonitorUSB: cfg.Devices.MonitorUSB, + }, stateManager) + services.DeviceService.SetBus(msgBus) + if err := services.DeviceService.Start(ctx); err != nil { + logger.WarnCF("device", "Failed to restart device service", map[string]any{"error": err.Error()}) + } else if cfg.Devices.Enabled { + fmt.Println(" ✓ Device event service restarted") + } + + // Wire up voice transcription with new config + transcriber := voice.DetectTranscriber(cfg) + al.SetTranscriber(transcriber) // This will set it to nil if disabled + if transcriber != nil { + logger.InfoCF("voice", "Transcription re-enabled (agent-level)", map[string]any{"provider": transcriber.Name()}) + } else { + logger.InfoCF("voice", "Transcription disabled", nil) + } return nil } +// setupConfigWatcherPolling sets up a simple polling-based config file watcher +// Returns a channel for config updates and a stop function +func setupConfigWatcherPolling(configPath string, debug bool) (chan *config.Config, func()) { + configChan := make(chan *config.Config, 1) + stop := make(chan struct{}) + var wg sync.WaitGroup + + wg.Add(1) + go func() { + defer wg.Done() + + // Get initial file info + lastModTime := getFileModTime(configPath) + lastSize := getFileSize(configPath) + + ticker := time.NewTicker(2 * time.Second) // Check every 2 seconds + defer ticker.Stop() + + for { + select { + case <-ticker.C: + currentModTime := getFileModTime(configPath) + currentSize := getFileSize(configPath) + + // Check if file changed (modification time or size changed) + if currentModTime.After(lastModTime) || currentSize != lastSize { + if debug { + logger.Debugf("🔍 Config file change detected") + } + + // Debounce - wait a bit to ensure file write is complete + time.Sleep(500 * time.Millisecond) + + // Validate and load new config + newCfg, err := config.LoadConfig(configPath) + if err != nil { + logger.Errorf("⚠ Error loading new config: %v", err) + logger.Warn(" Using previous valid config") + continue + } + + // Validate the new config + if err := newCfg.ValidateModelList(); err != nil { + logger.Errorf(" ⚠ New config validation failed: %v", err) + logger.Warn(" Using previous valid config") + continue + } + + logger.Info("✓ Config file validated and loaded") + + // Update last known state + lastModTime = currentModTime + lastSize = currentSize + + // Send new config to main loop (non-blocking) + select { + case configChan <- newCfg: + default: + // Channel full, skip this update + logger.Warn("⚠ Previous config reload still in progress, skipping") + } + } + + case <-stop: + return + } + } + }() + + stopFunc := func() { + close(stop) + wg.Wait() + } + + return configChan, stopFunc +} + +// getFileModTime returns the modification time of a file, or zero time if file doesn't exist +func getFileModTime(path string) time.Time { + info, err := os.Stat(path) + if err != nil { + return time.Time{} + } + return info.ModTime() +} + +// getFileSize returns the size of a file, or 0 if file doesn't exist +func getFileSize(path string) int64 { + info, err := os.Stat(path) + if err != nil { + return 0 + } + return info.Size() +} + func setupCronTool( agentLoop *agent.AgentLoop, msgBus *bus.MessageBus, @@ -346,7 +730,7 @@ func setupCronTool( var err error cronTool, err = tools.NewCronTool(cronService, agentLoop, msgBus, workspace, restrict, execTimeout, cfg) if err != nil { - log.Fatalf("Critical error during CronTool initialization: %v", err) + logger.Fatalf("Critical error during CronTool initialization: %v", err) } agentLoop.RegisterTool(cronTool) diff --git a/cmd/picoclaw/internal/model/command.go b/cmd/picoclaw/internal/model/command.go new file mode 100644 index 000000000..cad106fd5 --- /dev/null +++ b/cmd/picoclaw/internal/model/command.go @@ -0,0 +1,138 @@ +package model + +import ( + "fmt" + + "github.com/spf13/cobra" + + "github.com/sipeed/picoclaw/cmd/picoclaw/internal" + "github.com/sipeed/picoclaw/pkg/config" +) + +// LocalModel is a special model name that indicates that the model is local and with or without api_key. +const LocalModel = "local-model" + +func NewModelCommand() *cobra.Command { + cmd := &cobra.Command{ + Use: "model [model_name]", + Short: "Show or change the default model", + Long: `Show or change the default model configuration. + +If no argument is provided, shows the current default model. +If a model name is provided, sets it as the default model. + +Examples: + picoclaw model # Show current default model + picoclaw model gpt-5.2 # Set gpt-5.2 as default + picoclaw model claude-sonnet-4.6 # Set claude-sonnet-4.6 as default + picoclaw model local-model # Set local VLLM server as default + +Note: 'local-model' is a special value for using a local VLLM server +(running at localhost:8000 by default) which does not require an API key.`, + Args: cobra.MaximumNArgs(1), + RunE: func(cmd *cobra.Command, args []string) error { + configPath := internal.GetConfigPath() + + // Load current config + cfg, err := config.LoadConfig(configPath) + if err != nil { + return fmt.Errorf("failed to load config: %w", err) + } + + if len(args) == 0 { + // Show current default model + showCurrentModel(cfg) + return nil + } + + // Set new default model + modelName := args[0] + return setDefaultModel(configPath, cfg, modelName) + }, + } + + return cmd +} + +func showCurrentModel(cfg *config.Config) { + defaultModel := cfg.Agents.Defaults.ModelName + if defaultModel == "" { + defaultModel = cfg.Agents.Defaults.Model + } + + if defaultModel == "" { + fmt.Println("No default model is currently set.") + fmt.Println("\nAvailable models in your config:") + listAvailableModels(cfg) + } else { + fmt.Printf("Current default model: %s\n", defaultModel) + fmt.Println("\nAvailable models in your config:") + listAvailableModels(cfg) + } +} + +func listAvailableModels(cfg *config.Config) { + if len(cfg.ModelList) == 0 { + fmt.Println(" No models configured in model_list") + return + } + + defaultModel := cfg.Agents.Defaults.ModelName + if defaultModel == "" { + defaultModel = cfg.Agents.Defaults.Model + } + + for _, model := range cfg.ModelList { + marker := " " + if model.ModelName == defaultModel { + marker = "> " + } + if model.APIKey == "" { + continue + } + fmt.Printf("%s- %s (%s)\n", marker, model.ModelName, model.Model) + } +} + +func setDefaultModel(configPath string, cfg *config.Config, modelName string) error { + // Validate that the model exists in model_list + modelFound := false + for _, model := range cfg.ModelList { + if model.APIKey != "" && model.ModelName == modelName { + modelFound = true + break + } + } + + if !modelFound && modelName != LocalModel { + return fmt.Errorf("cannot found model '%s' in config", modelName) + } + + // Update the default model + // Clear old model field and set new model_name + oldModel := cfg.Agents.Defaults.ModelName + if oldModel == "" { + oldModel = cfg.Agents.Defaults.Model + } + + cfg.Agents.Defaults.ModelName = modelName + cfg.Agents.Defaults.Model = "" // Clear deprecated field + + // Save config back to file + if err := config.SaveConfig(configPath, cfg); err != nil { + return fmt.Errorf("failed to save config: %w", err) + } + + fmt.Printf("✓ Default model changed from '%s' to '%s'\n", + formatModelName(oldModel), modelName) + fmt.Println("\nThe new default model will be used for all agent interactions.") + + return nil +} + +func formatModelName(name string) string { + if name == "" { + return "(none)" + } + return name +} diff --git a/cmd/picoclaw/internal/model/command_test.go b/cmd/picoclaw/internal/model/command_test.go new file mode 100644 index 000000000..82943e4a6 --- /dev/null +++ b/cmd/picoclaw/internal/model/command_test.go @@ -0,0 +1,369 @@ +package model + +import ( + "bytes" + "io" + "os" + "path/filepath" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/sipeed/picoclaw/pkg/config" +) + +var configPath = "" + +func initTest(t *testing.T) { + tmpDir := t.TempDir() + configPath = filepath.Join(tmpDir, "config.json") + _ = os.Setenv("PICOCLAW_CONFIG", configPath) +} + +// captureStdout captures stdout during the execution of fn and returns the captured output +func captureStdout(fn func()) string { + oldStdout := os.Stdout + r, w, _ := os.Pipe() + os.Stdout = w + + fn() + + w.Close() + os.Stdout = oldStdout + + var buf bytes.Buffer + io.Copy(&buf, r) + return buf.String() +} + +func TestNewModelCommand(t *testing.T) { + cmd := NewModelCommand() + + require.NotNil(t, cmd) + + assert.Equal(t, "model [model_name]", cmd.Use) + assert.Equal(t, "Show or change the default model", cmd.Short) + + assert.Len(t, cmd.Aliases, 0) + + assert.False(t, cmd.HasFlags()) + + assert.Nil(t, cmd.Run) + assert.NotNil(t, cmd.RunE) + + assert.Nil(t, cmd.PersistentPreRunE) + assert.Nil(t, cmd.PersistentPreRun) + assert.Nil(t, cmd.PersistentPostRun) +} + +func TestShowCurrentModel_WithDefaultModel(t *testing.T) { + cfg := &config.Config{ + Agents: config.AgentsConfig{ + Defaults: config.AgentDefaults{ + ModelName: "gpt-4", + }, + }, + ModelList: []config.ModelConfig{ + {ModelName: "gpt-4", Model: "openai/gpt-4", APIKey: "test"}, + {ModelName: "claude-3", Model: "anthropic/claude-3", APIKey: "test"}, + }, + } + + output := captureStdout(func() { + showCurrentModel(cfg) + }) + + assert.Contains(t, output, "Current default model: gpt-4") + assert.Contains(t, output, "Available models in your config:") + assert.Contains(t, output, "gpt-4") + assert.Contains(t, output, "claude-3") +} + +func TestShowCurrentModel_NoDefaultModel(t *testing.T) { + cfg := &config.Config{ + Agents: config.AgentsConfig{ + Defaults: config.AgentDefaults{ + ModelName: "", + Model: "", + }, + }, + ModelList: []config.ModelConfig{ + {ModelName: "gpt-4", Model: "openai/gpt-4", APIKey: "test"}, + }, + } + + output := captureStdout(func() { + showCurrentModel(cfg) + }) + + assert.Contains(t, output, "No default model is currently set.") + assert.Contains(t, output, "Available models in your config:") +} + +func TestShowCurrentModel_BackwardCompatibility(t *testing.T) { + cfg := &config.Config{ + Agents: config.AgentsConfig{ + Defaults: config.AgentDefaults{ + Model: "legacy-model", + }, + }, + ModelList: []config.ModelConfig{}, + } + + output := captureStdout(func() { + showCurrentModel(cfg) + }) + + assert.Contains(t, output, "Current default model: legacy-model") +} + +func TestListAvailableModels_Empty(t *testing.T) { + cfg := &config.Config{ + ModelList: []config.ModelConfig{}, + } + + output := captureStdout(func() { + listAvailableModels(cfg) + }) + + assert.Contains(t, output, "No models configured in model_list") +} + +func TestListAvailableModels_WithModels(t *testing.T) { + cfg := &config.Config{ + Agents: config.AgentsConfig{ + Defaults: config.AgentDefaults{ + ModelName: "gpt-4", + }, + }, + ModelList: []config.ModelConfig{ + {ModelName: "gpt-4", Model: "openai/gpt-4", APIKey: "test"}, + {ModelName: "claude-3", Model: "anthropic/claude-3", APIKey: "test"}, + {ModelName: "no-key-model", Model: "openai/test", APIKey: ""}, + }, + } + + output := captureStdout(func() { + listAvailableModels(cfg) + }) + + assert.NotEmpty(t, output) + assert.Contains(t, output, "> - gpt-4 (openai/gpt-4)") + assert.Contains(t, output, "claude-3 (anthropic/claude-3)") + assert.NotContains(t, output, "no-key-model") +} + +func TestSetDefaultModel_ValidModel(t *testing.T) { + initTest(t) + + cfg := &config.Config{ + Agents: config.AgentsConfig{ + Defaults: config.AgentDefaults{ + ModelName: "old-model", + }, + }, + ModelList: []config.ModelConfig{ + {ModelName: "new-model", Model: "openai/new-model", APIKey: "test"}, + {ModelName: "old-model", Model: "openai/old-model", APIKey: "test"}, + }, + } + + output := captureStdout(func() { + err := setDefaultModel(configPath, cfg, "new-model") + assert.NoError(t, err) + }) + + assert.Contains(t, output, "Default model changed from 'old-model' to 'new-model'") + + // Verify config was updated + updatedCfg, err := config.LoadConfig(configPath) + require.NoError(t, err) + assert.Equal(t, "new-model", updatedCfg.Agents.Defaults.ModelName) + assert.Empty(t, updatedCfg.Agents.Defaults.Model) +} + +func TestSetDefaultModel_LegacyModelField(t *testing.T) { + initTest(t) + + cfg := &config.Config{ + Agents: config.AgentsConfig{ + Defaults: config.AgentDefaults{ + Model: "legacy-old", + }, + }, + ModelList: []config.ModelConfig{ + {ModelName: "new-model", Model: "openai/new-model", APIKey: "test"}, + }, + } + + output := captureStdout(func() { + err := setDefaultModel(configPath, cfg, "new-model") + assert.NoError(t, err) + }) + + assert.Contains(t, output, "Default model changed from 'legacy-old' to 'new-model'") +} + +func TestSetDefaultModel_InvalidModel(t *testing.T) { + initTest(t) + + cfg := &config.Config{ + Agents: config.AgentsConfig{ + Defaults: config.AgentDefaults{ + ModelName: "existing-model", + }, + }, + ModelList: []config.ModelConfig{ + {ModelName: "existing-model", Model: "openai/existing", APIKey: "test"}, + }, + } + + assert.Error(t, setDefaultModel(configPath, cfg, "nonexistent-model")) +} + +func TestSetDefaultModel_ModelWithoutAPIKey(t *testing.T) { + initTest(t) + + cfg := &config.Config{ + Agents: config.AgentsConfig{ + Defaults: config.AgentDefaults{ + ModelName: "existing-model", + }, + }, + ModelList: []config.ModelConfig{ + {ModelName: "existing-model", Model: "openai/existing", APIKey: "test"}, + {ModelName: "no-key-model", Model: "openai/nokey", APIKey: ""}, + }, + } + + assert.Error(t, setDefaultModel(configPath, cfg, "no-key-model")) +} + +func TestSetDefaultModel_SaveConfigError(t *testing.T) { + // Use an invalid path to trigger save error + invalidPath := "/nonexistent/directory/config.json" + + cfg := &config.Config{ + Agents: config.AgentsConfig{ + Defaults: config.AgentDefaults{ + ModelName: "old-model", + }, + }, + ModelList: []config.ModelConfig{ + {ModelName: "new-model", Model: "openai/new-model", APIKey: "test"}, + }, + } + + err := setDefaultModel(invalidPath, cfg, "new-model") + + assert.Error(t, err) + assert.Contains(t, err.Error(), "failed to save config") +} + +func TestFormatModelName(t *testing.T) { + tests := []struct { + name string + input string + expected string + }{ + {"empty string", "", "(none)"}, + {"simple model", "gpt-4", "gpt-4"}, + {"model with version", "claude-sonnet-4.6", "claude-sonnet-4.6"}, + {"model with spaces", "my model", "my model"}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + result := formatModelName(tt.input) + assert.Equal(t, tt.expected, result) + }) + } +} + +func TestModelCommandExecution_Show(t *testing.T) { + initTest(t) + + // Create a test config + cfg := &config.Config{ + Agents: config.AgentsConfig{ + Defaults: config.AgentDefaults{ + ModelName: "test-model", + }, + }, + ModelList: []config.ModelConfig{ + {ModelName: "test-model", Model: "openai/test", APIKey: "test"}, + }, + } + + err := config.SaveConfig(configPath, cfg) + require.NoError(t, err) + + cmd := NewModelCommand() + + output := captureStdout(func() { + err = cmd.RunE(cmd, []string{}) + assert.NoError(t, err) + }) + + assert.Contains(t, output, "Current default model: test-model") +} + +func TestModelCommandExecution_Set(t *testing.T) { + initTest(t) + + cfg := &config.Config{ + Agents: config.AgentsConfig{ + Defaults: config.AgentDefaults{ + ModelName: "old-model", + }, + }, + ModelList: []config.ModelConfig{ + {ModelName: "old-model", Model: "openai/old", APIKey: "test"}, + {ModelName: "new-model", Model: "openai/new", APIKey: "test"}, + }, + } + + err := config.SaveConfig(configPath, cfg) + require.NoError(t, err) + + cmd := NewModelCommand() + + output := captureStdout(func() { + err = cmd.RunE(cmd, []string{"new-model"}) + assert.NoError(t, err) + }) + + assert.Contains(t, output, "Default model changed from 'old-model' to 'new-model'") +} + +func TestModelCommandExecution_TooManyArgs(t *testing.T) { + cmd := NewModelCommand() + + err := cmd.RunE(cmd, []string{"model1", "model2"}) + + assert.Error(t, err) +} + +func TestListAvailableModels_MarkerLogic(t *testing.T) { + cfg := &config.Config{ + Agents: config.AgentsConfig{ + Defaults: config.AgentDefaults{ + ModelName: "middle-model", + }, + }, + ModelList: []config.ModelConfig{ + {ModelName: "first-model", Model: "openai/first", APIKey: "test"}, + {ModelName: "middle-model", Model: "openai/middle", APIKey: "test"}, + {ModelName: "last-model", Model: "openai/last", APIKey: "test"}, + }, + } + + output := captureStdout(func() { + listAvailableModels(cfg) + }) + + assert.Contains(t, output, " - first-model (openai/first)") + assert.Contains(t, output, "> - middle-model (openai/middle)") + assert.Contains(t, output, " - last-model (openai/last)") +} diff --git a/cmd/picoclaw/internal/skills/command.go b/cmd/picoclaw/internal/skills/command.go index 65eb127b9..8c666b810 100644 --- a/cmd/picoclaw/internal/skills/command.go +++ b/cmd/picoclaw/internal/skills/command.go @@ -29,7 +29,15 @@ func NewSkillsCommand() *cobra.Command { } d.workspace = cfg.WorkspacePath() - d.installer = skills.NewSkillInstaller(d.workspace) + installer, err := skills.NewSkillInstaller( + d.workspace, + cfg.Tools.Skills.Github.Token, + cfg.Tools.Skills.Github.Proxy, + ) + if err != nil { + return fmt.Errorf("error creating skills installer: %w", err) + } + d.installer = installer // get global config directory and builtin skills directory globalDir := filepath.Dir(internal.GetConfigPath()) diff --git a/cmd/picoclaw/main.go b/cmd/picoclaw/main.go index b82475905..bf9c0389f 100644 --- a/cmd/picoclaw/main.go +++ b/cmd/picoclaw/main.go @@ -18,6 +18,7 @@ import ( "github.com/sipeed/picoclaw/cmd/picoclaw/internal/cron" "github.com/sipeed/picoclaw/cmd/picoclaw/internal/gateway" "github.com/sipeed/picoclaw/cmd/picoclaw/internal/migrate" + "github.com/sipeed/picoclaw/cmd/picoclaw/internal/model" "github.com/sipeed/picoclaw/cmd/picoclaw/internal/onboard" "github.com/sipeed/picoclaw/cmd/picoclaw/internal/skills" "github.com/sipeed/picoclaw/cmd/picoclaw/internal/status" @@ -43,6 +44,7 @@ func NewPicoclawCommand() *cobra.Command { cron.NewCronCommand(), migrate.NewMigrateCommand(), skills.NewSkillsCommand(), + model.NewModelCommand(), version.NewVersionCommand(), ) diff --git a/cmd/picoclaw/main_test.go b/cmd/picoclaw/main_test.go index e622675ee..ad18cb330 100644 --- a/cmd/picoclaw/main_test.go +++ b/cmd/picoclaw/main_test.go @@ -39,6 +39,7 @@ func TestNewPicoclawCommand(t *testing.T) { "cron", "gateway", "migrate", + "model", "onboard", "skills", "status", diff --git a/config/config.example.json b/config/config.example.json index b259df6f6..094aa46df 100644 --- a/config/config.example.json +++ b/config/config.example.json @@ -25,6 +25,13 @@ "api_base": "https://api.anthropic.com/v1", "thinking_level": "high" }, + { + "_comment": "Anthropic Messages API - use native format for direct Anthropic API access", + "model_name": "claude-opus-4-6", + "model": "anthropic-messages/claude-opus-4-6", + "api_key": "sk-ant-your-key", + "api_base": "https://api.anthropic.com" + }, { "model_name": "gemini", "model": "antigravity/gemini-2.0-flash", @@ -40,6 +47,12 @@ "model": "longcat/LongCat-Flash-Thinking", "api_key": "your-longcat-api-key" }, + { + "model_name": "modelscope-qwen", + "model": "modelscope/Qwen/Qwen3-235B-A22B-Instruct-2507", + "api_key": "your-modelscope-access-token", + "api_base": "https://api-inference.modelscope.cn/v1" + }, { "model_name": "loadbalanced-gpt-5.4", "model": "openai/gpt-5.4", @@ -283,6 +296,10 @@ "longcat": { "api_key": "", "api_base": "https://api.longcat.chat/openai" + }, + "modelscope": { + "api_key": "", + "api_base": "https://api-inference.modelscope.cn/v1" } }, "tools": { @@ -427,6 +444,10 @@ "max_response_size": 0 } }, + "github": { + "proxy": "http://127.0.0.1:7891", + "token": "" + }, "max_concurrent_searches": 2, "search_cache": { "max_size": 50, diff --git a/go.mod b/go.mod index a6d3de4b8..5f0b6cadc 100644 --- a/go.mod +++ b/go.mod @@ -7,8 +7,8 @@ require ( github.com/anthropics/anthropic-sdk-go v1.22.1 github.com/bwmarrin/discordgo v0.29.0 github.com/caarlos0/env/v11 v11.3.1 - github.com/chzyer/readline v1.5.1 github.com/ergochat/irc-go v0.5.0 + github.com/ergochat/readline v0.1.3 github.com/gdamore/tcell/v2 v2.13.8 github.com/gomarkdown/markdown v0.0.0-20260217112301-37c66b85d6ab github.com/google/uuid v1.6.0 diff --git a/go.sum b/go.sum index cdca4fc12..addbab56c 100644 --- a/go.sum +++ b/go.sum @@ -27,12 +27,6 @@ github.com/caarlos0/env/v11 v11.3.1 h1:cArPWC15hWmEt+gWk7YBi7lEXTXCvpaSdCiZE2X5m github.com/caarlos0/env/v11 v11.3.1/go.mod h1:qupehSf/Y0TUTsxKywqRt/vJjN5nz6vauiYEUUr8P4U= github.com/cespare/xxhash/v2 v2.1.2/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= -github.com/chzyer/logex v1.2.1 h1:XHDu3E6q+gdHgsdTPH6ImJMIp436vR6MPtH8gP05QzM= -github.com/chzyer/logex v1.2.1/go.mod h1:JLbx6lG2kDbNRFnfkgvh4eRJRPX1QCoOIWomwysCBrQ= -github.com/chzyer/readline v1.5.1 h1:upd/6fQk4src78LMRzh5vItIt361/o4uq553V8B5sGI= -github.com/chzyer/readline v1.5.1/go.mod h1:Eh+b79XXUwfKfcPLepksvw2tcLE/Ct21YObkaSkeBlk= -github.com/chzyer/test v1.0.0 h1:p3BQDXSxOhOG0P9z6/hGnII4LGiEPOYBhs8asl/fC04= -github.com/chzyer/test v1.0.0/go.mod h1:2JlltgoNkt4TW/z9V/IzDdFaMTM2JPIi26O1pF38GC8= github.com/cloudwego/base64x v0.1.6 h1:t11wG9AECkCDk5fMSoxmufanudBtJ+/HemLstXDLI2M= github.com/cloudwego/base64x v0.1.6/go.mod h1:OFcloc187FXDaYHvrNIjxSe8ncn0OOM8gEHfghB2IPU= github.com/coder/websocket v1.8.14 h1:9L0p0iKiNOibykf283eHkKUHHrpG7f65OE3BhhO7v9g= @@ -50,6 +44,8 @@ github.com/elliotchance/orderedmap/v3 v3.1.0 h1:j4DJ5ObEmMBt/lcwIecKcoRxIQUEnw0L github.com/elliotchance/orderedmap/v3 v3.1.0/go.mod h1:G+Hc2RwaZvJMcS4JpGCOyViCnGeKf0bTYCGTO4uhjSo= github.com/ergochat/irc-go v0.5.0 h1:woQ1RS9YbfgqPgSpPBBQeczXGIGzR0aC7dEgk469fTw= github.com/ergochat/irc-go v0.5.0/go.mod h1:2vi7KNpIPWnReB5hmLpl92eMywQvuIeIIGdt/FQCph0= +github.com/ergochat/readline v0.1.3 h1:/DytGTmwdUJcLAe3k3VJgowh5vNnsdifYT6uVaf4pSo= +github.com/ergochat/readline v0.1.3/go.mod h1:o3ux9QLHLm77bq7hDB21UTm6HlV2++IPDMfIfKDuOgY= github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ= github.com/gdamore/encoding v1.0.1 h1:YzKZckdBL6jVt2Gc+5p82qhrGiqMdG/eNs6Wy0u3Uhw= @@ -297,7 +293,6 @@ golang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220310020820-b874c991c1a5/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= diff --git a/pkg/agent/loop.go b/pkg/agent/loop.go index 2117e2f39..1f7e32c85 100644 --- a/pkg/agent/loop.go +++ b/pkg/agent/loop.go @@ -65,6 +65,10 @@ type AgentLoop struct { mcp mcpRuntime + mu sync.RWMutex + // Track active requests for safe provider cleanup + activeRequests sync.WaitGroup + providerCache map[string]providers.LLMProvider lastSystemPrompt atomic.Value // string — last system prompt sent to LLM @@ -658,18 +662,20 @@ func (al *AgentLoop) Close() { al.stats.Close() } - for _, agentID := range al.registry.ListAgentIDs() { - if agent, ok := al.registry.GetAgent(agentID); ok { + registry := al.GetRegistry() + for _, agentID := range registry.ListAgentIDs() { + if agent, ok := registry.GetAgent(agentID); ok { agent.Sessions.Close() } } - al.registry.Close() + registry.Close() } func (al *AgentLoop) RegisterTool(tool tools.Tool) { - for _, agentID := range al.registry.ListAgentIDs() { - if agent, ok := al.registry.GetAgent(agentID); ok { + registry := al.GetRegistry() + for _, agentID := range registry.ListAgentIDs() { + if agent, ok := registry.GetAgent(agentID); ok { agent.Tools.Register(tool) } } @@ -680,16 +686,11 @@ func (al *AgentLoop) SetChannelManager(cm *channels.Manager) { } // resolveProvider returns the LLMProvider for the given provider/model pair. - // It caches created providers by "provider/model" key so each combination is - // only resolved once. Looks up model_list first (new format), then falls back - // to the legacy providers section via CreateProviderByName. - func (al *AgentLoop) resolveProvider( providerName, modelName string, - fallback providers.LLMProvider, ) providers.LLMProvider { key := strings.ToLower(providerName + "/" + modelName) @@ -703,44 +704,147 @@ func (al *AgentLoop) resolveProvider( } // Try model_list first (new config format). - if mc := al.cfg.FindModelConfigByRef(providerName, modelName); mc != nil { p, _, err := providers.CreateProviderFromConfig(mc) - if err == nil { al.providerCache[key] = p - return p } logger.WarnCF("agent", "Failed to create provider from model_list, trying legacy", - map[string]any{"provider": providerName, "model": modelName, "error": err.Error()}) } // Fall back to legacy providers section. - p, err := providers.CreateProviderByName(al.cfg, providerName) if err != nil { logger.WarnCF("agent", "Failed to create provider for fallback, using primary", - map[string]any{"provider": providerName, "error": err.Error()}) - return fallback } al.providerCache[key] = p - return p } +// ReloadProviderAndConfig atomically swaps the provider and config with proper synchronization. +// It uses a context to allow timeout control from the caller. +// Returns an error if the reload fails or context is canceled. +func (al *AgentLoop) ReloadProviderAndConfig( + ctx context.Context, + provider providers.LLMProvider, + cfg *config.Config, +) error { + // Validate inputs + if provider == nil { + return fmt.Errorf("provider cannot be nil") + } + if cfg == nil { + return fmt.Errorf("config cannot be nil") + } + + // Create new registry with updated config and provider + // Wrap in defer/recover to handle any panics gracefully + var registry *AgentRegistry + var panicErr error + done := make(chan struct{}, 1) + + go func() { + defer func() { + if r := recover(); r != nil { + panicErr = fmt.Errorf("panic during registry creation: %v", r) + logger.ErrorCF("agent", "Panic during registry creation", + map[string]any{"panic": r}) + } + close(done) + }() + + registry = NewAgentRegistry(cfg, provider) + }() + + // Wait for completion or context cancellation + select { + case <-done: + if registry == nil { + if panicErr != nil { + return fmt.Errorf("registry creation failed: %w", panicErr) + } + return fmt.Errorf("registry creation failed (nil result)") + } + case <-ctx.Done(): + return fmt.Errorf("context canceled during registry creation: %w", ctx.Err()) + } + + // Check context again before proceeding + if err := ctx.Err(); err != nil { + return fmt.Errorf("context canceled after registry creation: %w", err) + } + + // Ensure shared tools are re-registered on the new registry + registerSharedTools(cfg, al.bus, registry, provider, al) + + // Atomically swap the config and registry under write lock + // This ensures readers see a consistent pair + al.mu.Lock() + oldRegistry := al.registry + + // Store new values + al.cfg = cfg + al.registry = registry + + // Also update fallback chain with new config + al.fallback = providers.NewFallbackChain(providers.NewCooldownTracker()) + + al.mu.Unlock() + + // Close old provider after releasing the lock + // This prevents blocking readers while closing + if oldProvider, ok := extractProvider(oldRegistry); ok { + if stateful, ok := oldProvider.(providers.StatefulProvider); ok { + // Give in-flight requests a moment to complete + // Use a reasonable timeout that balances cleanup vs resource usage + select { + case <-time.After(100 * time.Millisecond): + stateful.Close() + case <-ctx.Done(): + // Context canceled, close immediately but log warning + logger.WarnCF("agent", "Context canceled during provider cleanup, forcing close", + map[string]any{"error": ctx.Err()}) + stateful.Close() + } + } + } + + logger.InfoCF("agent", "Provider and config reloaded successfully", + map[string]any{ + "model": cfg.Agents.Defaults.GetModelName(), + }) + + return nil +} + +// GetRegistry returns the current registry (thread-safe) +func (al *AgentLoop) GetRegistry() *AgentRegistry { + al.mu.RLock() + defer al.mu.RUnlock() + return al.registry +} + +// GetConfig returns the current config (thread-safe) +func (al *AgentLoop) GetConfig() *config.Config { + al.mu.RLock() + defer al.mu.RUnlock() + return al.cfg +} + // SetMediaStore injects a MediaStore for media lifecycle management. func (al *AgentLoop) SetMediaStore(s media.MediaStore) { al.mediaStore = s // Propagate store to send_file tools in all agents. - al.registry.ForEachTool("send_file", func(t tools.Tool) { + registry := al.GetRegistry() + registry.ForEachTool("send_file", func(t tools.Tool) { if sf, ok := t.(*tools.SendFileTool); ok { sf.SetMediaStore(s) } @@ -944,9 +1048,8 @@ func (al *AgentLoop) ProcessDirectWithChannel( // ProcessHeartbeat processes a heartbeat request without session history. // Each heartbeat is independent and doesn't accumulate context. - func (al *AgentLoop) ProcessHeartbeat(ctx context.Context, content, channel, chatID string) (string, error) { - agent := al.registry.GetDefaultAgent() + agent := al.GetRegistry().GetDefaultAgent() if agent == nil { return "", fmt.Errorf("no default agent for heartbeat") } @@ -1110,7 +1213,8 @@ func (al *AgentLoop) processMessage(ctx context.Context, msg bus.InboundMessage) // Route to determine agent and session key - route := al.registry.ResolveRoute(routing.RouteInput{ + registry := al.GetRegistry() + route := registry.ResolveRoute(routing.RouteInput{ Channel: msg.Channel, AccountID: msg.Metadata["account_id"], @@ -1124,9 +1228,9 @@ func (al *AgentLoop) processMessage(ctx context.Context, msg bus.InboundMessage) TeamID: msg.Metadata["team_id"], }) - agent, ok := al.registry.GetAgent(route.AgentID) + agent, ok := registry.GetAgent(route.AgentID) if !ok { - agent = al.registry.GetDefaultAgent() + agent = registry.GetDefaultAgent() } if agent == nil { return "", fmt.Errorf("no agent available for route (agent_id=%s)", route.AgentID) @@ -1626,8 +1730,9 @@ func (al *AgentLoop) runAgentLoop(ctx context.Context, agent *AgentInstance, opt opts.ChatID, ) - // Resolve media:// refs to base64 data URLs (streaming) - maxMediaSize := al.cfg.Agents.Defaults.GetMaxMediaSize() + // Resolve media:// refs: images→base64 data URLs, non-images→local paths in content + cfg := al.GetConfig() + maxMediaSize := cfg.Agents.Defaults.GetMaxMediaSize() messages = resolveMediaRefs(messages, al.mediaStore, maxMediaSize) // 2b. Interview staleness nudge: if MEMORY.md hasn't been updated for @@ -2053,6 +2158,7 @@ func (al *AgentLoop) runLLMIteration( map[string]any{ "agent_id": agent.ID, "iteration": iteration, + "model": activeModel, "error": err.Error(), }) return "", iteration, fmt.Errorf("LLM call failed after retries: %w", err) @@ -2157,6 +2263,69 @@ func (al *AgentLoop) runLLMIteration( return finalContent, iteration, nil } +// selectCandidates returns the model candidates and resolved model name to use +// for a conversation turn. When model routing is configured and the incoming +// message scores below the complexity threshold, it returns the light model +// candidates instead of the primary ones. +// +// The returned (candidates, model) pair is used for all LLM calls within one +// turn — tool follow-up iterations use the same tier as the initial call so +// that a multi-step tool chain doesn't switch models mid-way. +func (al *AgentLoop) selectCandidates( + agent *AgentInstance, + userMsg string, + history []providers.Message, +) (candidates []providers.FallbackCandidate, model string) { + if agent.Router == nil || len(agent.LightCandidates) == 0 { + return agent.Candidates, agent.Model + } + + _, usedLight, score := agent.Router.SelectModel(userMsg, history, agent.Model) + if !usedLight { + logger.DebugCF("agent", "Model routing: primary model selected", + map[string]any{ + "agent_id": agent.ID, + "score": score, + "threshold": agent.Router.Threshold(), + }) + return agent.Candidates, agent.Model + } + + logger.InfoCF("agent", "Model routing: light model selected", + map[string]any{ + "agent_id": agent.ID, + "light_model": agent.Router.LightModel(), + "score": score, + "threshold": agent.Router.Threshold(), + }) + return agent.LightCandidates, agent.Router.LightModel() +} + +// findNearestUserMessage finds the nearest user message to the given index. +// It searches backward first, then forward if no user message is found. +func (al *AgentLoop) findNearestUserMessage(messages []providers.Message, mid int) int { + originalMid := mid + + for mid > 0 && messages[mid].Role != "user" { + mid-- + } + + if messages[mid].Role == "user" { + return mid + } + + mid = originalMid + for mid < len(messages) && messages[mid].Role != "user" { + mid++ + } + + if mid < len(messages) { + return mid + } + + return originalMid +} + // callLLMWithRetry calls the LLM with streaming support, fallback chain, // and retry logic for timeout and context window errors. func (al *AgentLoop) callLLMWithRetry( @@ -2488,6 +2657,42 @@ func (al *AgentLoop) publishToolMedia(ctx context.Context, result *tools.ToolRes }) } +func (al *AgentLoop) buildCommandsRuntime(agent *AgentInstance, opts *processOptions) *commands.Runtime { + registry := al.GetRegistry() + cfg := al.GetConfig() + rt := &commands.Runtime{ + Config: cfg, + ListAgentIDs: registry.ListAgentIDs, + ListDefinitions: al.cmdRegistry.Definitions, + GetEnabledChannels: func() []string { + if al.channelManager == nil { + return nil + } + return al.channelManager.GetEnabledChannels() + }, + SwitchChannel: func(value string) error { + if al.channelManager == nil { + return fmt.Errorf("channel manager not initialized") + } + if _, exists := al.channelManager.GetChannel(value); !exists && value != "cli" { + return fmt.Errorf("channel '%s' not found or not enabled", value) + } + return nil + }, + } + if agent != nil { + rt.GetModelInfo = func() (string, string) { + return agent.Model, cfg.Agents.Defaults.Provider + } + rt.SwitchModel = func(value string) (string, error) { + oldModel := agent.Model + agent.Model = value + return oldModel, nil + } + } + return rt +} + // forceTextResponse makes a final LLM call without tools when max iterations // are exhausted, forcing a text response. func (al *AgentLoop) forceTextResponse(ctx context.Context, agent *AgentInstance, messages []providers.Message) string { @@ -2536,3 +2741,16 @@ func (al *AgentLoop) updateToolContexts(agent *AgentInstance, channel, chatID st } } } + +// Helper to extract provider from registry for cleanup +func extractProvider(registry *AgentRegistry) (providers.LLMProvider, bool) { + if registry == nil { + return nil, false + } + // Get any agent to access the provider + defaultAgent := registry.GetDefaultAgent() + if defaultAgent == nil { + return nil, false + } + return defaultAgent.Provider, true +} diff --git a/pkg/agent/loop_mcp.go b/pkg/agent/loop_mcp.go index 2795db52a..962789a06 100644 --- a/pkg/agent/loop_mcp.go +++ b/pkg/agent/loop_mcp.go @@ -63,6 +63,22 @@ func (al *AgentLoop) ensureMCPInitialized(ctx context.Context) error { return nil } + if al.cfg.Tools.MCP.Servers == nil || len(al.cfg.Tools.MCP.Servers) == 0 { + logger.WarnCF("agent", "MCP is enabled but no servers are configured, skipping MCP initialization", nil) + return nil + } + + findValidServer := false + for _, serverCfg := range al.cfg.Tools.MCP.Servers { + if serverCfg.Enabled { + findValidServer = true + } + } + if !findValidServer { + logger.WarnCF("agent", "MCP is enabled but no valid servers are configured, skipping MCP initialization", nil) + return nil + } + al.mcp.initOnce.Do(func() { mcpManager := mcp.NewManager() diff --git a/pkg/agent/loop_media.go b/pkg/agent/loop_media.go index 82547a008..1380f0214 100644 --- a/pkg/agent/loop_media.go +++ b/pkg/agent/loop_media.go @@ -20,9 +20,10 @@ import ( "github.com/sipeed/picoclaw/pkg/providers" ) -// resolveMediaRefs replaces media:// refs in message Media fields with base64 data URLs. -// Uses streaming base64 encoding (file handle → encoder → buffer) to avoid holding -// both raw bytes and encoded string in memory simultaneously. +// resolveMediaRefs resolves media:// refs in messages. +// Images are base64-encoded into the Media array for multimodal LLMs. +// Non-image files (documents, audio, video) have their local path injected +// into Content so the agent can access them via file tools like read_file. // Returns a new slice; original messages are not mutated. func resolveMediaRefs(messages []providers.Message, store media.MediaStore, maxSize int) []providers.Message { if store == nil { @@ -38,6 +39,8 @@ func resolveMediaRefs(messages []providers.Message, store media.MediaStore, maxS } resolved := make([]string, 0, len(m.Media)) + var pathTags []string + for _, ref := range m.Media { if !strings.HasPrefix(ref, "media://") { resolved = append(resolved, ref) @@ -61,62 +64,117 @@ func resolveMediaRefs(messages []providers.Message, store media.MediaStore, maxS }) continue } - if info.Size() > int64(maxSize) { - logger.WarnCF("agent", "Media file too large, skipping", map[string]any{ - "path": localPath, - "size": info.Size(), - "max_size": maxSize, - }) - continue - } - // Determine MIME type: prefer metadata, fallback to magic-bytes detection - mime := meta.ContentType - if mime == "" { - kind, ftErr := filetype.MatchFile(localPath) - if ftErr != nil || kind == filetype.Unknown { - logger.WarnCF("agent", "Unknown media type, skipping", map[string]any{ - "path": localPath, - }) - continue + mime := detectMIME(localPath, meta) + + if strings.HasPrefix(mime, "image/") { + dataURL := encodeImageToDataURL(localPath, mime, info, maxSize) + if dataURL != "" { + resolved = append(resolved, dataURL) } - mime = kind.MIME.Value - } - - // Streaming base64: open file → base64 encoder → buffer - // Peak memory: ~1.33x file size (buffer only, no raw bytes copy) - f, err := os.Open(localPath) - if err != nil { - logger.WarnCF("agent", "Failed to open media file", map[string]any{ - "path": localPath, - "error": err.Error(), - }) continue } - prefix := "data:" + mime + ";base64," - encodedLen := base64.StdEncoding.EncodedLen(int(info.Size())) - var buf bytes.Buffer - buf.Grow(len(prefix) + encodedLen) - buf.WriteString(prefix) - - encoder := base64.NewEncoder(base64.StdEncoding, &buf) - if _, err := io.Copy(encoder, f); err != nil { - f.Close() - logger.WarnCF("agent", "Failed to encode media file", map[string]any{ - "path": localPath, - "error": err.Error(), - }) - continue - } - encoder.Close() - f.Close() - - resolved = append(resolved, buf.String()) + pathTags = append(pathTags, buildPathTag(mime, localPath)) } result[i].Media = resolved + if len(pathTags) > 0 { + result[i].Content = injectPathTags(result[i].Content, pathTags) + } } return result } + +// detectMIME determines the MIME type from metadata or magic-bytes detection. +// Returns empty string if detection fails. +func detectMIME(localPath string, meta media.MediaMeta) string { + if meta.ContentType != "" { + return meta.ContentType + } + kind, err := filetype.MatchFile(localPath) + if err != nil || kind == filetype.Unknown { + return "" + } + return kind.MIME.Value +} + +// encodeImageToDataURL base64-encodes an image file into a data URL. +// Returns empty string if the file exceeds maxSize or encoding fails. +func encodeImageToDataURL(localPath, mime string, info os.FileInfo, maxSize int) string { + if info.Size() > int64(maxSize) { + logger.WarnCF("agent", "Media file too large, skipping", map[string]any{ + "path": localPath, + "size": info.Size(), + "max_size": maxSize, + }) + return "" + } + + f, err := os.Open(localPath) + if err != nil { + logger.WarnCF("agent", "Failed to open media file", map[string]any{ + "path": localPath, + "error": err.Error(), + }) + return "" + } + defer f.Close() + + prefix := "data:" + mime + ";base64," + encodedLen := base64.StdEncoding.EncodedLen(int(info.Size())) + var buf bytes.Buffer + buf.Grow(len(prefix) + encodedLen) + buf.WriteString(prefix) + + encoder := base64.NewEncoder(base64.StdEncoding, &buf) + if _, err := io.Copy(encoder, f); err != nil { + logger.WarnCF("agent", "Failed to encode media file", map[string]any{ + "path": localPath, + "error": err.Error(), + }) + return "" + } + encoder.Close() + + return buf.String() +} + +// buildPathTag creates a structured tag exposing the local file path. +// Tag type is derived from MIME: [audio:/path], [video:/path], or [file:/path]. +func buildPathTag(mime, localPath string) string { + switch { + case strings.HasPrefix(mime, "audio/"): + return "[audio:" + localPath + "]" + case strings.HasPrefix(mime, "video/"): + return "[video:" + localPath + "]" + default: + return "[file:" + localPath + "]" + } +} + +// injectPathTags replaces generic media tags in content with path-bearing versions, +// or appends if no matching generic tag is found. +func injectPathTags(content string, tags []string) string { + for _, tag := range tags { + var generic string + switch { + case strings.HasPrefix(tag, "[audio:"): + generic = "[audio]" + case strings.HasPrefix(tag, "[video:"): + generic = "[video]" + case strings.HasPrefix(tag, "[file:"): + generic = "[file]" + } + + if generic != "" && strings.Contains(content, generic) { + content = strings.Replace(content, generic, tag, 1) + } else if content == "" { + content = tag + } else { + content += " " + tag + } + } + return content +} diff --git a/pkg/agent/loop_test.go b/pkg/agent/loop_test.go index e2781e935..b165f53c4 100644 --- a/pkg/agent/loop_test.go +++ b/pkg/agent/loop_test.go @@ -770,13 +770,18 @@ func TestAgentLoop_ContextExhaustionRetry(t *testing.T) { } } -func TestProcessDirectWithChannel_InitializesMCPInAgentMode(t *testing.T) { +// TestProcessDirectWithChannel_TriggersMCPInitialization verifies that +// ProcessDirectWithChannel triggers MCP initialization when MCP is enabled. +// Note: Manager is only initialized when at least one MCP server is configured +// and successfully connected. +func TestProcessDirectWithChannel_TriggersMCPInitialization(t *testing.T) { tmpDir, err := os.MkdirTemp("", "agent-test-*") if err != nil { t.Fatalf("Failed to create temp dir: %v", err) } defer os.RemoveAll(tmpDir) + // Test with MCP enabled but no servers - should not initialize manager cfg := &config.Config{ Agents: config.AgentsConfig{ Defaults: config.AgentDefaults{ @@ -791,6 +796,7 @@ func TestProcessDirectWithChannel_InitializesMCPInAgentMode(t *testing.T) { ToolConfig: config.ToolConfig{ Enabled: true, }, + // No servers configured - manager should not be initialized }, }, } @@ -815,8 +821,9 @@ func TestProcessDirectWithChannel_InitializesMCPInAgentMode(t *testing.T) { t.Fatalf("ProcessDirectWithChannel failed: %v", err) } - if !al.mcp.hasManager() { - t.Fatal("expected MCP manager to be initialized in direct agent mode") + // Manager should not be initialized when no servers are configured + if al.mcp.hasManager() { + t.Fatal("expected MCP manager to be nil when no servers are configured") } } @@ -1088,7 +1095,7 @@ func TestResolveMediaRefs_SkipsOversizedFile(t *testing.T) { } } -func TestResolveMediaRefs_SkipsUnknownType(t *testing.T) { +func TestResolveMediaRefs_UnknownTypeInjectsPath(t *testing.T) { store := media.NewFileMediaStore() dir := t.TempDir() @@ -1104,7 +1111,11 @@ func TestResolveMediaRefs_SkipsUnknownType(t *testing.T) { result := resolveMediaRefs(messages, store, config.DefaultMaxMediaSize) if len(result[0].Media) != 0 { - t.Fatalf("expected 0 media (unknown type), got %d", len(result[0].Media)) + t.Fatalf("expected 0 media entries, got %d", len(result[0].Media)) + } + expected := "hi [file:" + txtPath + "]" + if result[0].Content != expected { + t.Fatalf("expected content %q, got %q", expected, result[0].Content) } } @@ -1166,3 +1177,144 @@ func TestResolveMediaRefs_UsesMetaContentType(t *testing.T) { t.Fatalf("expected jpeg prefix, got %q", result[0].Media[0][:30]) } } + +func TestResolveMediaRefs_PDFInjectsFilePath(t *testing.T) { + store := media.NewFileMediaStore() + dir := t.TempDir() + + pdfPath := filepath.Join(dir, "report.pdf") + // PDF magic bytes + os.WriteFile(pdfPath, []byte("%PDF-1.4 test content"), 0o644) + ref, _ := store.Store(pdfPath, media.MediaMeta{ContentType: "application/pdf"}, "test") + + messages := []providers.Message{ + {Role: "user", Content: "report.pdf [file]", Media: []string{ref}}, + } + result := resolveMediaRefs(messages, store, config.DefaultMaxMediaSize) + + if len(result[0].Media) != 0 { + t.Fatalf("expected 0 media (non-image), got %d", len(result[0].Media)) + } + expected := "report.pdf [file:" + pdfPath + "]" + if result[0].Content != expected { + t.Fatalf("expected content %q, got %q", expected, result[0].Content) + } +} + +func TestResolveMediaRefs_AudioInjectsAudioPath(t *testing.T) { + store := media.NewFileMediaStore() + dir := t.TempDir() + + oggPath := filepath.Join(dir, "voice.ogg") + os.WriteFile(oggPath, []byte("fake audio"), 0o644) + ref, _ := store.Store(oggPath, media.MediaMeta{ContentType: "audio/ogg"}, "test") + + messages := []providers.Message{ + {Role: "user", Content: "voice.ogg [audio]", Media: []string{ref}}, + } + result := resolveMediaRefs(messages, store, config.DefaultMaxMediaSize) + + if len(result[0].Media) != 0 { + t.Fatalf("expected 0 media, got %d", len(result[0].Media)) + } + expected := "voice.ogg [audio:" + oggPath + "]" + if result[0].Content != expected { + t.Fatalf("expected content %q, got %q", expected, result[0].Content) + } +} + +func TestResolveMediaRefs_VideoInjectsVideoPath(t *testing.T) { + store := media.NewFileMediaStore() + dir := t.TempDir() + + mp4Path := filepath.Join(dir, "clip.mp4") + os.WriteFile(mp4Path, []byte("fake video"), 0o644) + ref, _ := store.Store(mp4Path, media.MediaMeta{ContentType: "video/mp4"}, "test") + + messages := []providers.Message{ + {Role: "user", Content: "clip.mp4 [video]", Media: []string{ref}}, + } + result := resolveMediaRefs(messages, store, config.DefaultMaxMediaSize) + + if len(result[0].Media) != 0 { + t.Fatalf("expected 0 media, got %d", len(result[0].Media)) + } + expected := "clip.mp4 [video:" + mp4Path + "]" + if result[0].Content != expected { + t.Fatalf("expected content %q, got %q", expected, result[0].Content) + } +} + +func TestResolveMediaRefs_NoGenericTagAppendsPath(t *testing.T) { + store := media.NewFileMediaStore() + dir := t.TempDir() + + csvPath := filepath.Join(dir, "data.csv") + os.WriteFile(csvPath, []byte("a,b,c"), 0o644) + ref, _ := store.Store(csvPath, media.MediaMeta{ContentType: "text/csv"}, "test") + + messages := []providers.Message{ + {Role: "user", Content: "here is my data", Media: []string{ref}}, + } + result := resolveMediaRefs(messages, store, config.DefaultMaxMediaSize) + + expected := "here is my data [file:" + csvPath + "]" + if result[0].Content != expected { + t.Fatalf("expected content %q, got %q", expected, result[0].Content) + } +} + +func TestResolveMediaRefs_EmptyContentGetsPathTag(t *testing.T) { + store := media.NewFileMediaStore() + dir := t.TempDir() + + docPath := filepath.Join(dir, "doc.docx") + os.WriteFile(docPath, []byte("fake docx"), 0o644) + docxMIME := "application/vnd.openxmlformats-officedocument.wordprocessingml.document" + ref, _ := store.Store(docPath, media.MediaMeta{ContentType: docxMIME}, "test") + + messages := []providers.Message{ + {Role: "user", Content: "", Media: []string{ref}}, + } + result := resolveMediaRefs(messages, store, config.DefaultMaxMediaSize) + + expected := "[file:" + docPath + "]" + if result[0].Content != expected { + t.Fatalf("expected content %q, got %q", expected, result[0].Content) + } +} + +func TestResolveMediaRefs_MixedImageAndFile(t *testing.T) { + store := media.NewFileMediaStore() + dir := t.TempDir() + + pngPath := filepath.Join(dir, "photo.png") + pngHeader := []byte{ + 0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A, + 0x00, 0x00, 0x00, 0x0D, 0x49, 0x48, 0x44, 0x52, + 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x01, 0x08, 0x02, + 0x00, 0x00, 0x00, 0x90, 0x77, 0x53, 0xDE, + } + os.WriteFile(pngPath, pngHeader, 0o644) + imgRef, _ := store.Store(pngPath, media.MediaMeta{}, "test") + + pdfPath := filepath.Join(dir, "report.pdf") + os.WriteFile(pdfPath, []byte("%PDF-1.4 test"), 0o644) + fileRef, _ := store.Store(pdfPath, media.MediaMeta{ContentType: "application/pdf"}, "test") + + messages := []providers.Message{ + {Role: "user", Content: "check these [file]", Media: []string{imgRef, fileRef}}, + } + result := resolveMediaRefs(messages, store, config.DefaultMaxMediaSize) + + if len(result[0].Media) != 1 { + t.Fatalf("expected 1 media (image only), got %d", len(result[0].Media)) + } + if !strings.HasPrefix(result[0].Media[0], "data:image/png;base64,") { + t.Fatal("expected image to be base64 encoded") + } + expectedContent := "check these [file:" + pdfPath + "]" + if result[0].Content != expectedContent { + t.Fatalf("expected content %q, got %q", expectedContent, result[0].Content) + } +} diff --git a/pkg/channels/qq/qq.go b/pkg/channels/qq/qq.go index 73200f64e..4cb4db3c6 100644 --- a/pkg/channels/qq/qq.go +++ b/pkg/channels/qq/qq.go @@ -423,7 +423,9 @@ func (c *QQChannel) handleC2CMessage() event.C2CMessageEventHandler { // Reset msg_seq counter for new inbound message. c.msgSeqCounters.Store(senderID, new(atomic.Uint64)) - metadata := map[string]string{} + metadata := map[string]string{ + "account_id": senderID, + } sender := bus.SenderInfo{ Platform: "qq", @@ -495,7 +497,8 @@ func (c *QQChannel) handleGroupATMessage() event.GroupATMessageEventHandler { c.msgSeqCounters.Store(data.GroupID, new(atomic.Uint64)) metadata := map[string]string{ - "group_id": data.GroupID, + "account_id": senderID, + "group_id": data.GroupID, } sender := bus.SenderInfo{ diff --git a/pkg/channels/qq/qq_test.go b/pkg/channels/qq/qq_test.go new file mode 100644 index 000000000..3ceee0d09 --- /dev/null +++ b/pkg/channels/qq/qq_test.go @@ -0,0 +1,44 @@ +package qq + +import ( + "context" + "testing" + "time" + + "github.com/tencent-connect/botgo/dto" + + "github.com/sipeed/picoclaw/pkg/bus" + "github.com/sipeed/picoclaw/pkg/channels" +) + +func TestHandleC2CMessage_IncludesAccountIDMetadata(t *testing.T) { + messageBus := bus.NewMessageBus() + ch := &QQChannel{ + BaseChannel: channels.NewBaseChannel("qq", nil, messageBus, nil), + dedup: make(map[string]time.Time), + done: make(chan struct{}), + ctx: context.Background(), + } + + err := ch.handleC2CMessage()(nil, &dto.WSC2CMessageData{ + ID: "msg-1", + Content: "hello", + Author: &dto.User{ + ID: "7750283E123456", + }, + }) + if err != nil { + t.Fatalf("handleC2CMessage() error = %v", err) + } + + ctx, cancel := context.WithTimeout(context.Background(), time.Second) + defer cancel() + + inbound, ok := messageBus.ConsumeInbound(ctx) + if !ok { + t.Fatal("expected inbound message") + } + if inbound.Metadata["account_id"] != "7750283E123456" { + t.Fatalf("account_id metadata = %q, want %q", inbound.Metadata["account_id"], "7750283E123456") + } +} diff --git a/pkg/config/config.go b/pkg/config/config.go index 58b3215cc..9649d519f 100644 --- a/pkg/config/config.go +++ b/pkg/config/config.go @@ -224,8 +224,8 @@ type AgentDefaults struct { RestrictToWorkspace bool `json:"restrict_to_workspace" env:"PICOCLAW_AGENTS_DEFAULTS_RESTRICT_TO_WORKSPACE"` AllowReadOutsideWorkspace bool `json:"allow_read_outside_workspace" env:"PICOCLAW_AGENTS_DEFAULTS_ALLOW_READ_OUTSIDE_WORKSPACE"` Provider string `json:"provider" env:"PICOCLAW_AGENTS_DEFAULTS_PROVIDER"` - ModelName string `json:"model_name,omitempty" env:"PICOCLAW_AGENTS_DEFAULTS_MODEL_NAME"` - Model string `json:"model" env:"PICOCLAW_AGENTS_DEFAULTS_MODEL"` // Deprecated: use model_name instead + ModelName string `json:"model_name" env:"PICOCLAW_AGENTS_DEFAULTS_MODEL_NAME"` + Model string `json:"model,omitempty" env:"PICOCLAW_AGENTS_DEFAULTS_MODEL"` // Deprecated: use model_name instead ModelFallbacks []string `json:"model_fallbacks,omitempty"` PlanModel string `json:"plan_model,omitempty" env:"PICOCLAW_AGENTS_DEFAULTS_PLAN_MODEL"` PlanModelFallbacks []string `json:"plan_model_fallbacks,omitempty"` @@ -537,6 +537,7 @@ type ProvidersConfig struct { Avian ProviderConfig `json:"avian"` Minimax ProviderConfig `json:"minimax"` LongCat ProviderConfig `json:"longcat"` + ModelScope ProviderConfig `json:"modelscope"` } // IsEmpty checks if all provider configs are empty (no API keys or API bases set) @@ -564,7 +565,8 @@ func (p ProvidersConfig) IsEmpty() bool { p.Mistral.APIKey == "" && p.Mistral.APIBase == "" && p.Avian.APIKey == "" && p.Avian.APIBase == "" && p.Minimax.APIKey == "" && p.Minimax.APIBase == "" && - p.LongCat.APIKey == "" && p.LongCat.APIBase == "" + p.LongCat.APIKey == "" && p.LongCat.APIBase == "" && + p.ModelScope.APIKey == "" && p.ModelScope.APIBase == "" } // MarshalJSON implements custom JSON marshaling for ProvidersConfig @@ -721,6 +723,7 @@ type ExecConfig struct { type SkillsToolsConfig struct { ToolConfig ` envPrefix:"PICOCLAW_TOOLS_SKILLS_"` Registries SkillsRegistriesConfig ` json:"registries"` + Github SkillsGithubConfig ` json:"github"` MaxConcurrentSearches int ` json:"max_concurrent_searches" env:"PICOCLAW_TOOLS_SKILLS_MAX_CONCURRENT_SEARCHES"` SearchCache SearchCacheConfig ` json:"search_cache"` } @@ -770,6 +773,11 @@ type SkillsRegistriesConfig struct { ClawHub ClawHubRegistryConfig `json:"clawhub"` } +type SkillsGithubConfig struct { + Token string `json:"token,omitempty" env:"PICOCLAW_TOOLS_SKILLS_GITHUB_AUTH_TOKEN"` + Proxy string `json:"proxy,omitempty" env:"PICOCLAW_TOOLS_SKILLS_GITHUB_PROXY"` +} + type ClawHubRegistryConfig struct { Enabled bool `json:"enabled" env:"PICOCLAW_SKILLS_REGISTRIES_CLAWHUB_ENABLED"` BaseURL string `json:"base_url" env:"PICOCLAW_SKILLS_REGISTRIES_CLAWHUB_BASE_URL"` diff --git a/pkg/config/config_test.go b/pkg/config/config_test.go index 1c93028c7..c5bdbf3c3 100644 --- a/pkg/config/config_test.go +++ b/pkg/config/config_test.go @@ -342,8 +342,8 @@ func TestSaveConfig_IncludesEmptyLegacyModelField(t *testing.T) { t.Fatalf("ReadFile failed: %v", err) } - if !strings.Contains(string(data), `"model": ""`) { - t.Fatalf("saved config should include empty legacy model field, got: %s", string(data)) + if !strings.Contains(string(data), `"model_name": ""`) { + t.Fatalf("saved config should include empty legacy model_name field, got: %s", string(data)) } } diff --git a/pkg/config/defaults.go b/pkg/config/defaults.go index 2a3e66043..189af0a84 100644 --- a/pkg/config/defaults.go +++ b/pkg/config/defaults.go @@ -369,6 +369,14 @@ func DefaultConfig() *Config { APIKey: "", }, + // ModelScope (魔搭社区) - https://modelscope.cn/my/tokens + { + ModelName: "modelscope-qwen", + Model: "modelscope/Qwen/Qwen3-235B-A22B-Instruct-2507", + APIBase: "https://api-inference.modelscope.cn/v1", + APIKey: "", + }, + // VLLM (local) - http://localhost:8000 { ModelName: "local-model", diff --git a/pkg/config/migration.go b/pkg/config/migration.go index af6391651..c7fc214d5 100644 --- a/pkg/config/migration.go +++ b/pkg/config/migration.go @@ -424,6 +424,23 @@ func ConvertProvidersToModelList(cfg *Config) []ModelConfig { }, true }, }, + { + providerNames: []string{"modelscope"}, + protocol: "modelscope", + buildConfig: func(p ProvidersConfig) (ModelConfig, bool) { + if p.ModelScope.APIKey == "" && p.ModelScope.APIBase == "" { + return ModelConfig{}, false + } + return ModelConfig{ + ModelName: "modelscope", + Model: "modelscope/Qwen/Qwen3-235B-A22B-Instruct-2507", + APIKey: p.ModelScope.APIKey, + APIBase: p.ModelScope.APIBase, + Proxy: p.ModelScope.Proxy, + RequestTimeout: p.ModelScope.RequestTimeout, + }, true + }, + }, } // Process each provider migration diff --git a/pkg/config/migration_test.go b/pkg/config/migration_test.go index 0665ededa..1b6e5b032 100644 --- a/pkg/config/migration_test.go +++ b/pkg/config/migration_test.go @@ -163,14 +163,15 @@ func TestConvertProvidersToModelList_AllProviders(t *testing.T) { Mistral: ProviderConfig{APIKey: "key18"}, Avian: ProviderConfig{APIKey: "key19"}, LongCat: ProviderConfig{APIKey: "key-longcat"}, + ModelScope: ProviderConfig{APIKey: "key-modelscope"}, }, } result := ConvertProvidersToModelList(cfg) - // All 22 providers should be converted - if len(result) != 22 { - t.Errorf("len(result) = %d, want 22", len(result)) + // All 23 providers should be converted + if len(result) != 23 { + t.Errorf("len(result) = %d, want 23", len(result)) } } diff --git a/pkg/identity/identity.go b/pkg/identity/identity.go index 6bc09c210..372bbe38b 100644 --- a/pkg/identity/identity.go +++ b/pkg/identity/identity.go @@ -59,6 +59,9 @@ func MatchAllowed(sender bus.SenderInfo, allowed string) bool { } } + // Keep track of explicit username format + isAtUsername := strings.HasPrefix(allowed, "@") + // Strip leading "@" for username matching trimmed := strings.TrimPrefix(allowed, "@") @@ -75,11 +78,9 @@ func MatchAllowed(sender bus.SenderInfo, allowed string) bool { return true } - // Match against Username - if sender.Username != "" { - if sender.Username == trimmed || sender.Username == allowedUser { - return true - } + // Match against Username only when explicitly requested via "@username" + if isAtUsername && sender.Username != "" && sender.Username == trimmed { + return true } // Match compound sender format against allowed parts diff --git a/pkg/identity/identity_test.go b/pkg/identity/identity_test.go index 3d24bd794..a588f1484 100644 --- a/pkg/identity/identity_test.go +++ b/pkg/identity/identity_test.go @@ -104,6 +104,16 @@ func TestMatchAllowed(t *testing.T) { allowed: "@alice", want: true, }, + { + name: "plain entry does not match username", + sender: bus.SenderInfo{ + Platform: "discord", + PlatformID: "999999", + Username: "123456", + }, + allowed: "123456", + want: false, + }, { name: "@username does not match", sender: telegramSender, @@ -123,6 +133,16 @@ func TestMatchAllowed(t *testing.T) { allowed: "999|alice", want: true, }, + { + name: "compound matches by ID when username differs", + sender: bus.SenderInfo{ + Platform: "discord", + PlatformID: "123456", + Username: "not123456", + }, + allowed: "123456|alice", + want: true, + }, { name: "compound does not match", sender: telegramSender, diff --git a/pkg/logger/logger.go b/pkg/logger/logger.go index 0c5894ea0..e8f408037 100644 --- a/pkg/logger/logger.go +++ b/pkg/logger/logger.go @@ -317,6 +317,10 @@ func DebugC(component string, message string) { logMessage(DEBUG, component, message, nil) } +func Debugf(message string, ss ...any) { + logMessage(DEBUG, "", fmt.Sprintf(message, ss...), nil) +} + func DebugF(message string, fields map[string]any) { logMessage(DEBUG, "", message, fields) } @@ -337,6 +341,10 @@ func InfoF(message string, fields map[string]any) { logMessage(INFO, "", message, fields) } +func Infof(message string, ss ...any) { + logMessage(INFO, "", fmt.Sprintf(message, ss...), nil) +} + func InfoCF(component string, message string, fields map[string]any) { logMessage(INFO, component, message, fields) } @@ -365,6 +373,10 @@ func ErrorC(component string, message string) { logMessage(ERROR, component, message, nil) } +func Errorf(message string, ss ...any) { + logMessage(ERROR, "", fmt.Sprintf(message, ss...), nil) +} + func ErrorF(message string, fields map[string]any) { logMessage(ERROR, "", message, fields) } diff --git a/pkg/logger/logger_test.go b/pkg/logger/logger_test.go index 6e6f8dfa8..8170a618b 100644 --- a/pkg/logger/logger_test.go +++ b/pkg/logger/logger_test.go @@ -123,17 +123,21 @@ func TestLoggerHelperFunctions(t *testing.T) { SetLevel(INFO) Debug("This should not log") + Debugf("this should not log") Info("This should log") Warn("This should log") Error("This should log") InfoC("test", "Component message") InfoF("Fields message", map[string]any{"key": "value"}) + Infof("test from %v", "Infof") WarnC("test", "Warning with component") ErrorF("Error with fields", map[string]any{"error": "test"}) + Errorf("test from %v", "Errorf") SetLevel(DEBUG) DebugC("test", "Debug with component") + Debugf("test from %v", "Debugf") WarnF("Warning with fields", map[string]any{"key": "value"}) } diff --git a/pkg/providers/anthropic_messages/provider.go b/pkg/providers/anthropic_messages/provider.go new file mode 100644 index 000000000..7f1b92ce0 --- /dev/null +++ b/pkg/providers/anthropic_messages/provider.go @@ -0,0 +1,420 @@ +// PicoClaw - Ultra-lightweight personal AI agent +// License: MIT +// +// Copyright (c) 2026 PicoClaw contributors + +package anthropicmessages + +import ( + "bytes" + "context" + "encoding/json" + "fmt" + "io" + "net/http" + "net/url" + "strings" + "time" + + "github.com/sipeed/picoclaw/pkg/providers/protocoltypes" +) + +type ( + ToolCall = protocoltypes.ToolCall + FunctionCall = protocoltypes.FunctionCall + LLMResponse = protocoltypes.LLMResponse + UsageInfo = protocoltypes.UsageInfo + Message = protocoltypes.Message + ToolDefinition = protocoltypes.ToolDefinition + ToolFunctionDefinition = protocoltypes.ToolFunctionDefinition +) + +const ( + defaultAPIVersion = "2023-06-01" + defaultBaseURL = "https://api.anthropic.com/v1" + defaultRequestTimeout = 120 * time.Second +) + +// Provider implements Anthropic Messages API via HTTP (without SDK). +// It supports custom endpoints that use Anthropic's native message format. +type Provider struct { + apiKey string + apiBase string + httpClient *http.Client +} + +// NewProvider creates a new Anthropic Messages API provider. +func NewProvider(apiKey, apiBase string) *Provider { + return NewProviderWithTimeout(apiKey, apiBase, 0) +} + +// NewProviderWithTimeout creates a provider with custom request timeout. +func NewProviderWithTimeout(apiKey, apiBase string, timeoutSeconds int) *Provider { + baseURL := normalizeBaseURL(apiBase) + timeout := defaultRequestTimeout + if timeoutSeconds > 0 { + timeout = time.Duration(timeoutSeconds) * time.Second + } + + return &Provider{ + apiKey: apiKey, + apiBase: baseURL, + httpClient: &http.Client{ + Timeout: timeout, + }, + } +} + +// Chat sends messages to the Anthropic Messages API and returns the response. +func (p *Provider) Chat( + ctx context.Context, + messages []Message, + tools []ToolDefinition, + model string, + options map[string]any, +) (*LLMResponse, error) { + if p.apiKey == "" { + return nil, fmt.Errorf("API key not configured") + } + + // Build request body + requestBody, err := buildRequestBody(messages, tools, model, options) + if err != nil { + return nil, fmt.Errorf("building request body: %w", err) + } + + // Serialize to JSON + jsonBody, err := json.Marshal(requestBody) + if err != nil { + return nil, fmt.Errorf("serializing request body: %w", err) + } + + // Build request URL + endpointURL, err := url.JoinPath(p.apiBase, "messages") + if err != nil { + return nil, fmt.Errorf("building endpoint URL: %w", err) + } + + // Create HTTP request + req, err := http.NewRequestWithContext(ctx, "POST", endpointURL, bytes.NewReader(jsonBody)) + if err != nil { + return nil, fmt.Errorf("creating HTTP request: %w", err) + } + + // Set headers + req.Header.Set("Content-Type", "application/json") + req.Header.Set("X-API-Key", p.apiKey) //nolint:canonicalheader // Anthropic API requires exact header name + req.Header.Set("Anthropic-Version", defaultAPIVersion) + + // Execute request + resp, err := p.httpClient.Do(req) + if err != nil { + return nil, fmt.Errorf("executing HTTP request: %w", err) + } + defer resp.Body.Close() + + // Read response body + body, err := io.ReadAll(resp.Body) + if err != nil { + return nil, fmt.Errorf("reading response body: %w", err) + } + + // Check for HTTP errors with detailed messages + switch resp.StatusCode { + case http.StatusUnauthorized: + return nil, fmt.Errorf("authentication failed (401): check your API key") + case http.StatusTooManyRequests: + return nil, fmt.Errorf("rate limited (429): %s", string(body)) + case http.StatusBadRequest: + return nil, fmt.Errorf("bad request (400): %s", string(body)) + case http.StatusNotFound: + return nil, fmt.Errorf("endpoint not found (404): %s", string(body)) + case http.StatusInternalServerError: + return nil, fmt.Errorf("internal server error (500): %s", string(body)) + case http.StatusServiceUnavailable: + return nil, fmt.Errorf("service unavailable (503): %s", string(body)) + default: + if resp.StatusCode != http.StatusOK { + return nil, fmt.Errorf("API request failed with status %d: %s", resp.StatusCode, string(body)) + } + } + + // Parse response + return parseResponseBody(body) +} + +// GetDefaultModel returns the default model for this provider. +func (p *Provider) GetDefaultModel() string { + return "claude-sonnet-4.6" +} + +// buildRequestBody converts internal message format to Anthropic Messages API format. +func buildRequestBody( + messages []Message, + tools []ToolDefinition, + model string, + options map[string]any, +) (map[string]any, error) { + // max_tokens is required and guaranteed by agent loop + maxTokens, ok := asInt(options["max_tokens"]) + if !ok { + return nil, fmt.Errorf("max_tokens is required in options") + } + + result := map[string]any{ + "model": model, + "max_tokens": int64(maxTokens), + "messages": []any{}, + } + + // Set temperature from options + if temp, ok := asFloat(options["temperature"]); ok { + result["temperature"] = temp + } + + // Process messages + var systemPrompt string + var apiMessages []any + + for _, msg := range messages { + switch msg.Role { + case "system": + // Accumulate system messages + if systemPrompt != "" { + systemPrompt += "\n\n" + msg.Content + } else { + systemPrompt = msg.Content + } + + case "user": + if msg.ToolCallID != "" { + // Tool result message + content := []map[string]any{ + { + "type": "tool_result", + "tool_use_id": msg.ToolCallID, + "content": msg.Content, + }, + } + apiMessages = append(apiMessages, map[string]any{ + "role": "user", + "content": content, + }) + } else { + // Regular user message + apiMessages = append(apiMessages, map[string]any{ + "role": "user", + "content": msg.Content, + }) + } + + case "assistant": + content := []any{} + + // Add text content if present + if msg.Content != "" { + content = append(content, map[string]any{ + "type": "text", + "text": msg.Content, + }) + } + + // Add tool_use blocks + for _, tc := range msg.ToolCalls { + toolUse := map[string]any{ + "type": "tool_use", + "id": tc.ID, + "name": tc.Name, + "input": tc.Arguments, + } + content = append(content, toolUse) + } + + apiMessages = append(apiMessages, map[string]any{ + "role": "assistant", + "content": content, + }) + + case "tool": + // Tool result (alternative format) + content := []map[string]any{ + { + "type": "tool_result", + "tool_use_id": msg.ToolCallID, + "content": msg.Content, + }, + } + apiMessages = append(apiMessages, map[string]any{ + "role": "user", + "content": content, + }) + } + } + + result["messages"] = apiMessages + + // Set system prompt if present + if systemPrompt != "" { + result["system"] = systemPrompt + } + + // Add tools if present + if len(tools) > 0 { + result["tools"] = buildTools(tools) + } + + return result, nil +} + +// buildTools converts tool definitions to Anthropic format. +func buildTools(tools []ToolDefinition) []any { + result := make([]any, len(tools)) + for i, tool := range tools { + // Unmarshal Parameters (json.RawMessage) into map so it serialises + // correctly as a nested JSON object rather than a raw byte string. + var schema any + if len(tool.Function.Parameters) > 0 { + _ = json.Unmarshal(tool.Function.Parameters, &schema) + } + toolDef := map[string]any{ + "name": tool.Function.Name, + "description": tool.Function.Description, + "input_schema": schema, + } + result[i] = toolDef + } + return result +} + +// parseResponseBody parses Anthropic Messages API response. +func parseResponseBody(body []byte) (*LLMResponse, error) { + var resp anthropicMessageResponse + if err := json.Unmarshal(body, &resp); err != nil { + return nil, fmt.Errorf("parsing JSON response: %w", err) + } + + // Extract content and tool calls + var content strings.Builder + toolCalls := make([]ToolCall, 0) // Initialize as empty slice (not nil) for consistent JSON serialization + + for _, block := range resp.Content { + switch block.Type { + case "text": + content.WriteString(block.Text) + case "tool_use": + toolCalls = append(toolCalls, ToolCall{ + ID: block.ID, + Name: block.Name, + Arguments: block.Input, + Function: &FunctionCall{ + Name: block.Name, + Arguments: block.Input, + }, + }) + } + } + + // Map stop_reason + finishReason := "stop" + switch resp.StopReason { + case "tool_use": + finishReason = "tool_calls" + case "max_tokens": + finishReason = "length" + case "end_turn": + finishReason = "stop" + case "stop_sequence": + finishReason = "stop" + } + + return &LLMResponse{ + Content: content.String(), + ToolCalls: toolCalls, + FinishReason: finishReason, + Usage: &UsageInfo{ + PromptTokens: int(resp.Usage.InputTokens), + CompletionTokens: int(resp.Usage.OutputTokens), + TotalTokens: int(resp.Usage.InputTokens + resp.Usage.OutputTokens), + }, + }, nil +} + +// normalizeBaseURL ensures the base URL is properly formatted. +// It removes /v1 suffix if present (to avoid duplication) and always appends /v1. +// This handles edge cases like "https://api.example.com/v1/proxy" correctly. +func normalizeBaseURL(apiBase string) string { + base := strings.TrimSpace(apiBase) + if base == "" { + return defaultBaseURL + } + + // Remove trailing slashes + base = strings.TrimRight(base, "/") + + // Remove /v1 suffix if present (will be re-added) + // This prevents duplication for URLs like "https://api.example.com/v1/proxy" + if before, ok := strings.CutSuffix(base, "/v1"); ok { + base = before + } + + // Ensure we don't have an empty string after cutting + if base == "" { + return defaultBaseURL + } + + // Add /v1 suffix (required by Anthropic Messages API) + return base + "/v1" +} + +// Helper functions for type conversion + +func asInt(v any) (int, bool) { + switch val := v.(type) { + case int: + return val, true + case float64: + return int(val), true + case int64: + return int(val), true + default: + return 0, false + } +} + +func asFloat(v any) (float64, bool) { + switch val := v.(type) { + case float64: + return val, true + case int: + return float64(val), true + case int64: + return float64(val), true + default: + return 0, false + } +} + +// Anthropic API response structures + +type anthropicMessageResponse struct { + ID string `json:"id"` + Type string `json:"type"` + Role string `json:"role"` + Content []contentBlock `json:"content"` + StopReason string `json:"stop_reason"` + Model string `json:"model"` + Usage usageInfo `json:"usage"` +} + +type contentBlock struct { + Type string `json:"type"` + Text string `json:"text,omitempty"` + ID string `json:"id,omitempty"` + Name string `json:"name,omitempty"` + Input map[string]any `json:"input,omitempty"` +} + +type usageInfo struct { + InputTokens int64 `json:"input_tokens"` + OutputTokens int64 `json:"output_tokens"` +} diff --git a/pkg/providers/anthropic_messages/provider_test.go b/pkg/providers/anthropic_messages/provider_test.go new file mode 100644 index 000000000..bde2d47d1 --- /dev/null +++ b/pkg/providers/anthropic_messages/provider_test.go @@ -0,0 +1,616 @@ +// PicoClaw - Ultra-lightweight personal AI agent +// License: MIT +// +// Copyright (c) 2026 PicoClaw contributors + +package anthropicmessages + +import ( + "context" + "encoding/json" + "reflect" + "strings" + "testing" +) + +func TestBuildRequestBody(t *testing.T) { + tests := []struct { + name string + messages []Message + tools []ToolDefinition + model string + options map[string]any + want map[string]any + wantErr bool + }{ + { + name: "basic user message", + messages: []Message{ + {Role: "user", Content: "Hello, world!"}, + }, + model: "test-model", + options: map[string]any{ + "max_tokens": 8192, + }, + want: map[string]any{ + "model": "test-model", + "max_tokens": int64(8192), + "messages": []any{ + map[string]any{ + "role": "user", + "content": "Hello, world!", + }, + }, + }, + }, + { + name: "user and assistant messages", + messages: []Message{ + {Role: "user", Content: "What is 2+2?"}, + {Role: "assistant", Content: "4"}, + }, + model: "test-model", + options: map[string]any{ + "max_tokens": 8192, + }, + want: map[string]any{ + "model": "test-model", + "max_tokens": int64(8192), + "messages": []any{ + map[string]any{ + "role": "user", + "content": "What is 2+2?", + }, + map[string]any{ + "role": "assistant", + "content": []any{ + map[string]any{ + "type": "text", + "text": "4", + }, + }, + }, + }, + }, + }, + { + name: "with system message", + messages: []Message{ + {Role: "system", Content: "You are a helpful assistant."}, + {Role: "user", Content: "Hello"}, + }, + model: "test-model", + options: map[string]any{ + "max_tokens": 8192, + }, + want: map[string]any{ + "model": "test-model", + "max_tokens": int64(8192), + "system": "You are a helpful assistant.", + "messages": []any{ + map[string]any{ + "role": "user", + "content": "Hello", + }, + }, + }, + }, + { + name: "with custom max_tokens and temperature", + messages: []Message{ + {Role: "user", Content: "Test"}, + }, + model: "test-model", + options: map[string]any{ + "max_tokens": 2048, + "temperature": 0.5, + }, + want: map[string]any{ + "model": "test-model", + "max_tokens": int64(2048), + "temperature": 0.5, + "messages": []any{ + map[string]any{ + "role": "user", + "content": "Test", + }, + }, + }, + }, + { + name: "missing max_tokens returns error", + messages: []Message{ + {Role: "user", Content: "Test"}, + }, + model: "test-model", + options: map[string]any{}, + want: nil, + wantErr: true, + }, + { + name: "with tools", + messages: []Message{ + {Role: "user", Content: "What's the weather?"}, + }, + tools: []ToolDefinition{ + { + Function: ToolFunctionDefinition{ + Name: "get_weather", + Description: "Get current weather", + Parameters: json.RawMessage(`{"type":"object","properties":{"location":{"type":"string","description":"City name"}}}`), + }, + }, + }, + model: "test-model", + options: map[string]any{ + "max_tokens": 8192, + }, + want: map[string]any{ + "model": "test-model", + "max_tokens": int64(8192), + "messages": []any{ + map[string]any{ + "role": "user", + "content": "What's the weather?", + }, + }, + "tools": []any{ + map[string]any{ + "name": "get_weather", + "description": "Get current weather", + "input_schema": map[string]any{ + "type": "object", + "properties": map[string]any{ + "location": map[string]any{ + "type": "string", + "description": "City name", + }, + }, + }, + }, + }, + }, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, err := buildRequestBody(tt.messages, tt.tools, tt.model, tt.options) + if (err != nil) != tt.wantErr { + t.Errorf("buildRequestBody() error = %v, wantErr %v", err, tt.wantErr) + return + } + if !reflect.DeepEqual(got, tt.want) { + gotJSON, _ := json.MarshalIndent(got, "", " ") + wantJSON, _ := json.MarshalIndent(tt.want, "", " ") + t.Errorf("buildRequestBody() mismatch:\ngot:\n%s\nwant:\n%s", gotJSON, wantJSON) + } + }) + } +} + +func TestParseResponseBody(t *testing.T) { + tests := []struct { + name string + body []byte + want *LLMResponse + wantErr bool + }{ + { + name: "basic text response", + body: []byte(`{ + "id": "msg-123", + "type": "message", + "role": "assistant", + "content": [ + {"type": "text", "text": "Hello, how can I help?"} + ], + "stop_reason": "end_turn", + "model": "test-model", + "usage": { + "input_tokens": 10, + "output_tokens": 5 + } + }`), + want: &LLMResponse{ + Content: "Hello, how can I help?", + ToolCalls: []ToolCall{}, + FinishReason: "stop", + Usage: &UsageInfo{ + PromptTokens: 10, + CompletionTokens: 5, + TotalTokens: 15, + }, + Reasoning: "", + ReasoningDetails: nil, + }, + wantErr: false, + }, + { + name: "response with tool use", + body: []byte(`{ + "id": "msg-456", + "type": "message", + "role": "assistant", + "content": [ + {"type": "text", "text": "I'll check the weather for you."}, + { + "type": "tool_use", + "id": "toolu-123", + "name": "get_weather", + "input": {"location": "Tokyo"} + } + ], + "stop_reason": "tool_use", + "model": "test-model", + "usage": { + "input_tokens": 20, + "output_tokens": 15 + } + }`), + want: &LLMResponse{ + Content: "I'll check the weather for you.", + ToolCalls: []ToolCall{ + { + ID: "toolu-123", + Name: "get_weather", + Arguments: map[string]any{ + "location": "Tokyo", + }, + Function: &FunctionCall{ + Name: "get_weather", + Arguments: map[string]any{ + "location": "Tokyo", + }, + }, + }, + }, + FinishReason: "tool_calls", + Usage: &UsageInfo{ + PromptTokens: 20, + CompletionTokens: 15, + TotalTokens: 35, + }, + Reasoning: "", + ReasoningDetails: nil, + }, + wantErr: false, + }, + { + name: "invalid JSON", + body: []byte(`invalid json`), + want: nil, + wantErr: true, + }, + { + name: "max_tokens stop reason", + body: []byte(`{ + "id": "msg-789", + "type": "message", + "role": "assistant", + "content": [ + {"type": "text", "text": "Partial response"} + ], + "stop_reason": "max_tokens", + "model": "test-model", + "usage": { + "input_tokens": 100, + "output_tokens": 4096 + } + }`), + want: &LLMResponse{ + Content: "Partial response", + ToolCalls: []ToolCall{}, + FinishReason: "length", + Usage: &UsageInfo{ + PromptTokens: 100, + CompletionTokens: 4096, + TotalTokens: 4196, + }, + Reasoning: "", + ReasoningDetails: nil, + }, + wantErr: false, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, err := parseResponseBody(tt.body) + if (err != nil) != tt.wantErr { + t.Errorf("parseResponseBody() error = %v, wantErr %v", err, tt.wantErr) + return + } + if err != nil { + return + } + + // Compare individual fields + if got.Content != tt.want.Content { + t.Errorf("Content = %q, want %q", got.Content, tt.want.Content) + } + if got.FinishReason != tt.want.FinishReason { + t.Errorf("FinishReason = %q, want %q", got.FinishReason, tt.want.FinishReason) + } + if got.Usage == nil && tt.want.Usage != nil { + t.Errorf("Usage = nil, want non-nil") + } else if got.Usage != nil && tt.want.Usage == nil { + t.Errorf("Usage = non-nil, want nil") + } else if got.Usage != nil && tt.want.Usage != nil { + if got.Usage.PromptTokens != tt.want.Usage.PromptTokens { + t.Errorf("Usage.PromptTokens = %d, want %d", got.Usage.PromptTokens, tt.want.Usage.PromptTokens) + } + if got.Usage.CompletionTokens != tt.want.Usage.CompletionTokens { + t.Errorf("Usage.CompletionTokens = %d, want %d", + got.Usage.CompletionTokens, tt.want.Usage.CompletionTokens) + } + if got.Usage.TotalTokens != tt.want.Usage.TotalTokens { + t.Errorf("Usage.TotalTokens = %d, want %d", got.Usage.TotalTokens, tt.want.Usage.TotalTokens) + } + } + if len(got.ToolCalls) != len(tt.want.ToolCalls) { + t.Errorf("ToolCalls length = %d, want %d", len(got.ToolCalls), len(tt.want.ToolCalls)) + } else { + for i := range got.ToolCalls { + if got.ToolCalls[i].ID != tt.want.ToolCalls[i].ID { + t.Errorf("ToolCalls[%d].ID = %q, want %q", + i, got.ToolCalls[i].ID, tt.want.ToolCalls[i].ID) + } + if got.ToolCalls[i].Name != tt.want.ToolCalls[i].Name { + t.Errorf("ToolCalls[%d].Name = %q, want %q", + i, got.ToolCalls[i].Name, tt.want.ToolCalls[i].Name) + } + } + } + }) + } +} + +func TestNormalizeBaseURL(t *testing.T) { + tests := []struct { + name string + apiBase string + expected string + }{ + { + name: "empty string defaults to official API", + apiBase: "", + expected: "https://api.anthropic.com/v1", + }, + { + name: "URL without /v1 gets it appended", + apiBase: "https://api.example.com/anthropic", + expected: "https://api.example.com/anthropic/v1", + }, + { + name: "URL with /v1 remains unchanged", + apiBase: "https://api.example.com/v1", + expected: "https://api.example.com/v1", + }, + { + name: "URL with trailing slash gets cleaned", + apiBase: "https://api.example.com/anthropic/", + expected: "https://api.example.com/anthropic/v1", + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got := normalizeBaseURL(tt.apiBase) + if got != tt.expected { + t.Errorf("normalizeBaseURL(%q) = %q, want %q", tt.apiBase, got, tt.expected) + } + }) + } +} + +func TestNewProvider(t *testing.T) { + provider := NewProvider("test-key", "https://api.example.com") + if provider == nil { + t.Fatal("NewProvider() returned nil") + } + if provider.apiKey != "test-key" { + t.Errorf("provider.apiKey = %q, want %q", provider.apiKey, "test-key") + } + if provider.apiBase != "https://api.example.com/v1" { + t.Errorf("provider.apiBase = %q, want %q", provider.apiBase, "https://api.example.com/v1") + } +} + +func TestGetDefaultModel(t *testing.T) { + provider := NewProvider("test-key", "") + got := provider.GetDefaultModel() + expected := "claude-sonnet-4.6" + if got != expected { + t.Errorf("GetDefaultModel() = %q, want %q", got, expected) + } +} + +// TestBuildRequestBodyEdgeCases tests edge cases for buildRequestBody. +func TestBuildRequestBodyEdgeCases(t *testing.T) { + tests := []struct { + name string + messages []Message + tools []ToolDefinition + model string + options map[string]any + wantErr bool + }{ + { + name: "empty message list", + messages: []Message{}, + model: "test-model", + options: map[string]any{ + "max_tokens": 8192, + }, + wantErr: false, + }, + { + name: "very long system message", + messages: []Message{ + {Role: "system", Content: strings.Repeat("This is a very long system prompt. ", 1000)}, + {Role: "user", Content: "Hello"}, + }, + model: "test-model", + options: map[string]any{ + "max_tokens": 8192, + }, + wantErr: false, + }, + { + name: "multiple consecutive system messages", + messages: []Message{ + {Role: "system", Content: "First system message"}, + {Role: "system", Content: "Second system message"}, + {Role: "system", Content: "Third system message"}, + {Role: "user", Content: "Hello"}, + }, + model: "test-model", + options: map[string]any{ + "max_tokens": 8192, + }, + wantErr: false, + }, + { + name: "tool result without tool call", + messages: []Message{ + {Role: "user", Content: "Use a tool"}, + {Role: "assistant", Content: "", ToolCalls: []ToolCall{ + {ID: "tool-1", Name: "test_tool", Arguments: map[string]any{"arg": "value"}}, + }}, + {Role: "user", ToolCallID: "tool-1", Content: "Tool result"}, + }, + model: "test-model", + options: map[string]any{ + "max_tokens": 8192, + }, + wantErr: false, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, err := buildRequestBody(tt.messages, tt.tools, tt.model, tt.options) + if (err != nil) != tt.wantErr { + t.Errorf("buildRequestBody() error = %v, wantErr %v", err, tt.wantErr) + return + } + if err != nil { + return + } + + // Verify basic structure + if got == nil { + t.Error("buildRequestBody() returned nil") + return + } + if got["model"] != tt.model { + t.Errorf("model = %v, want %v", got["model"], tt.model) + } + }) + } +} + +// TestParseResponseBodyEdgeCases tests edge cases for parseResponseBody. +func TestParseResponseBodyEdgeCases(t *testing.T) { + tests := []struct { + name string + body []byte + wantErr bool + check func(*testing.T, *LLMResponse) + }{ + { + name: "empty content blocks", + body: []byte(`{ + "id": "msg-empty", + "type": "message", + "role": "assistant", + "content": [], + "stop_reason": "end_turn", + "model": "test-model", + "usage": {"input_tokens": 5, "output_tokens": 0} + }`), + wantErr: false, + check: func(t *testing.T, resp *LLMResponse) { + if resp.Content != "" { + t.Errorf("Content = %q, want empty string", resp.Content) + } + if len(resp.ToolCalls) != 0 { + t.Errorf("ToolCalls length = %d, want 0", len(resp.ToolCalls)) + } + }, + }, + { + name: "multiple tool use blocks", + body: []byte(`{ + "id": "msg-multi", + "type": "message", + "role": "assistant", + "content": [ + {"type": "tool_use", "id": "tool-1", "name": "func1", "input": {"arg": "val1"}}, + {"type": "tool_use", "id": "tool-2", "name": "func2", "input": {"arg": "val2"}} + ], + "stop_reason": "tool_use", + "model": "test-model", + "usage": {"input_tokens": 10, "output_tokens": 20} + }`), + wantErr: false, + check: func(t *testing.T, resp *LLMResponse) { + if len(resp.ToolCalls) != 2 { + t.Errorf("ToolCalls length = %d, want 2", len(resp.ToolCalls)) + } + }, + }, + { + name: "malformed JSON response", + body: []byte(`{invalid json`), + wantErr: true, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, err := parseResponseBody(tt.body) + if (err != nil) != tt.wantErr { + t.Errorf("parseResponseBody() error = %v, wantErr %v", err, tt.wantErr) + return + } + if tt.check != nil && err == nil { + tt.check(t, got) + } + }) + } +} + +// TestProviderChatErrors tests error handling in Chat. +// Note: apiBase check removed as it's dead code - normalizeBaseURL() always provides a default. +func TestProviderChatErrors(t *testing.T) { + tests := []struct { + name string + apiKey string + messages []Message + wantErrMsg string + }{ + { + name: "missing API key", + apiKey: "", + messages: []Message{{Role: "user", Content: "Test"}}, + wantErrMsg: "API key not configured", + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + // Create provider using constructor to ensure proper initialization + provider := NewProvider(tt.apiKey, "https://api.example.com") + + _, err := provider.Chat(context.Background(), tt.messages, nil, "test-model", nil) + if err == nil { + t.Fatal("Chat() expected error, got nil") + } + if err.Error() != tt.wantErrMsg { + t.Errorf("Chat() error = %q, want %q", err.Error(), tt.wantErrMsg) + } + }) + } +} diff --git a/pkg/providers/factory_provider.go b/pkg/providers/factory_provider.go index 217d8f1de..b375f921a 100644 --- a/pkg/providers/factory_provider.go +++ b/pkg/providers/factory_provider.go @@ -10,6 +10,7 @@ import ( "strings" "github.com/sipeed/picoclaw/pkg/config" + anthropicmessages "github.com/sipeed/picoclaw/pkg/providers/anthropic_messages" ) // createClaudeAuthProvider creates a Claude provider using OAuth credentials from auth store. @@ -53,7 +54,8 @@ func ExtractProtocol(model string) (protocol, modelID string) { // CreateProviderFromConfig creates a provider based on the ModelConfig. // It uses the protocol prefix in the Model field to determine which provider to create. -// Supported protocols: openai, litellm, anthropic, antigravity, claude-cli, codex-cli, github-copilot +// Supported protocols: openai, litellm, anthropic, anthropic-messages, antigravity, +// claude-cli, codex-cli, github-copilot // Returns the provider, the model ID (without protocol prefix), and any error. func CreateProviderFromConfig(cfg *config.ModelConfig) (LLMProvider, string, error) { if cfg == nil { @@ -95,7 +97,7 @@ func CreateProviderFromConfig(cfg *config.ModelConfig) (LLMProvider, string, err case "litellm", "openrouter", "groq", "zhipu", "gemini", "nvidia", "ollama", "moonshot", "shengsuanyun", "deepseek", "cerebras", "vivgrid", "volcengine", "vllm", "qwen", "mistral", "avian", - "minimax", "longcat": + "minimax", "longcat", "modelscope": // All other OpenAI-compatible HTTP providers if cfg.APIKey == "" && cfg.APIBase == "" { return nil, "", fmt.Errorf("api_key or api_base is required for HTTP-based protocol %q", protocol) @@ -137,6 +139,21 @@ func CreateProviderFromConfig(cfg *config.ModelConfig) (LLMProvider, string, err cfg.RequestTimeout, ), modelID, nil + case "anthropic-messages": + // Anthropic Messages API with native format (HTTP-based, no SDK) + apiBase := cfg.APIBase + if apiBase == "" { + apiBase = "https://api.anthropic.com/v1" + } + if cfg.APIKey == "" { + return nil, "", fmt.Errorf("api_key is required for anthropic-messages protocol (model: %s)", cfg.Model) + } + return anthropicmessages.NewProviderWithTimeout( + cfg.APIKey, + apiBase, + cfg.RequestTimeout, + ), modelID, nil + case "antigravity": return NewAntigravityProvider(), modelID, nil @@ -304,6 +321,8 @@ func getDefaultAPIBase(protocol string) string { return "https://api.minimaxi.com/v1" case "longcat": return "https://api.longcat.chat/openai" + case "modelscope": + return "https://api-inference.modelscope.cn/v1" default: return "" } diff --git a/pkg/providers/factory_provider_test.go b/pkg/providers/factory_provider_test.go index 6c7bb4795..00676ebf9 100644 --- a/pkg/providers/factory_provider_test.go +++ b/pkg/providers/factory_provider_test.go @@ -114,6 +114,7 @@ func TestCreateProviderFromConfig_DefaultAPIBase(t *testing.T) { {"deepseek", "deepseek"}, {"ollama", "ollama"}, {"longcat", "longcat"}, + {"modelscope", "modelscope"}, } for _, tt := range tests { @@ -186,6 +187,35 @@ func TestCreateProviderFromConfig_LongCat(t *testing.T) { } } +func TestCreateProviderFromConfig_ModelScope(t *testing.T) { + cfg := &config.ModelConfig{ + ModelName: "test-modelscope", + Model: "modelscope/Qwen/Qwen3-235B-A22B-Instruct-2507", + APIKey: "test-key", + APIBase: "https://api-inference.modelscope.cn/v1", + } + + provider, modelID, err := CreateProviderFromConfig(cfg) + if err != nil { + t.Fatalf("CreateProviderFromConfig() error = %v", err) + } + if provider == nil { + t.Fatal("CreateProviderFromConfig() returned nil provider") + } + if modelID != "Qwen/Qwen3-235B-A22B-Instruct-2507" { + t.Errorf("modelID = %q, want %q", modelID, "Qwen/Qwen3-235B-A22B-Instruct-2507") + } + if _, ok := provider.(*HTTPProvider); !ok { + t.Fatalf("expected *HTTPProvider, got %T", provider) + } +} + +func TestGetDefaultAPIBase_ModelScope(t *testing.T) { + if got := getDefaultAPIBase("modelscope"); got != "https://api-inference.modelscope.cn/v1" { + t.Fatalf("getDefaultAPIBase(%q) = %q, want %q", "modelscope", got, "https://api-inference.modelscope.cn/v1") + } +} + func TestCreateProviderFromConfig_Anthropic(t *testing.T) { cfg := &config.ModelConfig{ ModelName: "test-anthropic", diff --git a/pkg/skills/installer.go b/pkg/skills/installer.go index c9f19f25d..f6cdee3a6 100644 --- a/pkg/skills/installer.go +++ b/pkg/skills/installer.go @@ -2,80 +2,289 @@ package skills import ( "context" + "encoding/json" "fmt" - "io" "net/http" + "net/url" "os" + "path" "path/filepath" + "strings" "time" - "github.com/sipeed/picoclaw/pkg/fileutil" "github.com/sipeed/picoclaw/pkg/utils" ) -type SkillInstaller struct { - workspace string +// GitHubContent represents a file or directory in GitHub API response +type GitHubContent struct { + Name string `json:"name"` + Path string `json:"path"` + Type string `json:"type"` // "file" or "dir" + DownloadURL string `json:"download_url"` + URL string `json:"url"` // API URL for subdirectories } -func NewSkillInstaller(workspace string) *SkillInstaller { - return &SkillInstaller{ - workspace: workspace, +// GitHubRef represents a parsed GitHub reference +type GitHubRef struct { + Owner string // Repository owner + RepoName string // Repository name + Ref string // Git reference (branch, tag, or commit) + SubPath string // Path within the repository +} + +type SkillInstaller struct { + workspace string + client *http.Client + githubToken string + proxy string +} + +// NewSkillInstaller creates a new skill installer. +// proxy is an optional HTTP/HTTPS/SOCKS5 proxy URL for downloading skills. +func NewSkillInstaller(workspace, githubToken, proxy string) (*SkillInstaller, error) { + client, err := utils.CreateHTTPClient(proxy, 15*time.Second) + if err != nil { + return nil, fmt.Errorf("failed to create HTTP client: %w", err) } + + return &SkillInstaller{ + workspace: workspace, + client: client, + githubToken: githubToken, + proxy: proxy, + }, nil +} + +// parseGitHubRef parses a GitHub reference. +// Supports: "owner/repo", "owner/repo/path", or full URL like "https://github.com/owner/repo/tree/ref/path" +func parseGitHubRef(repo string) (GitHubRef, error) { + repo = strings.TrimSpace(repo) + + // Handle full URL + if strings.HasPrefix(repo, "http://") || strings.HasPrefix(repo, "https://") { + u, err := url.Parse(repo) + if err != nil { + return GitHubRef{}, fmt.Errorf("invalid URL: %w", err) + } + parts := strings.Split(strings.Trim(u.Path, "/"), "/") + if len(parts) < 2 { + return GitHubRef{}, fmt.Errorf("invalid GitHub URL") + } + ref := GitHubRef{ + Owner: parts[0], + RepoName: parts[1], + Ref: "main", + } + // Look for /tree/ or /blob/ in the path + for i := 2; i < len(parts); i++ { + if parts[i] == "tree" || parts[i] == "blob" { + if i+1 < len(parts) { + ref.Ref = parts[i+1] + ref.SubPath = strings.Join(parts[i+2:], "/") + } + break + } + } + return ref, nil + } + + // Handle shorthand format + parts := strings.Split(strings.Trim(repo, "/"), "/") + if len(parts) < 2 { + return GitHubRef{}, fmt.Errorf("invalid format %q: expected 'owner/repo'", repo) + } + ref := GitHubRef{ + Owner: parts[0], + RepoName: parts[1], + Ref: "main", + } + if len(parts) > 2 { + ref.SubPath = strings.Join(parts[2:], "/") + } + return ref, nil } func (si *SkillInstaller) InstallFromGitHub(ctx context.Context, repo string) error { - skillDir := filepath.Join(si.workspace, "skills", filepath.Base(repo)) - - if _, err := os.Stat(skillDir); err == nil { - return fmt.Errorf("skill '%s' already exists", filepath.Base(repo)) + ref, err := parseGitHubRef(repo) + if err != nil { + return err } - url := fmt.Sprintf("https://raw.githubusercontent.com/%s/main/SKILL.md", repo) + skillName := ref.RepoName + if ref.SubPath != "" { + skillName = filepath.Base(ref.SubPath) + } + skillDirectory := filepath.Join(si.workspace, "skills", skillName) + + if _, err := os.Stat(skillDirectory); err == nil { + return fmt.Errorf("skill '%s' already exists", skillName) + } + + // Build GitHub API URL + apiPath := path.Join(ref.Owner, ref.RepoName, "contents") + if ref.SubPath != "" { + apiPath = path.Join(apiPath, ref.SubPath) + } + apiURL := fmt.Sprintf("https://api.github.com/repos/%s?ref=%s", apiPath, ref.Ref) + + if err := si.getGithubDirAllFiles(ctx, apiURL, skillDirectory, true); err != nil { + // Fallback to raw download + return si.downloadRaw(ctx, ref.Owner, ref.RepoName, ref.Ref, ref.SubPath, skillDirectory) + } + + if _, err := os.Stat(filepath.Join(skillDirectory, "SKILL.md")); err != nil { + return fmt.Errorf("SKILL.md not found in repository") + } + return nil +} + +// downloadDir recursively downloads a directory from GitHub API +// isRoot: true if this is the skill root directory (only download SKILL.md at root) +func (si *SkillInstaller) getGithubDirAllFiles(ctx context.Context, apiURL, localDir string, isRoot bool) error { + req, err := http.NewRequestWithContext(ctx, "GET", apiURL, nil) + if err != nil { + return err + } + if si.githubToken != "" { + req.Header.Set("Authorization", "Bearer "+si.githubToken) + } + + resp, err := utils.DoRequestWithRetry(si.client, req) + if err != nil { + return err + } + defer resp.Body.Close() + + if resp.StatusCode != 200 { + return fmt.Errorf("HTTP %d", resp.StatusCode) + } + + var items []GitHubContent + if err := json.NewDecoder(resp.Body).Decode(&items); err != nil { + return err + } + + for _, item := range items { + localPath := filepath.Join(localDir, item.Name) + + switch item.Type { + case "file": + if !shouldDownload(item.Name, isRoot) { + continue + } + if err := si.downloadFile(ctx, item.DownloadURL, localPath); err != nil { + return fmt.Errorf("download %s: %w", item.Name, err) + } + case "dir": + if !isSkillDirectory(item.Name) { + continue + } + if err := si.getGithubDirAllFiles(ctx, item.URL, localPath, false); err != nil { + return err + } + } + } + return nil +} + +// downloadRaw is a fallback that downloads just SKILL.md from raw.githubusercontent.com +func (si *SkillInstaller) downloadRaw(ctx context.Context, owner, repo, ref, subPath, localDir string) error { + urlPath := path.Join(owner, repo, ref) + if subPath != "" { + urlPath = path.Join(urlPath, subPath) + } + url := fmt.Sprintf("https://raw.githubusercontent.com/%s/SKILL.md", urlPath) - client := &http.Client{Timeout: 15 * time.Second} req, err := http.NewRequestWithContext(ctx, "GET", url, nil) if err != nil { return fmt.Errorf("failed to create request: %w", err) } - resp, err := utils.DoRequestWithRetry(client, req) + // Use chunked download to temporary file. + tmpPath, err := utils.DownloadToFile(ctx, si.client, req, 0) if err != nil { return fmt.Errorf("failed to fetch skill: %w", err) } - defer resp.Body.Close() + defer os.Remove(tmpPath) - if resp.StatusCode != 200 { - return fmt.Errorf("failed to fetch skill: HTTP %d", resp.StatusCode) - } - - body, err := io.ReadAll(resp.Body) - if err != nil { - return fmt.Errorf("failed to read response: %w", err) - } - - if err := os.MkdirAll(skillDir, 0o755); err != nil { + if err := os.MkdirAll(localDir, 0o755); err != nil { return fmt.Errorf("failed to create skill directory: %w", err) } - skillPath := filepath.Join(skillDir, "SKILL.md") + localPath := filepath.Join(localDir, "SKILL.md") - // Use unified atomic write utility with explicit sync for flash storage reliability. - if err := fileutil.WriteFileAtomic(skillPath, body, 0o600); err != nil { + // Atomic move from temp to final location. + if err := os.Rename(tmpPath, localPath); err != nil { return fmt.Errorf("failed to write skill file: %w", err) } - return nil + return os.Chmod(localPath, 0o600) +} + +func (si *SkillInstaller) downloadFile(ctx context.Context, url, localPath string) error { + req, err := http.NewRequestWithContext(ctx, "GET", url, nil) + if err != nil { + return err + } + + // Use chunked download to temporary file, then move atomically to target. + tmpPath, err := utils.DownloadToFile(ctx, si.client, req, 0) + if err != nil { + return err + } + defer os.Remove(tmpPath) + + if err := os.MkdirAll(filepath.Dir(localPath), 0o755); err != nil { + return err + } + + // Atomic move from temp to final location. + if err := os.Rename(tmpPath, localPath); err != nil { + return fmt.Errorf("failed to move downloaded file: %w", err) + } + + return os.Chmod(localPath, 0o600) +} + +// shouldDownload determines if a file should be downloaded +// root: true if we're at the skill root directory +func shouldDownload(name string, root bool) bool { + if root { + return name == "SKILL.md" + } + return true +} + +// isSkillDir checks if a directory is a standard skill resource directory +func isSkillDirectory(name string) bool { + switch name { + case "scripts", "references", "assets", "templates", "docs": + return true + } + return false } func (si *SkillInstaller) Uninstall(skillName string) error { - skillDir := filepath.Join(si.workspace, "skills", skillName) + parts := strings.Split(skillName, "/") + var finalSkillName string + for i := len(parts) - 1; i >= 0; i-- { + if parts[i] != "" { + finalSkillName = parts[i] + break + } + } + if finalSkillName == "" { + finalSkillName = skillName + } + + skillDir := filepath.Join(si.workspace, "skills", finalSkillName) if _, err := os.Stat(skillDir); os.IsNotExist(err) { - return fmt.Errorf("skill '%s' not found", skillName) + return fmt.Errorf("skill '%s' not found (processed as '%s')", skillName, finalSkillName) } if err := os.RemoveAll(skillDir); err != nil { - return fmt.Errorf("failed to remove skill: %w", err) + return fmt.Errorf("failed to remove skill '%s': %w", finalSkillName, err) } return nil diff --git a/pkg/skills/installer_test.go b/pkg/skills/installer_test.go new file mode 100644 index 000000000..759cfc489 --- /dev/null +++ b/pkg/skills/installer_test.go @@ -0,0 +1,665 @@ +package skills + +import ( + "context" + "encoding/json" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "strings" + "testing" + "time" +) + +func TestParseGitHubRef(t *testing.T) { + tests := []struct { + name string + repo string + wantOwner string + wantRepoName string + wantRef string + wantSubPath string + wantErr bool + wantErrContain string + }{ + { + name: "simple owner/repo", + repo: "sipeed/picoclaw", + wantOwner: "sipeed", + wantRepoName: "picoclaw", + wantRef: "main", + wantSubPath: "", + }, + { + name: "owner/repo with subpath", + repo: "sipeed/picoclaw/skills/test", + wantOwner: "sipeed", + wantRepoName: "picoclaw", + wantRef: "main", + wantSubPath: "skills/test", + }, + { + name: "full URL with tree", + repo: "https://github.com/sipeed/picoclaw/tree/dev/skills/test", + wantOwner: "sipeed", + wantRepoName: "picoclaw", + wantRef: "dev", + wantSubPath: "skills/test", + }, + { + name: "full URL with blob", + repo: "https://github.com/sipeed/picoclaw/blob/main/README.md", + wantOwner: "sipeed", + wantRepoName: "picoclaw", + wantRef: "main", + wantSubPath: "README.md", + }, + { + name: "full URL without ref", + repo: "https://github.com/sipeed/picoclaw", + wantOwner: "sipeed", + wantRepoName: "picoclaw", + wantRef: "main", + wantSubPath: "", + }, + { + name: "invalid format - single part", + repo: "sipeed", + wantErr: true, + wantErrContain: "expected 'owner/repo'", + }, + { + name: "invalid URL", + repo: "http://[invalid", + wantErr: true, + wantErrContain: "invalid URL", + }, + { + name: "invalid GitHub URL - only one path part", + repo: "https://github.com/sipeed", + wantErr: true, + wantErrContain: "invalid GitHub URL", + }, + { + name: "with whitespace", + repo: " sipeed/picoclaw ", + wantOwner: "sipeed", + wantRepoName: "picoclaw", + wantRef: "main", + wantSubPath: "", + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + ref, err := parseGitHubRef(tt.repo) + + if tt.wantErr { + if err == nil { + t.Errorf("parseGitHubRef() error = nil, wantErr = true") + return + } + if tt.wantErrContain != "" && !strings.Contains(err.Error(), tt.wantErrContain) { + t.Errorf("parseGitHubRef() error = %v, want error containing %v", err, tt.wantErrContain) + } + return + } + + if err != nil { + t.Errorf("parseGitHubRef() unexpected error = %v", err) + return + } + + if ref.Owner != tt.wantOwner { + t.Errorf("parseGitHubRef() owner = %v, want %v", ref.Owner, tt.wantOwner) + } + if ref.RepoName != tt.wantRepoName { + t.Errorf("parseGitHubRef() repoName = %v, want %v", ref.RepoName, tt.wantRepoName) + } + if ref.Ref != tt.wantRef { + t.Errorf("parseGitHubRef() ref = %v, want %v", ref.Ref, tt.wantRef) + } + if ref.SubPath != tt.wantSubPath { + t.Errorf("parseGitHubRef() subPath = %v, want %v", ref.SubPath, tt.wantSubPath) + } + }) + } +} + +func TestShouldDownload(t *testing.T) { + tests := []struct { + name string + file string + root bool + want bool + }{ + {"SKILL.md at root", "SKILL.md", true, true}, + {"other file at root", "README.md", true, false}, + {"script at root", "script.py", true, false}, + {"SKILL.md not at root", "SKILL.md", false, true}, + {"any file not at root", "any.txt", false, true}, + {"script not at root", "script.py", false, true}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got := shouldDownload(tt.file, tt.root) + if got != tt.want { + t.Errorf("shouldDownload(%q, %v) = %v, want %v", tt.file, tt.root, got, tt.want) + } + }) + } +} + +func TestIsSkillDirectory(t *testing.T) { + tests := []struct { + name string + dir string + want bool + }{ + {"scripts dir", "scripts", true}, + {"references dir", "references", true}, + {"assets dir", "assets", true}, + {"templates dir", "templates", true}, + {"docs dir", "docs", true}, + {"other dir", "other", false}, + {"src dir", "src", false}, + {"empty string", "", false}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got := isSkillDirectory(tt.dir) + if got != tt.want { + t.Errorf("isSkillDirectory(%q) = %v, want %v", tt.dir, got, tt.want) + } + }) + } +} + +func TestNewSkillInstaller(t *testing.T) { + tmpDir := t.TempDir() + installer, err := NewSkillInstaller(tmpDir, "test-token", "") + if err != nil { + t.Fatalf("NewSkillInstaller() error = %v", err) + } + + if installer == nil { + t.Fatal("NewSkillInstaller() returned nil") + } + + if installer.workspace != tmpDir { + t.Errorf("workspace = %v, want %v", installer.workspace, tmpDir) + } + + if installer.githubToken != "test-token" { + t.Errorf("githubToken = %v, want 'test-token'", installer.githubToken) + } + + if installer.proxy != "" { + t.Errorf("proxy = %v, want empty", installer.proxy) + } + + if installer.client == nil { + t.Error("client is nil") + } else if installer.client.Timeout != 15*time.Second { + t.Errorf("client.Timeout = %v, want 15s", installer.client.Timeout) + } +} + +func TestNewSkillInstaller_WithProxy(t *testing.T) { + tmpDir := t.TempDir() + installer, err := NewSkillInstaller(tmpDir, "test-token", "http://127.0.0.1:7890") + if err != nil { + t.Fatalf("NewSkillInstaller() error = %v", err) + } + + if installer.proxy != "http://127.0.0.1:7890" { + t.Errorf("proxy = %v, want 'http://127.0.0.1:7890'", installer.proxy) + } + + if installer.client == nil { + t.Fatal("client is nil") + } + + // Verify the transport has proxy configured + transport, ok := installer.client.Transport.(*http.Transport) + if !ok { + t.Fatal("client.Transport is not *http.Transport") + } + + if transport.Proxy == nil { + t.Error("transport.Proxy is nil, expected non-nil") + } +} + +func TestNewSkillInstaller_InvalidProxy(t *testing.T) { + tmpDir := t.TempDir() + installer, err := NewSkillInstaller(tmpDir, "test-token", "://invalid-proxy") + if err == nil { + t.Error("NewSkillInstaller() expected error for invalid proxy, got nil") + } + if installer != nil { + t.Error("expected nil installer on error") + } +} + +func TestSkillInstaller_DownloadFile(t *testing.T) { + // Create a test server that serves files + content := "test file content for skill download" + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.Method != http.MethodGet { + t.Errorf("expected GET, got %s", r.Method) + } + w.WriteHeader(http.StatusOK) + w.Write([]byte(content)) + })) + defer server.Close() + + tmpDir := t.TempDir() + installer, err := NewSkillInstaller(tmpDir, "", "") + if err != nil { + t.Fatalf("NewSkillInstaller() error = %v", err) + } + + t.Run("successful download", func(t *testing.T) { + localPath := filepath.Join(tmpDir, "test-skill", "SKILL.md") + err := installer.downloadFile(context.Background(), server.URL, localPath) + if err != nil { + t.Errorf("downloadFile() error = %v", err) + return + } + + // Verify file was downloaded + data, err := os.ReadFile(localPath) + if err != nil { + t.Errorf("failed to read downloaded file: %v", err) + return + } + + if string(data) != content { + t.Errorf("downloaded content = %q, want %q", string(data), content) + } + + // Check file permissions + info, err := os.Stat(localPath) + if err != nil { + t.Errorf("failed to stat file: %v", err) + return + } + + if info.Mode().Perm() != 0o600 { + t.Errorf("file permissions = %o, want %o", info.Mode().Perm(), 0o600) + } + }) + + t.Run("http error", func(t *testing.T) { + errorServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.WriteHeader(http.StatusNotFound) + w.Write([]byte("not found")) + })) + defer errorServer.Close() + + localPath := filepath.Join(tmpDir, "error-test", "SKILL.md") + err := installer.downloadFile(context.Background(), errorServer.URL, localPath) + if err == nil { + t.Error("downloadFile() expected error for 404, got nil") + } + }) +} + +func TestSkillInstaller_DownloadRaw(t *testing.T) { + content := "raw skill content" + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.WriteHeader(http.StatusOK) + w.Write([]byte(content)) + })) + defer server.Close() + + tmpDir := t.TempDir() + installer, err := NewSkillInstaller(tmpDir, "", "") + if err != nil { + t.Fatalf("NewSkillInstaller() error = %v", err) + } + + // Replace the client with one that points to our test server + // We need to modify the URL in the function, so we'll test indirectly + + localDir := filepath.Join(tmpDir, "raw-test") + ctx := context.Background() + + // Create a simple test by calling downloadFile directly since downloadRaw + // constructs its own URL + testFile := filepath.Join(localDir, "SKILL.md") + err = installer.downloadFile(ctx, server.URL, testFile) + if err != nil { + t.Errorf("downloadFile() error = %v", err) + } + + // Verify file content + data, err := os.ReadFile(testFile) + if err != nil { + t.Errorf("failed to read file: %v", err) + return + } + + if string(data) != content { + t.Errorf("content = %q, want %q", string(data), content) + } +} + +func TestSkillInstaller_Uninstall(t *testing.T) { + tmpDir := t.TempDir() + skillsDir := filepath.Join(tmpDir, "skills") + os.MkdirAll(skillsDir, 0o755) + + installer, err := NewSkillInstaller(tmpDir, "", "") + if err != nil { + t.Fatalf("NewSkillInstaller() error = %v", err) + } + + t.Run("uninstall existing skill", func(t *testing.T) { + skillName := "test-skill" + skillDir := filepath.Join(skillsDir, skillName) + + // Create skill directory with a file + os.MkdirAll(skillDir, 0o755) + os.WriteFile(filepath.Join(skillDir, "SKILL.md"), []byte("test"), 0o644) + + if err := installer.Uninstall(skillName); err != nil { + t.Errorf("Uninstall() error = %v", err) + } + + // Verify directory was removed + if _, err := os.Stat(skillDir); !os.IsNotExist(err) { + t.Error("skill directory still exists after uninstall") + } + }) + + t.Run("uninstall non-existent skill", func(t *testing.T) { + if err := installer.Uninstall("non-existent-skill"); err == nil { + t.Error("Uninstall() expected error for non-existent skill, got nil") + } else if !strings.Contains(err.Error(), "not found") { + t.Errorf("error message = %q, want 'not found'", err.Error()) + } + }) + + t.Run("uninstall with path separator", func(t *testing.T) { + skillName := "owner/repo/skill-name" + skillDir := filepath.Join(skillsDir, "skill-name") + + // Create skill directory + os.MkdirAll(skillDir, 0o755) + os.WriteFile(filepath.Join(skillDir, "SKILL.md"), []byte("test"), 0o644) + + if err := installer.Uninstall(skillName); err != nil { + t.Errorf("Uninstall() error = %v", err) + } + + if _, err := os.Stat(skillDir); !os.IsNotExist(err) { + t.Error("skill directory still exists after uninstall") + } + }) + + t.Run("uninstall with trailing slash", func(t *testing.T) { + skillName := "skill-name/" + skillDir := filepath.Join(skillsDir, "skill-name") + + // Create skill directory + os.MkdirAll(skillDir, 0o755) + os.WriteFile(filepath.Join(skillDir, "SKILL.md"), []byte("test"), 0o644) + + if err := installer.Uninstall(skillName); err != nil { + t.Errorf("Uninstall() error = %v", err) + } + + if _, err := os.Stat(skillDir); !os.IsNotExist(err) { + t.Error("skill directory still exists after uninstall") + } + }) +} + +func TestSkillInstaller_InstallFromGitHub_SkillAlreadyExists(t *testing.T) { + tmpDir := t.TempDir() + skillsDir := filepath.Join(tmpDir, "skills") + os.MkdirAll(skillsDir, 0o755) + + installer, err := NewSkillInstaller(tmpDir, "", "") + if err != nil { + t.Fatalf("NewSkillInstaller() error = %v", err) + } + + // Create an existing skill directory + existingSkill := filepath.Join(skillsDir, "picoclaw") + os.MkdirAll(existingSkill, 0o755) + os.WriteFile(filepath.Join(existingSkill, "SKILL.md"), []byte("existing"), 0o644) + + // Try to install the same skill - should fail + err = installer.InstallFromGitHub(context.Background(), "sipeed/picoclaw") + if err == nil { + t.Error("InstallFromGitHub() expected error for existing skill, got nil") + } + if !strings.Contains(err.Error(), "already exists") { + t.Errorf("error message = %q, want 'already exists'", err.Error()) + } +} + +func TestGitHubContent_Struct(t *testing.T) { + // Test that GitHubContent struct can be properly unmarshaled + jsonData := `{ + "name": "test.md", + "path": "skills/test.md", + "type": "file", + "download_url": "https://example.com/download", + "url": "https://api.github.com/contents/skills/test.md" + }` + + var content GitHubContent + err := json.Unmarshal([]byte(jsonData), &content) + if err != nil { + t.Errorf("failed to unmarshal GitHubContent: %v", err) + } + + if content.Name != "test.md" { + t.Errorf("Name = %q, want 'test.md'", content.Name) + } + if content.Type != "file" { + t.Errorf("Type = %q, want 'file'", content.Type) + } + if content.DownloadURL != "https://example.com/download" { + t.Errorf("DownloadURL = %q, want 'https://example.com/download'", content.DownloadURL) + } +} + +func TestSkillInstaller_GetGithubDirAllFiles(t *testing.T) { + tmpDir := t.TempDir() + installer, err := NewSkillInstaller(tmpDir, "", "") + if err != nil { + t.Fatalf("NewSkillInstaller() error = %v", err) + } + + // Create a test server that mimics GitHub API + fileContent := "skill file content" + var serverURL string + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + // Check for authorization header + authHeader := r.Header.Get("Authorization") + if authHeader != "" && !strings.HasPrefix(authHeader, "Bearer ") { + t.Errorf("expected Bearer token, got: %s", authHeader) + } + + // Return different responses based on path + if strings.Contains(r.URL.Path, "/contents") { + // API response for directory listing + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusOK) + + items := []map[string]any{ + { + "name": "SKILL.md", + "path": "SKILL.md", + "type": "file", + "download_url": serverURL + "/download/SKILL.md", + }, + { + "name": "scripts", + "path": "scripts", + "type": "dir", + "url": serverURL + "/api/scripts", + }, + } + json.NewEncoder(w).Encode(items) + } else if strings.Contains(r.URL.Path, "/api/scripts") { + // API response for scripts subdirectory + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusOK) + + items := []map[string]any{ + { + "name": "test.py", + "path": "scripts/test.py", + "type": "file", + "download_url": serverURL + "/download/test.py", + }, + } + json.NewEncoder(w).Encode(items) + } else if strings.Contains(r.URL.Path, "/download/") { + // Raw file download + w.WriteHeader(http.StatusOK) + w.Write([]byte(fileContent)) + } else { + w.WriteHeader(http.StatusNotFound) + } + })) + serverURL = server.URL + defer server.Close() + + localDir := filepath.Join(tmpDir, "test-skill") + + t.Run("download from GitHub API", func(t *testing.T) { + err := installer.getGithubDirAllFiles(context.Background(), server.URL+"/contents", localDir, true) + if err != nil { + t.Errorf("getGithubDirAllFiles() error = %v", err) + return + } + + // Verify SKILL.md was downloaded + skillMd := filepath.Join(localDir, "SKILL.md") + data, err := os.ReadFile(skillMd) + if err != nil { + t.Errorf("failed to read SKILL.md: %v", err) + return + } + if string(data) != fileContent { + t.Errorf("SKILL.md content = %q, want %q", string(data), fileContent) + } + + // Verify scripts directory and file + scriptFile := filepath.Join(localDir, "scripts", "test.py") + data, err = os.ReadFile(scriptFile) + if err != nil { + t.Errorf("failed to read test.py: %v", err) + return + } + if string(data) != fileContent { + t.Errorf("test.py content = %q, want %q", string(data), fileContent) + } + }) + + t.Run("http error response", func(t *testing.T) { + errorServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.WriteHeader(http.StatusForbidden) + })) + defer errorServer.Close() + + err := installer.getGithubDirAllFiles( + context.Background(), + errorServer.URL, + filepath.Join(tmpDir, "error-test"), + true, + ) + if err == nil { + t.Error("getGithubDirAllFiles() expected error for 403, got nil") + } + }) +} + +func TestSkillInstaller_InstallFromGitHub_WithToken(t *testing.T) { + tmpDir := t.TempDir() + skillsDir := filepath.Join(tmpDir, "skills") + os.MkdirAll(skillsDir, 0o755) + + var serverURL string + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + // Capture the authorization header + authHeader := r.Header.Get("Authorization") + if authHeader != "" { + tokenReceived := strings.TrimPrefix(authHeader, "Bearer ") + t.Fatalf("github token is %s", tokenReceived) + } + + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusOK) + + items := []map[string]any{ + { + "name": "SKILL.md", + "path": "SKILL.md", + "type": "file", + "download_url": serverURL + "/download/SKILL.md", + }, + } + json.NewEncoder(w).Encode(items) + })) + serverURL = server.URL + defer server.Close() + + installer, err := NewSkillInstaller(tmpDir, "test-github-token", "") + if err != nil { + t.Fatalf("NewSkillInstaller() error = %v", err) + } + + // We need to test the token is passed - the actual install will fail + // because we're not fully mocking the download, but we can verify + // the token is sent in the request + + // Use a simple context with timeout + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + + // The install will fail because download URL isn't properly set up, + // but the token should be sent in the API request + _ = installer.InstallFromGitHub(ctx, "owner/repo") + + // Note: We can't easily intercept the download request since it's a different URL, + // but the fact that the API request was made verifies the token flow + // In a real scenario, the token would be sent to both API and raw downloads +} + +func TestSkillInstaller_ContextCancellation(t *testing.T) { + tmpDir := t.TempDir() + installer, err := NewSkillInstaller(tmpDir, "", "") + if err != nil { + t.Fatalf("NewSkillInstaller() error = %v", err) + } + + // Create a slow server + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + time.Sleep(100 * time.Millisecond) + w.WriteHeader(http.StatusOK) + w.Write([]byte("response")) + })) + defer server.Close() + + // Create a canceled context + ctx, cancel := context.WithCancel(context.Background()) + cancel() // Cancel immediately + + localPath := filepath.Join(tmpDir, "cancel-test", "file.txt") + err = installer.downloadFile(ctx, server.URL, localPath) + + if err == nil { + t.Error("downloadFile() expected error for canceled context, got nil") + } +} diff --git a/pkg/tools/shell.go b/pkg/tools/shell.go index 16bb82b64..a04602a32 100644 --- a/pkg/tools/shell.go +++ b/pkg/tools/shell.go @@ -533,15 +533,49 @@ func (t *ExecTool) guardCommand(command, cwd string) string { // before blocking. + // Web URL schemes whose path components (starting with //) should be exempt + // from workspace sandbox checks. file: is intentionally excluded so that + // file:// URIs are still validated against the workspace boundary. + webSchemes := []string{"http:", "https:", "ftp:", "ftps:", "sftp:", "ssh:", "git:"} + agentCLI := isAgentCLICommand(cmd) - for _, token := range strings.Fields(cmd) { + fields := strings.Fields(cmd) + for i, token := range fields { token = strings.Trim(token, "\"'") - if !filepath.IsAbs(token) { + // Handle file:// URIs: extract the path and validate it. + if strings.HasPrefix(token, "file://") { + token = strings.TrimPrefix(token, "file://") + if !filepath.IsAbs(token) { + continue + } + } else if !filepath.IsAbs(token) { continue } + // Skip URL path components that look like they're from web URLs. + // When a URL like "https://github.com/repo" is tokenised, the path + // portion may appear absolute. Check whether the previous token is + // a complete web URL containing this path component. + if strings.HasPrefix(token, "//") { + // Check if the previous non-operator token ends with a web scheme + // that makes this token part of a URL rather than a real path. + isPartOfURL := false + if i > 0 { + prev := strings.Trim(fields[i-1], "\"'") + for _, scheme := range webSchemes { + if strings.HasSuffix(prev, scheme) || strings.Contains(prev, scheme+"//") { + isPartOfURL = true + break + } + } + } + if isPartOfURL { + continue + } + } + p := filepath.Clean(token) if safePaths[p] { diff --git a/pkg/tools/shell_test.go b/pkg/tools/shell_test.go index 90265e5bd..c4553020f 100644 --- a/pkg/tools/shell_test.go +++ b/pkg/tools/shell_test.go @@ -522,3 +522,101 @@ func TestShellTool_CustomAllowPatterns(t *testing.T) { t.Errorf("'git push upstream main' should still be blocked by deny pattern") } } + +// TestShellTool_URLsNotBlocked verifies that commands containing URLs are not +// incorrectly blocked by the workspace restriction safety guard (issue #1203). +func TestShellTool_URLsNotBlocked(t *testing.T) { + tmpDir := t.TempDir() + tool, err := NewExecTool(tmpDir, true) + if err != nil { + t.Fatalf("unable to configure exec tool: %s", err) + } + + // These commands contain URLs and should NOT be blocked by workspace restriction. + // The URL path components (e.g., "//github.com") should be recognized as URLs, + // not as file system paths. + commands := []string{ + "agent-browser open https://github.com", + "curl https://api.example.com/data", + "wget http://example.com/file", + "browser open https://github.com/user/repo", + "fetch ftp://ftp.example.com/file.txt", + "git clone https://github.com/sipeed/picoclaw.git", + } + + for _, cmd := range commands { + result := tool.Execute(context.Background(), map[string]any{"command": cmd}) + if result.IsError && strings.Contains(result.ForLLM, "path outside working dir") { + t.Errorf("command with URL should not be blocked by workspace check: %s\n error: %s", cmd, result.ForLLM) + } + } +} + +// TestShellTool_FileURISandboxing verifies that file:// URIs that escape the +// workspace are still blocked, even though other URLs are allowed (issue #1254). +func TestShellTool_FileURISandboxing(t *testing.T) { + tmpDir := t.TempDir() + tool, err := NewExecTool(tmpDir, true) + if err != nil { + t.Fatalf("unable to configure exec tool: %s", err) + } + + // These file:// URIs should be blocked if they reference paths outside the workspace. + // Unlike web URLs (http://, https://, ftp://), file:// URIs can be used to escape the sandbox. + blockedCommands := []string{ + "cat file:///etc/passwd", + "cat file:///etc/hosts", + "cat file:///root/.ssh/id_rsa", + } + + for _, cmd := range blockedCommands { + result := tool.Execute(context.Background(), map[string]any{"command": cmd}) + if !result.IsError || !strings.Contains(result.ForLLM, "path outside working dir") { + t.Errorf("file:// URI outside workspace should be blocked: %s", cmd) + } + } + + // These file:// URIs should be allowed if they reference paths inside the workspace. + // Create a test file inside the temp directory + testFile := filepath.Join(tmpDir, "test.txt") + if err := os.WriteFile(testFile, []byte("test content"), 0o644); err != nil { + t.Fatalf("failed to create test file: %s", err) + } + + allowedCommands := []string{ + "cat file://" + testFile, + } + + for _, cmd := range allowedCommands { + result := tool.Execute(context.Background(), map[string]any{"command": cmd}) + if result.IsError && strings.Contains(result.ForLLM, "path outside working dir") { + t.Errorf("file:// URI inside workspace should be allowed: %s\n error: %s", cmd, result.ForLLM) + } + } +} + +// TestShellTool_URLBypassPrevented verifies that a command cannot bypass the workspace +// sandbox by smuggling a real path after a URL that contains the same //path substring. +// e.g. "echo https://etc/passwd && cat //etc/passwd" must still be blocked. +func TestShellTool_URLBypassPrevented(t *testing.T) { + tmpDir := t.TempDir() + tool, err := NewExecTool(tmpDir, true) + if err != nil { + t.Fatalf("unable to configure exec tool: %s", err) + } + + // The path //etc/passwd appears twice: once as the host part of an https URL + // and once as a real (escaped) absolute path. The guard must block the command + // because the second occurrence is a genuine out-of-workspace path. + blockedCommands := []string{ + "echo https://etc/passwd && cat //etc/passwd", + "curl https://host/file && ls //etc", + } + + for _, cmd := range blockedCommands { + result := tool.Execute(context.Background(), map[string]any{"command": cmd}) + if !result.IsError || !strings.Contains(result.ForLLM, "path outside working dir") { + t.Errorf("bypass attempt should be blocked: %q\n got: %s", cmd, result.ForLLM) + } + } +} diff --git a/pkg/tools/web.go b/pkg/tools/web.go index 70a912cb7..26d6b3b7e 100644 --- a/pkg/tools/web.go +++ b/pkg/tools/web.go @@ -14,6 +14,8 @@ import ( "strings" "sync/atomic" "time" + + "github.com/sipeed/picoclaw/pkg/utils" ) const ( @@ -739,7 +741,7 @@ func NewWebSearchTool(opts WebSearchToolOptions) (*WebSearchTool, error) { // Priority: Perplexity > Brave > SearXNG > Tavily > DuckDuckGo > GLM Search if opts.PerplexityEnabled && len(opts.PerplexityAPIKeys) > 0 { - client, err := createHTTPClient(opts.Proxy, perplexityTimeout) + client, err := utils.CreateHTTPClient(opts.Proxy, perplexityTimeout) if err != nil { return nil, fmt.Errorf("failed to create HTTP client for Perplexity: %w", err) } @@ -755,7 +757,7 @@ func NewWebSearchTool(opts WebSearchToolOptions) (*WebSearchTool, error) { maxResults = opts.PerplexityMaxResults } } else if opts.BraveEnabled && len(opts.BraveAPIKeys) > 0 { - client, err := createHTTPClient(opts.Proxy, searchTimeout) + client, err := utils.CreateHTTPClient(opts.Proxy, searchTimeout) if err != nil { return nil, fmt.Errorf("failed to create HTTP client for Brave: %w", err) } @@ -775,7 +777,7 @@ func NewWebSearchTool(opts WebSearchToolOptions) (*WebSearchTool, error) { maxResults = opts.SearXNGMaxResults } } else if opts.TavilyEnabled && len(opts.TavilyAPIKeys) > 0 { - client, err := createHTTPClient(opts.Proxy, searchTimeout) + client, err := utils.CreateHTTPClient(opts.Proxy, searchTimeout) if err != nil { return nil, fmt.Errorf("failed to create HTTP client for Tavily: %w", err) } @@ -794,7 +796,7 @@ func NewWebSearchTool(opts WebSearchToolOptions) (*WebSearchTool, error) { maxResults = opts.TavilyMaxResults } } else if opts.DuckDuckGoEnabled { - client, err := createHTTPClient(opts.Proxy, searchTimeout) + client, err := utils.CreateHTTPClient(opts.Proxy, searchTimeout) if err != nil { return nil, fmt.Errorf("failed to create HTTP client for DuckDuckGo: %w", err) } @@ -806,7 +808,7 @@ func NewWebSearchTool(opts WebSearchToolOptions) (*WebSearchTool, error) { maxResults = opts.DuckDuckGoMaxResults } } else if opts.GLMSearchEnabled && opts.GLMSearchAPIKey != "" { - client, err := createHTTPClient(opts.Proxy, searchTimeout) + client, err := utils.CreateHTTPClient(opts.Proxy, searchTimeout) if err != nil { return nil, fmt.Errorf("failed to create HTTP client for GLM Search: %w", err) } @@ -924,7 +926,7 @@ func NewWebFetchToolWithProxy(maxChars int, proxy string, fetchLimitBytes int64) if maxChars <= 0 { maxChars = defaultMaxChars } - client, err := createHTTPClient(proxy, fetchTimeout) + client, err := utils.CreateHTTPClient(proxy, fetchTimeout) if err != nil { return nil, fmt.Errorf("failed to create HTTP client for web fetch: %w", err) } diff --git a/pkg/tools/web_test.go b/pkg/tools/web_test.go index 0737d2087..41d83e6f5 100644 --- a/pkg/tools/web_test.go +++ b/pkg/tools/web_test.go @@ -10,7 +10,6 @@ import ( "net/http/httptest" "strings" "testing" - "time" "github.com/sipeed/picoclaw/pkg/logger" ) @@ -639,108 +638,6 @@ func TestWebTool_WebFetch_MissingDomain(t *testing.T) { } } -func TestCreateHTTPClient_ProxyConfigured(t *testing.T) { - client, err := createHTTPClient("http://127.0.0.1:7890", 12*time.Second) - if err != nil { - t.Fatalf("createHTTPClient() error: %v", err) - } - if client.Timeout != 12*time.Second { - t.Fatalf("client.Timeout = %v, want %v", client.Timeout, 12*time.Second) - } - - tr, ok := client.Transport.(*http.Transport) - if !ok { - t.Fatalf("client.Transport type = %T, want *http.Transport", client.Transport) - } - if tr.Proxy == nil { - t.Fatal("transport.Proxy is nil, want non-nil") - } - - req, err := http.NewRequest("GET", "https://example.com", nil) - if err != nil { - t.Fatalf("http.NewRequest() error: %v", err) - } - proxyURL, err := tr.Proxy(req) - if err != nil { - t.Fatalf("transport.Proxy(req) error: %v", err) - } - if proxyURL == nil || proxyURL.String() != "http://127.0.0.1:7890" { - t.Fatalf("proxy URL = %v, want %q", proxyURL, "http://127.0.0.1:7890") - } -} - -func TestCreateHTTPClient_InvalidProxy(t *testing.T) { - _, err := createHTTPClient("://bad-proxy", 10*time.Second) - if err == nil { - t.Fatal("createHTTPClient() expected error for invalid proxy URL, got nil") - } -} - -func TestCreateHTTPClient_Socks5ProxyConfigured(t *testing.T) { - client, err := createHTTPClient("socks5://127.0.0.1:1080", 8*time.Second) - if err != nil { - t.Fatalf("createHTTPClient() error: %v", err) - } - - tr, ok := client.Transport.(*http.Transport) - if !ok { - t.Fatalf("client.Transport type = %T, want *http.Transport", client.Transport) - } - req, err := http.NewRequest("GET", "https://example.com", nil) - if err != nil { - t.Fatalf("http.NewRequest() error: %v", err) - } - proxyURL, err := tr.Proxy(req) - if err != nil { - t.Fatalf("transport.Proxy(req) error: %v", err) - } - if proxyURL == nil || proxyURL.String() != "socks5://127.0.0.1:1080" { - t.Fatalf("proxy URL = %v, want %q", proxyURL, "socks5://127.0.0.1:1080") - } -} - -func TestCreateHTTPClient_UnsupportedProxyScheme(t *testing.T) { - _, err := createHTTPClient("ftp://127.0.0.1:21", 10*time.Second) - if err == nil { - t.Fatal("createHTTPClient() expected error for unsupported scheme, got nil") - } - if !strings.Contains(err.Error(), "unsupported proxy scheme") { - t.Fatalf("error = %q, want to contain %q", err.Error(), "unsupported proxy scheme") - } -} - -func TestCreateHTTPClient_ProxyFromEnvironmentWhenConfigEmpty(t *testing.T) { - t.Setenv("HTTP_PROXY", "http://127.0.0.1:8888") - t.Setenv("http_proxy", "http://127.0.0.1:8888") - t.Setenv("HTTPS_PROXY", "http://127.0.0.1:8888") - t.Setenv("https_proxy", "http://127.0.0.1:8888") - t.Setenv("ALL_PROXY", "") - t.Setenv("all_proxy", "") - t.Setenv("NO_PROXY", "") - t.Setenv("no_proxy", "") - - client, err := createHTTPClient("", 10*time.Second) - if err != nil { - t.Fatalf("createHTTPClient() error: %v", err) - } - - tr, ok := client.Transport.(*http.Transport) - if !ok { - t.Fatalf("client.Transport type = %T, want *http.Transport", client.Transport) - } - if tr.Proxy == nil { - t.Fatal("transport.Proxy is nil, want proxy function from environment") - } - - req, err := http.NewRequest("GET", "https://example.com", nil) - if err != nil { - t.Fatalf("http.NewRequest() error: %v", err) - } - if _, err := tr.Proxy(req); err != nil { - t.Fatalf("transport.Proxy(req) error: %v", err) - } -} - func TestNewWebFetchToolWithProxy(t *testing.T) { tool, err := NewWebFetchToolWithProxy(1024, "http://127.0.0.1:7890", testFetchLimit) if err != nil { diff --git a/pkg/utils/http_client.go b/pkg/utils/http_client.go new file mode 100644 index 000000000..bda7c5c83 --- /dev/null +++ b/pkg/utils/http_client.go @@ -0,0 +1,48 @@ +package utils + +import ( + "fmt" + "net/http" + "net/url" + "strings" + "time" +) + +// CreateHTTPClient creates an HTTP client with optional proxy support. +// If proxyURL is empty, it uses the system environment proxy settings. +// Supported proxy schemes: http, https, socks5, socks5h. +func CreateHTTPClient(proxyURL string, timeout time.Duration) (*http.Client, error) { + client := &http.Client{ + Timeout: timeout, + Transport: &http.Transport{ + MaxIdleConns: 10, + IdleConnTimeout: 30 * time.Second, + DisableCompression: false, + TLSHandshakeTimeout: 15 * time.Second, + }, + } + + if proxyURL != "" { + proxy, err := url.Parse(proxyURL) + if err != nil { + return nil, fmt.Errorf("invalid proxy URL: %w", err) + } + scheme := strings.ToLower(proxy.Scheme) + switch scheme { + case "http", "https", "socks5", "socks5h": + default: + return nil, fmt.Errorf( + "unsupported proxy scheme %q (supported: http, https, socks5, socks5h)", + proxy.Scheme, + ) + } + if proxy.Host == "" { + return nil, fmt.Errorf("invalid proxy URL: missing host") + } + client.Transport.(*http.Transport).Proxy = http.ProxyURL(proxy) + } else { + client.Transport.(*http.Transport).Proxy = http.ProxyFromEnvironment + } + + return client, nil +} diff --git a/pkg/utils/http_client_test.go b/pkg/utils/http_client_test.go new file mode 100644 index 000000000..ff3d0429b --- /dev/null +++ b/pkg/utils/http_client_test.go @@ -0,0 +1,110 @@ +package utils + +import ( + "net/http" + "strings" + "testing" + "time" +) + +func TestCreateHTTPClient_ProxyConfigured(t *testing.T) { + client, err := CreateHTTPClient("http://127.0.0.1:7890", 12*time.Second) + if err != nil { + t.Fatalf("createHTTPClient() error: %v", err) + } + if client.Timeout != 12*time.Second { + t.Fatalf("client.Timeout = %v, want %v", client.Timeout, 12*time.Second) + } + + tr, ok := client.Transport.(*http.Transport) + if !ok { + t.Fatalf("client.Transport type = %T, want *http.Transport", client.Transport) + } + if tr.Proxy == nil { + t.Fatal("transport.Proxy is nil, want non-nil") + } + + req, err := http.NewRequest("GET", "https://example.com", nil) + if err != nil { + t.Fatalf("http.NewRequest() error: %v", err) + } + proxyURL, err := tr.Proxy(req) + if err != nil { + t.Fatalf("transport.Proxy(req) error: %v", err) + } + if proxyURL == nil || proxyURL.String() != "http://127.0.0.1:7890" { + t.Fatalf("proxy URL = %v, want %q", proxyURL, "http://127.0.0.1:7890") + } +} + +func TestCreateHTTPClient_InvalidProxy(t *testing.T) { + _, err := CreateHTTPClient("://bad-proxy", 10*time.Second) + if err == nil { + t.Fatal("createHTTPClient() expected error for invalid proxy URL, got nil") + } +} + +func TestCreateHTTPClient_Socks5ProxyConfigured(t *testing.T) { + client, err := CreateHTTPClient("socks5://127.0.0.1:1080", 8*time.Second) + if err != nil { + t.Fatalf("createHTTPClient() error: %v", err) + } + + tr, ok := client.Transport.(*http.Transport) + if !ok { + t.Fatalf("client.Transport type = %T, want *http.Transport", client.Transport) + } + req, err := http.NewRequest("GET", "https://example.com", nil) + if err != nil { + t.Fatalf("http.NewRequest() error: %v", err) + } + proxyURL, err := tr.Proxy(req) + if err != nil { + t.Fatalf("transport.Proxy(req) error: %v", err) + } + if proxyURL == nil || proxyURL.String() != "socks5://127.0.0.1:1080" { + t.Fatalf("proxy URL = %v, want %q", proxyURL, "socks5://127.0.0.1:1080") + } +} + +func TestCreateHTTPClient_UnsupportedProxyScheme(t *testing.T) { + _, err := CreateHTTPClient("ftp://127.0.0.1:21", 10*time.Second) + if err == nil { + t.Fatal("createHTTPClient() expected error for unsupported scheme, got nil") + } + if !strings.Contains(err.Error(), "unsupported proxy scheme") { + t.Fatalf("error = %q, want to contain %q", err.Error(), "unsupported proxy scheme") + } +} + +func TestCreateHTTPClient_ProxyFromEnvironmentWhenConfigEmpty(t *testing.T) { + t.Setenv("HTTP_PROXY", "http://127.0.0.1:8888") + t.Setenv("http_proxy", "http://127.0.0.1:8888") + t.Setenv("HTTPS_PROXY", "http://127.0.0.1:8888") + t.Setenv("https_proxy", "http://127.0.0.1:8888") + t.Setenv("ALL_PROXY", "") + t.Setenv("all_proxy", "") + t.Setenv("NO_PROXY", "") + t.Setenv("no_proxy", "") + + client, err := CreateHTTPClient("", 10*time.Second) + if err != nil { + t.Fatalf("createHTTPClient() error: %v", err) + } + + tr, ok := client.Transport.(*http.Transport) + if !ok { + t.Fatalf("client.Transport type = %T, want *http.Transport", client.Transport) + } + if tr.Proxy == nil { + t.Fatal("transport.Proxy is nil, want proxy function from environment") + } + + req, err := http.NewRequest("GET", "https://example.com", nil) + if err != nil { + t.Fatalf("http.NewRequest() error: %v", err) + } + if _, err := tr.Proxy(req); err != nil { + t.Fatalf("transport.Proxy(req) error: %v", err) + } +} diff --git a/web/backend/api/events.go b/web/backend/api/events.go index 0a8d4a9bb..af44d1824 100644 --- a/web/backend/api/events.go +++ b/web/backend/api/events.go @@ -7,8 +7,11 @@ import ( // GatewayEvent represents a state change event for the gateway process. type GatewayEvent struct { - Status string `json:"gateway_status"` // "running", "starting", "stopped", "error" - PID int `json:"pid,omitempty"` + Status string `json:"gateway_status"` // "running", "starting", "restarting", "stopped", "error" + PID int `json:"pid,omitempty"` + BootDefaultModel string `json:"boot_default_model,omitempty"` + ConfigDefaultModel string `json:"config_default_model,omitempty"` + RestartRequired bool `json:"gateway_restart_required,omitempty"` } // EventBroadcaster manages SSE client subscriptions and broadcasts events. diff --git a/web/backend/api/gateway.go b/web/backend/api/gateway.go index 41f702e32..1813cac92 100644 --- a/web/backend/api/gateway.go +++ b/web/backend/api/gateway.go @@ -23,19 +23,36 @@ import ( // gateway holds the state for the managed gateway process. var gateway = struct { - mu sync.Mutex - cmd *exec.Cmd - logs *LogBuffer - events *EventBroadcaster + mu sync.Mutex + cmd *exec.Cmd + bootDefaultModel string + runtimeStatus string + startupDeadline time.Time + logs *LogBuffer + events *EventBroadcaster }{ - logs: NewLogBuffer(200), - events: NewEventBroadcaster(), + runtimeStatus: "stopped", + logs: NewLogBuffer(200), + events: NewEventBroadcaster(), +} + +var ( + gatewayStartupWindow = 15 * time.Second + gatewayRestartGracePeriod = 5 * time.Second + gatewayRestartForceKillWindow = 3 * time.Second + gatewayRestartPollInterval = 100 * time.Millisecond +) + +var gatewayHealthGet = func(url string, timeout time.Duration) (*http.Response, error) { + client := http.Client{Timeout: timeout} + return client.Get(url) } // registerGatewayRoutes binds gateway lifecycle endpoints to the ServeMux. func (h *Handler) registerGatewayRoutes(mux *http.ServeMux) { mux.HandleFunc("GET /api/gateway/status", h.handleGatewayStatus) mux.HandleFunc("GET /api/gateway/events", h.handleGatewayEvents) + mux.HandleFunc("GET /api/gateway/logs", h.handleGatewayLogs) mux.HandleFunc("POST /api/gateway/logs/clear", h.handleGatewayClearLogs) mux.HandleFunc("POST /api/gateway/start", h.handleGatewayStart) mux.HandleFunc("POST /api/gateway/stop", h.handleGatewayStop) @@ -65,7 +82,7 @@ func (h *Handler) TryAutoStartGateway() { return } - pid, err := h.startGatewayLocked() + pid, err := h.startGatewayLocked("starting") if err != nil { log.Printf("Failed to auto-start gateway: %v", err) return @@ -131,7 +148,110 @@ func isCmdProcessAliveLocked(cmd *exec.Cmd) bool { return cmd.Process.Signal(syscall.Signal(0)) == nil } -func (h *Handler) startGatewayLocked() (int, error) { +func setGatewayRuntimeStatusLocked(status string) { + gateway.runtimeStatus = status + if status == "starting" || status == "restarting" { + gateway.startupDeadline = time.Now().Add(gatewayStartupWindow) + return + } + gateway.startupDeadline = time.Time{} +} + +func gatewayStatusOnHealthFailureLocked() string { + if gateway.runtimeStatus == "starting" || gateway.runtimeStatus == "restarting" { + if gateway.startupDeadline.IsZero() || time.Now().Before(gateway.startupDeadline) { + return gateway.runtimeStatus + } + return "error" + } + if gateway.runtimeStatus == "running" { + return "running" + } + if gateway.runtimeStatus == "error" { + return "error" + } + return "error" +} + +func currentGatewayStatusLocked(processAlive bool) string { + if !processAlive { + if gateway.runtimeStatus == "restarting" { + if gateway.startupDeadline.IsZero() || time.Now().Before(gateway.startupDeadline) { + return "restarting" + } + return "error" + } + if gateway.runtimeStatus == "error" { + return "error" + } + return "stopped" + } + return gatewayStatusOnHealthFailureLocked() +} + +func waitForGatewayProcessExit(cmd *exec.Cmd, timeout time.Duration) bool { + if cmd == nil || cmd.Process == nil { + return true + } + + deadline := time.Now().Add(timeout) + for { + if !isCmdProcessAliveLocked(cmd) { + return true + } + if time.Now().After(deadline) { + return false + } + time.Sleep(gatewayRestartPollInterval) + } +} + +func stopGatewayProcessForRestart(cmd *exec.Cmd) error { + if cmd == nil || cmd.Process == nil || !isCmdProcessAliveLocked(cmd) { + return nil + } + + var stopErr error + if runtime.GOOS == "windows" { + stopErr = cmd.Process.Kill() + } else { + stopErr = cmd.Process.Signal(syscall.SIGTERM) + } + if stopErr != nil && isCmdProcessAliveLocked(cmd) { + return fmt.Errorf("failed to stop existing gateway: %w", stopErr) + } + + if waitForGatewayProcessExit(cmd, gatewayRestartGracePeriod) { + return nil + } + + if runtime.GOOS != "windows" { + killErr := cmd.Process.Signal(syscall.SIGKILL) + if killErr != nil && isCmdProcessAliveLocked(cmd) { + return fmt.Errorf("failed to force-stop existing gateway: %w", killErr) + } + if waitForGatewayProcessExit(cmd, gatewayRestartForceKillWindow) { + return nil + } + } + + return fmt.Errorf("existing gateway did not exit before restart") +} + +func gatewayRestartRequired(status, bootDefaultModel, configDefaultModel string) bool { + return status == "running" && + bootDefaultModel != "" && + configDefaultModel != "" && + bootDefaultModel != configDefaultModel +} + +func (h *Handler) startGatewayLocked(initialStatus string) (int, error) { + cfg, err := config.LoadConfig(h.configPath) + if err != nil { + return 0, fmt.Errorf("failed to load config: %w", err) + } + defaultModelName := strings.TrimSpace(cfg.Agents.Defaults.GetModelName()) + // Locate the picoclaw executable execPath := utils.FindPicoclawBinary() @@ -171,11 +291,19 @@ func (h *Handler) startGatewayLocked() (int, error) { } gateway.cmd = cmd + gateway.bootDefaultModel = defaultModelName + setGatewayRuntimeStatusLocked(initialStatus) pid := cmd.Process.Pid log.Printf("Started picoclaw gateway (PID: %d) from %s", pid, execPath) - // Broadcast starting event - gateway.events.Broadcast(GatewayEvent{Status: "starting", PID: pid}) + // Broadcast the launch state immediately so clients can reflect it without polling. + gateway.events.Broadcast(GatewayEvent{ + Status: initialStatus, + PID: pid, + BootDefaultModel: defaultModelName, + ConfigDefaultModel: defaultModelName, + RestartRequired: false, + }) // Capture stdout/stderr in background go scanPipe(stdoutPipe, gateway.logs) @@ -190,13 +318,23 @@ func (h *Handler) startGatewayLocked() (int, error) { } gateway.mu.Lock() + shouldBroadcastStopped := false if gateway.cmd == cmd { gateway.cmd = nil + gateway.bootDefaultModel = "" + if gateway.runtimeStatus != "restarting" { + setGatewayRuntimeStatusLocked("stopped") + shouldBroadcastStopped = true + } } gateway.mu.Unlock() - // Broadcast stopped event - gateway.events.Broadcast(GatewayEvent{Status: "stopped"}) + if shouldBroadcastStopped { + gateway.events.Broadcast(GatewayEvent{ + Status: "stopped", + RestartRequired: false, + }) + } }() // Start a goroutine to probe health and broadcast "running" once ready @@ -219,12 +357,22 @@ func (h *Handler) startGatewayLocked() (int, error) { healthPort = 18790 } healthURL := fmt.Sprintf("http://%s/health", net.JoinHostPort(healthHost, strconv.Itoa(healthPort))) - client := http.Client{Timeout: 1 * time.Second} - resp, err := client.Get(healthURL) + resp, err := gatewayHealthGet(healthURL, 1*time.Second) if err == nil { resp.Body.Close() if resp.StatusCode == http.StatusOK { - gateway.events.Broadcast(GatewayEvent{Status: "running", PID: pid}) + gateway.mu.Lock() + if gateway.cmd == cmd { + setGatewayRuntimeStatusLocked("running") + } + gateway.mu.Unlock() + gateway.events.Broadcast(GatewayEvent{ + Status: "running", + PID: pid, + BootDefaultModel: defaultModelName, + ConfigDefaultModel: defaultModelName, + RestartRequired: false, + }) return } } @@ -253,6 +401,7 @@ func (h *Handler) handleGatewayStart(w http.ResponseWriter, r *http.Request) { } if gateway.cmd != nil && gateway.cmd.Process != nil { gateway.cmd = nil + setGatewayRuntimeStatusLocked("stopped") } ready, reason, err := h.gatewayStartReady() @@ -274,7 +423,7 @@ func (h *Handler) handleGatewayStart(w http.ResponseWriter, r *http.Request) { return } - pid, err := h.startGatewayLocked() + pid, err := h.startGatewayLocked("starting") if err != nil { http.Error(w, fmt.Sprintf("Failed to start gateway: %v", err), http.StatusInternalServerError) return @@ -330,30 +479,72 @@ func (h *Handler) handleGatewayStop(w http.ResponseWriter, r *http.Request) { // // POST /api/gateway/restart func (h *Handler) handleGatewayRestart(w http.ResponseWriter, r *http.Request) { - gateway.mu.Lock() - - // Stop existing process if running - if gateway.cmd != nil && gateway.cmd.Process != nil { - if isCmdProcessAliveLocked(gateway.cmd) { - // Process is alive, send SIGTERM - if runtime.GOOS == "windows" { - gateway.cmd.Process.Kill() - } else { - gateway.cmd.Process.Signal(syscall.SIGTERM) - } - - // Wait briefly for it to exit - gateway.mu.Unlock() - time.Sleep(2 * time.Second) - gateway.mu.Lock() - } - gateway.cmd = nil + ready, reason, err := h.gatewayStartReady() + if err != nil { + http.Error( + w, + fmt.Sprintf("Failed to validate gateway start conditions: %v", err), + http.StatusInternalServerError, + ) + return + } + if !ready { + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusBadRequest) + json.NewEncoder(w).Encode(map[string]any{ + "status": "precondition_failed", + "message": reason, + }) + return } + gateway.mu.Lock() + previousCmd := gateway.cmd + setGatewayRuntimeStatusLocked("restarting") + gateway.events.Broadcast(GatewayEvent{ + Status: "restarting", + RestartRequired: false, + }) gateway.mu.Unlock() - // Start fresh via the existing handler - h.handleGatewayStart(w, r) + if err = stopGatewayProcessForRestart(previousCmd); err != nil { + gateway.mu.Lock() + if gateway.cmd == previousCmd { + if isCmdProcessAliveLocked(previousCmd) { + setGatewayRuntimeStatusLocked("running") + } else { + gateway.cmd = nil + gateway.bootDefaultModel = "" + setGatewayRuntimeStatusLocked("error") + } + } + gateway.mu.Unlock() + http.Error(w, fmt.Sprintf("Failed to restart gateway: %v", err), http.StatusInternalServerError) + return + } + + gateway.mu.Lock() + if gateway.cmd == previousCmd { + gateway.cmd = nil + gateway.bootDefaultModel = "" + } + pid, err := h.startGatewayLocked("restarting") + if err != nil { + gateway.cmd = nil + gateway.bootDefaultModel = "" + setGatewayRuntimeStatusLocked("error") + } + gateway.mu.Unlock() + if err != nil { + http.Error(w, fmt.Sprintf("Failed to restart gateway: %v", err), http.StatusInternalServerError) + return + } + + w.Header().Set("Content-Type", "application/json") + json.NewEncoder(w).Encode(map[string]any{ + "status": "ok", + "pid": pid, + }) } // handleGatewayClearLogs clears the in-memory gateway log buffer. @@ -370,28 +561,48 @@ func (h *Handler) handleGatewayClearLogs(w http.ResponseWriter, r *http.Request) }) } -// handleGatewayStatus returns the gateway run status, health info, and logs. +// handleGatewayStatus returns the gateway run status and health info. // // GET /api/gateway/status func (h *Handler) handleGatewayStatus(w http.ResponseWriter, r *http.Request) { + data := h.gatewayStatusData() + w.Header().Set("Content-Type", "application/json") + json.NewEncoder(w).Encode(data) +} + +func (h *Handler) gatewayStatusData() map[string]any { data := map[string]any{} + cfg, cfgErr := config.LoadConfig(h.configPath) + configDefaultModel := "" + if cfgErr == nil && cfg != nil { + configDefaultModel = strings.TrimSpace(cfg.Agents.Defaults.GetModelName()) + if configDefaultModel != "" { + data["config_default_model"] = configDefaultModel + } + } // Check process state gateway.mu.Lock() processAlive := isGatewayProcessAliveLocked() + bootDefaultModel := "" if processAlive { data["pid"] = gateway.cmd.Process.Pid + if gateway.bootDefaultModel != "" { + data["boot_default_model"] = gateway.bootDefaultModel + bootDefaultModel = gateway.bootDefaultModel + } } gateway.mu.Unlock() if !processAlive { - data["gateway_status"] = "stopped" + gateway.mu.Lock() + data["gateway_status"] = currentGatewayStatusLocked(false) + gateway.mu.Unlock() } else { // Process is alive — probe its health endpoint - cfg, err := config.LoadConfig(h.configPath) host := "127.0.0.1" port := 18790 - if err == nil && cfg != nil { + if cfgErr == nil && cfg != nil { host = gatewayProbeHost(h.effectiveGatewayBindHost(cfg)) if cfg.Gateway.Port != 0 { port = cfg.Gateway.Port @@ -399,21 +610,31 @@ func (h *Handler) handleGatewayStatus(w http.ResponseWriter, r *http.Request) { } url := fmt.Sprintf("http://%s/health", net.JoinHostPort(host, strconv.Itoa(port))) - client := http.Client{Timeout: 2 * time.Second} - resp, err := client.Get(url) + resp, err := gatewayHealthGet(url, 2*time.Second) if err != nil { - data["gateway_status"] = "starting" + gateway.mu.Lock() + data["gateway_status"] = currentGatewayStatusLocked(true) + gateway.mu.Unlock() } else { defer resp.Body.Close() if resp.StatusCode != http.StatusOK { + gateway.mu.Lock() + setGatewayRuntimeStatusLocked("error") + gateway.mu.Unlock() data["gateway_status"] = "error" data["status_code"] = resp.StatusCode } else { var healthData map[string]any if decErr := json.NewDecoder(resp.Body).Decode(&healthData); decErr != nil { + gateway.mu.Lock() + setGatewayRuntimeStatusLocked("error") + gateway.mu.Unlock() data["gateway_status"] = "error" } else { + gateway.mu.Lock() + setGatewayRuntimeStatusLocked("running") + gateway.mu.Unlock() for k, v := range healthData { data[k] = v } @@ -423,6 +644,13 @@ func (h *Handler) handleGatewayStatus(w http.ResponseWriter, r *http.Request) { } } + status, _ := data["gateway_status"].(string) + data["gateway_restart_required"] = gatewayRestartRequired( + status, + bootDefaultModel, + configDefaultModel, + ) + ready, reason, readyErr := h.gatewayStartReady() if readyErr != nil { data["gateway_start_allowed"] = false @@ -434,16 +662,22 @@ func (h *Handler) handleGatewayStatus(w http.ResponseWriter, r *http.Request) { } } - // Append incremental log data - appendGatewayLogs(r, data) + return data +} +// handleGatewayLogs returns buffered gateway logs, optionally incrementally. +// +// GET /api/gateway/logs +func (h *Handler) handleGatewayLogs(w http.ResponseWriter, r *http.Request) { + data := gatewayLogsData(r) w.Header().Set("Content-Type", "application/json") json.NewEncoder(w).Encode(data) } -// appendGatewayLogs reads log_offset and log_run_id query params from the request -// and populates the response data map with incremental log lines. -func appendGatewayLogs(r *http.Request, data map[string]any) { +// gatewayLogsData reads log_offset and log_run_id query params from the request +// and returns incremental log lines. +func gatewayLogsData(r *http.Request) map[string]any { + data := map[string]any{} clientOffset := 0 clientRunID := -1 @@ -465,7 +699,7 @@ func appendGatewayLogs(r *http.Request, data map[string]any) { data["logs"] = []string{} data["log_total"] = 0 data["log_run_id"] = 0 - return + return data } // If runID changed, reset offset to get all logs from new run @@ -482,6 +716,7 @@ func appendGatewayLogs(r *http.Request, data map[string]any) { data["logs"] = lines data["log_total"] = total data["log_run_id"] = runID + return data } // handleGatewayEvents serves an SSE stream of gateway state change events. @@ -524,28 +759,7 @@ func (h *Handler) handleGatewayEvents(w http.ResponseWriter, r *http.Request) { // currentGatewayStatus returns the current gateway status as a JSON string. func (h *Handler) currentGatewayStatus() string { - gateway.mu.Lock() - defer gateway.mu.Unlock() - - data := map[string]any{ - "gateway_status": "stopped", - } - if isGatewayProcessAliveLocked() { - data["gateway_status"] = "running" - data["pid"] = gateway.cmd.Process.Pid - } - - ready, reason, readyErr := h.gatewayStartReady() - if readyErr != nil { - data["gateway_start_allowed"] = false - data["gateway_start_reason"] = readyErr.Error() - } else { - data["gateway_start_allowed"] = ready - if !ready { - data["gateway_start_reason"] = reason - } - } - + data := h.gatewayStatusData() encoded, _ := json.Marshal(data) return string(encoded) } diff --git a/web/backend/api/gateway_test.go b/web/backend/api/gateway_test.go index d4265776a..06803722d 100644 --- a/web/backend/api/gateway_test.go +++ b/web/backend/api/gateway_test.go @@ -2,19 +2,76 @@ package api import ( "encoding/json" + "errors" "net/http" "net/http/httptest" "os" + "os/exec" "path/filepath" + "runtime" "strconv" "strings" "testing" + "time" "github.com/sipeed/picoclaw/pkg/auth" "github.com/sipeed/picoclaw/pkg/config" "github.com/sipeed/picoclaw/web/backend/utils" ) +func startLongRunningProcess(t *testing.T) *exec.Cmd { + t.Helper() + + var cmd *exec.Cmd + if runtime.GOOS == "windows" { + cmd = exec.Command("powershell", "-NoProfile", "-Command", "Start-Sleep -Seconds 30") + } else { + cmd = exec.Command("sleep", "30") + } + + if err := cmd.Start(); err != nil { + t.Fatalf("Start() error = %v", err) + } + + return cmd +} + +func startIgnoringTermProcess(t *testing.T) *exec.Cmd { + t.Helper() + + if runtime.GOOS == "windows" { + t.Skip("TERM handling differs on Windows") + } + + cmd := exec.Command("sh", "-c", "trap '' TERM; sleep 30") + if err := cmd.Start(); err != nil { + t.Fatalf("Start() error = %v", err) + } + + return cmd +} + +func resetGatewayTestState(t *testing.T) { + t.Helper() + + originalHealthGet := gatewayHealthGet + originalRestartGracePeriod := gatewayRestartGracePeriod + originalRestartForceKillWindow := gatewayRestartForceKillWindow + originalRestartPollInterval := gatewayRestartPollInterval + t.Cleanup(func() { + gatewayHealthGet = originalHealthGet + gatewayRestartGracePeriod = originalRestartGracePeriod + gatewayRestartForceKillWindow = originalRestartForceKillWindow + gatewayRestartPollInterval = originalRestartPollInterval + + gateway.mu.Lock() + gateway.cmd = nil + gateway.bootDefaultModel = "" + setGatewayRuntimeStatusLocked("stopped") + gateway.mu.Unlock() + }) +} + func TestGatewayStartReady_NoDefaultModel(t *testing.T) { configPath := filepath.Join(t.TempDir(), "config.json") h := NewHandler(configPath) @@ -317,6 +374,412 @@ func TestGatewayStatusIncludesStartConditionWhenNotReady(t *testing.T) { } } +func TestGatewayStatusKeepsRunningWhenHealthProbeFailsAfterRunning(t *testing.T) { + resetGatewayTestState(t) + + configPath := filepath.Join(t.TempDir(), "config.json") + h := NewHandler(configPath) + mux := http.NewServeMux() + h.RegisterRoutes(mux) + + cmd := startLongRunningProcess(t) + t.Cleanup(func() { + if cmd.Process != nil { + _ = cmd.Process.Kill() + } + _ = cmd.Wait() + }) + + gateway.mu.Lock() + gateway.cmd = cmd + gateway.bootDefaultModel = "existing-model" + // Simulate a process that has already reached the running state. + setGatewayRuntimeStatusLocked("running") + gateway.mu.Unlock() + + gatewayHealthGet = func(string, time.Duration) (*http.Response, error) { + return nil, errors.New("probe failed") + } + + rec := httptest.NewRecorder() + req := httptest.NewRequest(http.MethodGet, "/api/gateway/status", nil) + mux.ServeHTTP(rec, req) + + if rec.Code != http.StatusOK { + t.Fatalf("status = %d, want %d", rec.Code, http.StatusOK) + } + + var body map[string]any + if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil { + t.Fatalf("unmarshal response: %v", err) + } + + if got := body["gateway_status"]; got != "running" { + t.Fatalf("gateway_status = %#v, want %q", got, "running") + } +} + +func TestGatewayStatusReturnsErrorAfterStartupWindowExpires(t *testing.T) { + resetGatewayTestState(t) + + configPath := filepath.Join(t.TempDir(), "config.json") + h := NewHandler(configPath) + mux := http.NewServeMux() + h.RegisterRoutes(mux) + + cmd := startLongRunningProcess(t) + t.Cleanup(func() { + if cmd.Process != nil { + _ = cmd.Process.Kill() + } + _ = cmd.Wait() + }) + + gateway.mu.Lock() + gateway.cmd = cmd + gateway.bootDefaultModel = "existing-model" + setGatewayRuntimeStatusLocked("starting") + gateway.startupDeadline = time.Now().Add(-time.Second) + gateway.mu.Unlock() + + gatewayHealthGet = func(string, time.Duration) (*http.Response, error) { + return nil, errors.New("probe failed") + } + + rec := httptest.NewRecorder() + req := httptest.NewRequest(http.MethodGet, "/api/gateway/status", nil) + mux.ServeHTTP(rec, req) + + if rec.Code != http.StatusOK { + t.Fatalf("status = %d, want %d", rec.Code, http.StatusOK) + } + + var body map[string]any + if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil { + t.Fatalf("unmarshal response: %v", err) + } + + if got := body["gateway_status"]; got != "error" { + t.Fatalf("gateway_status = %#v, want %q", got, "error") + } +} + +func TestGatewayStatusReturnsRestartingDuringRestartGap(t *testing.T) { + resetGatewayTestState(t) + + configPath := filepath.Join(t.TempDir(), "config.json") + h := NewHandler(configPath) + mux := http.NewServeMux() + h.RegisterRoutes(mux) + + gateway.mu.Lock() + setGatewayRuntimeStatusLocked("restarting") + gateway.mu.Unlock() + + rec := httptest.NewRecorder() + req := httptest.NewRequest(http.MethodGet, "/api/gateway/status", nil) + mux.ServeHTTP(rec, req) + + if rec.Code != http.StatusOK { + t.Fatalf("status = %d, want %d", rec.Code, http.StatusOK) + } + + var body map[string]any + if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil { + t.Fatalf("unmarshal response: %v", err) + } + + if got := body["gateway_status"]; got != "restarting" { + t.Fatalf("gateway_status = %#v, want %q", got, "restarting") + } +} + +func TestGatewayStatusIncludesRestartRequiredWhenModelsDiffer(t *testing.T) { + resetGatewayTestState(t) + + configPath := filepath.Join(t.TempDir(), "config.json") + cfg := config.DefaultConfig() + cfg.Agents.Defaults.ModelName = cfg.ModelList[0].ModelName + cfg.ModelList[0].APIKey = "test-key" + if err := config.SaveConfig(configPath, cfg); err != nil { + t.Fatalf("SaveConfig() error = %v", err) + } + + h := NewHandler(configPath) + mux := http.NewServeMux() + h.RegisterRoutes(mux) + + cmd := startLongRunningProcess(t) + t.Cleanup(func() { + if cmd.Process != nil { + _ = cmd.Process.Kill() + } + _ = cmd.Wait() + }) + + gateway.mu.Lock() + gateway.cmd = cmd + gateway.bootDefaultModel = "previous-model" + setGatewayRuntimeStatusLocked("running") + gateway.mu.Unlock() + + gatewayHealthGet = func(string, time.Duration) (*http.Response, error) { + rec := httptest.NewRecorder() + rec.WriteHeader(http.StatusOK) + _, _ = rec.WriteString(`{"ok":true}`) + return rec.Result(), nil + } + + rec := httptest.NewRecorder() + req := httptest.NewRequest(http.MethodGet, "/api/gateway/status", nil) + mux.ServeHTTP(rec, req) + + if rec.Code != http.StatusOK { + t.Fatalf("status = %d, want %d", rec.Code, http.StatusOK) + } + + var body map[string]any + if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil { + t.Fatalf("unmarshal response: %v", err) + } + + if got := body["gateway_restart_required"]; got != true { + t.Fatalf("gateway_restart_required = %#v, want true", got) + } +} + +func TestGatewayRestartKeepsRunningProcessWhenPreconditionsFail(t *testing.T) { + configPath := filepath.Join(t.TempDir(), "config.json") + cfg := config.DefaultConfig() + cfg.Agents.Defaults.ModelName = cfg.ModelList[0].ModelName + cfg.ModelList[0].APIKey = "" + cfg.ModelList[0].AuthMethod = "" + if err := config.SaveConfig(configPath, cfg); err != nil { + t.Fatalf("SaveConfig() error = %v", err) + } + + h := NewHandler(configPath) + mux := http.NewServeMux() + h.RegisterRoutes(mux) + + cmd := startLongRunningProcess(t) + t.Cleanup(func() { + gateway.mu.Lock() + if gateway.cmd == cmd { + gateway.cmd = nil + gateway.bootDefaultModel = "" + } + gateway.mu.Unlock() + + if cmd.Process != nil { + _ = cmd.Process.Kill() + } + _ = cmd.Wait() + }) + + gateway.mu.Lock() + gateway.cmd = cmd + gateway.bootDefaultModel = "existing-model" + gateway.mu.Unlock() + + rec := httptest.NewRecorder() + req := httptest.NewRequest(http.MethodPost, "/api/gateway/restart", nil) + mux.ServeHTTP(rec, req) + + if rec.Code != http.StatusBadRequest { + t.Fatalf("status = %d, want %d", rec.Code, http.StatusBadRequest) + } + + gateway.mu.Lock() + stillRunning := gateway.cmd == cmd && isCmdProcessAliveLocked(cmd) + gateway.mu.Unlock() + + if !stillRunning { + t.Fatalf("gateway process was stopped when restart preconditions failed") + } +} + +func TestGatewayRestartKeepsOldProcessWhenItDoesNotExitInTime(t *testing.T) { + resetGatewayTestState(t) + + configPath := filepath.Join(t.TempDir(), "config.json") + cfg := config.DefaultConfig() + cfg.Agents.Defaults.ModelName = cfg.ModelList[0].ModelName + cfg.ModelList[0].APIKey = "test-key" + if err := config.SaveConfig(configPath, cfg); err != nil { + t.Fatalf("SaveConfig() error = %v", err) + } + + h := NewHandler(configPath) + mux := http.NewServeMux() + h.RegisterRoutes(mux) + + cmd := startIgnoringTermProcess(t) + t.Cleanup(func() { + gateway.mu.Lock() + if gateway.cmd == cmd { + gateway.cmd = nil + gateway.bootDefaultModel = "" + } + gateway.mu.Unlock() + + if cmd.Process != nil { + _ = cmd.Process.Kill() + } + _ = cmd.Wait() + }) + + gatewayRestartGracePeriod = 150 * time.Millisecond + gatewayRestartForceKillWindow = 150 * time.Millisecond + gatewayRestartPollInterval = 10 * time.Millisecond + + gateway.mu.Lock() + gateway.cmd = cmd + gateway.bootDefaultModel = "existing-model" + setGatewayRuntimeStatusLocked("running") + gateway.mu.Unlock() + + rec := httptest.NewRecorder() + req := httptest.NewRequest(http.MethodPost, "/api/gateway/restart", nil) + mux.ServeHTTP(rec, req) + + if rec.Code != http.StatusInternalServerError { + t.Fatalf("status = %d, want %d", rec.Code, http.StatusInternalServerError) + } + + gateway.mu.Lock() + stillRunning := gateway.cmd == cmd && isCmdProcessAliveLocked(cmd) + status := gateway.runtimeStatus + gateway.mu.Unlock() + + if !stillRunning { + t.Fatalf("gateway process was replaced before the old process exited") + } + if status != "running" { + t.Fatalf("runtimeStatus = %q, want %q", status, "running") + } +} + +func TestGatewayRestartReturnsErrorStatusWhenReplacementFailsToStart(t *testing.T) { + resetGatewayTestState(t) + + configPath := filepath.Join(t.TempDir(), "config.json") + cfg := config.DefaultConfig() + cfg.Agents.Defaults.ModelName = cfg.ModelList[0].ModelName + cfg.ModelList[0].APIKey = "test-key" + if err := config.SaveConfig(configPath, cfg); err != nil { + t.Fatalf("SaveConfig() error = %v", err) + } + + invalidBinaryPath := filepath.Join(t.TempDir(), "fake-picoclaw") + if err := os.WriteFile(invalidBinaryPath, []byte("#!/bin/sh\n"), 0o644); err != nil { + t.Fatalf("WriteFile() error = %v", err) + } + t.Setenv("PICOCLAW_BINARY", invalidBinaryPath) + + h := NewHandler(configPath) + mux := http.NewServeMux() + h.RegisterRoutes(mux) + + rec := httptest.NewRecorder() + req := httptest.NewRequest(http.MethodPost, "/api/gateway/restart", nil) + mux.ServeHTTP(rec, req) + + if rec.Code != http.StatusInternalServerError { + t.Fatalf("restart status = %d, want %d", rec.Code, http.StatusInternalServerError) + } + + statusRec := httptest.NewRecorder() + statusReq := httptest.NewRequest(http.MethodGet, "/api/gateway/status", nil) + mux.ServeHTTP(statusRec, statusReq) + + if statusRec.Code != http.StatusOK { + t.Fatalf("status = %d, want %d", statusRec.Code, http.StatusOK) + } + + var body map[string]any + if err := json.Unmarshal(statusRec.Body.Bytes(), &body); err != nil { + t.Fatalf("unmarshal response: %v", err) + } + + if got := body["gateway_status"]; got != "error" { + t.Fatalf("gateway_status = %#v, want %q", got, "error") + } +} + +func TestGatewayStatusExcludesLogsFields(t *testing.T) { + configPath := filepath.Join(t.TempDir(), "config.json") + h := NewHandler(configPath) + mux := http.NewServeMux() + h.RegisterRoutes(mux) + + rec := httptest.NewRecorder() + req := httptest.NewRequest(http.MethodGet, "/api/gateway/status", nil) + mux.ServeHTTP(rec, req) + + if rec.Code != http.StatusOK { + t.Fatalf("status = %d, want %d", rec.Code, http.StatusOK) + } + + var body map[string]any + if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil { + t.Fatalf("unmarshal response: %v", err) + } + + if _, ok := body["logs"]; ok { + t.Fatalf("logs unexpectedly present in status response: %#v", body["logs"]) + } + if _, ok := body["log_total"]; ok { + t.Fatalf("log_total unexpectedly present in status response: %#v", body["log_total"]) + } + if _, ok := body["log_run_id"]; ok { + t.Fatalf("log_run_id unexpectedly present in status response: %#v", body["log_run_id"]) + } +} + +func TestGatewayLogsReturnsIncrementalHistory(t *testing.T) { + configPath := filepath.Join(t.TempDir(), "config.json") + h := NewHandler(configPath) + mux := http.NewServeMux() + h.RegisterRoutes(mux) + + gateway.logs.Clear() + gateway.logs.Append("first line") + gateway.logs.Append("second line") + runID := gateway.logs.RunID() + + rec := httptest.NewRecorder() + req := httptest.NewRequest( + http.MethodGet, + "/api/gateway/logs?log_offset=1&log_run_id="+strconv.Itoa(runID), + nil, + ) + mux.ServeHTTP(rec, req) + + if rec.Code != http.StatusOK { + t.Fatalf("logs status = %d, want %d", rec.Code, http.StatusOK) + } + + var body map[string]any + if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil { + t.Fatalf("unmarshal logs response: %v", err) + } + + logs, ok := body["logs"].([]any) + if !ok { + t.Fatalf("logs missing or not array: %#v", body["logs"]) + } + if len(logs) != 1 || logs[0] != "second line" { + t.Fatalf("logs = %#v, want [\"second line\"]", logs) + } + if got := body["log_total"]; got != float64(2) { + t.Fatalf("log_total = %#v, want 2", got) + } + if got := body["log_run_id"]; got != float64(runID) { + t.Fatalf("log_run_id = %#v, want %d", got, runID) + } +} + func TestGatewayClearLogsResetsBufferedHistory(t *testing.T) { configPath := filepath.Join(t.TempDir(), "config.json") h := NewHandler(configPath) @@ -353,33 +816,36 @@ func TestGatewayClearLogsResetsBufferedHistory(t *testing.T) { t.Fatalf("log_run_id = %d, want > %d", int(clearRunID), previousRunID) } - statusRec := httptest.NewRecorder() - statusReq := httptest.NewRequest( + logsRec := httptest.NewRecorder() + logsReq := httptest.NewRequest( http.MethodGet, - "/api/gateway/status?log_offset=0&log_run_id="+strconv.Itoa(previousRunID), + "/api/gateway/logs?log_offset=0&log_run_id="+strconv.Itoa(previousRunID), nil, ) - mux.ServeHTTP(statusRec, statusReq) + mux.ServeHTTP(logsRec, logsReq) - if statusRec.Code != http.StatusOK { - t.Fatalf("status code = %d, want %d", statusRec.Code, http.StatusOK) + if logsRec.Code != http.StatusOK { + t.Fatalf("logs code = %d, want %d", logsRec.Code, http.StatusOK) } - var statusBody map[string]any - if err := json.Unmarshal(statusRec.Body.Bytes(), &statusBody); err != nil { - t.Fatalf("unmarshal status response: %v", err) + var logsBody map[string]any + if err := json.Unmarshal(logsRec.Body.Bytes(), &logsBody); err != nil { + t.Fatalf("unmarshal logs response: %v", err) } - logs, ok := statusBody["logs"].([]any) + logs, ok := logsBody["logs"].([]any) if !ok { - t.Fatalf("logs missing or not array: %#v", statusBody["logs"]) + t.Fatalf("logs missing or not array: %#v", logsBody["logs"]) } if len(logs) != 0 { t.Fatalf("logs len = %d, want 0", len(logs)) } - if got := statusBody["log_total"]; got != float64(0) { + if got := logsBody["log_total"]; got != float64(0) { t.Fatalf("log_total = %#v, want 0", got) } + if got := logsBody["log_run_id"]; got != clearBody["log_run_id"] { + t.Fatalf("log_run_id = %#v, want %#v", got, clearBody["log_run_id"]) + } } func TestFindPicoclawBinary_EnvOverride(t *testing.T) { diff --git a/web/frontend/src/api/gateway.ts b/web/frontend/src/api/gateway.ts index 020e92e3a..9e02a02b5 100644 --- a/web/frontend/src/api/gateway.ts +++ b/web/frontend/src/api/gateway.ts @@ -1,14 +1,20 @@ // API client for gateway process management. interface GatewayStatusResponse { - gateway_status: "running" | "starting" | "stopped" | "error" + gateway_status: "running" | "starting" | "restarting" | "stopped" | "error" gateway_start_allowed?: boolean gateway_start_reason?: string + gateway_restart_required?: boolean pid?: number + boot_default_model?: string + config_default_model?: string + [key: string]: unknown +} + +interface GatewayLogsResponse { logs?: string[] log_total?: number log_run_id?: number - [key: string]: unknown } interface GatewayActionResponse { @@ -28,10 +34,14 @@ async function request(path: string, options?: RequestInit): Promise { return res.json() as Promise } -export async function getGatewayStatus(options?: { +export async function getGatewayStatus(): Promise { + return request("/api/gateway/status") +} + +export async function getGatewayLogs(options?: { log_offset?: number log_run_id?: number -}): Promise { +}): Promise { const params = new URLSearchParams() if (options?.log_offset !== undefined) { params.set("log_offset", options.log_offset.toString()) @@ -40,7 +50,7 @@ export async function getGatewayStatus(options?: { params.set("log_run_id", options.log_run_id.toString()) } const queryString = params.toString() ? `?${params.toString()}` : "" - return request(`/api/gateway/status${queryString}`) + return request(`/api/gateway/logs${queryString}`) } export async function startGateway(): Promise { @@ -67,4 +77,8 @@ export async function clearGatewayLogs(): Promise { }) } -export type { GatewayStatusResponse, GatewayActionResponse } +export type { + GatewayStatusResponse, + GatewayLogsResponse, + GatewayActionResponse, +} diff --git a/web/frontend/src/api/models.ts b/web/frontend/src/api/models.ts index 6a4544c65..8e49b48b4 100644 --- a/web/frontend/src/api/models.ts +++ b/web/frontend/src/api/models.ts @@ -84,7 +84,7 @@ export async function setDefaultModel( body: JSON.stringify({ model_name: modelName }), }) - void refreshGatewayState() + await refreshGatewayState() return response } diff --git a/web/frontend/src/components/app-header.tsx b/web/frontend/src/components/app-header.tsx index 7a50fe0fb..fe0c84e69 100644 --- a/web/frontend/src/components/app-header.tsx +++ b/web/frontend/src/components/app-header.tsx @@ -6,6 +6,7 @@ import { IconMoon, IconPlayerPlay, IconPower, + IconRefresh, IconSun, } from "@tabler/icons-react" import { Link } from "@tanstack/react-router" @@ -31,6 +32,11 @@ import { } from "@/components/ui/dropdown-menu.tsx" import { Separator } from "@/components/ui/separator.tsx" import { SidebarTrigger } from "@/components/ui/sidebar" +import { + Tooltip, + TooltipContent, + TooltipTrigger, +} from "@/components/ui/tooltip" import { useGateway } from "@/hooks/use-gateway.ts" import { useTheme } from "@/hooks/use-theme.ts" @@ -41,27 +47,35 @@ export function AppHeader() { state: gwState, loading: gwLoading, canStart, + restartRequired, start, + restart, stop, } = useGateway() const isRunning = gwState === "running" const isStarting = gwState === "starting" + const isRestarting = gwState === "restarting" const isStopped = gwState === "stopped" || gwState === "unknown" const showNotConnectedHint = - canStart && (gwState === "stopped" || gwState === "error") + !isRestarting && canStart && (gwState === "stopped" || gwState === "error") const [showStopDialog, setShowStopDialog] = React.useState(false) const handleGatewayToggle = () => { - if (gwLoading || (!isRunning && !canStart)) return + if (gwLoading || isRestarting || (!isRunning && !canStart)) return if (isRunning) { setShowStopDialog(true) } else { - start() + void start() } } + const handleGatewayRestart = () => { + if (gwLoading || isRestarting || !restartRequired || !canStart) return + void restart() + } + const confirmStop = () => { setShowStopDialog(false) stop() @@ -115,35 +129,67 @@ export function AppHeader() {
+ {restartRequired && ( + + + + + + {t("header.gateway.restartRequired")} + + + )} + {/* Gateway Start/Stop */} - + {isRunning ? ( + + + + + {t("header.gateway.action.stop")} + + ) : ( + + )} (null) const [isAtBottom, setIsAtBottom] = useState(true) + const [hasScrolled, setHasScrolled] = useState(false) const [input, setInput] = useState("") const { @@ -56,14 +58,26 @@ export function ChatPage() { onDeletedActiveSession: newChat, }) - const handleScroll = (e: React.UIEvent) => { - const { scrollTop, scrollHeight, clientHeight } = e.currentTarget + const syncScrollState = (element: HTMLDivElement) => { + const { scrollTop, scrollHeight, clientHeight } = element + setHasScrolled(scrollTop > 0) setIsAtBottom(scrollHeight - scrollTop <= clientHeight + 10) } + const handleScroll = (e: React.UIEvent) => { + syncScrollState(e.currentTarget) + } + useEffect(() => { - if (isAtBottom && scrollRef.current) { - scrollRef.current.scrollTop = scrollRef.current.scrollHeight + void hydrateActiveSession() + }, []) + + useEffect(() => { + if (scrollRef.current) { + if (isAtBottom) { + scrollRef.current.scrollTop = scrollRef.current.scrollHeight + } + syncScrollState(scrollRef.current) } }, [messages, isTyping, isAtBottom]) @@ -77,6 +91,9 @@ export function ChatPage() {
diff --git a/web/frontend/src/components/models/edit-model-sheet.tsx b/web/frontend/src/components/models/edit-model-sheet.tsx index 4c77944a9..237991a9f 100644 --- a/web/frontend/src/components/models/edit-model-sheet.tsx +++ b/web/frontend/src/components/models/edit-model-sheet.tsx @@ -110,7 +110,7 @@ export function EditModelSheet({ : undefined, thinking_level: form.thinkingLevel || undefined, }) - if (setAsDefault) { + if (setAsDefault && !model.is_default) { await setDefaultModel(model.model_name) } onSaved() diff --git a/web/frontend/src/components/models/models-page.tsx b/web/frontend/src/components/models/models-page.tsx index b8e80e709..6776e5ca8 100644 --- a/web/frontend/src/components/models/models-page.tsx +++ b/web/frontend/src/components/models/models-page.tsx @@ -79,6 +79,8 @@ export function ModelsPage() { }, [fetchModels]) const handleSetDefault = async (model: ModelInfo) => { + if (model.is_default) return + setSettingDefaultIndex(model.index) try { await setDefaultModel(model.model_name) diff --git a/web/frontend/src/components/page-header.tsx b/web/frontend/src/components/page-header.tsx index 9d4aa6975..656551f39 100644 --- a/web/frontend/src/components/page-header.tsx +++ b/web/frontend/src/components/page-header.tsx @@ -2,16 +2,28 @@ import { IconMenu2 } from "@tabler/icons-react" import type { ReactNode } from "react" import { SidebarTrigger } from "@/components/ui/sidebar" +import { cn } from "@/lib/utils" interface PageHeaderProps { title: string titleExtra?: ReactNode children?: ReactNode + className?: string } -export function PageHeader({ title, titleExtra, children }: PageHeaderProps) { +export function PageHeader({ + title, + titleExtra, + children, + className, +}: PageHeaderProps) { return ( -
+
diff --git a/web/frontend/src/hooks/use-chat-models.ts b/web/frontend/src/hooks/use-chat-models.ts index 8a82ceaf3..9afa882db 100644 --- a/web/frontend/src/hooks/use-chat-models.ts +++ b/web/frontend/src/hooks/use-chat-models.ts @@ -1,4 +1,4 @@ -import { useCallback, useEffect, useMemo, useState } from "react" +import { useCallback, useEffect, useMemo, useRef, useState } from "react" import { type ModelInfo, getModels, setDefaultModel } from "@/api/models" @@ -20,6 +20,7 @@ function isLocalModel(model: ModelInfo): boolean { export function useChatModels({ isConnected }: UseChatModelsOptions) { const [modelList, setModelList] = useState([]) const [defaultModelName, setDefaultModelName] = useState("") + const setDefaultRequestIdRef = useRef(0) const loadModels = useCallback(async () => { try { @@ -41,17 +42,28 @@ export function useChatModels({ isConnected }: UseChatModelsOptions) { return () => clearTimeout(timerId) }, [isConnected, loadModels]) - const handleSetDefault = useCallback(async (modelName: string) => { - try { - await setDefaultModel(modelName) - setDefaultModelName(modelName) - setModelList((prev) => - prev.map((m) => ({ ...m, is_default: m.model_name === modelName })), - ) - } catch (err) { - console.error("Failed to set default model:", err) - } - }, []) + const handleSetDefault = useCallback( + async (modelName: string) => { + if (modelName === defaultModelName) return + const requestId = ++setDefaultRequestIdRef.current + + try { + await setDefaultModel(modelName) + const data = await getModels() + if (requestId !== setDefaultRequestIdRef.current) { + return + } + + setModelList(data.models) + if (data.models.some((m) => m.model_name === data.default_model)) { + setDefaultModelName(data.default_model) + } + } catch (err) { + console.error("Failed to set default model:", err) + } + }, + [defaultModelName], + ) const hasConfiguredModels = useMemo( () => modelList.some((m) => m.configured), diff --git a/web/frontend/src/hooks/use-gateway-logs.ts b/web/frontend/src/hooks/use-gateway-logs.ts index a39e6e930..15cbca4ae 100644 --- a/web/frontend/src/hooks/use-gateway-logs.ts +++ b/web/frontend/src/hooks/use-gateway-logs.ts @@ -1,7 +1,7 @@ import { useAtomValue } from "jotai" import { useEffect, useRef, useState } from "react" -import { clearGatewayLogs, getGatewayStatus } from "@/api/gateway" +import { clearGatewayLogs, getGatewayLogs } from "@/api/gateway" import { gatewayAtom } from "@/store/gateway" export function useGatewayLogs() { @@ -37,7 +37,7 @@ export function useGatewayLogs() { const fetchLogs = async () => { if ( !mounted || - (gateway.status !== "running" && gateway.status !== "starting") + !["running", "starting", "restarting"].includes(gateway.status) ) { if (mounted) { timeout = setTimeout(fetchLogs, 1000) @@ -49,7 +49,7 @@ export function useGatewayLogs() { const requestToken = syncTokenRef.current const requestOffset = logOffsetRef.current const requestRunId = logRunIdRef.current - const data = await getGatewayStatus({ + const data = await getGatewayLogs({ log_offset: requestOffset, log_run_id: requestRunId, }) diff --git a/web/frontend/src/hooks/use-gateway.ts b/web/frontend/src/hooks/use-gateway.ts index 097dc3598..848f4d59c 100644 --- a/web/frontend/src/hooks/use-gateway.ts +++ b/web/frontend/src/hooks/use-gateway.ts @@ -1,31 +1,30 @@ -import { useAtom } from "jotai" +import { useAtomValue } from "jotai" import { useCallback, useEffect, useState } from "react" import { type GatewayStatusResponse, getGatewayStatus, + restartGateway, startGateway, stopGateway, } from "@/api/gateway" -import { gatewayAtom } from "@/store" +import { + applyGatewayStatusToStore, + gatewayAtom, + updateGatewayStore, +} from "@/store" // Global variable to ensure we only have one SSE connection let sseInitialized = false export function useGateway() { - const [{ status: state, canStart }, setGateway] = useAtom(gatewayAtom) + const gateway = useAtomValue(gatewayAtom) + const { status: state, canStart, restartRequired } = gateway const [loading, setLoading] = useState(false) - const applyGatewayStatus = useCallback( - (data: GatewayStatusResponse) => { - setGateway((prev) => ({ - ...prev, - status: data.gateway_status ?? "unknown", - canStart: data.gateway_start_allowed ?? true, - })) - }, - [setGateway], - ) + const applyGatewayStatus = useCallback((data: GatewayStatusResponse) => { + applyGatewayStatusToStore(data) + }, []) // Initialize global SSE connection once useEffect(() => { @@ -35,9 +34,10 @@ export function useGateway() { getGatewayStatus() .then((data) => applyGatewayStatus(data)) .catch(() => { - setGateway({ + updateGatewayStore({ status: "unknown", canStart: true, + restartRequired: false, }) }) @@ -59,14 +59,7 @@ export function useGateway() { data.gateway_status || typeof data.gateway_start_allowed === "boolean" ) { - setGateway((prev) => ({ - ...prev, - status: data.gateway_status ?? prev.status, - canStart: - typeof data.gateway_start_allowed === "boolean" - ? data.gateway_start_allowed - : prev.canStart, - })) + applyGatewayStatus(data) } } catch { // ignore @@ -75,7 +68,9 @@ export function useGateway() { es.onerror = () => { // EventSource will auto-reconnect - setGateway((prev) => ({ ...prev, status: "unknown" })) + updateGatewayStore((prev) => + prev.status === "restarting" ? {} : { status: "unknown" }, + ) } return () => { @@ -83,7 +78,7 @@ export function useGateway() { es.close() sseInitialized = false } - }, [applyGatewayStatus, setGateway]) + }, [applyGatewayStatus]) const start = useCallback(async () => { if (!canStart) return @@ -92,19 +87,19 @@ export function useGateway() { try { await startGateway() // SSE will push the real state changes, but set optimistic state - setGateway((prev) => ({ ...prev, status: "starting" })) + updateGatewayStore({ status: "starting" }) } catch (err) { console.error("Failed to start gateway:", err) try { const status = await getGatewayStatus() applyGatewayStatus(status) } catch { - setGateway((prev) => ({ ...prev, status: "unknown" })) + updateGatewayStore({ status: "unknown" }) } } finally { setLoading(false) } - }, [applyGatewayStatus, canStart, setGateway]) + }, [applyGatewayStatus, canStart]) const stop = useCallback(async () => { setLoading(true) @@ -117,5 +112,37 @@ export function useGateway() { } }, []) - return { state, loading, canStart, start, stop } + const restart = useCallback(async () => { + if (state !== "running") return + + const previousState = state + const previousCanStart = canStart + const previousRestartRequired = restartRequired + + setLoading(true) + updateGatewayStore({ + status: "restarting", + restartRequired: false, + }) + + try { + await restartGateway() + } catch (err) { + console.error("Failed to restart gateway:", err) + try { + const status = await getGatewayStatus() + applyGatewayStatus(status) + } catch { + updateGatewayStore({ + status: previousState, + canStart: previousCanStart, + restartRequired: previousRestartRequired, + }) + } + } finally { + setLoading(false) + } + }, [applyGatewayStatus, canStart, restartRequired, state]) + + return { state, loading, canStart, restartRequired, start, stop, restart } } diff --git a/web/frontend/src/hooks/use-pico-chat.ts b/web/frontend/src/hooks/use-pico-chat.ts index 7e3066177..1b97a2a9c 100644 --- a/web/frontend/src/hooks/use-pico-chat.ts +++ b/web/frontend/src/hooks/use-pico-chat.ts @@ -1,79 +1,12 @@ import dayjs from "dayjs" import { useAtomValue } from "jotai" + import { - type SetStateAction, - useCallback, - useEffect, - useRef, - useState, -} from "react" -import { useTranslation } from "react-i18next" -import { toast } from "sonner" - -import { getPicoToken } from "@/api/pico" -import { getSessionHistory } from "@/api/sessions" -import { gatewayAtom } from "@/store" - -// Pico Protocol message types -interface PicoMessage { - type: string - id?: string - session_id?: string - timestamp?: number | string - payload?: Record -} - -export interface ChatMessage { - id: string - role: "user" | "assistant" - content: string - timestamp: number | string -} - -type ConnectionState = "disconnected" | "connecting" | "connected" | "error" - -const LAST_SESSION_STORAGE_KEY = "picoclaw:last-session-id" - -function readStoredSessionId(): string { - const value = localStorage.getItem(LAST_SESSION_STORAGE_KEY)?.trim() - return value || "" -} - -function writeStoredSessionId(sessionId: string) { - if (sessionId) { - localStorage.setItem(LAST_SESSION_STORAGE_KEY, sessionId) - return - } - - localStorage.removeItem(LAST_SESSION_STORAGE_KEY) -} - -function generateSessionId(): string { - const webCrypto = globalThis.crypto - if (webCrypto && typeof webCrypto.randomUUID === "function") { - return webCrypto.randomUUID() - } - - if (webCrypto && typeof webCrypto.getRandomValues === "function") { - const bytes = new Uint8Array(16) - webCrypto.getRandomValues(bytes) - - // RFC4122 v4: set version and variant bits. - bytes[6] = (bytes[6] & 0x0f) | 0x40 - bytes[8] = (bytes[8] & 0x3f) | 0x80 - - const hex = Array.from(bytes, (b) => b.toString(16).padStart(2, "0")) - return ( - `${hex[0]}${hex[1]}${hex[2]}${hex[3]}-` + - `${hex[4]}${hex[5]}-` + - `${hex[6]}${hex[7]}-` + - `${hex[8]}${hex[9]}-` + - `${hex[10]}${hex[11]}${hex[12]}${hex[13]}${hex[14]}${hex[15]}` - ) - } - - return `session-${Date.now()}-${Math.random().toString(16).slice(2, 10)}` -} + newChatSession, + sendChatMessage, + switchChatSession, +} from "@/lib/pico-chat-controller" +import { chatAtom } from "@/store/chat" const UNIX_MS_THRESHOLD = 1e12 @@ -124,355 +57,16 @@ export function formatMessageTime(dateRaw: number | string | Date): string { } export function usePicoChat() { - const { t } = useTranslation() - const { status: gatewayState } = useAtomValue(gatewayAtom) - const [messages, setMessages] = useState([]) - const [connectionState, setConnectionState] = - useState("disconnected") - const [isTyping, setIsTyping] = useState(false) - const [activeSessionId, setActiveSessionId] = - useState(() => readStoredSessionId() || generateSessionId()) - - const wsRef = useRef(null) - const isConnectingRef = useRef(false) - const msgIdCounter = useRef(0) - const activeSessionIdRef = useRef(activeSessionId) - const messagesRevisionRef = useRef(0) - - const setTrackedMessages = useCallback( - (nextState: SetStateAction) => { - setMessages((prev) => { - const next = - typeof nextState === "function" - ? ( - nextState as (prevState: ChatMessage[]) => ChatMessage[] - )(prev) - : nextState - - if (next !== prev) { - messagesRevisionRef.current += 1 - } - - return next - }) - }, - [], - ) - - // Keep ref in sync - useEffect(() => { - activeSessionIdRef.current = activeSessionId - writeStoredSessionId(activeSessionId) - }, [activeSessionId]) - - const loadSessionMessages = useCallback(async (sessionId: string) => { - const detail = await getSessionHistory(sessionId) - const fallbackTime = detail.updated - - return detail.messages.map((m, i) => ({ - id: `hist-${i}-${Date.now()}`, - role: m.role as "user" | "assistant", - content: m.content, - timestamp: fallbackTime, - })) - }, []) - - useEffect(() => { - const storedSessionId = readStoredSessionId() - if (!storedSessionId) { - return - } - - const restoreRevision = messagesRevisionRef.current - let cancelled = false - void loadSessionMessages(storedSessionId) - .then((historyMessages) => { - if (cancelled) { - return - } - if (activeSessionIdRef.current !== storedSessionId) { - return - } - if (messagesRevisionRef.current !== restoreRevision) { - return - } - setTrackedMessages(historyMessages) - setIsTyping(false) - }) - .catch((err) => { - console.error("Failed to restore last session history:", err) - if (cancelled) { - return - } - if (activeSessionIdRef.current !== storedSessionId) { - return - } - if (messagesRevisionRef.current !== restoreRevision) { - return - } - localStorage.removeItem(LAST_SESSION_STORAGE_KEY) - setTrackedMessages([]) - setIsTyping(false) - }) - - return () => { - cancelled = true - } - }, [loadSessionMessages, setTrackedMessages]) - - const handlePicoMessage = useCallback((msg: PicoMessage) => { - const payload = msg.payload || {} - - switch (msg.type) { - case "message.create": { - const content = (payload.content as string) || "" - const messageId = (payload.message_id as string) || `pico-${Date.now()}` - // Use provided timestamp or current time - const timestampRaw = - msg.timestamp !== undefined && Number.isFinite(Number(msg.timestamp)) - ? normalizeUnixTimestamp(Number(msg.timestamp)) - : Date.now() - - setTrackedMessages((prev) => [ - ...prev, - { - id: messageId, - role: "assistant", - content, - timestamp: timestampRaw, - }, - ]) - setIsTyping(false) - break - } - - case "message.update": { - const content = (payload.content as string) || "" - const messageId = payload.message_id as string - if (!messageId) break - - setTrackedMessages((prev) => - prev.map((m) => (m.id === messageId ? { ...m, content } : m)), - ) - break - } - - case "typing.start": - setIsTyping(true) - break - - case "typing.stop": - setIsTyping(false) - break - - case "error": - console.error("Pico error:", payload) - setIsTyping(false) - break - - case "pong": - // heartbeat response, ignore - break - - default: - console.log("Unknown pico message type:", msg.type) - } - }, [setTrackedMessages]) - - const connect = useCallback(async () => { - if ( - isConnectingRef.current || - (wsRef.current && - (wsRef.current.readyState === WebSocket.OPEN || - wsRef.current.readyState === WebSocket.CONNECTING)) - ) { - return - } - - isConnectingRef.current = true - setConnectionState("connecting") - - try { - const { token, ws_url } = await getPicoToken() - - if (!token) { - console.error("No pico token available") - setConnectionState("error") - isConnectingRef.current = false - return - } - - // If the backend returns a localhost URL but we are accessing it via a LAN IP - // (e.g., from a mobile device during dev), rewrite the hostname to match. - let finalWsUrl = ws_url - try { - const parsedUrl = new URL(ws_url) - const isLocalHost = - parsedUrl.hostname === "localhost" || - parsedUrl.hostname === "127.0.0.1" || - parsedUrl.hostname === "0.0.0.0" - const isBrowserLocal = - window.location.hostname === "localhost" || - window.location.hostname === "127.0.0.1" - - if (isLocalHost && !isBrowserLocal) { - parsedUrl.hostname = window.location.hostname - finalWsUrl = parsedUrl.toString() - } - } catch (e) { - console.warn("Could not parse ws_url:", e) - } - - // Build WebSocket URL with session_id - const sessionId = activeSessionIdRef.current - const url = `${finalWsUrl}?token=${encodeURIComponent(token)}&session_id=${encodeURIComponent(sessionId)}` - const socket = new WebSocket(url) - - socket.onopen = () => { - setConnectionState("connected") - isConnectingRef.current = false - } - - socket.onmessage = (event) => { - try { - const msg: PicoMessage = JSON.parse(event.data) - handlePicoMessage(msg) - } catch { - console.warn("Non-JSON message from pico:", event.data) - } - } - - socket.onclose = () => { - setConnectionState("disconnected") - wsRef.current = null - isConnectingRef.current = false - } - - socket.onerror = () => { - setConnectionState("error") - isConnectingRef.current = false - } - - wsRef.current = socket - } catch (err) { - console.error("Failed to connect to pico:", err) - setConnectionState("error") - isConnectingRef.current = false - } - }, [handlePicoMessage]) - - const disconnect = useCallback(() => { - if (wsRef.current) { - wsRef.current.close() - wsRef.current = null - } - setConnectionState("disconnected") - isConnectingRef.current = false - }, []) - - // Auto connect/disconnect based on gateway state - useEffect(() => { - // Wrap in setTimeout to avoid React calling setState synchronously during render - const timerId = setTimeout(() => { - if (gatewayState === "running") { - connect() - } else { - disconnect() - } - }, 0) - - return () => clearTimeout(timerId) - }, [gatewayState, connect, disconnect]) - - // Cleanup on unmount - useEffect(() => { - return () => disconnect() - }, [disconnect]) - - const sendMessage = useCallback((content: string) => { - if (!wsRef.current || wsRef.current.readyState !== WebSocket.OPEN) { - console.warn("WebSocket not connected") - return - } - - const id = `msg-${++msgIdCounter.current}-${Date.now()}` - const timestampRaw = Date.now() - - // Add user message to local state - setTrackedMessages((prev) => [ - ...prev, - { id, role: "user", content, timestamp: timestampRaw }, - ]) - - // Show typing indicator immediately - setIsTyping(true) - - // Send via Pico Protocol - const picoMsg: PicoMessage = { - type: "message.send", - id, - payload: { content }, - } - wsRef.current.send(JSON.stringify(picoMsg)) - }, [setTrackedMessages]) - - // Switch to a historical session - const switchSession = useCallback( - async (sessionId: string) => { - if (sessionId === activeSessionIdRef.current) { - return - } - - try { - const historyMessages = await loadSessionMessages(sessionId) - - // Only switch the active websocket session after history has loaded successfully. - disconnect() - setActiveSessionId(sessionId) - setIsTyping(false) - setTrackedMessages(historyMessages) - } catch (err) { - console.error("Failed to load session history:", err) - toast.error(t("chat.historyOpenFailed")) - return - } - - setTimeout(() => { - if (gatewayState === "running") { - connect() - } - }, 100) - }, - [connect, disconnect, gatewayState, loadSessionMessages, setTrackedMessages, t], - ) - - // Start a new empty chat - const newChat = useCallback(() => { - if (messages.length === 0) { - return - } - - disconnect() - const newId = generateSessionId() - setActiveSessionId(newId) - setTrackedMessages([]) - setIsTyping(false) - - // Reconnect with the fresh session - setTimeout(() => { - if (gatewayState === "running") { - connect() - } - }, 100) - }, [disconnect, connect, gatewayState, messages.length, setTrackedMessages]) + const { messages, connectionState, isTyping, activeSessionId } = + useAtomValue(chatAtom) return { messages, connectionState, isTyping, activeSessionId, - sendMessage, - switchSession, - newChat, + sendMessage: sendChatMessage, + switchSession: switchChatSession, + newChat: newChatSession, } } diff --git a/web/frontend/src/i18n/locales/en.json b/web/frontend/src/i18n/locales/en.json index 453c5905f..b099dec13 100644 --- a/web/frontend/src/i18n/locales/en.json +++ b/web/frontend/src/i18n/locales/en.json @@ -58,11 +58,14 @@ }, "action": { "start": "Start Gateway", - "stop": "Stop Gateway" + "stop": "Stop Gateway", + "restart": "Restart Gateway" }, "status": { - "starting": "Starting Gateway..." - } + "starting": "Starting Gateway...", + "restarting": "Restarting Gateway..." + }, + "restartRequired": "Model changes require a gateway restart to take effect." } }, "common": { diff --git a/web/frontend/src/i18n/locales/zh.json b/web/frontend/src/i18n/locales/zh.json index b6bdedbfa..78093e5c7 100644 --- a/web/frontend/src/i18n/locales/zh.json +++ b/web/frontend/src/i18n/locales/zh.json @@ -58,11 +58,14 @@ }, "action": { "start": "启动服务", - "stop": "停止服务" + "stop": "停止服务", + "restart": "重启服务" }, "status": { - "starting": "服务启动中..." - } + "starting": "服务启动中...", + "restarting": "服务重启中..." + }, + "restartRequired": "切换默认模型后需要重启服务才能生效。" } }, "common": { diff --git a/web/frontend/src/lib/pico-chat-controller.ts b/web/frontend/src/lib/pico-chat-controller.ts new file mode 100644 index 000000000..be3397bae --- /dev/null +++ b/web/frontend/src/lib/pico-chat-controller.ts @@ -0,0 +1,405 @@ +import { getDefaultStore } from "jotai" +import { toast } from "sonner" + +import { getPicoToken } from "@/api/pico" +import { getSessionHistory } from "@/api/sessions" +import i18n from "@/i18n" +import { + clearStoredSessionId, + generateSessionId, + normalizeUnixTimestamp, + readStoredSessionId, +} from "@/lib/pico-chat-state" +import { type ChatMessage, getChatState, updateChatStore } from "@/store/chat" +import { gatewayAtom } from "@/store/gateway" + +interface PicoMessage { + type: string + id?: string + session_id?: string + timestamp?: number | string + payload?: Record +} + +const store = getDefaultStore() + +let wsRef: WebSocket | null = null +let isConnecting = false +let msgIdCounter = 0 +let activeSessionIdRef = getChatState().activeSessionId +let initialized = false +let unsubscribeGateway: (() => void) | null = null +let hydratePromise: Promise | null = null +let connectionGeneration = 0 + +async function loadSessionMessages(sessionId: string): Promise { + const detail = await getSessionHistory(sessionId) + const fallbackTime = detail.updated + + return detail.messages.map((message, index) => ({ + id: `hist-${index}-${Date.now()}`, + role: message.role, + content: message.content, + timestamp: fallbackTime, + })) +} + +function handlePicoMessage(message: PicoMessage) { + const payload = message.payload || {} + + switch (message.type) { + case "message.create": { + const content = (payload.content as string) || "" + const messageId = (payload.message_id as string) || `pico-${Date.now()}` + const timestamp = + message.timestamp !== undefined && + Number.isFinite(Number(message.timestamp)) + ? normalizeUnixTimestamp(Number(message.timestamp)) + : Date.now() + + updateChatStore((prev) => ({ + messages: [ + ...prev.messages, + { + id: messageId, + role: "assistant", + content, + timestamp, + }, + ], + isTyping: false, + })) + break + } + + case "message.update": { + const content = (payload.content as string) || "" + const messageId = payload.message_id as string + if (!messageId) { + break + } + + updateChatStore((prev) => ({ + messages: prev.messages.map((msg) => + msg.id === messageId ? { ...msg, content } : msg, + ), + })) + break + } + + case "typing.start": + updateChatStore({ isTyping: true }) + break + + case "typing.stop": + updateChatStore({ isTyping: false }) + break + + case "error": + console.error("Pico error:", payload) + updateChatStore({ isTyping: false }) + break + + case "pong": + break + + default: + console.log("Unknown pico message type:", message.type) + } +} + +function setActiveSessionId(sessionId: string) { + activeSessionIdRef = sessionId + updateChatStore({ activeSessionId: sessionId }) +} + +export async function connectChat() { + if (store.get(gatewayAtom).status !== "running") { + return + } + + if ( + isConnecting || + (wsRef && + (wsRef.readyState === WebSocket.OPEN || + wsRef.readyState === WebSocket.CONNECTING)) + ) { + return + } + + const generation = connectionGeneration + 1 + connectionGeneration = generation + isConnecting = true + updateChatStore({ connectionState: "connecting" }) + + try { + const { token, ws_url } = await getPicoToken() + + if (generation !== connectionGeneration) { + return + } + + if (!token) { + console.error("No pico token available") + updateChatStore({ connectionState: "error" }) + isConnecting = false + return + } + + let finalWsUrl = ws_url + try { + const parsedUrl = new URL(ws_url) + const isLocalHost = + parsedUrl.hostname === "localhost" || + parsedUrl.hostname === "127.0.0.1" || + parsedUrl.hostname === "0.0.0.0" + const isBrowserLocal = + window.location.hostname === "localhost" || + window.location.hostname === "127.0.0.1" + + if (isLocalHost && !isBrowserLocal) { + parsedUrl.hostname = window.location.hostname + finalWsUrl = parsedUrl.toString() + } + } catch (error) { + console.warn("Could not parse ws_url:", error) + } + + const url = `${finalWsUrl}?token=${encodeURIComponent(token)}&session_id=${encodeURIComponent(activeSessionIdRef)}` + const socket = new WebSocket(url) + + if (generation !== connectionGeneration) { + socket.close() + return + } + + socket.onopen = () => { + if (wsRef !== socket) { + return + } + updateChatStore({ connectionState: "connected" }) + isConnecting = false + } + + socket.onmessage = (event) => { + try { + const message: PicoMessage = JSON.parse(event.data) + handlePicoMessage(message) + } catch { + console.warn("Non-JSON message from pico:", event.data) + } + } + + socket.onclose = () => { + if (wsRef !== socket) { + return + } + wsRef = null + isConnecting = false + updateChatStore({ + connectionState: "disconnected", + isTyping: false, + }) + } + + socket.onerror = () => { + if (wsRef !== socket) { + return + } + isConnecting = false + updateChatStore({ connectionState: "error" }) + } + + wsRef = socket + } catch (error) { + if (generation !== connectionGeneration) { + return + } + console.error("Failed to connect to pico:", error) + updateChatStore({ connectionState: "error" }) + isConnecting = false + } +} + +export function disconnectChat() { + connectionGeneration += 1 + + const socket = wsRef + wsRef = null + isConnecting = false + + if (socket) { + socket.close() + } + + updateChatStore({ + connectionState: "disconnected", + isTyping: false, + }) +} + +export async function hydrateActiveSession() { + if (hydratePromise) { + return hydratePromise + } + + const state = getChatState() + const storedSessionId = readStoredSessionId() + + if ( + !storedSessionId || + state.hasHydratedActiveSession || + state.messages.length > 0 || + storedSessionId !== state.activeSessionId + ) { + if (!state.hasHydratedActiveSession) { + updateChatStore({ hasHydratedActiveSession: true }) + } + return + } + + hydratePromise = loadSessionMessages(storedSessionId) + .then((historyMessages) => { + const currentState = getChatState() + if (currentState.activeSessionId !== storedSessionId) { + return + } + + if (currentState.messages.length > 0) { + updateChatStore({ hasHydratedActiveSession: true }) + return + } + + updateChatStore({ + messages: historyMessages, + isTyping: false, + hasHydratedActiveSession: true, + }) + }) + .catch((error) => { + console.error("Failed to restore last session history:", error) + + const currentState = getChatState() + if (currentState.activeSessionId !== storedSessionId) { + return + } + + if (currentState.messages.length > 0) { + updateChatStore({ hasHydratedActiveSession: true }) + return + } + + clearStoredSessionId() + updateChatStore({ + messages: [], + isTyping: false, + hasHydratedActiveSession: true, + }) + }) + .finally(() => { + hydratePromise = null + }) + + return hydratePromise +} + +export function sendChatMessage(content: string) { + if (!wsRef || wsRef.readyState !== WebSocket.OPEN) { + console.warn("WebSocket not connected") + return + } + + const id = `msg-${++msgIdCounter}-${Date.now()}` + + updateChatStore((prev) => ({ + messages: [ + ...prev.messages, + { id, role: "user", content, timestamp: Date.now() }, + ], + isTyping: true, + })) + + wsRef.send( + JSON.stringify({ + type: "message.send", + id, + payload: { content }, + }), + ) +} + +export async function switchChatSession(sessionId: string) { + if (sessionId === activeSessionIdRef) { + return + } + + try { + const historyMessages = await loadSessionMessages(sessionId) + + disconnectChat() + setActiveSessionId(sessionId) + updateChatStore({ + messages: historyMessages, + isTyping: false, + hasHydratedActiveSession: true, + }) + + if (store.get(gatewayAtom).status === "running") { + await connectChat() + } + } catch (error) { + console.error("Failed to load session history:", error) + toast.error(i18n.t("chat.historyOpenFailed")) + } +} + +export async function newChatSession() { + if (getChatState().messages.length === 0) { + return + } + + disconnectChat() + setActiveSessionId(generateSessionId()) + updateChatStore({ + messages: [], + isTyping: false, + hasHydratedActiveSession: true, + }) + + if (store.get(gatewayAtom).status === "running") { + await connectChat() + } +} + +export function initializeChatStore() { + if (initialized) { + return + } + + initialized = true + activeSessionIdRef = getChatState().activeSessionId + + const syncConnectionWithGateway = () => { + if (store.get(gatewayAtom).status === "running") { + void connectChat() + return + } + + disconnectChat() + } + + unsubscribeGateway = store.sub(gatewayAtom, syncConnectionWithGateway) + + if (!readStoredSessionId()) { + updateChatStore({ hasHydratedActiveSession: true }) + } + + syncConnectionWithGateway() +} + +export function teardownChatStore() { + unsubscribeGateway?.() + unsubscribeGateway = null + initialized = false + disconnectChat() +} diff --git a/web/frontend/src/lib/pico-chat-state.ts b/web/frontend/src/lib/pico-chat-state.ts new file mode 100644 index 000000000..5b7d6c6cd --- /dev/null +++ b/web/frontend/src/lib/pico-chat-state.ts @@ -0,0 +1,59 @@ +const LAST_SESSION_STORAGE_KEY = "picoclaw:last-session-id" +const UNIX_MS_THRESHOLD = 1e12 + +function readStorageValue() { + return ( + globalThis.localStorage?.getItem(LAST_SESSION_STORAGE_KEY)?.trim() || "" + ) +} + +export function readStoredSessionId(): string { + return readStorageValue() +} + +export function writeStoredSessionId(sessionId: string) { + if (sessionId) { + globalThis.localStorage?.setItem(LAST_SESSION_STORAGE_KEY, sessionId) + return + } + + globalThis.localStorage?.removeItem(LAST_SESSION_STORAGE_KEY) +} + +export function clearStoredSessionId() { + globalThis.localStorage?.removeItem(LAST_SESSION_STORAGE_KEY) +} + +export function generateSessionId(): string { + const webCrypto = globalThis.crypto + if (webCrypto && typeof webCrypto.randomUUID === "function") { + return webCrypto.randomUUID() + } + + if (webCrypto && typeof webCrypto.getRandomValues === "function") { + const bytes = new Uint8Array(16) + webCrypto.getRandomValues(bytes) + + bytes[6] = (bytes[6] & 0x0f) | 0x40 + bytes[8] = (bytes[8] & 0x3f) | 0x80 + + const hex = Array.from(bytes, (b) => b.toString(16).padStart(2, "0")) + return ( + `${hex[0]}${hex[1]}${hex[2]}${hex[3]}-` + + `${hex[4]}${hex[5]}-` + + `${hex[6]}${hex[7]}-` + + `${hex[8]}${hex[9]}-` + + `${hex[10]}${hex[11]}${hex[12]}${hex[13]}${hex[14]}${hex[15]}` + ) + } + + return `session-${Date.now()}-${Math.random().toString(16).slice(2, 10)}` +} + +export function getInitialActiveSessionId(): string { + return readStorageValue() || generateSessionId() +} + +export function normalizeUnixTimestamp(timestamp: number): number { + return timestamp < UNIX_MS_THRESHOLD ? timestamp * 1000 : timestamp +} diff --git a/web/frontend/src/routes/__root.tsx b/web/frontend/src/routes/__root.tsx index 48f228d84..6431d9490 100644 --- a/web/frontend/src/routes/__root.tsx +++ b/web/frontend/src/routes/__root.tsx @@ -1,9 +1,15 @@ import { Outlet, createRootRoute } from "@tanstack/react-router" import { TanStackRouterDevtools } from "@tanstack/react-router-devtools" +import { useEffect } from "react" import { AppLayout } from "@/components/app-layout" +import { initializeChatStore } from "@/lib/pico-chat-controller" const RootLayout = () => { + useEffect(() => { + initializeChatStore() + }, []) + return ( diff --git a/web/frontend/src/store/chat.ts b/web/frontend/src/store/chat.ts new file mode 100644 index 000000000..d79a1a93b --- /dev/null +++ b/web/frontend/src/store/chat.ts @@ -0,0 +1,62 @@ +import { atom, getDefaultStore } from "jotai" + +import { + getInitialActiveSessionId, + writeStoredSessionId, +} from "@/lib/pico-chat-state" + +export interface ChatMessage { + id: string + role: "user" | "assistant" + content: string + timestamp: number | string +} + +export type ConnectionState = + | "disconnected" + | "connecting" + | "connected" + | "error" + +export interface ChatStoreState { + messages: ChatMessage[] + connectionState: ConnectionState + isTyping: boolean + activeSessionId: string + hasHydratedActiveSession: boolean +} + +type ChatStorePatch = Partial + +const DEFAULT_CHAT_STATE: ChatStoreState = { + messages: [], + connectionState: "disconnected", + isTyping: false, + activeSessionId: getInitialActiveSessionId(), + hasHydratedActiveSession: false, +} + +export const chatAtom = atom(DEFAULT_CHAT_STATE) + +const store = getDefaultStore() + +export function getChatState() { + return store.get(chatAtom) +} + +export function updateChatStore( + patch: + | ChatStorePatch + | ((prev: ChatStoreState) => ChatStorePatch | ChatStoreState), +) { + store.set(chatAtom, (prev) => { + const nextPatch = typeof patch === "function" ? patch(prev) : patch + const next = { ...prev, ...nextPatch } + + if (next.activeSessionId !== prev.activeSessionId) { + writeStoredSessionId(next.activeSessionId) + } + + return next + }) +} diff --git a/web/frontend/src/store/gateway.ts b/web/frontend/src/store/gateway.ts index 89da9d7fd..b7655839c 100644 --- a/web/frontend/src/store/gateway.ts +++ b/web/frontend/src/store/gateway.ts @@ -5,6 +5,7 @@ import { type GatewayStatusResponse, getGatewayStatus } from "@/api/gateway" export type GatewayState = | "running" | "starting" + | "restarting" | "stopped" | "error" | "unknown" @@ -12,19 +13,54 @@ export type GatewayState = export interface GatewayStoreState { status: GatewayState canStart: boolean + restartRequired: boolean +} + +type GatewayStorePatch = Partial + +const DEFAULT_GATEWAY_STATE: GatewayStoreState = { + status: "unknown", + canStart: true, + restartRequired: false, } // Global atom for gateway state -export const gatewayAtom = atom({ - status: "unknown", - canStart: true, -}) +export const gatewayAtom = atom(DEFAULT_GATEWAY_STATE) -function applyGatewayStatusToStore(data: GatewayStatusResponse) { - getDefaultStore().set(gatewayAtom, (prev) => ({ - ...prev, - status: data.gateway_status ?? "unknown", - canStart: data.gateway_start_allowed ?? true, +function normalizeGatewayStoreState( + prev: GatewayStoreState, + patch: GatewayStorePatch, +) { + return { ...prev, ...patch } +} + +export function updateGatewayStore( + patch: + | GatewayStorePatch + | ((prev: GatewayStoreState) => GatewayStorePatch | GatewayStoreState), +) { + getDefaultStore().set(gatewayAtom, (prev) => { + const nextPatch = typeof patch === "function" ? patch(prev) : patch + return normalizeGatewayStoreState(prev, nextPatch) + }) +} + +export function applyGatewayStatusToStore( + data: Partial< + Pick< + GatewayStatusResponse, + "gateway_status" | "gateway_start_allowed" | "gateway_restart_required" + > + >, +) { + updateGatewayStore((prev) => ({ + status: data.gateway_status ?? prev.status, + canStart: data.gateway_start_allowed ?? prev.canStart, + restartRequired: + data.gateway_restart_required ?? + (data.gateway_status && data.gateway_status !== "running" + ? false + : prev.restartRequired), })) } @@ -33,6 +69,6 @@ export async function refreshGatewayState() { const status = await getGatewayStatus() applyGatewayStatusToStore(status) } catch { - // Best-effort refresh only; keep current state on error. + updateGatewayStore(DEFAULT_GATEWAY_STATE) } } diff --git a/web/frontend/src/store/index.ts b/web/frontend/src/store/index.ts index 9dfcdf3c7..d377cdace 100644 --- a/web/frontend/src/store/index.ts +++ b/web/frontend/src/store/index.ts @@ -1 +1,2 @@ export * from "./gateway" +export * from "./chat" diff --git a/workspace/skills/weather/SKILL.md b/workspace/skills/weather/SKILL.md index 8073de192..aa90a9b20 100644 --- a/workspace/skills/weather/SKILL.md +++ b/workspace/skills/weather/SKILL.md @@ -1,49 +1,59 @@ --- name: weather -description: Get current weather and forecasts (no API key required). +description: Get current weather and forecasts with verified location matching (no API key required). homepage: https://wttr.in/:help metadata: {"nanobot":{"emoji":"🌤️","requires":{"bins":["curl"]}}} --- # Weather -Two free services, no API keys needed. +Use the most reliable location match first. For Chinese city names or other non-Latin input, prefer `wttr.in` with the original query because it resolves native names directly. Use Open-Meteo for structured current conditions and forecasts only after you have confirmed the exact city. -## wttr.in (primary) +## Accuracy Rules -Quick one-liner: +- Always restate the matched location, region/country, and observation time in the final answer. +- Do not trust the first geocoding hit blindly. Check `country`, `admin1`, `admin2`, and `population`. +- For Chinese city queries, do not send Hanzi directly to Open-Meteo geocoding unless the top result is obviously correct. Prefer `wttr.in` with the original Chinese name, or geocode the English/pinyin city name instead. +- If multiple plausible matches remain, ask a follow-up question or state the assumption clearly. +- Use `timezone=auto` when calling Open-Meteo so the reported time matches the location. + +## wttr.in (best for direct city-name queries) + +Quick current conditions: ```bash -curl -s "wttr.in/London?format=3" -# Output: London: ⛅️ +8°C +curl -s "https://wttr.in/London?format=%l:+%c+%t+%h+%w" ``` -Compact format: +Chinese city example: ```bash -curl -s "wttr.in/London?format=%l:+%c+%t+%h+%w" -# Output: London: ⛅️ +8°C 71% ↙5km/h +curl -s "https://wttr.in/%E6%88%90%E9%83%BD?format=%l:+%c+%t+%h+%w" +curl -s "https://wttr.in/%E4%B8%8A%E6%B5%B7?format=%l:+%c+%t+%h+%w" ``` -Full forecast: +JSON output if you need more detail: ```bash -curl -s "wttr.in/London?T" +curl -s "https://wttr.in/Chengdu?format=j1" ``` -Format codes: `%c` condition · `%t` temp · `%h` humidity · `%w` wind · `%l` location · `%m` moon - Tips: -- URL-encode spaces: `wttr.in/New+York` -- Airport codes: `wttr.in/JFK` -- Units: `?m` (metric) `?u` (USCS) -- Today only: `?1` · Current only: `?0` -- PNG: `curl -s "wttr.in/Berlin.png" -o /tmp/weather.png` +- URL-encode spaces: `New York` -> `New+York` +- URL-encode non-ASCII text before sending the request +- Use `?m` for metric units and `?u` for US units -## Open-Meteo (fallback, JSON) +## Open-Meteo (best for structured forecasts) -Free, no key, good for programmatic use: +1. Geocode the city and verify the returned location metadata: ```bash -curl -s "https://api.open-meteo.com/v1/forecast?latitude=51.5&longitude=-0.12¤t_weather=true" +curl -s "https://geocoding-api.open-meteo.com/v1/search?name=Chengdu&count=3&language=en&format=json" ``` -Find coordinates for a city, then query. Returns JSON with temp, windspeed, weathercode. +2. Query current weather and today's forecast with the verified coordinates: +```bash +curl -s "https://api.open-meteo.com/v1/forecast?latitude=30.66667&longitude=104.06667¤t=temperature_2m,relative_humidity_2m,weather_code,wind_speed_10m&daily=weather_code,temperature_2m_max,temperature_2m_min&forecast_days=1&timezone=auto" +``` + +Important: +- For Chinese inputs like `成都`, geocoding `name=%E6%88%90%E9%83%BD` may return smaller homonym locations first. Prefer `Chengdu` after verifying it matches Sichuan, China. +- If geocoding looks suspicious, fall back to `wttr.in` for the original city name instead of presenting a likely wrong result. Docs: https://open-meteo.com/en/docs