picoclaw/pkg/tools/filesystem.go
nuestraai ac6e77d7c9
CLI overrides, workspace config, MagicForm channel, and security hardening (#1)
* feat(cli): add workspace, config-dir, tools, and skills override flags

Add --workspace, --config-dir, --tools, and --skills flags to
`picoclaw agent` for single-shot invocations. Supports workspace
override with bootstrap file injection (AGENTS.md, IDENTITY.md,
SOUL.md, USER.md), tool allowlisting, and skills filtering.

Also extracts FormatSkillsSummary from SkillsLoader for reuse,
adds SetSkillsFilter to ContextBuilder, and wires SkillsFilter
from agent config into the context builder.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(channels): add MagicForm webhook channel plugin

Implement MagicForm channel for webhook-based agentic task delegation.
Supports Bearer token auth, async processing with HTTP callback,
workspace path validation against configurable workspace_root,
bootstrap file injection, tool/skill filtering via metadata, and
per-conversation session key isolation.

Includes Bus() accessor on BaseChannel for direct message publishing,
MagicFormConfig with workspace_root for path traversal prevention,
request body size limits, and TTL cleanup for stale request contexts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(agent): support per-request workspace, tool, and skill overrides

Add workspace override, tool allowlisting, and skills filtering to the
agent loop via processOptions metadata. When a workspace override is
active (e.g. from MagicForm channel), creates isolated SessionManager
and ContextBuilder instances per request.

Threads effSessions/effContextBuilder through all downstream paths
including runLLMIteration, forceCompression, maybeSummarize, and
summarizeSession to ensure full workspace isolation. Adds defense-in-
depth tool execution guard alongside LLM-facing tool definition filter.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(auth): add response body size limits to OAuth HTTP calls

Wrap all io.ReadAll(resp.Body) calls in the OAuth flow with
io.LimitReader capped at 1 MB to prevent memory exhaustion from
malicious or unexpectedly large server responses.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(tools): add response body size limits to search providers

Add 2 MB LimitReader to BraveSearch, Tavily, DuckDuckGo, and
Perplexity search providers to prevent memory exhaustion from
unexpectedly large responses. Matches existing pattern used by
GLMSearchProvider.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(tools): add 20 MB file write size limit

Reject writes exceeding 20 MB in WriteFileTool to prevent disk
exhaustion from unexpectedly large LLM-generated file content.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(tools): use crypto/rand for temp file naming

Replace predictable PID+nanosecond temp file names with
cryptographically random hex strings to prevent symlink
pre-placement attacks.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(skills): validate GitHub repo format and limit download size

Add regex validation for 'owner/repo' format to prevent URL injection
in InstallFromGitHub. Add 5 MB LimitReader on skill file downloads
to prevent memory exhaustion.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(auth): allow OAuth credentials to be overridden via env vars

Add PICOCLAW_OPENAI_CLIENT_ID, PICOCLAW_GOOGLE_CLIENT_ID, and
PICOCLAW_GOOGLE_CLIENT_SECRET env var overrides. Hardcoded values
remain as fallback defaults for backward compatibility.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(auth): sanitize OAuth error messages

Replace raw HTTP response bodies in error messages with generic
status-code-only messages. Full response details are logged at
debug level for troubleshooting.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(launcher): HTML-escape error messages in auth callback

Use html.EscapeString on all error text rendered in HTML responses
to prevent XSS via crafted OAuth error parameters or error messages.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(launcher): remove internal details from HTTP error responses

Log detailed errors server-side and return generic messages to
clients to prevent information disclosure of file paths and
internal state.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(launcher): add HTTP security headers middleware

Add SecurityHeaders middleware setting X-Content-Type-Options,
X-Frame-Options, and Content-Security-Policy on all responses
to mitigate MIME sniffing, clickjacking, and content injection.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(skills): validate ClawHub registry base URL scheme

Reject non-HTTPS registry URLs (unless localhost) to prevent
config-based redirection to malicious skill servers. Falls back
to the default https://clawhub.ai with a warning.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(skills): prevent workspace skills from shadowing builtins

Builtin skill names are now reserved — workspace and global skills
with the same name as a builtin are skipped with a warning log.
LoadSkill also checks builtins first. This prevents supply chain
attacks where a malicious skill replaces a trusted builtin.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(skills): warn when safety metadata is unavailable

Add MetadataAvailable flag to InstallResult. When the registry
metadata fetch fails (silent fallback), the user now sees a
warning that safety checks could not be completed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* security(shell): add opt-in environment variable filtering

When filter_env is enabled in config, shell commands run with a minimal
allowlist of environment variables (PATH, HOME, LANG, TERM, etc. plus
PICOCLAW_* prefix), preventing accidental leakage of sensitive env vars
like API keys to spawned processes. Disabled by default for backward compat.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(config): add workspace-local config.json overlay for per-tenant isolation

Support per-workspace config overrides via --config-dir (CLI) or configDir
(webhook), enabling different API keys, models, and agent settings per tenant.

- Add LoadWorkspaceConfig/MergeWorkspaceConfig with raw JSON overlay to
  preserve unmentioned bool fields (prevents clobbering tool enabled flags)
- Add Config.Clone() for safe per-request config copies in gateway mode
- Gateway: per-request provider creation from workspace config, with
  effProvider/effModel threaded through all LLM call sites and summarization
- CLI: workspace config merged before provider creation, CLI flags win
- MagicForm: replace inline bootstrap fields with configDir path;
  agent loop copies bootstrap files and loads config.json from configDir
- Fix --session flag: format as agent:main:cli:{key} so router honors it
- Fix cron session key: format as agent:main:cron:{id} for isolation
- Add shared CopyBootstrapFiles helper in pkg/agent/bootstrap.go
- Add config/workspace.config.example.json

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(agent): honor workspace config overrides in model routing

selectCandidates uses agent.Model/Candidates baked in at startup, which
ignores per-request workspace config overrides (effProvider/effModel).
When the workspace overrides the provider, skip routing and fallback
candidates entirely to avoid cross-provider credential issues. When only
the model is overridden, allow routing but use the effective model.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: fix linter errors (gci, gofumpt, golines, govet, unused)

- gci: alphabetize magicform import in gateway/helpers.go
- gofumpt: reformat multi-line function signature in agent/helpers.go
- golines: wrap lines exceeding 120 chars across 5 files
- govet: rename shadowed variables (err → wcErr, s → trimmed)
- unused: remove maybeSummarize, forceCompression, summarizeSession
  wrappers superseded by their *With variants

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: nuestraai <nuestraai@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: admin-mf <admin@magicform.ai>
2026-03-06 10:16:36 -06:00

474 lines
12 KiB
Go

package tools
import (
"context"
"crypto/rand"
"encoding/hex"
"fmt"
"io/fs"
"os"
"path/filepath"
"regexp"
"strings"
"github.com/sipeed/picoclaw/pkg/fileutil"
)
const maxWriteSize = 20 * 1024 * 1024 // 20 MB — limit for file writes via the write tool
// validatePath ensures the given path is within the workspace if restrict is true.
func validatePath(path, workspace string, restrict bool) (string, error) {
if workspace == "" {
return path, fmt.Errorf("workspace is not defined")
}
absWorkspace, err := filepath.Abs(workspace)
if err != nil {
return "", fmt.Errorf("failed to resolve workspace path: %w", err)
}
var absPath string
if filepath.IsAbs(path) {
absPath = filepath.Clean(path)
} else {
absPath, err = filepath.Abs(filepath.Join(absWorkspace, path))
if err != nil {
return "", fmt.Errorf("failed to resolve file path: %w", err)
}
}
if restrict {
if !isWithinWorkspace(absPath, absWorkspace) {
return "", fmt.Errorf("access denied: path is outside the workspace")
}
var resolved string
workspaceReal := absWorkspace
if resolved, err = filepath.EvalSymlinks(absWorkspace); err == nil {
workspaceReal = resolved
}
if resolved, err = filepath.EvalSymlinks(absPath); err == nil {
if !isWithinWorkspace(resolved, workspaceReal) {
return "", fmt.Errorf("access denied: symlink resolves outside workspace")
}
} else if os.IsNotExist(err) {
var parentResolved string
if parentResolved, err = resolveExistingAncestor(filepath.Dir(absPath)); err == nil {
if !isWithinWorkspace(parentResolved, workspaceReal) {
return "", fmt.Errorf("access denied: symlink resolves outside workspace")
}
} else if !os.IsNotExist(err) {
return "", fmt.Errorf("failed to resolve path: %w", err)
}
} else {
return "", fmt.Errorf("failed to resolve path: %w", err)
}
}
return absPath, nil
}
func resolveExistingAncestor(path string) (string, error) {
for current := filepath.Clean(path); ; current = filepath.Dir(current) {
if resolved, err := filepath.EvalSymlinks(current); err == nil {
return resolved, nil
} else if !os.IsNotExist(err) {
return "", err
}
if filepath.Dir(current) == current {
return "", os.ErrNotExist
}
}
}
func isWithinWorkspace(candidate, workspace string) bool {
rel, err := filepath.Rel(filepath.Clean(workspace), filepath.Clean(candidate))
return err == nil && filepath.IsLocal(rel)
}
type ReadFileTool struct {
fs fileSystem
}
func NewReadFileTool(workspace string, restrict bool, allowPaths ...[]*regexp.Regexp) *ReadFileTool {
var patterns []*regexp.Regexp
if len(allowPaths) > 0 {
patterns = allowPaths[0]
}
return &ReadFileTool{fs: buildFs(workspace, restrict, patterns)}
}
func (t *ReadFileTool) Name() string {
return "read_file"
}
func (t *ReadFileTool) Description() string {
return "Read the contents of a file"
}
func (t *ReadFileTool) Parameters() map[string]any {
return map[string]any{
"type": "object",
"properties": map[string]any{
"path": map[string]any{
"type": "string",
"description": "Path to the file to read",
},
},
"required": []string{"path"},
}
}
func (t *ReadFileTool) Execute(ctx context.Context, args map[string]any) *ToolResult {
path, ok := args["path"].(string)
if !ok {
return ErrorResult("path is required")
}
content, err := t.fs.ReadFile(path)
if err != nil {
return ErrorResult(err.Error())
}
return NewToolResult(string(content))
}
type WriteFileTool struct {
fs fileSystem
}
func NewWriteFileTool(workspace string, restrict bool, allowPaths ...[]*regexp.Regexp) *WriteFileTool {
var patterns []*regexp.Regexp
if len(allowPaths) > 0 {
patterns = allowPaths[0]
}
return &WriteFileTool{fs: buildFs(workspace, restrict, patterns)}
}
func (t *WriteFileTool) Name() string {
return "write_file"
}
func (t *WriteFileTool) Description() string {
return "Write content to a file"
}
func (t *WriteFileTool) Parameters() map[string]any {
return map[string]any{
"type": "object",
"properties": map[string]any{
"path": map[string]any{
"type": "string",
"description": "Path to the file to write",
},
"content": map[string]any{
"type": "string",
"description": "Content to write to the file",
},
},
"required": []string{"path", "content"},
}
}
func (t *WriteFileTool) Execute(ctx context.Context, args map[string]any) *ToolResult {
path, ok := args["path"].(string)
if !ok {
return ErrorResult("path is required")
}
content, ok := args["content"].(string)
if !ok {
return ErrorResult("content is required")
}
if len(content) > maxWriteSize {
return ErrorResult(fmt.Sprintf("content too large: %d bytes exceeds %d byte limit", len(content), maxWriteSize))
}
if err := t.fs.WriteFile(path, []byte(content)); err != nil {
return ErrorResult(err.Error())
}
return SilentResult(fmt.Sprintf("File written: %s", path))
}
type ListDirTool struct {
fs fileSystem
}
func NewListDirTool(workspace string, restrict bool, allowPaths ...[]*regexp.Regexp) *ListDirTool {
var patterns []*regexp.Regexp
if len(allowPaths) > 0 {
patterns = allowPaths[0]
}
return &ListDirTool{fs: buildFs(workspace, restrict, patterns)}
}
func (t *ListDirTool) Name() string {
return "list_dir"
}
func (t *ListDirTool) Description() string {
return "List files and directories in a path"
}
func (t *ListDirTool) Parameters() map[string]any {
return map[string]any{
"type": "object",
"properties": map[string]any{
"path": map[string]any{
"type": "string",
"description": "Path to list",
},
},
"required": []string{"path"},
}
}
func (t *ListDirTool) Execute(ctx context.Context, args map[string]any) *ToolResult {
path, ok := args["path"].(string)
if !ok {
path = "."
}
entries, err := t.fs.ReadDir(path)
if err != nil {
return ErrorResult(fmt.Sprintf("failed to read directory: %v", err))
}
return formatDirEntries(entries)
}
func formatDirEntries(entries []os.DirEntry) *ToolResult {
var result strings.Builder
for _, entry := range entries {
if entry.IsDir() {
result.WriteString("DIR: " + entry.Name() + "\n")
} else {
result.WriteString("FILE: " + entry.Name() + "\n")
}
}
return NewToolResult(result.String())
}
// fileSystem abstracts reading, writing, and listing files, allowing both
// unrestricted (host filesystem) and sandbox (os.Root) implementations to share the same polymorphic interface.
type fileSystem interface {
ReadFile(path string) ([]byte, error)
WriteFile(path string, data []byte) error
ReadDir(path string) ([]os.DirEntry, error)
}
// hostFs is an unrestricted fileReadWriter that operates directly on the host filesystem.
type hostFs struct{}
func (h *hostFs) ReadFile(path string) ([]byte, error) {
content, err := os.ReadFile(path)
if err != nil {
if os.IsNotExist(err) {
return nil, fmt.Errorf("failed to read file: file not found: %w", err)
}
if os.IsPermission(err) {
return nil, fmt.Errorf("failed to read file: access denied: %w", err)
}
return nil, fmt.Errorf("failed to read file: %w", err)
}
return content, nil
}
func (h *hostFs) ReadDir(path string) ([]os.DirEntry, error) {
return os.ReadDir(path)
}
func (h *hostFs) WriteFile(path string, data []byte) error {
// Use unified atomic write utility with explicit sync for flash storage reliability.
// Using 0o600 (owner read/write only) for secure default permissions.
return fileutil.WriteFileAtomic(path, data, 0o600)
}
// sandboxFs is a sandboxed fileSystem that operates within a strictly defined workspace using os.Root.
type sandboxFs struct {
workspace string
}
func (r *sandboxFs) execute(path string, fn func(root *os.Root, relPath string) error) error {
if r.workspace == "" {
return fmt.Errorf("workspace is not defined")
}
root, err := os.OpenRoot(r.workspace)
if err != nil {
return fmt.Errorf("failed to open workspace: %w", err)
}
defer root.Close()
relPath, err := getSafeRelPath(r.workspace, path)
if err != nil {
return err
}
return fn(root, relPath)
}
func (r *sandboxFs) ReadFile(path string) ([]byte, error) {
var content []byte
err := r.execute(path, func(root *os.Root, relPath string) error {
fileContent, err := root.ReadFile(relPath)
if err != nil {
if os.IsNotExist(err) {
return fmt.Errorf("failed to read file: file not found: %w", err)
}
// os.Root returns "escapes from parent" for paths outside the root
if os.IsPermission(err) || strings.Contains(err.Error(), "escapes from parent") ||
strings.Contains(err.Error(), "permission denied") {
return fmt.Errorf("failed to read file: access denied: %w", err)
}
return fmt.Errorf("failed to read file: %w", err)
}
content = fileContent
return nil
})
return content, err
}
func (r *sandboxFs) WriteFile(path string, data []byte) error {
return r.execute(path, func(root *os.Root, relPath string) error {
dir := filepath.Dir(relPath)
if dir != "." && dir != "/" {
if err := root.MkdirAll(dir, 0o755); err != nil {
return fmt.Errorf("failed to create parent directories: %w", err)
}
}
// Use atomic write pattern with explicit sync for flash storage reliability.
// Using 0o600 (owner read/write only) for secure default permissions.
randBytes := make([]byte, 8)
if _, err := rand.Read(randBytes); err != nil {
return fmt.Errorf("failed to generate random bytes for temp file: %w", err)
}
tmpRelPath := fmt.Sprintf(".tmp-%s", hex.EncodeToString(randBytes))
tmpFile, err := root.OpenFile(tmpRelPath, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o600)
if err != nil {
root.Remove(tmpRelPath)
return fmt.Errorf("failed to open temp file: %w", err)
}
if _, err := tmpFile.Write(data); err != nil {
tmpFile.Close()
root.Remove(tmpRelPath)
return fmt.Errorf("failed to write temp file: %w", err)
}
// CRITICAL: Force sync to storage medium before rename.
// This ensures data is physically written to disk, not just cached.
if err := tmpFile.Sync(); err != nil {
tmpFile.Close()
root.Remove(tmpRelPath)
return fmt.Errorf("failed to sync temp file: %w", err)
}
if err := tmpFile.Close(); err != nil {
root.Remove(tmpRelPath)
return fmt.Errorf("failed to close temp file: %w", err)
}
if err := root.Rename(tmpRelPath, relPath); err != nil {
root.Remove(tmpRelPath)
return fmt.Errorf("failed to rename temp file over target: %w", err)
}
// Sync directory to ensure rename is durable
if dirFile, err := root.Open("."); err == nil {
_ = dirFile.Sync()
dirFile.Close()
}
return nil
})
}
func (r *sandboxFs) ReadDir(path string) ([]os.DirEntry, error) {
var entries []os.DirEntry
err := r.execute(path, func(root *os.Root, relPath string) error {
dirEntries, err := fs.ReadDir(root.FS(), relPath)
if err != nil {
return err
}
entries = dirEntries
return nil
})
return entries, err
}
// whitelistFs wraps a sandboxFs and allows access to specific paths outside
// the workspace when they match any of the provided patterns.
type whitelistFs struct {
sandbox *sandboxFs
host hostFs
patterns []*regexp.Regexp
}
func (w *whitelistFs) matches(path string) bool {
for _, p := range w.patterns {
if p.MatchString(path) {
return true
}
}
return false
}
func (w *whitelistFs) ReadFile(path string) ([]byte, error) {
if w.matches(path) {
return w.host.ReadFile(path)
}
return w.sandbox.ReadFile(path)
}
func (w *whitelistFs) WriteFile(path string, data []byte) error {
if w.matches(path) {
return w.host.WriteFile(path, data)
}
return w.sandbox.WriteFile(path, data)
}
func (w *whitelistFs) ReadDir(path string) ([]os.DirEntry, error) {
if w.matches(path) {
return w.host.ReadDir(path)
}
return w.sandbox.ReadDir(path)
}
// buildFs returns the appropriate fileSystem implementation based on restriction
// settings and optional path whitelist patterns.
func buildFs(workspace string, restrict bool, patterns []*regexp.Regexp) fileSystem {
if !restrict {
return &hostFs{}
}
sandbox := &sandboxFs{workspace: workspace}
if len(patterns) > 0 {
return &whitelistFs{sandbox: sandbox, patterns: patterns}
}
return sandbox
}
// Helper to get a safe relative path for os.Root usage
func getSafeRelPath(workspace, path string) (string, error) {
if workspace == "" {
return "", fmt.Errorf("workspace is not defined")
}
rel := filepath.Clean(path)
if filepath.IsAbs(rel) {
var err error
rel, err = filepath.Rel(workspace, rel)
if err != nil {
return "", fmt.Errorf("failed to calculate relative path: %w", err)
}
}
if !filepath.IsLocal(rel) {
return "", fmt.Errorf("path escapes workspace: %s", path)
}
return rel, nil
}