diff --git a/.dockerignore b/.dockerignore index d632da5ea..13e985a35 100644 --- a/.dockerignore +++ b/.dockerignore @@ -8,3 +8,4 @@ config/ *.md LICENSE assets/ +docker/data/ diff --git a/.github/AGENTS.md b/.github/AGENTS.md new file mode 100644 index 000000000..09bd88b4c --- /dev/null +++ b/.github/AGENTS.md @@ -0,0 +1,107 @@ +# AGENTS.md — AI Agent Instructions for PicoClaw + +> Instructions for AI coding agents working on this repository. + +## Project Overview + +PicoClaw is an ultra-lightweight personal AI assistant written in **Go**. It connects chat **Channels** (Telegram, Discord, WeChat, etc.) to LLM **Providers** (OpenAI, Anthropic, Ollama, etc.) via a central **Message Bus**. Designed to run on $10 hardware with <10MB RAM. + +## Architecture + +``` +cmd/picoclaw/main.go ← CLI entry (cobra): agent, gateway, onboard +cmd/picoclaw-launcher-tui/ ← TUI launcher +web/backend/ ← Web launcher + control panel + +pkg/gateway/ → Orchestrator: initializes bus, channels, agent, cron +pkg/bus/ → Message Bus: InboundMessage ↔ OutboundMessage decoupling +pkg/agent/ → Agent Loop: session history, LLM interaction, tool execution +pkg/channels/ → Channel Manager: chat integrations lifecycle + routing +pkg/providers/ → LLM Providers: OpenAI-compatible, Anthropic, etc. +pkg/tools/ → Tool/Skill definitions (shell, browser, MCP, etc.) +pkg/config/ → Config struct with env tag overrides, JSON merge on defaults +pkg/session/ → Session management and persistence +pkg/mcp/ → Model Context Protocol integration +pkg/routing/ → Smart model routing (simple → cheap, complex → SOTA) + +workspace/ → Agent identity files (SOUL.md, USER.md, AGENT.md) +config/ → config.example.json (reference for all options) +docker/ → Dockerfile.heavy, docker-compose.yml, entrypoint.sh +``` + +## Build & Test + +```bash +make build # Build binary (runs go generate first) +make generate # Run go generate only +make check # Full pre-commit: deps + fmt + vet + test +make test # Run all tests +make fmt # Format code +make vet # Static analysis +make lint # Full linter (golangci-lint) +make build-launcher # Build web launcher (requires pnpm for frontend) +``` + +Single test: `go test -run TestName -v ./pkg/session/` + +## Coding Conventions + +### Go Style +- **Go 1.25+**, module path `github.com/sipeed/picoclaw` +- Build tags: `-tags stdjson`; CGO disabled by default (`CGO_ENABLED=0`) +- Error wrapping: `fmt.Errorf("context: %w", err)` +- Logging: `zerolog` via `pkg/logger` — use `log.Debug()`, `log.Info()`, `log.Error()`, etc. +- Testing: `testify/assert` and `testify/require`, co-located `_test.go` files +- Commits: imperative mood, conventional-ish ("Add retry logic", "Fix session leak (#123)") + +### Config Pattern +- Central `Config` struct in `pkg/config/config.go` with `env` tags +- `LoadConfig`: starts from `DefaultConfig()`, merges user JSON on top +- Reference: `config/config.example.json` + +### Key Patterns +- Channels/providers registered via side-effect imports in `pkg/gateway/` +- Workspace files (`~/.picoclaw/workspace/`) are the agent's persistent home +- `workspace/` in the repo is the template; `docker/data/` is the Docker volume mount + +## Docker + +The "heavy" image (`docker/Dockerfile.heavy`) is a multi-stage build: +1. Go builder → builds `picoclaw` + `picoclaw-launcher` +2. Node builder → builds web frontend +3. Runtime: `node:24-slim` + Chromium, Python/uv, Xvfb + +**`docker/entrypoint.sh`**: runs `onboard` → starts Xvfb → starts Copilot proxy → execs launcher + +**Volume**: `docker/data/` mounts to `/root/.picoclaw` in the container. All workspace, skills, memory, and session files the agent uses live here. + +⚠️ **When editing files the Docker agent reads, edit under `docker/data/`, NOT `workspace/`** — the latter is the repo template, not what the running container sees. + +## Documentation + +Existing docs are in `docs/`. Key references: +- [docs/configuration.md](docs/configuration.md) — Config file guide +- [docs/providers.md](docs/providers.md) — LLM provider setup +- [docs/docker.md](docs/docker.md) — Docker deployment +- [docs/tools_configuration.md](docs/tools_configuration.md) — Tool/skill config +- [docs/spawn-tasks.md](docs/spawn-tasks.md) — Sub-agent spawning +- [docs/steering.md](docs/steering.md) — Model steering +- [docs/channels/](docs/channels/) — Per-channel setup guides +- [docs/hooks/](docs/hooks/) — Hook system +- [docs/agent-refactor/](docs/agent-refactor/) — Architecture deep-dives +- [CONTRIBUTING.md](CONTRIBUTING.md) — Contribution workflow, AI disclosure requirements +- [ROADMAP.md](ROADMAP.md) — Feature roadmap and priorities + +Link to these docs rather than duplicating their content. + +## Security Notes + +- File operations are sandboxed to workspace (`restrict_to_workspace: true` by default) +- Never hardcode API keys — use env vars or `config.json` (gitignored) +- The `exec` tool has `custom_allow_patterns` to whitelist commands +- Review AI-generated code for path traversal, injection, SSRF (see CONTRIBUTING.md) +- `.env` and `config.json` are gitignored; `config.example.json` is the reference + +## Cross-Platform + +PicoClaw builds for: linux/amd64, linux/arm64, linux/arm (ARMv7), linux/mipsle, linux/riscv64, linux/loong64, darwin/arm64, windows/amd64. The Makefile has per-platform targets and a MIPS ELF e_flags patch for NaN2008 kernels. diff --git a/.gitignore b/.gitignore index 8b5f95215..af097a6f0 100644 --- a/.gitignore +++ b/.gitignore @@ -63,3 +63,4 @@ web/backend/dist/* .claude/ docker/data +.learnings \ No newline at end of file diff --git a/docker/Dockerfile b/docker/Dockerfile deleted file mode 100644 index 480244127..000000000 --- a/docker/Dockerfile +++ /dev/null @@ -1,43 +0,0 @@ -# ============================================================ -# Stage 1: Build the picoclaw binary -# ============================================================ -FROM golang:1.25-alpine AS builder - -RUN apk add --no-cache git make - -WORKDIR /src - -# Cache dependencies -COPY go.mod go.sum ./ -RUN go mod download - -# Copy source and build -COPY . . -RUN make build - -# ============================================================ -# Stage 2: Minimal runtime image -# ============================================================ -FROM alpine:3.23 - -RUN apk add --no-cache ca-certificates tzdata curl - -# Health check -HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \ - CMD wget -q --spider http://localhost:18790/health || exit 1 - -# Copy binary -COPY --from=builder /src/build/picoclaw /usr/local/bin/picoclaw - -# Create non-root user and group -RUN addgroup -g 1000 picoclaw && \ - adduser -D -u 1000 -G picoclaw picoclaw - -# Switch to non-root user -USER picoclaw - -# Run onboard to create initial directories and config -RUN /usr/local/bin/picoclaw onboard - -ENTRYPOINT ["picoclaw"] -CMD ["gateway"] diff --git a/docker/Dockerfile.full b/docker/Dockerfile.full deleted file mode 100644 index 30e1680d5..000000000 --- a/docker/Dockerfile.full +++ /dev/null @@ -1,44 +0,0 @@ -# ============================================================ -# Stage 1: Build the picoclaw binary -# ============================================================ -FROM golang:1.26.0-alpine AS builder - -RUN apk add --no-cache git make - -WORKDIR /src - -# Cache dependencies -COPY go.mod go.sum ./ -RUN go mod download - -# Copy source and build -COPY . . -RUN make build - -# ============================================================ -# Stage 2: Node.js-based runtime with full MCP support -# ============================================================ -FROM node:24-alpine3.23 - -# Install runtime dependencies -RUN apk add --no-cache \ - ca-certificates \ - curl \ - git \ - python3 \ - py3-pip - -# Install uv and symlink to system path -RUN curl -LsSf https://astral.sh/uv/install.sh | sh && \ - ln -s /root/.local/bin/uv /usr/local/bin/uv && \ - ln -s /root/.local/bin/uvx /usr/local/bin/uvx && \ - uv --version - -# Copy binary -COPY --from=builder /src/build/picoclaw /usr/local/bin/picoclaw - -# Create picoclaw home directory -RUN /usr/local/bin/picoclaw onboard - -ENTRYPOINT ["picoclaw"] -CMD ["gateway"] diff --git a/docker/Dockerfile.goreleaser b/docker/Dockerfile.goreleaser deleted file mode 100644 index 68a02aae8..000000000 --- a/docker/Dockerfile.goreleaser +++ /dev/null @@ -1,12 +0,0 @@ -FROM alpine:3.21 - -ARG TARGETPLATFORM - -RUN apk add --no-cache ca-certificates tzdata - -COPY $TARGETPLATFORM/picoclaw /usr/local/bin/picoclaw -COPY docker/entrypoint.sh /entrypoint.sh - -RUN chmod +x /entrypoint.sh - -ENTRYPOINT ["/entrypoint.sh"] diff --git a/docker/Dockerfile.goreleaser.launcher b/docker/Dockerfile.goreleaser.launcher deleted file mode 100644 index 5d65576f7..000000000 --- a/docker/Dockerfile.goreleaser.launcher +++ /dev/null @@ -1,12 +0,0 @@ -FROM alpine:3.21 - -ARG TARGETPLATFORM - -RUN apk add --no-cache ca-certificates tzdata - -COPY $TARGETPLATFORM/picoclaw /usr/local/bin/picoclaw -COPY $TARGETPLATFORM/picoclaw-launcher /usr/local/bin/picoclaw-launcher -COPY $TARGETPLATFORM/picoclaw-launcher-tui /usr/local/bin/picoclaw-launcher-tui - -ENTRYPOINT ["picoclaw-launcher"] -CMD ["-public", "-no-browser"] diff --git a/docker/Dockerfile.heavy b/docker/Dockerfile.heavy index cbc243e39..3fa770f31 100644 --- a/docker/Dockerfile.heavy +++ b/docker/Dockerfile.heavy @@ -1,9 +1,10 @@ # ============================================================ -# Stage 1: Build the picoclaw binary +# Stage 1: Build picoclaw + picoclaw-launcher # ============================================================ FROM golang:1.26.0-alpine AS builder -RUN apk add --no-cache git make +RUN apk add --no-cache git make nodejs npm && \ + npm install -g pnpm WORKDIR /src @@ -11,29 +12,59 @@ WORKDIR /src COPY go.mod go.sum ./ RUN go mod download -# Copy source and build +# Copy source and build picoclaw gateway binary COPY . . RUN make build -# ============================================================ -# Stage 2: Node.js runtime with Python + MCP support -# ============================================================ -FROM node:24-alpine3.23 +# Build the launcher (frontend + backend) +RUN cd web/frontend && pnpm install --frozen-lockfile && pnpm build:backend +RUN CGO_ENABLED=0 go build -v -tags stdjson -o build/picoclaw-launcher ./web/backend -RUN apk add --no-cache \ +# ============================================================ +# Stage 2: Node.js runtime with Python + MCP + Homebrew support +# ============================================================ +FROM node:24-slim + +RUN apt-get update && apt-get install -y --no-install-recommends \ ca-certificates \ curl \ git \ python3 \ - py3-pip \ + python3-pip \ chromium \ - jq + jq \ + tmux \ + ffmpeg \ + wget \ + build-essential \ + procps \ + file \ + && curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg \ + | dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg \ + && echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" \ + | tee /etc/apt/sources.list.d/github-cli.list > /dev/null \ + && apt-get update && apt-get install -y gh \ + && rm -rf /var/lib/apt/lists/* -# Install Playwright browsers for agent-browser -ENV PLAYWRIGHT_BROWSERS_PATH=/opt/playwright-browsers -RUN npm install -g agent-browser && \ - npx playwright install chromium && \ - chmod -R o+rx $PLAYWRIGHT_BROWSERS_PATH +# Install Homebrew (requires non-root user) +ENV NONINTERACTIVE=1 +RUN useradd -m -s /bin/bash linuxbrew && \ + mkdir -p /home/linuxbrew/.linuxbrew && \ + chown -R linuxbrew: /home/linuxbrew +USER linuxbrew +RUN /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" +USER root +ENV PATH="/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:$PATH" + +# Install agent-browser CLI (uses system chromium via AGENT_BROWSER_EXECUTABLE_PATH) +RUN npm install -g agent-browser + +# Virtual framebuffer for headed chromium in headless container +RUN apt-get update && apt-get install -y --no-install-recommends xvfb \ + && rm -rf /var/lib/apt/lists/* + +# Install GitHub Copilot CLI (headless proxy) +RUN npm install -g @github/copilot@1.0.10 # Install uv RUN curl -LsSf https://astral.sh/uv/install.sh | sh && \ @@ -41,27 +72,25 @@ RUN curl -LsSf https://astral.sh/uv/install.sh | sh && \ ln -s /root/.local/bin/uvx /usr/local/bin/uvx && \ uv --version +# Install Python packages (system-wide via uv to avoid PEP 668 issues) +RUN uv pip install --system --break-system-packages atproto + # Health check HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \ CMD wget -q --spider http://localhost:18790/health || exit 1 -# Copy binary +# Copy binaries COPY --from=builder /src/build/picoclaw /usr/local/bin/picoclaw - -# Reuse existing node user (UID/GID 1000) — rename to picoclaw -RUN deluser node 2>/dev/null; delgroup node 2>/dev/null; \ - addgroup -g 1000 picoclaw 2>/dev/null; \ - adduser -D -u 1000 -G picoclaw -h /home/picoclaw picoclaw 2>/dev/null || true - -USER picoclaw +COPY --from=builder /src/build/picoclaw-launcher /usr/local/bin/picoclaw-launcher # Run onboard to create initial directories and config RUN /usr/local/bin/picoclaw onboard # Copy default workspace -COPY --chown=picoclaw:picoclaw workspace/ /home/picoclaw/.picoclaw/workspace/ +COPY workspace/ /root/.picoclaw/workspace/ -VOLUME /home/picoclaw/.picoclaw/workspace +COPY docker/entrypoint.sh /entrypoint.sh +RUN sed -i 's/\r$//' /entrypoint.sh && chmod +x /entrypoint.sh -ENTRYPOINT ["picoclaw"] -CMD ["gateway"] +ENTRYPOINT ["/entrypoint.sh"] +CMD [] diff --git a/docker/docker-compose.full.yml b/docker/docker-compose.full.yml deleted file mode 100644 index 6f34448c4..000000000 --- a/docker/docker-compose.full.yml +++ /dev/null @@ -1,44 +0,0 @@ -services: - # ───────────────────────────────────────────── - # PicoClaw Agent (one-shot query) - Full MCP Support - # docker compose -f docker/docker-compose.full.yml run --rm picoclaw-agent -m "Hello" - # ───────────────────────────────────────────── - picoclaw-agent: - build: - context: .. - dockerfile: docker/Dockerfile.full - container_name: picoclaw-agent-full - profiles: - - agent - volumes: - - ../config/config.json:/root/.picoclaw/config.json:ro - - picoclaw-workspace:/root/.picoclaw/workspace - - picoclaw-npm-cache:/root/.npm # npm cache for faster MCP server installs - entrypoint: ["picoclaw", "agent"] - stdin_open: true - tty: true - - # ───────────────────────────────────────────── - # PicoClaw Gateway (Long-running Bot) - Full MCP Support - # docker compose -f docker/docker-compose.full.yml --profile gateway up - # ───────────────────────────────────────────── - picoclaw-gateway: - build: - context: .. - dockerfile: docker/Dockerfile.full - container_name: picoclaw-gateway-full - restart: unless-stopped - profiles: - - gateway - volumes: - # Configuration file - - ../config/config.json:/root/.picoclaw/config.json:ro - # Persistent workspace (sessions, memory, logs) - - picoclaw-workspace:/root/.picoclaw/workspace - # NPM cache for faster MCP server installs - - picoclaw-npm-cache:/root/.npm - command: ["gateway"] - -volumes: - picoclaw-workspace: - picoclaw-npm-cache: # Cache npm packages to speed up MCP server installations diff --git a/docker/docker-compose.yml b/docker/docker-compose.yml index b26cf4199..1c4d92291 100644 --- a/docker/docker-compose.yml +++ b/docker/docker-compose.yml @@ -9,44 +9,45 @@ services: profiles: - agent # Uncomment to access host network; leave commented unless needed. - #extra_hosts: - # - "host.docker.internal:host-gateway" + extra_hosts: + - "host.docker.internal:host-gateway" volumes: - ./data:/root/.picoclaw + networks: + - picoclaw_net entrypoint: ["picoclaw", "agent"] stdin_open: true tty: true # ───────────────────────────────────────────── - # PicoClaw Gateway (Long-running Bot) - # docker compose -f docker/docker-compose.yml --profile gateway up + # PicoClaw Gateway + Launcher (Long-running Bot + Web Console) + # docker compose -f docker/docker-compose.yml up # ───────────────────────────────────────────── picoclaw-gateway: - image: docker.io/sipeed/picoclaw:latest - container_name: picoclaw-gateway - restart: on-failure - profiles: - - gateway - # Uncomment to access host network; leave commented unless needed. - #extra_hosts: - # - "host.docker.internal:host-gateway" - volumes: - - ./data:/root/.picoclaw - - # ───────────────────────────────────────────── - # PicoClaw Launcher (Web Console + Gateway) - # docker compose -f docker/docker-compose.yml --profile launcher up - # ───────────────────────────────────────────── - picoclaw-launcher: - image: docker.io/sipeed/picoclaw:launcher - container_name: picoclaw-launcher - restart: on-failure - profiles: - - launcher + build: + context: .. + dockerfile: docker/Dockerfile.heavy + container_name: picoclaw-gateway-full + restart: unless-stopped + extra_hosts: + - "host.docker.internal:host-gateway" environment: - - PICOCLAW_GATEWAY_HOST=0.0.0.0 + - COPILOT_GITHUB_TOKEN=${COPILOT_GITHUB_TOKEN} + - AGENT_BROWSER_PROFILE=/root/.picoclaw/workspace/browser/picoclaw + - AGENT_BROWSER_EXECUTABLE_PATH=/usr/bin/chromium + - AGENT_BROWSER_HEADED=true + - AGENT_BROWSER_ARGS=--no-sandbox,--disable-blink-features=AutomationControlled + - DISPLAY=:99 ports: - - "127.0.0.1:18800:18800" - "127.0.0.1:18790:18790" + - "192.168.1.169:18790:18790" + - "127.0.0.1:18800:18800" + - "192.168.1.169:18800:18800" volumes: - ./data:/root/.picoclaw + networks: + - picoclaw_net + +networks: + picoclaw_net: + external: false diff --git a/docker/entrypoint.sh b/docker/entrypoint.sh index b6fc724b5..f4af0c328 100644 --- a/docker/entrypoint.sh +++ b/docker/entrypoint.sh @@ -12,4 +12,31 @@ if [ ! -d "${HOME}/.picoclaw/workspace" ] && [ ! -f "${HOME}/.picoclaw/config.js exit 0 fi -exec picoclaw gateway "$@" +# Start virtual framebuffer for headed chromium (agent-browser) +if [ -n "$DISPLAY" ] && command -v Xvfb >/dev/null 2>&1; then + Xvfb "$DISPLAY" -screen 0 1280x1024x24 -ac & + sleep 1 + echo "Xvfb running on $DISPLAY" +fi + +# Start Copilot headless proxy if token is provided +if [ -n "$COPILOT_GITHUB_TOKEN" ] && command -v copilot >/dev/null 2>&1; then + COPILOT_PORT="${COPILOT_PORT:-4321}" + echo "Starting Copilot headless proxy on port ${COPILOT_PORT}..." + copilot --headless --port "$COPILOT_PORT" & + COPILOT_PID=$! + sleep 2 + if kill -0 "$COPILOT_PID" 2>/dev/null; then + echo "Copilot proxy running (PID ${COPILOT_PID})" + else + echo "WARNING: Copilot proxy failed to start, continuing without it" + fi +fi + +# Start launcher (web UI) if available, otherwise fall back to gateway only. +# The launcher auto-starts its own gateway subprocess. +if command -v picoclaw-launcher >/dev/null 2>&1; then + exec picoclaw-launcher -public -no-browser "$@" +else + exec picoclaw gateway -d "$@" +fi \ No newline at end of file