From d7b288410db6cebb47194114dbce9ac60322e007 Mon Sep 17 00:00:00 2001 From: Max Date: Wed, 14 Jan 2026 17:41:02 +0800 Subject: [PATCH] Refactor Task and TriggerRequest Structures for Enhanced Input Handling - Updated the Task struct in DESIGN.md to replace the Messages field with a new Input field, allowing for more flexible user input options. - Revised the TriggerRequest struct in TECHNICAL.md to incorporate the new Input field, improving clarity on user interactions. - Enhanced related code examples and documentation to reflect these changes, ensuring consistency in task management and execution processes. --- agent/robot/TODO.md | 636 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 636 insertions(+) create mode 100644 agent/robot/TODO.md diff --git a/agent/robot/TODO.md b/agent/robot/TODO.md new file mode 100644 index 00000000..08f3f6cc --- /dev/null +++ b/agent/robot/TODO.md @@ -0,0 +1,636 @@ +# Robot Agent - Implementation TODO + +> Based on DESIGN.md and TECHNICAL.md +> Test environment: `source yao/env.local.sh` +> Test assistants: `yao-dev-app/assistants/robot/` + +--- + +## Workflow: Human-AI Collaboration + +**Important:** Follow this workflow strictly for each sub-task. + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ Implementation Workflow │ +├─────────────────────────────────────────────────────────────────┤ +│ 1. AI: Implement code for current sub-task │ +│ 2. AI: Present code for review (DO NOT write tests yet) │ +│ 3. Human: Review code, provide feedback │ +│ 4. AI: Iterate based on feedback │ +│ 5. Human: Confirm "LGTM" or "Approved" │ +│ 6. AI: Write tests for the approved code │ +│ 7. Human: Review tests │ +│ 8. AI: Run tests, fix if needed │ +│ 9. Human: Confirm sub-task complete, move to next │ +└─────────────────────────────────────────────────────────────────┘ +``` + +**Rules:** + +| Rule | Description | +| ------------------------ | ------------------------------------------ | +| One sub-task at a time | Focus only on current sub-task | +| No tests before approval | Wait for human "LGTM" before writing tests | +| No jumping ahead | Do not implement future phases | +| Ask if unclear | When in doubt, ask before proceeding | + +--- + +## Core Principle + +- Phase 1-2: Types + Skeleton (code compiles) +- Phase 3: Complete scheduling system (Cache + Pool + Trigger + Dedup + Job), executor is stub +- Phase 4-9: Implement executor phases one by one (P0 → P5) +- Phase 10: API completion, end-to-end tests +- Monitoring: Provided by Job system, no separate implementation + +--- + +## Phase 1: Types & Interfaces + +**Goal:** Define all types, enums, interfaces. No logic, no external deps. + +### 1.1 Enums (`types/enums.go`) + +- [ ] `Phase` - execution phases (inspiration, goals, tasks, run, delivery, learning) +- [ ] `ClockMode` - clock trigger modes (times, interval, daemon) +- [ ] `TriggerType` - trigger sources (clock, human, event) +- [ ] `ExecStatus` - execution status (pending, running, completed, failed, cancelled) +- [ ] `RobotStatus` - robot status (idle, working, paused, error, maintenance) +- [ ] `InterventionAction` - human actions (task.add, goal.adjust, etc.) +- [ ] `Priority` - priority levels (high, normal, low) +- [ ] `DeliveryType` - delivery types (email, file, webhook, notify) +- [ ] `DedupResult` - dedup results (skip, merge, proceed) +- [ ] `EventSource` - event sources (webhook, database) +- [ ] `LearningType` - learning types (execution, feedback, insight) +- [ ] `TaskSource` - task sources (auto, human, event) +- [ ] `ExecutorType` - executor types (assistant, mcp, process) +- [ ] `TaskStatus` - task status (pending, running, completed, failed, skipped, cancelled) +- [ ] `InsertPosition` - insert positions (first, last, next, at) + +### 1.2 Context (`types/context.go`) + +- [ ] `Context` struct - robot execution context +- [ ] `NewContext()` - constructor +- [ ] `UserID()`, `TeamID()` - helper methods + +### 1.3 Config Types (`types/config.go`) + +- [ ] `Config` - main config struct +- [ ] `Triggers`, `TriggerSwitch` - trigger enable/disable +- [ ] `Clock` - clock config with validation +- [ ] `Identity` - role, duties, rules +- [ ] `Quota` - concurrency limits with defaults +- [ ] `KB`, `DB` - knowledge base and database config +- [ ] `Learn` - learning config +- [ ] `Resources`, `MCPConfig` - available agents and tools +- [ ] `Delivery` - output delivery config +- [ ] `Event` - event trigger config + +### 1.4 Core Types (`types/robot.go`) + +- [ ] `Robot` struct - runtime robot representation +- [ ] `Robot` methods - `CanRun()`, `RunningCount()`, `AddExecution()`, `RemoveExecution()`, `GetExecution()`, `GetExecutions()` +- [ ] `Execution` struct - single execution instance +- [ ] `TriggerInput` - stored trigger input +- [ ] `CurrentState` - current executing state +- [ ] `Goals` - P1 output (markdown) +- [ ] `Task` - planned task (structured) +- [ ] `TaskResult` - task execution result +- [ ] `DeliveryResult` - delivery output +- [ ] `LearningEntry` - knowledge to save + +### 1.5 Clock Context (`types/clock.go`) + +- [ ] `ClockContext` struct - time context for P0 +- [ ] `NewClockContext()` - constructor + +### 1.6 Inspiration (`types/inspiration.go`) + +- [ ] `InspirationReport` struct - P0 output + +### 1.7 Request/Response (`types/request.go`) + +- [ ] `InterveneRequest` - human intervention request +- [ ] `EventRequest` - event trigger request +- [ ] `ExecutionResult` - trigger result +- [ ] `RobotState` - robot status query result + +### 1.8 Interfaces (`types/interfaces.go`) + +- [ ] `Manager` interface +- [ ] `Executor` interface +- [ ] `Pool` interface +- [ ] `Cache` interface +- [ ] `Dedup` interface +- [ ] `Store` interface + +### 1.9 Errors (`types/errors.go`) + +- [ ] Config errors +- [ ] Runtime errors +- [ ] Phase errors + +### 1.10 Tests + +- [ ] `types/enums_test.go` - enum validation +- [ ] `types/config_test.go` - config validation +- [ ] `types/clock_test.go` - clock context creation +- [ ] `types/robot_test.go` - robot methods + +--- + +## Phase 2: Skeleton Implementation + +**Goal:** Create all packages with empty/stub implementations. Code compiles. + +### 2.1 Utils (`utils/`) + +- [ ] `utils/convert.go` - JSON, map, struct conversions (implement) +- [ ] `utils/time.go` - time parsing, formatting, timezone (implement) +- [ ] `utils/id.go` - ID generation (nanoid) (implement) +- [ ] `utils/validate.go` - validation helpers (implement) +- [ ] Test: `utils/utils_test.go` + +### 2.2 Package Skeletons (stubs only, implemented in Phase 3) + +Create empty structs and stub methods that return nil/empty/success: + +- [ ] `cache/cache.go` - Cache struct, stub methods +- [ ] `dedup/dedup.go` - Dedup struct, stub methods +- [ ] `store/store.go` - Store struct, stub methods +- [ ] `pool/pool.go` - Pool struct, stub methods +- [ ] `job/job.go` - job helper stubs +- [ ] `plan/plan.go` - Plan struct, stub methods +- [ ] `trigger/trigger.go` - trigger dispatcher stub +- [ ] `executor/executor.go` - Executor struct, stub `Execute()` +- [ ] `manager/manager.go` - Manager struct, stub methods + +### 2.3 API Skeletons + +- [ ] `api/api.go` - Go API facade (all function signatures, return errors) +- [ ] `api/process.go` - Yao Process registration (all processes, return errors) +- [ ] `api/jsapi.go` - JSAPI registration (all methods, return errors) + +### 2.4 Root + +- [ ] `robot.go` - package entry + - [ ] `Init()` - placeholder + - [ ] `Shutdown()` - placeholder + +### 2.5 Compile Test + +- [ ] All packages compile without errors +- [ ] All imports resolve correctly +- [ ] No circular dependencies + +--- + +## Phase 3: Complete Scheduling System + +**Goal:** Implement complete scheduling system. Executor is stub (simulates success). + +This phase delivers a fully working scheduling pipeline: + +``` +Trigger → Manager → Cache → Dedup → Pool → Worker → Executor(stub) → Job +``` + +### 3.1 Cache Implementation + +- [ ] `cache/cache.go` - Cache struct with thread-safe map +- [ ] `cache/load.go` - load robots from `__yao.member` where `member_type='robot'` and `autonomous_mode=true` +- [ ] `cache/refresh.go` - refresh single robot, periodic full refresh (every hour) +- [ ] Test: load/refresh with real DB + +### 3.2 Pool Implementation + +- [ ] `pool/pool.go` - worker pool with configurable size (global limit) +- [ ] `pool/queue.go` - priority queue (sorted by: robot priority, trigger type, wait time) +- [ ] `pool/worker.go` - worker goroutines, dispatch to executor +- [ ] Test: submit jobs, verify execution order, verify concurrency limits + +### 3.3 Trigger Implementation + +- [ ] `trigger/trigger.go` - trigger dispatcher (routes to clock/intervene/event) +- [ ] `trigger/clock.go` - clock trigger + - [ ] `times` mode: match specific times (09:00, 14:00) + - [ ] `interval` mode: run every X duration (30m, 1h) + - [ ] `daemon` mode: restart immediately after completion + - [ ] Timezone handling +- [ ] `trigger/intervene.go` - human intervention + - [ ] Parse action (task.add, goal.adjust, etc.) + - [ ] Build TriggerInput with Messages +- [ ] `trigger/event.go` - event handling + - [ ] Webhook event dispatch + - [ ] Database change event dispatch +- [ ] `trigger/control.go` - execution control + - [ ] Pause execution + - [ ] Resume execution + - [ ] Cancel/Stop execution +- [ ] Test: clock matching (all modes), intervention handling, event dispatch + +### 3.4 Dedup Implementation + +- [ ] `dedup/dedup.go` - Dedup struct +- [ ] `dedup/fast.go` - fast in-memory time-window dedup + - [ ] Key: `memberID:triggerType:window` + - [ ] Check before submit + - [ ] Mark after submit +- [ ] Test: dedup check/mark, window expiry + +### 3.5 Job Integration + +- [ ] `job/job.go` - create job + - [ ] `job_id`: `robot_exec_{execID}` + - [ ] `category_id`: `autonomous_robot` + - [ ] Metadata: member_id, team_id, trigger_type, exec_id +- [ ] `job/execution.go` - execution lifecycle + - [ ] Create execution on trigger + - [ ] Update status on phase change + - [ ] Complete/fail on finish +- [ ] `job/log.go` - write phase logs + - [ ] Log phase start/end + - [ ] Log errors +- [ ] Test: job creation, execution tracking, log writing + +### 3.6 Manager Implementation + +- [ ] `manager/manager.go` - Manager struct + - [ ] `Start()` - start ticker goroutine, start pool + - [ ] `Stop()` - graceful shutdown (wait for running, drain queue) + - [ ] `Tick()` - main loop: + 1. Get all cached robots + 2. For each robot with clock trigger enabled + 3. Check if should execute (schedule match + dedup) + 4. Submit to pool +- [ ] Test: manager start/stop, tick cycle + +### 3.7 Executor Stub + +- [ ] `executor/executor.go` - stub implementation + - [ ] `Execute()` - simulate full execution + 1. Create Execution record + 2. Update phase: P0 → P1 → P2 → P3 → P4 → P5 + 3. Sleep briefly between phases (simulate work) + 4. Return success with mock data +- [ ] Test: verify stub called, verify phase progression + +### 3.8 Integration Test (End-to-End Scheduling) + +- [ ] Create test robot in `__yao.member` with clock config +- [ ] Start manager +- [ ] Wait for clock trigger +- [ ] Verify: + - [ ] Robot loaded to cache + - [ ] Clock trigger matched + - [ ] Dedup checked + - [ ] Job submitted to pool + - [ ] Worker picked up job + - [ ] Executor stub called + - [ ] Job execution recorded + - [ ] Logs written +- [ ] Test human intervention trigger +- [ ] Test event trigger +- [ ] Test concurrent executions (multiple robots) +- [ ] Test quota enforcement (per-robot limit) +- [ ] Test pause/resume/stop + +--- + +## Phase 4: Executor - P0 Inspiration + +**Goal:** Implement P0 (Inspiration Agent). Clock trigger → P0 → stub P1-P5. + +### 4.1 Test Assistant Setup + +Create `yao-dev-app/assistants/robot/` directory: + +- [ ] `inspiration/package.yao` - Inspiration Agent config +- [ ] `inspiration/prompts.yml` - P0 prompts +- [ ] `inspiration/src/index.ts` - hooks if needed + +### 4.2 P0 Implementation + +- [ ] `executor/inspiration.go` - build prompt with `ClockContext` +- [ ] `executor/inspiration.go` - call Inspiration Agent +- [ ] `executor/inspiration.go` - parse response to `InspirationReport` +- [ ] `executor/prompt.go` - `BuildInspirationPrompt()` + +### 4.3 Tests + +- [ ] `executor/inspiration_test.go` - P0 with real LLM call +- [ ] Verify: clock context in prompt +- [ ] Verify: markdown report generated + +--- + +## Phase 5: Executor - P1 Goals + +**Goal:** Implement P1 (Goal Generation Agent). P0 → P1 → stub P2-P5. + +### 5.1 Test Assistant Setup + +- [ ] `goals/package.yao` - Goal Generation Agent config +- [ ] `goals/prompts.yml` - P1 prompts + +### 5.2 P1 Implementation + +- [ ] `executor/goals.go` - build prompt with inspiration report +- [ ] `executor/goals.go` - call Goal Agent +- [ ] `executor/goals.go` - parse response to `Goals` (markdown) +- [ ] `executor/prompt.go` - `BuildGoalsPrompt()` + +### 5.3 Tests + +- [ ] `executor/goals_test.go` - P1 with real LLM call +- [ ] Verify: inspiration report in prompt +- [ ] Verify: goals markdown generated + +--- + +## Phase 6: Executor - P2 Tasks + +**Goal:** Implement P2 (Task Planning Agent). P1 → P2 → stub P3-P5. + +### 6.1 Test Assistant Setup + +- [ ] `tasks/package.yao` - Task Planning Agent config +- [ ] `tasks/prompts.yml` - P2 prompts + +### 6.2 P2 Implementation + +- [ ] `executor/tasks.go` - build prompt with goals +- [ ] `executor/tasks.go` - call Task Agent +- [ ] `executor/tasks.go` - parse response to `[]Task` (structured) +- [ ] `executor/prompt.go` - `BuildTasksPrompt()` + +### 6.3 Tests + +- [ ] `executor/tasks_test.go` - P2 with real LLM call +- [ ] Verify: goals in prompt +- [ ] Verify: structured tasks generated + +--- + +## Phase 7: Executor - P3 Run + +**Goal:** Implement P3 (Task Execution). P2 → P3 → stub P4-P5. + +### 7.1 Implementation + +- [ ] `executor/run.go` - iterate tasks +- [ ] `executor/run.go` - call executor (assistant/mcp/process) +- [ ] `executor/run.go` - collect results +- [ ] `executor/agent.go` - unified agent call method + +### 7.2 Validation Agent Setup + +- [ ] `validation/package.yao` - Validation Agent config +- [ ] `validation/prompts.yml` - validation prompts + +### 7.3 Tests + +- [ ] `executor/run_test.go` - P3 with real agent calls +- [ ] Verify: tasks executed in order +- [ ] Verify: results collected + +--- + +## Phase 8: Executor - P4 Delivery + +**Goal:** Implement P4 (Delivery). P3 → P4 → stub P5. + +### 8.1 Test Assistant Setup + +- [ ] `delivery/package.yao` - Delivery Agent config +- [ ] `delivery/prompts.yml` - delivery prompts + +### 8.2 Implementation + +- [ ] `executor/delivery.go` - build delivery content +- [ ] `executor/delivery.go` - send via configured channel (email/file/webhook/notify) + +### 8.3 Tests + +- [ ] `executor/delivery_test.go` - P4 delivery +- [ ] Verify: delivery sent (mock or real) + +--- + +## Phase 9: Executor - P5 Learning + +**Goal:** Implement P5 (Learning). Full execution flow complete. + +### 9.1 Test Assistant Setup + +- [ ] `learning/package.yao` - Learning Agent config +- [ ] `learning/prompts.yml` - learning prompts + +### 9.2 Store Implementation + +- [ ] `store/kb.go` - KB operations (create, save, search) +- [ ] `store/learning.go` - save learning entries to private KB + +### 9.3 Implementation + +- [ ] `executor/learning.go` - extract learnings from execution +- [ ] `executor/learning.go` - call Learning Agent +- [ ] `executor/learning.go` - save to private KB + +### 9.4 Tests + +- [ ] `executor/learning_test.go` - P5 learning +- [ ] Verify: learnings saved to KB + +--- + +## Phase 10: API & Integration + +**Goal:** Complete API implementation, end-to-end tests. + +### 10.1 API Implementation + +- [ ] `api/api.go` - implement all Go API functions +- [ ] `api/process.go` - implement all Process handlers +- [ ] `api/jsapi.go` - implement JSAPI + +### 10.2 End-to-End Tests + +- [ ] Full clock trigger flow (P0 → P5) +- [ ] Human intervention flow (P1 → P5) +- [ ] Event trigger flow (P1 → P5) +- [ ] Concurrent execution test +- [ ] Pause/Resume/Stop test + +### 10.3 Integration with OpenAPI + +- [ ] HTTP endpoints for human intervention +- [ ] Webhook endpoints for events + +--- + +## Phase 11: Advanced Features + +**Goal:** Implement semantic dedup, plan queue. + +### 11.1 Semantic Dedup + +- [ ] `dedup/semantic.go` - call Dedup Agent for goal/task level dedup +- [ ] Dedup Agent setup (`assistants/robot/dedup/`) +- [ ] Test: semantic dedup with real LLM + +### 11.2 Plan Queue + +- [ ] `plan/plan.go` - plan queue implementation + - [ ] Store planned tasks/goals + - [ ] Execute at next cycle or specified time +- [ ] `plan/schedule.go` - schedule for later +- [ ] Test: plan queue operations + +> **Note:** Monitoring is provided by Job system (Activity Monitor UI). No separate implementation needed. + +--- + +## Test Assistants Structure + +``` +yao-dev-app/assistants/robot/ +├── inspiration/ # P0: Inspiration Agent +│ ├── package.yao +│ └── prompts.yml +├── goals/ # P1: Goal Generation Agent +│ ├── package.yao +│ └── prompts.yml +├── tasks/ # P2: Task Planning Agent +│ ├── package.yao +│ └── prompts.yml +├── validation/ # P3: Validation Agent +│ ├── package.yao +│ └── prompts.yml +├── delivery/ # P4: Delivery Agent +│ ├── package.yao +│ └── prompts.yml +├── learning/ # P5: Learning Agent +│ ├── package.yao +│ └── prompts.yml +└── dedup/ # Deduplication Agent + ├── package.yao + └── prompts.yml +``` + +--- + +## Notes + +### Test Environment Setup + +1. **Environment Variables:** Run `source yao/env.local.sh` before tests +2. **Test Preparation:** Use `testutils.Prepare(t)` to load config, KB, and agents + +```go +package robot_test + +import ( + "testing" + "github.com/yaoapp/yao/agent/testutils" +) + +func TestExample(t *testing.T) { + // Load environment config (from YAO_TEST_APPLICATION) + // This loads: config, connectors, KB, agents, models, etc. + testutils.Prepare(t) + defer testutils.Clean(t) + + // Your test code here +} +``` + +### Test Conventions + +1. **Black-box Tests:** All tests in `*_test` package (external package) +2. **Real LLM Calls:** Use `gpt-4o` or `deepseek` connectors for agent tests +3. **Incremental:** Each phase builds on previous, all tests must pass before next phase +4. **No Skip:** Do NOT use `t.Skip()` except for `testing.Short()` (CI mode) +5. **Must Assert:** Every test MUST have result validation assertions + +```go +func TestWithLLM(t *testing.T) { + // Only allowed Skip: testing.Short() for CI + if testing.Short() { + t.Skip("Skipping integration test") + } + + testutils.Prepare(t) + defer testutils.Clean(t) + + // Your test code... + result, err := SomeFunction() + + // MUST have assertions - no empty tests! + assert.NoError(t, err) + assert.NotNil(t, result) + assert.Equal(t, expected, result.Field) +} +``` + +### Test Rules + +| Rule | Description | +| ----------------- | ----------------------------------------- | +| No arbitrary Skip | Only `testing.Short()` skip allowed | +| Must assert | Every test must validate results | +| No empty tests | Tests without assertions will fail review | +| Real calls | LLM tests use real API calls, not mocks | + +### Key Environment Variables + +| Variable | Description | +| ---------------------- | ------------------------------- | +| `YAO_TEST_APPLICATION` | Test app path (`yao-dev-app`) | +| `OPENAI_TEST_KEY` | OpenAI API key | +| `DEEPSEEK_API_KEY` | DeepSeek API key | +| `YAO_DB_DRIVER` | Database driver (mysql/sqlite3) | +| `YAO_DB_PRIMARY` | Database connection string | + +--- + +## Progress Tracking + +| Phase | Status | Description | +| --------------------- | ------ | ---------------------------------------------------- | +| 1. Types & Interfaces | ⬜ | All types, enums, interfaces | +| 2. Skeleton | ⬜ | Empty stubs, code compiles | +| 3. Scheduling System | ⬜ | Cache + Pool + Trigger + Dedup + Job (executor stub) | +| 4. P0 Inspiration | ⬜ | Inspiration Agent integration | +| 5. P1 Goals | ⬜ | Goal Generation Agent integration | +| 6. P2 Tasks | ⬜ | Task Planning Agent integration | +| 7. P3 Run | ⬜ | Task execution (assistant/mcp/process) | +| 8. P4 Delivery | ⬜ | Output delivery (email/file/webhook/notify) | +| 9. P5 Learning | ⬜ | Learning Agent + KB save | +| 10. API & Integration | ⬜ | Complete API, end-to-end tests | +| 11. Advanced | ⬜ | Semantic dedup, plan queue | + +Legend: ⬜ Not started | 🟡 In progress | ✅ Complete + +--- + +## Quick Commands + +```bash +# Setup environment +source yao/env.local.sh + +# Run all robot tests +go test -v ./agent/robot/... + +# Run specific phase tests +go test -v ./agent/robot/types/... +go test -v ./agent/robot/cache/... +go test -v ./agent/robot/pool/... +go test -v ./agent/robot/executor/... + +# Run with coverage +go test -cover ./agent/robot/... +```