Update DESIGN.md, TECHNICAL.md, and TODO.md for P3 Implementation Enhancements
- Revised DESIGN.md to clarify the architecture of the P3 entry point, including updated RunConfig parameters and task execution flow. - Expanded TECHNICAL.md with detailed implementation notes on multi-turn conversation flow, validation rules format, task dependencies, and resource management. - Removed outdated architecture diagrams from TODO.md and added comprehensive notes on the new multi-turn conversation handling and validation mechanisms. - Documented the functionality of the new `yao/assert` package and its integration into the validation process.
This commit is contained in:
Max
parent
1dd1785dc0
commit
9cc5be78ef
3 changed files with 131 additions and 86 deletions
|
|
@ -372,8 +372,9 @@ type Task struct {
|
||||||
```
|
```
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
┌─────────────────────────────────────────────────────────────┐
|
||||||
│ run.go (P3 Entry) │
|
│ run.go (P3 Entry) │
|
||||||
│ - RunConfig: threshold, continue-on-failure, max-turns │
|
│ - RunConfig: ContinueOnFailure, ValidationThreshold, │
|
||||||
│ - RunExecution: main execution loop │
|
│ MaxTurnsPerTask │
|
||||||
|
│ - RunExecution: main loop with task dependency passing │
|
||||||
└─────────────────────┬───────────────────────────────────────┘
|
└─────────────────────┬───────────────────────────────────────┘
|
||||||
│
|
│
|
||||||
┌────────────┴────────────┐
|
┌────────────┴────────────┐
|
||||||
|
|
@ -381,23 +382,25 @@ type Task struct {
|
||||||
┌─────────────────┐ ┌─────────────────┐
|
┌─────────────────┐ ┌─────────────────┐
|
||||||
│ runner.go │ │ validator.go │
|
│ runner.go │ │ validator.go │
|
||||||
│ - Runner │ │ - Validator │
|
│ - Runner │ │ - Validator │
|
||||||
│ - Task exec │ │ - Two-layer │
|
│ - Multi-turn │ │ - Two-layer │
|
||||||
│ - Multi-turn │ │ - Rule+Semantic│
|
│ conversation │ │ - Rule+Semantic│
|
||||||
│ conversation │ │ - NeedReply │
|
│ - Task context │ │ - NeedReply │
|
||||||
|
│ building │ │ - ReplyContent │
|
||||||
└────────┬────────┘ └────────┬────────┘
|
└────────┬────────┘ └────────┬────────┘
|
||||||
│ │
|
│ │
|
||||||
│ ▼
|
│ ▼
|
||||||
│ ┌─────────────────┐
|
│ ┌─────────────────┐
|
||||||
│ │ yao/assert │
|
│ │ yao/assert │
|
||||||
│ │ - 8 assertion │
|
│ │ - Asserter │
|
||||||
│ │ types │
|
│ │ - 8 types │
|
||||||
|
│ │ - Extensible │
|
||||||
│ └─────────────────┘
|
│ └─────────────────┘
|
||||||
▼
|
▼
|
||||||
┌─────────────────────────────────────────┐
|
┌─────────────────────────────────────────┐
|
||||||
│ Executor Types │
|
│ Executor Types │
|
||||||
│ - assistant: AI Agent (multi-turn) │
|
│ - ExecutorAssistant → Multi-turn AI │
|
||||||
│ - mcp: MCP Tool (single-call) │
|
│ - ExecutorMCP → Single-call MCP tool │
|
||||||
│ - process: Yao Process (single-call) │
|
│ - ExecutorProcess → Single-call Process │
|
||||||
└─────────────────────────────────────────┘
|
└─────────────────────────────────────────┘
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1775,3 +1775,121 @@ var (
|
||||||
ErrDeliveryFailed = errors.New("delivery failed")
|
ErrDeliveryFailed = errors.New("delivery failed")
|
||||||
)
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. P3 Implementation Details
|
||||||
|
|
||||||
|
### 5.1 Multi-Turn Conversation Flow
|
||||||
|
|
||||||
|
For assistant tasks, P3 uses a validator-driven multi-turn conversation:
|
||||||
|
|
||||||
|
```
|
||||||
|
┌──────────────────────────────────────────────────────────────┐
|
||||||
|
│ executeAssistantWithMultiTurn │
|
||||||
|
├──────────────────────────────────────────────────────────────┤
|
||||||
|
│ 1. Create Conversation (single instance for entire task) │
|
||||||
|
│ 2. Build initial messages with task context │
|
||||||
|
│ │
|
||||||
|
│ ┌─────────────────── Turn Loop ───────────────────────────┐ │
|
||||||
|
│ │ Phase 1: Call assistant via conv.Turn() │ │
|
||||||
|
│ │ Phase 2: ValidateWithContext() determines: │ │
|
||||||
|
│ │ - Complete: task done? │ │
|
||||||
|
│ │ - NeedReply: continue conversation? │ │
|
||||||
|
│ │ - ReplyContent: what to send next? │ │
|
||||||
|
│ │ Phase 3: If NeedReply, use ReplyContent as next input │ │
|
||||||
|
│ │ Break if: Complete && Passed, or !NeedReply │ │
|
||||||
|
│ └──────────────────────────────────────────────────────────┘ │
|
||||||
|
│ │
|
||||||
|
│ 3. Return output, validation, error │
|
||||||
|
└──────────────────────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
Key points:
|
||||||
|
- `ValidateWithContext()` returns `NeedReply` and `ReplyContent`
|
||||||
|
- Conversation continues until `Complete && Passed` or `!NeedReply`
|
||||||
|
- Max turns controlled by `RunConfig.MaxTurnsPerTask`
|
||||||
|
|
||||||
|
### 5.2 Validation Rules Format
|
||||||
|
|
||||||
|
Validation rules support two formats:
|
||||||
|
|
||||||
|
1. **Natural language**: `"output must be valid JSON"`, `"must contain 'field'"`
|
||||||
|
2. **Structured JSON**: `{"type": "type", "path": "field", "value": "array"}`
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
```json
|
||||||
|
// Natural language rules (converted to semantic validation)
|
||||||
|
"output must be valid JSON"
|
||||||
|
"must contain product name"
|
||||||
|
|
||||||
|
// Structured JSON assertions
|
||||||
|
{"type": "equals", "value": "success"}
|
||||||
|
{"type": "contains", "value": "total"}
|
||||||
|
{"type": "regex", "value": "^[A-Z].*"}
|
||||||
|
{"type": "json_path", "path": "data.items", "value": 10}
|
||||||
|
{"type": "type", "path": "result", "value": "object"}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.3 Task Dependencies
|
||||||
|
|
||||||
|
Task dependencies are handled automatically:
|
||||||
|
|
||||||
|
1. `BuildTaskContext()` collects previous task results
|
||||||
|
2. `FormatPreviousResultsAsContext()` formats them for assistant
|
||||||
|
|
||||||
|
```go
|
||||||
|
// Previous results are passed as context
|
||||||
|
func (r *Runner) BuildTaskContext(exec *robottypes.Execution, taskIndex int) *RunnerContext {
|
||||||
|
ctx := &RunnerContext{}
|
||||||
|
if taskIndex > 0 {
|
||||||
|
ctx.PreviousResults = exec.Results[:taskIndex]
|
||||||
|
}
|
||||||
|
return ctx
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.4 Resource Management
|
||||||
|
|
||||||
|
Agent context is properly released to prevent resource leaks:
|
||||||
|
|
||||||
|
```go
|
||||||
|
func (c *AgentCaller) Call(ctx *robottypes.Context, assistantID string, messages []agentcontext.Message) (*CallResult, error) {
|
||||||
|
agentCtx := c.buildAgentContext(ctx)
|
||||||
|
defer agentCtx.Release() // IMPORTANT: Release agent context
|
||||||
|
|
||||||
|
response, err := ast.Stream(agentCtx, messages, opts)
|
||||||
|
// ...
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.5 yao/assert Package
|
||||||
|
|
||||||
|
The `yao/assert` package is a standalone universal assertion library that can be used by other modules:
|
||||||
|
|
||||||
|
```go
|
||||||
|
import "github.com/yaoapp/yao/assert"
|
||||||
|
|
||||||
|
// Create asserter with optional callbacks
|
||||||
|
asserter := assert.NewAsserter(assert.AssertionOptions{
|
||||||
|
AgentValidator: myAgentValidator, // for "agent" type assertions
|
||||||
|
ScriptRunner: myScriptRunner, // for "script" type assertions
|
||||||
|
})
|
||||||
|
|
||||||
|
// Run assertions
|
||||||
|
results := asserter.Assert(output, []assert.Assertion{
|
||||||
|
{Type: "type", Value: "object"},
|
||||||
|
{Type: "contains", Value: "success"},
|
||||||
|
{Type: "json_path", Path: "data.count", Value: 10},
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
Supported assertion types:
|
||||||
|
- `equals` - exact match
|
||||||
|
- `contains` - substring check
|
||||||
|
- `not_contains` - negative substring check
|
||||||
|
- `json_path` - JSON path extraction and comparison
|
||||||
|
- `regex` - regex pattern matching
|
||||||
|
- `type` - type checking (with optional path)
|
||||||
|
- `script` - custom script validation
|
||||||
|
- `agent` - AI agent validation
|
||||||
|
|
|
||||||
|
|
@ -858,82 +858,6 @@ Created new `yao/assert` package for universal assertion/validation:
|
||||||
**TODO (Future):**
|
**TODO (Future):**
|
||||||
- [ ] Test: ContinueOnFailure option (run_test.go)
|
- [ ] Test: ContinueOnFailure option (run_test.go)
|
||||||
|
|
||||||
### 9.4 Architecture
|
|
||||||
|
|
||||||
```
|
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
|
||||||
│ run.go (P3 Entry) │
|
|
||||||
│ - RunConfig: ContinueOnFailure, ValidationThreshold, │
|
|
||||||
│ MaxTurnsPerTask │
|
|
||||||
│ - RunExecution: main loop with task dependency passing │
|
|
||||||
└─────────────────────┬───────────────────────────────────────┘
|
|
||||||
│
|
|
||||||
┌────────────┴────────────┐
|
|
||||||
▼ ▼
|
|
||||||
┌─────────────────┐ ┌─────────────────┐
|
|
||||||
│ runner.go │ │ validator.go │
|
|
||||||
│ - Runner │ │ - Validator │
|
|
||||||
│ - Multi-turn │ │ - Two-layer │
|
|
||||||
│ conversation │ │ - Rule+Semantic│
|
|
||||||
│ - Task context │ │ - NeedReply │
|
|
||||||
│ building │ │ - ReplyContent │
|
|
||||||
└────────┬────────┘ └────────┬────────┘
|
|
||||||
│ │
|
|
||||||
│ ▼
|
|
||||||
│ ┌─────────────────┐
|
|
||||||
│ │ yao/assert │
|
|
||||||
│ │ - Asserter │
|
|
||||||
│ │ - 8 types │
|
|
||||||
│ │ - Extensible │
|
|
||||||
│ └─────────────────┘
|
|
||||||
▼
|
|
||||||
┌─────────────────────────────────────────┐
|
|
||||||
│ Executor Types │
|
|
||||||
│ - ExecutorAssistant → Multi-turn AI │
|
|
||||||
│ - ExecutorMCP → Single-call MCP tool │
|
|
||||||
│ - ExecutorProcess → Single-call Process │
|
|
||||||
└─────────────────────────────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
**Multi-Turn Conversation Flow (Assistant Tasks):**
|
|
||||||
|
|
||||||
```
|
|
||||||
┌──────────────────────────────────────────────────────────────┐
|
|
||||||
│ executeAssistantWithMultiTurn │
|
|
||||||
├──────────────────────────────────────────────────────────────┤
|
|
||||||
│ 1. Create Conversation (single instance for entire task) │
|
|
||||||
│ 2. Build initial messages with task context │
|
|
||||||
│ │
|
|
||||||
│ ┌─────────────────── Turn Loop ───────────────────────────┐ │
|
|
||||||
│ │ Phase 1: Call assistant via conv.Turn() │ │
|
|
||||||
│ │ Phase 2: ValidateWithContext() determines: │ │
|
|
||||||
│ │ - Complete: task done? │ │
|
|
||||||
│ │ - NeedReply: continue conversation? │ │
|
|
||||||
│ │ - ReplyContent: what to send next? │ │
|
|
||||||
│ │ Phase 3: If NeedReply, use ReplyContent as next input │ │
|
|
||||||
│ │ Break if: Complete && Passed, or !NeedReply │ │
|
|
||||||
│ └──────────────────────────────────────────────────────────┘ │
|
|
||||||
│ │
|
|
||||||
│ 3. Return output, validation, error │
|
|
||||||
└──────────────────────────────────────────────────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
### 9.5 Notes
|
|
||||||
|
|
||||||
- Validation rules support two formats:
|
|
||||||
1. Natural language: `"output must be valid JSON"`, `"must contain 'field'"`
|
|
||||||
2. Structured JSON: `{"type": "type", "path": "field", "value": "array"}`
|
|
||||||
- Multi-turn conversation is validator-driven:
|
|
||||||
- `ValidateWithContext()` returns `NeedReply` and `ReplyContent`
|
|
||||||
- Conversation continues until `Complete && Passed` or `!NeedReply`
|
|
||||||
- Max turns controlled by `RunConfig.MaxTurnsPerTask`
|
|
||||||
- Task dependencies handled automatically:
|
|
||||||
- `BuildTaskContext()` collects previous task results
|
|
||||||
- `FormatPreviousResultsAsContext()` formats them for assistant
|
|
||||||
- MCP and Process tasks use single-call execution (no multi-turn)
|
|
||||||
- `yao/assert` is a standalone package, can be used by other modules
|
|
||||||
- Agent context is properly released via `defer agentCtx.Release()` in `AgentCaller.Call()`
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Phase 10: P4 Delivery Implementation
|
## Phase 10: P4 Delivery Implementation
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue