Update DESIGN.md, TECHNICAL.md, and TODO.md for P3 Implementation Enhancements
- Revised DESIGN.md to clarify the architecture of the P3 entry point, including updated RunConfig parameters and task execution flow. - Expanded TECHNICAL.md with detailed implementation notes on multi-turn conversation flow, validation rules format, task dependencies, and resource management. - Removed outdated architecture diagrams from TODO.md and added comprehensive notes on the new multi-turn conversation handling and validation mechanisms. - Documented the functionality of the new `yao/assert` package and its integration into the validation process.
This commit is contained in:
Max
parent
1dd1785dc0
commit
9cc5be78ef
3 changed files with 131 additions and 86 deletions
|
|
@ -372,8 +372,9 @@ type Task struct {
|
|||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ run.go (P3 Entry) │
|
||||
│ - RunConfig: threshold, continue-on-failure, max-turns │
|
||||
│ - RunExecution: main execution loop │
|
||||
│ - RunConfig: ContinueOnFailure, ValidationThreshold, │
|
||||
│ MaxTurnsPerTask │
|
||||
│ - RunExecution: main loop with task dependency passing │
|
||||
└─────────────────────┬───────────────────────────────────────┘
|
||||
│
|
||||
┌────────────┴────────────┐
|
||||
|
|
@ -381,23 +382,25 @@ type Task struct {
|
|||
┌─────────────────┐ ┌─────────────────┐
|
||||
│ runner.go │ │ validator.go │
|
||||
│ - Runner │ │ - Validator │
|
||||
│ - Task exec │ │ - Two-layer │
|
||||
│ - Multi-turn │ │ - Rule+Semantic│
|
||||
│ conversation │ │ - NeedReply │
|
||||
│ - Multi-turn │ │ - Two-layer │
|
||||
│ conversation │ │ - Rule+Semantic│
|
||||
│ - Task context │ │ - NeedReply │
|
||||
│ building │ │ - ReplyContent │
|
||||
└────────┬────────┘ └────────┬────────┘
|
||||
│ │
|
||||
│ ▼
|
||||
│ ┌─────────────────┐
|
||||
│ │ yao/assert │
|
||||
│ │ - 8 assertion │
|
||||
│ │ types │
|
||||
│ │ - Asserter │
|
||||
│ │ - 8 types │
|
||||
│ │ - Extensible │
|
||||
│ └─────────────────┘
|
||||
▼
|
||||
┌─────────────────────────────────────────┐
|
||||
│ Executor Types │
|
||||
│ - assistant: AI Agent (multi-turn) │
|
||||
│ - mcp: MCP Tool (single-call) │
|
||||
│ - process: Yao Process (single-call) │
|
||||
│ - ExecutorAssistant → Multi-turn AI │
|
||||
│ - ExecutorMCP → Single-call MCP tool │
|
||||
│ - ExecutorProcess → Single-call Process │
|
||||
└─────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
|
|
|
|||
|
|
@ -1775,3 +1775,121 @@ var (
|
|||
ErrDeliveryFailed = errors.New("delivery failed")
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. P3 Implementation Details
|
||||
|
||||
### 5.1 Multi-Turn Conversation Flow
|
||||
|
||||
For assistant tasks, P3 uses a validator-driven multi-turn conversation:
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ executeAssistantWithMultiTurn │
|
||||
├──────────────────────────────────────────────────────────────┤
|
||||
│ 1. Create Conversation (single instance for entire task) │
|
||||
│ 2. Build initial messages with task context │
|
||||
│ │
|
||||
│ ┌─────────────────── Turn Loop ───────────────────────────┐ │
|
||||
│ │ Phase 1: Call assistant via conv.Turn() │ │
|
||||
│ │ Phase 2: ValidateWithContext() determines: │ │
|
||||
│ │ - Complete: task done? │ │
|
||||
│ │ - NeedReply: continue conversation? │ │
|
||||
│ │ - ReplyContent: what to send next? │ │
|
||||
│ │ Phase 3: If NeedReply, use ReplyContent as next input │ │
|
||||
│ │ Break if: Complete && Passed, or !NeedReply │ │
|
||||
│ └──────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 3. Return output, validation, error │
|
||||
└──────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
Key points:
|
||||
- `ValidateWithContext()` returns `NeedReply` and `ReplyContent`
|
||||
- Conversation continues until `Complete && Passed` or `!NeedReply`
|
||||
- Max turns controlled by `RunConfig.MaxTurnsPerTask`
|
||||
|
||||
### 5.2 Validation Rules Format
|
||||
|
||||
Validation rules support two formats:
|
||||
|
||||
1. **Natural language**: `"output must be valid JSON"`, `"must contain 'field'"`
|
||||
2. **Structured JSON**: `{"type": "type", "path": "field", "value": "array"}`
|
||||
|
||||
Examples:
|
||||
```json
|
||||
// Natural language rules (converted to semantic validation)
|
||||
"output must be valid JSON"
|
||||
"must contain product name"
|
||||
|
||||
// Structured JSON assertions
|
||||
{"type": "equals", "value": "success"}
|
||||
{"type": "contains", "value": "total"}
|
||||
{"type": "regex", "value": "^[A-Z].*"}
|
||||
{"type": "json_path", "path": "data.items", "value": 10}
|
||||
{"type": "type", "path": "result", "value": "object"}
|
||||
```
|
||||
|
||||
### 5.3 Task Dependencies
|
||||
|
||||
Task dependencies are handled automatically:
|
||||
|
||||
1. `BuildTaskContext()` collects previous task results
|
||||
2. `FormatPreviousResultsAsContext()` formats them for assistant
|
||||
|
||||
```go
|
||||
// Previous results are passed as context
|
||||
func (r *Runner) BuildTaskContext(exec *robottypes.Execution, taskIndex int) *RunnerContext {
|
||||
ctx := &RunnerContext{}
|
||||
if taskIndex > 0 {
|
||||
ctx.PreviousResults = exec.Results[:taskIndex]
|
||||
}
|
||||
return ctx
|
||||
}
|
||||
```
|
||||
|
||||
### 5.4 Resource Management
|
||||
|
||||
Agent context is properly released to prevent resource leaks:
|
||||
|
||||
```go
|
||||
func (c *AgentCaller) Call(ctx *robottypes.Context, assistantID string, messages []agentcontext.Message) (*CallResult, error) {
|
||||
agentCtx := c.buildAgentContext(ctx)
|
||||
defer agentCtx.Release() // IMPORTANT: Release agent context
|
||||
|
||||
response, err := ast.Stream(agentCtx, messages, opts)
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
### 5.5 yao/assert Package
|
||||
|
||||
The `yao/assert` package is a standalone universal assertion library that can be used by other modules:
|
||||
|
||||
```go
|
||||
import "github.com/yaoapp/yao/assert"
|
||||
|
||||
// Create asserter with optional callbacks
|
||||
asserter := assert.NewAsserter(assert.AssertionOptions{
|
||||
AgentValidator: myAgentValidator, // for "agent" type assertions
|
||||
ScriptRunner: myScriptRunner, // for "script" type assertions
|
||||
})
|
||||
|
||||
// Run assertions
|
||||
results := asserter.Assert(output, []assert.Assertion{
|
||||
{Type: "type", Value: "object"},
|
||||
{Type: "contains", Value: "success"},
|
||||
{Type: "json_path", Path: "data.count", Value: 10},
|
||||
})
|
||||
```
|
||||
|
||||
Supported assertion types:
|
||||
- `equals` - exact match
|
||||
- `contains` - substring check
|
||||
- `not_contains` - negative substring check
|
||||
- `json_path` - JSON path extraction and comparison
|
||||
- `regex` - regex pattern matching
|
||||
- `type` - type checking (with optional path)
|
||||
- `script` - custom script validation
|
||||
- `agent` - AI agent validation
|
||||
|
|
|
|||
|
|
@ -858,82 +858,6 @@ Created new `yao/assert` package for universal assertion/validation:
|
|||
**TODO (Future):**
|
||||
- [ ] Test: ContinueOnFailure option (run_test.go)
|
||||
|
||||
### 9.4 Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ run.go (P3 Entry) │
|
||||
│ - RunConfig: ContinueOnFailure, ValidationThreshold, │
|
||||
│ MaxTurnsPerTask │
|
||||
│ - RunExecution: main loop with task dependency passing │
|
||||
└─────────────────────┬───────────────────────────────────────┘
|
||||
│
|
||||
┌────────────┴────────────┐
|
||||
▼ ▼
|
||||
┌─────────────────┐ ┌─────────────────┐
|
||||
│ runner.go │ │ validator.go │
|
||||
│ - Runner │ │ - Validator │
|
||||
│ - Multi-turn │ │ - Two-layer │
|
||||
│ conversation │ │ - Rule+Semantic│
|
||||
│ - Task context │ │ - NeedReply │
|
||||
│ building │ │ - ReplyContent │
|
||||
└────────┬────────┘ └────────┬────────┘
|
||||
│ │
|
||||
│ ▼
|
||||
│ ┌─────────────────┐
|
||||
│ │ yao/assert │
|
||||
│ │ - Asserter │
|
||||
│ │ - 8 types │
|
||||
│ │ - Extensible │
|
||||
│ └─────────────────┘
|
||||
▼
|
||||
┌─────────────────────────────────────────┐
|
||||
│ Executor Types │
|
||||
│ - ExecutorAssistant → Multi-turn AI │
|
||||
│ - ExecutorMCP → Single-call MCP tool │
|
||||
│ - ExecutorProcess → Single-call Process │
|
||||
└─────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Multi-Turn Conversation Flow (Assistant Tasks):**
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ executeAssistantWithMultiTurn │
|
||||
├──────────────────────────────────────────────────────────────┤
|
||||
│ 1. Create Conversation (single instance for entire task) │
|
||||
│ 2. Build initial messages with task context │
|
||||
│ │
|
||||
│ ┌─────────────────── Turn Loop ───────────────────────────┐ │
|
||||
│ │ Phase 1: Call assistant via conv.Turn() │ │
|
||||
│ │ Phase 2: ValidateWithContext() determines: │ │
|
||||
│ │ - Complete: task done? │ │
|
||||
│ │ - NeedReply: continue conversation? │ │
|
||||
│ │ - ReplyContent: what to send next? │ │
|
||||
│ │ Phase 3: If NeedReply, use ReplyContent as next input │ │
|
||||
│ │ Break if: Complete && Passed, or !NeedReply │ │
|
||||
│ └──────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 3. Return output, validation, error │
|
||||
└──────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### 9.5 Notes
|
||||
|
||||
- Validation rules support two formats:
|
||||
1. Natural language: `"output must be valid JSON"`, `"must contain 'field'"`
|
||||
2. Structured JSON: `{"type": "type", "path": "field", "value": "array"}`
|
||||
- Multi-turn conversation is validator-driven:
|
||||
- `ValidateWithContext()` returns `NeedReply` and `ReplyContent`
|
||||
- Conversation continues until `Complete && Passed` or `!NeedReply`
|
||||
- Max turns controlled by `RunConfig.MaxTurnsPerTask`
|
||||
- Task dependencies handled automatically:
|
||||
- `BuildTaskContext()` collects previous task results
|
||||
- `FormatPreviousResultsAsContext()` formats them for assistant
|
||||
- MCP and Process tasks use single-call execution (no multi-turn)
|
||||
- `yao/assert` is a standalone package, can be used by other modules
|
||||
- Agent context is properly released via `defer agentCtx.Release()` in `AgentCaller.Call()`
|
||||
|
||||
---
|
||||
|
||||
## Phase 10: P4 Delivery Implementation
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue