diff --git a/agent/README.md b/agent/README.md index 4825416f..588670fe 100644 --- a/agent/README.md +++ b/agent/README.md @@ -1,597 +1,149 @@ -# Agent API Documentation +# Yao Agent -Agent is a chat/AI assistant API that provides endpoints for managing conversations, assistants, file uploads, and more. +A powerful AI assistant framework for building intelligent conversational agents with tool integration, knowledge base search, and multi-agent orchestration. -## Base URL +## Quick Start -All endpoints are relative to your base URL + `/api/__yao/agent` +### 1. Create an Assistant -Example: `http://localhost:5099/api/__yao/agent` +``` +assistants/ +└── my-assistant/ + ├── package.yao # Configuration + ├── prompts.yml # System prompts + └── locales/ + └── en-us.yml # Translations +``` -## Authentication +**package.yao** -All endpoints require a `token` parameter for authentication. The token can be provided as: +```json +{ + "name": "{{ name }}", + "connector": "gpt-4o", + "description": "{{ description }}", + "placeholder": { + "title": "{{ chat.title }}", + "prompts": ["{{ chat.prompts.0 }}"] + } +} +``` -- Query parameter: `?token=your_token_here` -- Authorization header: `Authorization: Bearer your_token_here` +**prompts.yml** -## CORS Support +```yaml +- role: system + content: | + You are a helpful assistant. +``` -The API supports Cross-Origin Resource Sharing (CORS) and handles preflight OPTIONS requests. +**locales/en-us.yml** + +```yaml +name: My Assistant +description: A helpful AI assistant +chat: + title: New Chat + prompts: + - How can I help you today? +``` + +### 2. Add Hooks (Optional) + +Create `src/index.ts` for custom logic: + +```typescript +import { agent } from "@yao/runtime"; + +function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create { + // Preprocess messages before LLM call + return { messages }; +} + +function Next(ctx: agent.Context, payload: agent.Payload): agent.Next { + // Post-process LLM response + return null; +} +``` + +### 3. Test (Optional) + +```bash +yao agent test -i "Hello, how are you?" +``` + +### 4. Run + +```bash +yao start +``` + +Access via API: `POST /api/__yao/agent` + +## Documentation + +- [Configuration](docs/configuration.md) - Assistant settings, connectors, options +- [Prompts](docs/prompts.md) - System prompts and prompt presets +- [Hooks](docs/hooks.md) - Create/Next hooks and agent lifecycle +- [Context API](docs/context-api.md) - Messaging, memory, trace, MCP +- [MCP Integration](docs/mcp.md) - Tool servers and resources +- [Search](docs/search.md) - Web, knowledge base, and database search +- [Internationalization](docs/i18n.md) - Multi-language support +- [Testing](docs/testing.md) - Agent testing framework + +## Architecture + +```mermaid +flowchart LR + subgraph Request + A[User Request] + end + + subgraph Create["Create Hook"] + B1[Preprocess Messages] + B2[Configure LLM] + B3[Delegate to Agent] + end + + subgraph LLM["LLM Call"] + C1[Load Prompts] + C2[Generate Response] + end + + subgraph Tools["Tool Execution"] + D1[MCP Tools] + D2[Search] + D3[Memory] + end + + subgraph Next["Next Hook"] + E1[Process Results] + E2[Transform Output] + E3[Delegate to Agent] + end + + subgraph Response + F[Stream Response] + end + + A --> Create + Create --> LLM + LLM --> Tools + Tools --> Next + Next --> Response + Next -.->|Continue| LLM +``` ## API Endpoints -### 1. Chat Endpoints +| Endpoint | Method | Description | +| ------------------------------- | ------ | ------------------- | +| `/api/__yao/agent` | POST | Chat with assistant | +| `/api/__yao/agent/history` | GET | Get chat history | +| `/api/__yao/agent/chats` | GET | List chat sessions | +| `/api/__yao/agent/assistants` | GET | List assistants | +| `/api/__yao/agent/upload/:type` | POST | Upload files | -#### 1.1 Chat with AI +## License -Start or continue a conversation with an AI assistant. - -**Endpoints:** - -- `GET /` - Chat via query parameters -- `POST /` - Chat via JSON body - -**Parameters:** - -- `content` (required) - The message content -- `chat_id` (optional) - Chat session ID. If not provided, a new one will be generated -- `context` (optional) - Additional context for the conversation -- `assistant_id` (optional) - Specific assistant to use -- `silent` (optional) - Silent mode: `true` or `1` -- `history_visible` (optional) - Show history: `true` or `1` -- `client_type` (optional) - Client type identifier - -**Examples:** - -```bash -# GET request -curl -X GET 'http://localhost:5099/api/__yao/agent?content=Hello&chat_id=chat_123&token=xxx' - -# POST request -curl -X POST 'http://localhost:5099/api/__yao/agent' \ - -H 'Content-Type: application/json' \ - -d '{"content": "Hello", "chat_id": "chat_123", "token": "xxx"}' -``` - -**Response:** -Server-Sent Events (SSE) stream with chat messages. - -#### 1.2 Chat History - -Get conversation history for a specific chat. - -**Endpoint:** `GET /history` - -**Parameters:** - -- `chat_id` (required) - Chat session ID - -**Example:** - -```bash -curl -X GET 'http://localhost:5099/api/__yao/agent/history?chat_id=chat_123&token=xxx' -``` - -**Response:** - -```json -{ - "data": [ - { - "role": "user", - "content": "Hello", - "timestamp": "2024-01-01T00:00:00Z" - }, - { - "role": "assistant", - "content": "Hi there!", - "timestamp": "2024-01-01T00:00:01Z" - } - ] -} -``` - -### 2. Chat Management - -#### 2.1 List Chats - -Get a paginated list of chat conversations. - -**Endpoint:** `GET /chats` - -**Parameters:** - -- `page` (optional) - Page number (default: 1) -- `pagesize` (optional) - Items per page (default: 20) -- `keywords` (optional) - Search keywords -- `order` (optional) - Sort order (`asc` or `desc`) -- `locale` (optional) - Locale code (default: `en-us`) - -**Example:** - -```bash -curl -X GET 'http://localhost:5099/api/__yao/agent/chats?page=1&pagesize=20&keywords=search+term&order=desc&token=xxx' -``` - -**Response:** - -```json -{ - "data": { - "groups": [ - { - "date": "2024-01-01", - "chats": [ - { - "chat_id": "chat_123", - "title": "Chat Title", - "updated_at": "2024-01-01T00:00:00Z" - } - ] - } - ], - "total": 50, - "page": 1, - "pagesize": 20 - } -} -``` - -#### 2.2 Get Latest Chat - -Get the most recent chat or create a new one if none exists. - -**Endpoint:** `GET /chats/latest` - -**Parameters:** - -- `assistant_id` (optional) - Preferred assistant ID -- `locale` (optional) - Locale code (default: `en-us`) - -**Example:** - -```bash -curl -X GET 'http://localhost:5099/api/__yao/agent/chats/latest?assistant_id=assistant_123&token=xxx' -``` - -#### 2.3 Get Chat Details - -Get detailed information about a specific chat. - -**Endpoint:** `GET /chats/:id` - -**Parameters:** - -- `locale` (optional) - Locale code (default: `en-us`) - -**Example:** - -```bash -curl -X GET 'http://localhost:5099/api/__yao/agent/chats/chat_123?token=xxx' -``` - -#### 2.4 Update Chat - -Update chat metadata (e.g., title). - -**Endpoint:** `POST /chats/:id` - -**Body:** - -```json -{ - "title": "New Title", - "content": "Chat content for title generation" -} -``` - -**Example:** - -```bash -curl -X POST 'http://localhost:5099/api/__yao/agent/chats/chat_123' \ - -H 'Content-Type: application/json' \ - -d '{"title": "New Title", "content": "Chat content", "token": "xxx"}' -``` - -#### 2.5 Delete Chat - -Delete a specific chat conversation. - -**Endpoint:** `DELETE /chats/:id` - -**Example:** - -```bash -curl -X DELETE 'http://localhost:5099/api/__yao/agent/chats/chat_123?token=xxx' -``` - -### 3. Assistant Management - -#### 3.1 List Assistants - -Get a paginated list of available assistants. - -**Endpoint:** `GET /assistants` - -**Parameters:** - -- `page` (optional) - Page number (default: 1) -- `pagesize` (optional) - Items per page (default: 20) -- `tags` (optional) - Comma-separated list of tags -- `keywords` (optional) - Search keywords -- `connector` (optional) - Connector name filter -- `select` (optional) - Comma-separated fields to select -- `built_in` (optional) - Filter built-in assistants (`true`/`false`/`1`/`0`) -- `mentionable` (optional) - Filter mentionable assistants (`true`/`false`/`1`/`0`) -- `automated` (optional) - Filter automated assistants (`true`/`false`/`1`/`0`) -- `assistant_id` (optional) - Specific assistant ID -- `locale` (optional) - Locale code (default: `en-us`) - -**Example:** - -```bash -curl -X GET 'http://localhost:5099/api/__yao/agent/assistants?page=1&pagesize=20&tags=tag1,tag2&token=xxx' -``` - -#### 3.2 Get Assistant Tags - -Get all available assistant tags. - -**Endpoint:** `GET /assistants/tags` - -**Parameters:** - -- `locale` (optional) - Locale code (default: `en-us`) - -**Example:** - -```bash -curl -X GET 'http://localhost:5099/api/__yao/agent/assistants/tags?token=xxx' -``` - -#### 3.3 Get Assistant Details - -Get detailed information about a specific assistant. - -**Endpoint:** `GET /assistants/:id` - -**Parameters:** - -- `locale` (optional) - Locale code (default: `en-us`) - -**Example:** - -```bash -curl -X GET 'http://localhost:5099/api/__yao/agent/assistants/assistant_123?token=xxx' -``` - -#### 3.4 Execute Assistant API - -Call a specific assistant's API functionality. - -**Endpoint:** `POST /assistants/:id/call` - -**Body:** - -```json -{ - "name": "Test", - "payload": { - "name": "yao", - "age": 18 - } -} -``` - -**Example:** - -```bash -curl -X POST 'http://localhost:5099/api/__yao/agent/assistants/assistant_123/call' \ - -H 'Content-Type: application/json' \ - -d '{"name": "Test", "payload": {"name": "yao", "age": 18}}' -``` - -#### 3.5 Create/Update Assistant - -Create a new assistant or update an existing one. - -**Endpoint:** `POST /assistants` - -**Body:** - -```json -{ - "name": "My Assistant", - "type": "chat", - "tags": ["tag1", "tag2"], - "mentionable": true, - "avatar": "path/to/avatar.png" -} -``` - -**Example:** - -```bash -curl -X POST 'http://localhost:5099/api/__yao/agent/assistants' \ - -H 'Content-Type: application/json' \ - -d '{"name": "My Assistant", "type": "chat", "tags": ["tag1"], "token": "xxx"}' -``` - -#### 3.6 Delete Assistant - -Delete a specific assistant. - -**Endpoint:** `DELETE /assistants/:id` - -**Example:** - -```bash -curl -X DELETE 'http://localhost:5099/api/__yao/agent/assistants/assistant_123?token=xxx' -``` - -### 4. File Management - -#### 4.1 Upload File - -Upload files to different storage types. - -**Endpoint:** `POST /upload/:storage` - -**Storage Types:** - -- `chat` - Chat-related files -- `knowledge` - Knowledge base files -- `assets` - General assets - -**Form Data:** - -- `file` (required) - The file to upload -- `chat_id` (required for chat storage) - Chat session ID -- `collection_id` (required for knowledge storage) - Knowledge collection ID -- `public` (optional) - Make file public -- `gzip` (optional) - Enable gzip compression - -**Example:** - -```bash -curl -X POST 'http://localhost:5099/api/__yao/agent/upload/chat?chat_id=chat_123&token=xxx' \ - -F 'file=@/path/to/file.txt' -``` - -**Response:** - -```json -{ - "data": { - "id": "file_123", - "content_type": "text/plain", - "bytes": 1024, - "status": "uploaded" - } -} -``` - -#### 4.2 Download File - -Download a previously uploaded file. - -**Endpoint:** `GET /download` - -**Parameters:** - -- `file_id` (required) - File ID to download -- `disposition` (optional) - Content disposition (default: `attachment`) - -**Example:** - -```bash -curl -X GET 'http://localhost:5099/api/__yao/agent/download?file_id=file_123&disposition=attachment&token=xxx' \ - -o downloaded_file.txt -``` - -### 5. Mentions - -#### 5.1 Get Mentions - -Get mentionable assistants for autocomplete. - -**Endpoint:** `GET /mentions` - -**Parameters:** - -- `keywords` (optional) - Search keywords -- `locale` (optional) - Locale code (default: `en-us`) - -**Example:** - -```bash -curl -X GET 'http://localhost:5099/api/__yao/agent/mentions?keywords=assistant&token=xxx' -``` - -**Response:** - -```json -{ - "data": [ - { - "id": "assistant_123", - "name": "Assistant Name", - "type": "chat", - "avatar": "avatar_url" - } - ] -} -``` - -### 6. Generation Endpoints - -#### 6.1 Generate Title - -Generate a title for chat content. - -**Endpoints:** - -- `GET /generate/title` - Generate via query parameters -- `POST /generate/title` - Generate via JSON body - -**Parameters:** - -- `content` (required) - Content to generate title for -- `chat_id` (optional) - Associated chat ID -- `context` (optional) - Additional context - -**Examples:** - -```bash -# GET request -curl -X GET 'http://localhost:5099/api/__yao/agent/generate/title?content=Chat+content&chat_id=chat_123&token=xxx' - -# POST request -curl -X POST 'http://localhost:5099/api/__yao/agent/generate/title' \ - -H 'Content-Type: application/json' \ - -d '{"content": "Chat content", "chat_id": "chat_123", "token": "xxx"}' -``` - -**Response:** SSE stream with generated title - -#### 6.2 Generate Prompts - -Generate prompts based on content. - -**Endpoints:** - -- `GET /generate/prompts` - Generate via query parameters -- `POST /generate/prompts` - Generate via JSON body - -**Parameters:** - -- `content` (required) - Content to generate prompts for -- `chat_id` (optional) - Associated chat ID -- `context` (optional) - Additional context - -**Examples:** - -```bash -# GET request -curl -X GET 'http://localhost:5099/api/__yao/agent/generate/prompts?content=Generate+prompts&chat_id=chat_123&token=xxx' - -# POST request -curl -X POST 'http://localhost:5099/api/__yao/agent/generate/prompts' \ - -H 'Content-Type: application/json' \ - -d '{"content": "Generate prompts", "chat_id": "chat_123", "token": "xxx"}' -``` - -**Response:** SSE stream with generated prompts - -### 7. Utility Endpoints - -#### 7.1 List Connectors - -Get available AI connectors. - -**Endpoint:** `GET /utility/connectors` - -**Example:** - -```bash -curl -X GET 'http://localhost:5099/api/__yao/agent/utility/connectors?token=xxx' -``` - -**Response:** - -```json -{ - "data": [ - { - "label": "OpenAI", - "value": "openai" - }, - { - "label": "Custom API", - "value": "custom_api" - } - ] -} -``` - -#### 7.2 Status Check - -Check API service status. - -**Endpoint:** `GET /status` - -**Example:** - -```bash -curl -X GET 'http://localhost:5099/api/__yao/agent/status?token=xxx' -``` - -**Response:** HTTP 200 status code - -### 8. Dangerous Operations - -#### 8.1 Clear All Chats - -Delete all chat conversations for the authenticated user. - -**Endpoint:** `DELETE /dangerous/clear_chats` - -**Example:** - -```bash -curl -X DELETE 'http://localhost:5099/api/__yao/agent/dangerous/clear_chats?token=xxx' -``` - -**Response:** - -```json -{ - "message": "ok" -} -``` - -## Error Responses - -All endpoints return JSON error responses in the following format: - -```json -{ - "message": "Error description", - "code": 400 -} -``` - -Common error codes: - -- `400` - Bad Request (missing or invalid parameters) -- `401` - Unauthorized (invalid or missing token) -- `403` - Forbidden (access denied) -- `404` - Not Found (resource not found) -- `500` - Internal Server Error - -## Server-Sent Events (SSE) - -Chat and generation endpoints return Server-Sent Events for real-time streaming: - -**Headers:** - -- `Content-Type: text/event-stream;charset=utf-8` -- `Cache-Control: no-cache` -- `Connection: keep-alive` - -**Event Format:** - -``` -data: {"type": "message", "content": "Hello"} - -data: {"type": "done"} -``` - -## Rate Limiting - -Rate limiting may be applied based on your authentication token and usage patterns. - -## Support - -For support and questions, please refer to the Yao App Engine documentation. +This project is part of the Yao App Engine and follows the [Yao Open Source License](../LICENSE). diff --git a/agent/docs/configuration.md b/agent/docs/configuration.md new file mode 100644 index 00000000..e3d798f5 --- /dev/null +++ b/agent/docs/configuration.md @@ -0,0 +1,243 @@ +# Assistant Configuration + +## Directory Structure + +``` +assistants/ +└── / + ├── package.yao # Required: Configuration + ├── prompts.yml # Optional: Default prompts + ├── prompts/ # Optional: Prompt presets + │ ├── chat.yml + │ └── task.yml + ├── locales/ # Optional: Translations + │ ├── en-us.yml + │ └── zh-cn.yml + ├── src/ # Optional: Hook scripts + │ └── index.ts + └── mcps/ # Optional: MCP servers + └── tools.mcp.yao +``` + +## package.yao + +### Basic Fields + +```json +{ + "name": "{{ name }}", + "type": "assistant", + "avatar": "/assets/avatar.png", + "description": "{{ description }}", + "connector": "gpt-4o", + "tags": ["Category1", "Category2"], + "sort": 1 +} +``` + +| Field | Type | Description | +| ------------- | -------- | ------------------------------------ | +| `name` | string | Display name (supports i18n `{{ }}`) | +| `type` | string | Type: `assistant` (default) | +| `avatar` | string | Avatar image path | +| `description` | string | Description (supports i18n) | +| `connector` | string | LLM connector ID | +| `tags` | string[] | Categorization tags | +| `sort` | number | Display order | + +### Connector Options + +```json +{ + "connector": "gpt-4o", + "connector_options": { + "optional": true, + "connectors": ["gpt-4o", "gpt-4o-mini", "claude-3"], + "filters": ["tool_calls", "vision"] + } +} +``` + +| Field | Type | Description | +| ------------ | -------- | -------------------------------------------- | +| `optional` | boolean | Allow user to select connector | +| `connectors` | string[] | Available connectors (empty = all) | +| `filters` | string[] | Required capabilities: `vision`, `audio`, `tool_calls`, `reasoning` | + +### Generation Options + +```json +{ + "options": { + "temperature": 0.7, + "max_tokens": 4096 + } +} +``` + +### Placeholder (UI Hints) + +```json +{ + "placeholder": { + "title": "{{ chat.title }}", + "description": "{{ chat.description }}", + "prompts": [ + "{{ chat.prompts.0 }}", + "{{ chat.prompts.1 }}" + ] + } +} +``` + +### Visibility & Access + +```json +{ + "public": true, + "share": "team", + "readonly": true, + "built_in": true, + "mentionable": true, + "automated": false +} +``` + +| Field | Type | Description | +| ------------- | ------- | --------------------------------- | +| `public` | boolean | Visible to all users | +| `share` | string | Sharing scope: `private`, `team` | +| `readonly` | boolean | Prevent user modifications | +| `built_in` | boolean | System-managed assistant | +| `mentionable` | boolean | Can be @mentioned in chat | +| `automated` | boolean | Can be triggered automatically | + +### Modes + +```json +{ + "modes": ["chat", "task"], + "default_mode": "task" +} +``` + +### MCP Servers + +```json +{ + "mcp": { + "servers": [ + "server-id", + { "server_id": "tools", "tools": ["tool1", "tool2"] }, + { "server_id": "resources", "resources": ["uri://pattern"] } + ] + } +} +``` + +### Knowledge Base + +```json +{ + "kb": { + "collections": ["collection-id-1", "collection-id-2"] + } +} +``` + +### Database Models + +```json +{ + "db": { + "models": ["model.name", "another.model"] + } +} +``` + +### Uses (Wrapper Tools) + +```json +{ + "uses": { + "vision": "vision-agent", + "audio": "audio-agent", + "search": "disabled", + "fetch": "mcp:fetcher" + } +} +``` + +| Field | Description | +| -------- | -------------------------------------------------- | +| `vision` | Vision processing: `` or `mcp:` | +| `audio` | Audio processing: `` or `mcp:` | +| `search` | Search: `disabled`, ``, or `mcp:`| +| `fetch` | HTTP fetching: `` or `mcp:` | + +### Search Configuration + +```json +{ + "search": { + "web": { + "provider": "tavily", + "max_results": 10 + }, + "kb": { + "threshold": 0.7, + "graph": true + }, + "db": { + "max_results": 20 + }, + "citation": { + "format": "[{index}]", + "auto_inject_prompt": true + } + } +} +``` + +## Environment Variables + +Use `$ENV.VAR_NAME` for sensitive values: + +```json +{ + "connector": "$ENV.LLM_CONNECTOR" +} +``` + +## Complete Example + +```json +{ + "name": "{{ name }}", + "type": "assistant", + "avatar": "/assets/assistant.png", + "connector": "gpt-4o", + "connector_options": { + "optional": true, + "connectors": ["gpt-4o", "gpt-4o-mini"], + "filters": ["tool_calls"] + }, + "mcp": { + "servers": [{ "server_id": "tools", "tools": ["search", "calculate"] }] + }, + "description": "{{ description }}", + "options": { "temperature": 0.7 }, + "public": true, + "placeholder": { + "title": "{{ chat.title }}", + "description": "{{ chat.description }}", + "prompts": ["{{ chat.prompts.0 }}", "{{ chat.prompts.1 }}"] + }, + "tags": ["Productivity"], + "modes": ["chat", "task"], + "default_mode": "chat", + "sort": 1, + "readonly": true, + "mentionable": true +} +``` diff --git a/agent/docs/context-api.md b/agent/docs/context-api.md new file mode 100644 index 00000000..fb2a9a95 --- /dev/null +++ b/agent/docs/context-api.md @@ -0,0 +1,314 @@ +# Context API + +The `ctx` object provides access to messaging, memory, tracing, and MCP operations. + +## Properties + +```typescript +interface Context { + chat_id: string; // Chat session ID + assistant_id: string; // Assistant ID + locale: string; // User locale (e.g., "en-us") + theme: string; // UI theme + route: string; // Request route + referer: string; // Request source + metadata: Record; // Custom metadata + authorized: Record; // Auth info + + memory: Memory; // Memory namespaces + trace: Trace; // Tracing API + mcp: MCP; // MCP operations + search: Search; // Search API +} +``` + +## Messaging + +### Send Complete Message + +```typescript +ctx.Send({ type: "text", props: { content: "Hello!" } }); +ctx.Send("Hello!"); // Shorthand for text +``` + +### Streaming Messages + +```typescript +const msgId = ctx.SendStream("Starting..."); +ctx.Append(msgId, " processing..."); +ctx.Append(msgId, " done!"); +ctx.End(msgId); +``` + +### Update Streaming Message + +```typescript +const msgId = ctx.SendStream({ type: "loading", props: { message: "Loading..." } }); +// ... do work ... +ctx.Replace(msgId, { type: "text", props: { content: "Complete!" } }); +ctx.End(msgId); +``` + +### Merge Data + +```typescript +const msgId = ctx.SendStream({ type: "status", props: { progress: 0 } }); +ctx.Merge(msgId, { progress: 50 }, "props"); +ctx.Merge(msgId, { progress: 100, status: "done" }, "props"); +ctx.End(msgId); +``` + +### Set Field + +```typescript +const msgId = ctx.SendStream({ type: "result", props: {} }); +ctx.Set(msgId, "success", "props.status"); +ctx.Set(msgId, { count: 10 }, "props.data"); +ctx.End(msgId); +``` + +### Block Grouping + +```typescript +const blockId = ctx.BlockID(); +ctx.Send("Step 1", blockId); +ctx.Send("Step 2", blockId); +ctx.Send("Step 3", blockId); +ctx.EndBlock(blockId); +``` + +### ID Generators + +```typescript +const msgId = ctx.MessageID(); // "M1", "M2", ... +const blockId = ctx.BlockID(); // "B1", "B2", ... +const threadId = ctx.ThreadID(); // "T1", "T2", ... +``` + +## Memory + +Four-level hierarchical memory system: + +| Namespace | Scope | Persistence | +| -------------------- | ------------ | ----------- | +| `ctx.memory.user` | Per user | Persistent | +| `ctx.memory.team` | Per team | Persistent | +| `ctx.memory.chat` | Per chat | Persistent | +| `ctx.memory.context` | Per request | Temporary | + +### Basic Operations + +```typescript +// Get/Set +ctx.memory.user.Set("theme", "dark"); +const theme = ctx.memory.user.Get("theme"); + +// With TTL (seconds) +ctx.memory.context.Set("temp", data, 300); + +// Check/Delete +if (ctx.memory.chat.Has("topic")) { + ctx.memory.chat.Del("topic"); +} + +// Get and delete atomically +const token = ctx.memory.context.GetDel("one_time_token"); + +// Collection operations +const keys = ctx.memory.user.Keys(); +const count = ctx.memory.chat.Len(); +ctx.memory.context.Clear(); +``` + +### Counters + +```typescript +const views = ctx.memory.user.Incr("page_views"); +const credits = ctx.memory.user.Decr("credits", 5); +``` + +### Lists + +```typescript +ctx.memory.chat.Push("history", [msg1, msg2]); +const last = ctx.memory.chat.Pop("queue"); +const items = ctx.memory.chat.Pull("queue", 5); +const all = ctx.memory.chat.PullAll("queue"); +``` + +### Sets + +```typescript +ctx.memory.user.AddToSet("visited", ["/home", "/about"]); +``` + +### Array Access + +```typescript +const len = ctx.memory.chat.ArrayLen("messages"); +const first = ctx.memory.chat.ArrayGet("messages", 0); +const last = ctx.memory.chat.ArrayGet("messages", -1); +ctx.memory.chat.ArraySet("messages", 0, newMsg); +const slice = ctx.memory.chat.ArraySlice("messages", -10, -1); +const page = ctx.memory.chat.ArrayPage("messages", 1, 20); +const all = ctx.memory.chat.ArrayAll("messages"); +``` + +## Trace + +### Create Nodes + +```typescript +const node = ctx.trace.Add( + { query: "input data" }, + { + label: "Processing", + type: "process", + icon: "play", + description: "Processing user request" + } +); +``` + +### Logging + +```typescript +ctx.trace.Info("Starting process"); +ctx.trace.Debug("Variable: " + value); +ctx.trace.Warn("Deprecated feature"); +ctx.trace.Error("Operation failed"); + +// Or on node +node.Info("Step completed"); +``` + +### Node Lifecycle + +```typescript +node.SetOutput({ result: data }); +node.SetMetadata("duration", 1500); +node.Complete({ status: "done" }); +// or +node.Fail("Error message"); +``` + +### Parallel Nodes + +```typescript +const nodes = ctx.trace.Parallel([ + { input: { url: "api1" }, option: { label: "API 1" } }, + { input: { url: "api2" }, option: { label: "API 2" } } +]); +``` + +### Child Nodes + +```typescript +const parent = ctx.trace.Add({}, { label: "Parent" }); +const child = parent.Add({}, { label: "Child" }); +``` + +## MCP + +### Tools + +```typescript +// List tools +const tools = ctx.mcp.ListTools("server-id"); + +// Call single tool +const result = ctx.mcp.CallTool("server-id", "tool-name", { arg: "value" }); + +// Call multiple sequentially +const results = ctx.mcp.CallTools("server-id", [ + { name: "tool1", arguments: { a: 1 } }, + { name: "tool2", arguments: { b: 2 } } +]); + +// Call multiple in parallel +const results = ctx.mcp.CallToolsParallel("server-id", [ + { name: "tool1", arguments: {} }, + { name: "tool2", arguments: {} } +]); +``` + +### Resources + +```typescript +const resources = ctx.mcp.ListResources("server-id"); +const data = ctx.mcp.ReadResource("server-id", "resource://uri"); +``` + +### Prompts + +```typescript +const prompts = ctx.mcp.ListPrompts("server-id"); +const prompt = ctx.mcp.GetPrompt("server-id", "prompt-name", { arg: "value" }); +``` + +## Search + +### Single Search + +```typescript +// Web search +const webResult = ctx.search.Web("query", { + limit: 10, + sites: ["example.com"], + time_range: "week" +}); + +// Knowledge base +const kbResult = ctx.search.KB("query", { + collections: ["docs"], + threshold: 0.7, + graph: true +}); + +// Database +const dbResult = ctx.search.DB("query", { + models: ["model.name"], + wheres: [{ column: "status", value: "active" }], + limit: 20 +}); +``` + +### Parallel Search + +```typescript +// Wait for all +const results = ctx.search.All([ + { type: "web", query: "topic" }, + { type: "kb", query: "topic", collections: ["docs"] } +]); + +// First success +const results = ctx.search.Any([ + { type: "web", query: "topic" }, + { type: "kb", query: "topic" } +]); + +// First complete +const results = ctx.search.Race([ + { type: "web", query: "topic" }, + { type: "kb", query: "topic" } +]); +``` + +### Result Structure + +```typescript +interface SearchResult { + type: "web" | "kb" | "db"; + query: string; + source: "hook" | "auto" | "user"; + items: { + citation_id: string; + title: string; + url: string; + content: string; + score: number; + }[]; + error?: string; +} +``` diff --git a/agent/docs/hooks.md b/agent/docs/hooks.md new file mode 100644 index 00000000..1525a4be --- /dev/null +++ b/agent/docs/hooks.md @@ -0,0 +1,294 @@ +# Hooks + +Hooks allow you to customize agent behavior at key points in the execution lifecycle. + +## Lifecycle + +``` +User Input → Create Hook → LLM Call → Tool Execution → Next Hook → Response +``` + +## Create Hook + +Called before LLM call. Use to preprocess messages, configure request, or delegate. + +```typescript +function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create { + // Return null for default behavior + return null; + + // Or return configuration + return { + messages, // Modified messages + temperature: 0.7, // Override temperature + max_tokens: 2000, // Override max tokens + connector: "gpt-4o-mini", // Override connector + prompt_preset: "task", // Select prompt preset + disable_global_prompts: true,// Skip global prompts + mcp_servers: [ // Add MCP servers + { server_id: "tools", tools: ["search"] } + ], + uses: { // Override wrapper tools + vision: "vision-agent", + search: "disabled" + }, + force_uses: true, // Force use wrapper tools + locale: "zh-cn", // Override locale + metadata: { key: "value" }, // Pass data to context + }; +} +``` + +### Delegation (Skip LLM) + +Route to another agent immediately: + +```typescript +function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create { + if (shouldDelegate(messages)) { + return { + delegate: { + agent_id: "specialist.agent", + messages: messages, + options: { metadata: { source: "main" } } + } + }; + } + return { messages }; +} +``` + +## Next Hook + +Called after LLM response and tool execution. Use to post-process or delegate. + +```typescript +function Next(ctx: agent.Context, payload: agent.Payload): agent.Next { + const { messages, completion, tools, error } = payload; + + // Handle errors + if (error) { + return { data: { status: "error", message: error } }; + } + + // Process tool results + if (tools?.length > 0) { + const results = tools.map(t => t.result); + return { data: { status: "success", results } }; + } + + // Delegate based on response + if (completion?.content?.includes("transfer")) { + return { + delegate: { + agent_id: "transfer.agent", + messages: payload.messages + } + }; + } + + // Return null for standard response + return null; +} +``` + +### Payload Structure + +```typescript +interface Payload { + messages: Message[]; // Messages sent to LLM + completion?: { + content: string; // LLM text response + tool_calls?: ToolCall[]; // Tool calls from LLM + usage?: UsageInfo; // Token usage + }; + tools?: ToolCallResponse[]; // Tool execution results + error?: string; // Error message +} + +interface ToolCallResponse { + toolcall_id: string; + server: string; // MCP server ID + tool: string; // Tool name + arguments?: any; // Tool arguments + result?: any; // Tool result + error?: string; // Tool error +} +``` + +### Return Values + +```typescript +interface NextResponse { + delegate?: { // Route to another agent + agent_id: string; + messages: Message[]; + options?: Record; + }; + data?: any; // Custom response data + metadata?: Record;// Debug metadata +} +``` + +## Sending Messages + +Use `ctx` to send messages to the client: + +```typescript +function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create { + // Send complete message + ctx.Send({ type: "text", props: { content: "Processing..." } }); + + // Streaming message + const msgId = ctx.SendStream("Starting..."); + ctx.Append(msgId, " step 1..."); + ctx.Append(msgId, " step 2..."); + ctx.End(msgId); + + return { messages }; +} +``` + +## Memory + +Share data between hooks: + +```typescript +function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create { + // Store in request-scoped memory + ctx.memory.context.Set("start_time", Date.now()); + ctx.memory.context.Set("query", messages[0]?.content); + + return { messages }; +} + +function Next(ctx: agent.Context, payload: agent.Payload): agent.Next { + // Retrieve data + const startTime = ctx.memory.context.Get("start_time"); + const duration = Date.now() - startTime; + + return { data: { duration_ms: duration } }; +} +``` + +## Tracing + +Add trace nodes for debugging and UI: + +```typescript +function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create { + const node = ctx.trace.Add( + { query: messages[0]?.content }, + { label: "Preprocessing", type: "process", icon: "play" } + ); + + node.Info("Starting analysis"); + // ... processing ... + node.Complete({ status: "done" }); + + return { messages }; +} +``` + +## Error Handling + +```typescript +function Next(ctx: agent.Context, payload: agent.Payload): agent.Next { + try { + if (payload.error) { + ctx.trace.Error(payload.error); + return { + data: { status: "error", message: "Something went wrong" } + }; + } + // ... normal processing + } catch (e) { + ctx.trace.Error(e.message); + return { data: { status: "error", message: e.message } }; + } +} +``` + +## Multi-Agent Orchestration + +```typescript +// Main agent delegates based on intent +function Next(ctx: agent.Context, payload: agent.Payload): agent.Next { + const { tools } = payload; + + // Route based on tool result + const intent = tools?.[0]?.result?.intent; + const agentMap = { + "search": "search.agent", + "calculate": "calc.agent", + "translate": "translate.agent" + }; + + if (intent && agentMap[intent]) { + return { + delegate: { + agent_id: agentMap[intent], + messages: payload.messages + } + }; + } + + return null; +} +``` + +## Complete Example + +```typescript +import { agent } from "@yao/runtime"; + +function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create { + const query = messages[messages.length - 1]?.content || ""; + + // Store for Next hook + ctx.memory.context.Set("query", query); + ctx.memory.context.Set("start", Date.now()); + + // Add trace + ctx.trace.Add({ query }, { label: "Create", type: "hook" }); + + // Check if needs special handling + if (query.toLowerCase().includes("urgent")) { + return { + messages, + temperature: 0, + prompt_preset: "task" + }; + } + + return { messages }; +} + +function Next(ctx: agent.Context, payload: agent.Payload): agent.Next { + const { completion, tools, error } = payload; + const start = ctx.memory.context.Get("start"); + const duration = Date.now() - start; + + ctx.trace.Add( + { duration }, + { label: "Next", type: "hook" } + ).Complete(); + + if (error) { + return { data: { status: "error", error } }; + } + + if (tools?.length > 0) { + return { + data: { + status: "success", + response: completion?.content, + tools: tools.map(t => ({ name: t.tool, result: t.result })), + duration_ms: duration + } + }; + } + + return null; +} +``` diff --git a/agent/docs/i18n.md b/agent/docs/i18n.md new file mode 100644 index 00000000..ad31213f --- /dev/null +++ b/agent/docs/i18n.md @@ -0,0 +1,199 @@ +# Internationalization (i18n) + +## Locale Files + +Create `locales/` directory in the assistant: + +``` +assistants/my-assistant/ +└── locales/ + ├── en-us.yml + ├── zh-cn.yml + └── ja.yml +``` + +## Locale File Format + +```yaml +# locales/en-us.yml +name: My Assistant +description: A helpful AI assistant + +chat: + title: New Chat + description: How can I help you today? + prompts: + - What can you do? + - Help me with a task + - Tell me about yourself + +messages: + welcome: Welcome back! + error: Something went wrong + processing: Processing your request... +``` + +```yaml +# locales/zh-cn.yml +name: 我的助手 +description: 一个有帮助的AI助手 + +chat: + title: 新对话 + description: 今天我能帮您什么？ + prompts: + - 你能做什么？ + - 帮我完成一个任务 + - 介绍一下你自己 + +messages: + welcome: 欢迎回来！ + error: 出了点问题 + processing: 正在处理您的请求... +``` + +## Using Translations + +### In package.yao + +Use `{{ key }}` syntax: + +```json +{ + "name": "{{ name }}", + "description": "{{ description }}", + "placeholder": { + "title": "{{ chat.title }}", + "description": "{{ chat.description }}", + "prompts": [ + "{{ chat.prompts.0 }}", + "{{ chat.prompts.1 }}", + "{{ chat.prompts.2 }}" + ] + } +} +``` + +### In Prompts + +```yaml +- role: system + content: | + You are {{ name }}. + {{ description }} + + Respond in the user's language. +``` + +## Locale Detection + +The system detects locale from: + +1. Request header `Accept-Language` +2. User preference (stored in memory) +3. Default: `en-us` + +### Override in Hook + +```typescript +function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create { + // Get user preference + const userLocale = ctx.memory.user.Get("preferred_locale"); + + return { + messages, + locale: userLocale || "en-us" + }; +} +``` + +## Global Translations + +Define global translations in `agent/locales/`: + +``` +agent/ +└── locales/ + ├── en-us.yml + └── zh-cn.yml +``` + +These are available to all assistants via the `__global__` namespace. + +## Accessing Translations in Hooks + +```typescript +function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create { + const locale = ctx.locale; // e.g., "en-us" + + // Use locale for custom logic + if (locale.startsWith("zh")) { + return { + messages, + prompt_preset: "chinese" + }; + } + + return { messages }; +} +``` + +## Nested Keys + +Access nested values with dot notation: + +```yaml +# locales/en-us.yml +errors: + validation: + required: This field is required + invalid: Invalid value + network: + timeout: Connection timed out +``` + +```json +{ + "placeholder": { + "title": "{{ errors.validation.required }}" + } +} +``` + +## Fallback Behavior + +If a translation key is not found: + +1. Try the requested locale +2. Fall back to `en-us` +3. Return the key itself if not found + +## Dynamic Locale Content + +```typescript +function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create { + // Add locale-specific system message + const localeGreeting = { + "en-us": "Hello! How can I help you?", + "zh-cn": "你好！有什么可以帮您的？", + "ja": "こんにちは！何かお手伝いできますか？" + }; + + const greeting = localeGreeting[ctx.locale] || localeGreeting["en-us"]; + + return { + messages: [ + { role: "system", content: `Greeting: ${greeting}` }, + ...messages + ] + }; +} +``` + +## Best Practices + +1. **Keep keys consistent** - Use the same keys across all locale files +2. **Use nested structure** - Organize related translations together +3. **Provide fallbacks** - Always have `en-us` as the base locale +4. **Test all locales** - Verify translations render correctly +5. **Use context variables** - Combine with `$CTX.locale` in prompts diff --git a/agent/docs/mcp.md b/agent/docs/mcp.md new file mode 100644 index 00000000..5b030384 --- /dev/null +++ b/agent/docs/mcp.md @@ -0,0 +1,268 @@ +# MCP Integration + +Model Context Protocol (MCP) enables tool integration with external services. + +## Defining MCP Servers + +Create `mcps/tools.mcp.yao` in the assistant directory: + +```json +{ + "name": "Tools", + "description": "Custom tools for the assistant", + "transport": "process", + "process": { + "command": "node", + "args": ["server.js"] + } +} +``` + +### Transport Types + +**Process (Local)** + +```json +{ + "transport": "process", + "process": { + "command": "python", + "args": ["mcp_server.py"], + "env": { "API_KEY": "$ENV.API_KEY" } + } +} +``` + +**HTTP** + +```json +{ + "transport": "http", + "http": { + "url": "https://mcp.example.com", + "headers": { "Authorization": "Bearer $ENV.TOKEN" } + } +} +``` + +**SSE (Server-Sent Events)** + +```json +{ + "transport": "sse", + "sse": { + "url": "https://mcp.example.com/events" + } +} +``` + +## Configuring in package.yao + +### All Tools + +```json +{ + "mcp": { + "servers": ["tools"] + } +} +``` + +### Specific Tools + +```json +{ + "mcp": { + "servers": [ + { "server_id": "tools", "tools": ["search", "calculate"] } + ] + } +} +``` + +### With Resources + +```json +{ + "mcp": { + "servers": [ + { + "server_id": "data", + "tools": ["query"], + "resources": ["data://users/*"] + } + ] + } +} +``` + +## Dynamic Configuration in Hooks + +```typescript +function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create { + return { + messages, + mcp_servers: [ + { server_id: "tools", tools: ["search"] }, + { server_id: "data", resources: ["data://reports"] } + ] + }; +} +``` + +## Using MCP in Hooks + +### List Available Tools + +```typescript +const tools = ctx.mcp.ListTools("server-id"); +// { tools: [{ name: "search", description: "...", inputSchema: {...} }] } +``` + +### Call Tool + +```typescript +const result = ctx.mcp.CallTool("server-id", "search", { + query: "example", + limit: 10 +}); +// { content: [{ type: "text", text: "..." }] } +``` + +### Batch Tool Calls + +```typescript +// Sequential +const results = ctx.mcp.CallTools("server-id", [ + { name: "step1", arguments: { input: "a" } }, + { name: "step2", arguments: { input: "b" } } +]); + +// Parallel +const results = ctx.mcp.CallToolsParallel("server-id", [ + { name: "api1", arguments: {} }, + { name: "api2", arguments: {} } +]); +``` + +### Read Resources + +```typescript +const resources = ctx.mcp.ListResources("server-id"); +const data = ctx.mcp.ReadResource("server-id", "data://users/123"); +``` + +### Get Prompts + +```typescript +const prompts = ctx.mcp.ListPrompts("server-id"); +const prompt = ctx.mcp.GetPrompt("server-id", "system", { role: "helper" }); +``` + +## Tool Mapping + +Map MCP tools to Yao processes: + +``` +mcps/ +└── mapping/ + └── tools/ + ├── search.yao + └── calculate.yao +``` + +**mapping/tools/search.yao** + +```json +{ + "process": "scripts.search.Execute", + "args": ["{{input}}"] +} +``` + +## Error Handling + +```typescript +function Next(ctx: agent.Context, payload: agent.Payload): agent.Next { + const { tools } = payload; + + if (tools) { + for (const tool of tools) { + if (tool.error) { + ctx.trace.Error(`Tool ${tool.tool} failed: ${tool.error}`); + // Handle error + } else { + // Process result + console.log(tool.result); + } + } + } + + return null; +} +``` + +## Complete Example + +**mcps/calculator.mcp.yao** + +```json +{ + "name": "Calculator", + "description": "Math operations", + "transport": "process", + "process": { + "command": "node", + "args": ["calc-server.js"] + } +} +``` + +**package.yao** + +```json +{ + "name": "Math Assistant", + "connector": "gpt-4o", + "mcp": { + "servers": [ + { "server_id": "agents.assistant.calculator", "tools": ["add", "multiply"] } + ] + } +} +``` + +**src/index.ts** + +```typescript +function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create { + // Check if calculation is needed + const query = messages[messages.length - 1]?.content || ""; + if (/\d+\s*[\+\-\*\/]\s*\d+/.test(query)) { + // Enable calculator + return { + messages, + mcp_servers: [{ server_id: "agents.assistant.calculator" }] + }; + } + return { messages }; +} + +function Next(ctx: agent.Context, payload: agent.Payload): agent.Next { + const { tools } = payload; + + if (tools?.length > 0) { + const calcResult = tools.find(t => t.server.includes("calculator")); + if (calcResult?.result) { + return { + data: { + answer: calcResult.result, + expression: calcResult.arguments + } + }; + } + } + + return null; +} +``` diff --git a/agent/docs/prompts.md b/agent/docs/prompts.md new file mode 100644 index 00000000..1e7dcb53 --- /dev/null +++ b/agent/docs/prompts.md @@ -0,0 +1,177 @@ +# Prompts + +## Default Prompts + +Create `prompts.yml` in the assistant directory: + +```yaml +- role: system + content: | + You are a helpful assistant. + + ## Guidelines + - Be concise and accurate + - Ask clarifying questions when needed + +- role: system + name: context + content: | + Current date: {{ $CTX.date }} + User locale: {{ $CTX.locale }} +``` + +### Prompt Structure + +```yaml +- role: system | user | assistant + content: string + name: string (optional) +``` + +### Context Variables + +Use `$CTX.*` for runtime context: + +| Variable | Description | +| ---------------- | -------------------------- | +| `$CTX.date` | Current date | +| `$CTX.time` | Current time | +| `$CTX.locale` | User locale (e.g., en-us) | +| `$CTX.timezone` | User timezone | +| `$CTX.user_id` | Current user ID | +| `$CTX.team_id` | Current team ID | +| `$CTX.chat_id` | Current chat session ID | + +## Prompt Presets + +Create presets in `prompts/` directory for different scenarios: + +``` +prompts/ +├── chat.yml # Casual conversation +├── task.yml # Task-oriented +└── analysis.yml # Data analysis +``` + +**prompts/chat.yml** + +```yaml +- role: system + content: | + You are a friendly conversational assistant. + Be warm and engaging. +``` + +**prompts/task.yml** + +```yaml +- role: system + content: | + You are a task-focused assistant. + Be precise and efficient. +``` + +### Using Presets + +Select preset in Create hook: + +```typescript +function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create { + return { + messages, + prompt_preset: "task", // Use prompts/task.yml + }; +} +``` + +Or via mode configuration in `package.yao`: + +```json +{ + "modes": ["chat", "task"], + "default_mode": "task" +} +``` + +## Global Prompts + +Define global prompts in `agent/prompts.yml` (applies to all assistants): + +```yaml +- role: system + content: | + # Global Guidelines + - Always be helpful and respectful + - Follow company policies +``` + +### Disabling Global Prompts + +Per assistant: + +```json +{ + "disable_global_prompts": true +} +``` + +Per request (in Create hook): + +```typescript +function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create { + return { + messages, + disable_global_prompts: true, + }; +} +``` + +## Multi-line Content + +Use YAML block scalars for long content: + +```yaml +- role: system + content: | + # Assistant Role + + You are an expert in data analysis. + + ## Capabilities + - Statistical analysis + - Data visualization + - Report generation + + ## Guidelines + 1. Always validate input data + 2. Explain your methodology + 3. Provide actionable insights +``` + +## Dynamic Prompts + +Inject dynamic content in Create hook: + +```typescript +function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create { + const userPrefs = ctx.memory.user.Get("preferences"); + + // Add dynamic system message + const dynamicPrompt = { + role: "system", + content: `User preferences: ${JSON.stringify(userPrefs)}`, + }; + + return { + messages: [dynamicPrompt, ...messages], + }; +} +``` + +## Prompt Best Practices + +1. **Be specific** - Clear instructions produce better results +2. **Use structure** - Headers, lists, and sections improve readability +3. **Set boundaries** - Define what the assistant should and shouldn't do +4. **Include examples** - Show expected input/output formats +5. **Layer prompts** - Use global + assistant + dynamic prompts together diff --git a/agent/docs/search.md b/agent/docs/search.md new file mode 100644 index 00000000..ff71f9e3 --- /dev/null +++ b/agent/docs/search.md @@ -0,0 +1,283 @@ +# Search + +The agent search system provides automatic search across web, knowledge base (KB), and database (DB). + +## Auto Search Flow + +1. **Intent Detection** - `__yao.needsearch` agent analyzes user message +2. **Search Execution** - Executes web/kb/db searches based on intent +3. **Context Injection** - Results injected as system message +4. **Citation** - LLM can cite results using `[1]`, `[2]` format + +## Configuration + +### Global (agent/search.yml) + +```yaml +web: + provider: tavily # tavily, serper, serpapi + api_key_env: TAVILY_API_KEY + max_results: 10 + +kb: + threshold: 0.7 + graph: true + +db: + max_results: 20 + +keyword: + max_keywords: 5 + language: en + +citation: + format: "[{index}]" + auto_inject_prompt: true +``` + +### Per Assistant (package.yao) + +```json +{ + "search": { + "web": { "max_results": 5 }, + "kb": { "threshold": 0.8 }, + "citation": { "format": "[{index}]" } + }, + "kb": { + "collections": ["docs", "faq"] + }, + "db": { + "models": ["articles", "products"] + } +} +``` + +## Controlling Search in Hooks + +### Disable Search + +```typescript +function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create { + return { + messages, + search: false + }; +} +``` + +### Enable Specific Types + +```typescript +function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create { + return { + messages, + search: { + need_search: true, + search_types: ["kb", "db"], // Only KB and DB + confidence: 1.0, + reason: "controlled by hook" + } + }; +} +``` + +### Disable via Uses + +```typescript +function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create { + return { + messages, + uses: { search: "disabled" } + }; +} +``` + +## Search API (ctx.search) + +### Web Search + +```typescript +const result = ctx.search.Web("query", { + limit: 10, + sites: ["example.com", "docs.example.com"], + time_range: "week", // day, week, month, year + rerank: { top_n: 5 } +}); +``` + +### Knowledge Base Search + +```typescript +const result = ctx.search.KB("query", { + collections: ["docs", "faq"], + threshold: 0.7, + limit: 10, + graph: true, + rerank: { top_n: 5 } +}); +``` + +### Database Search + +```typescript +const result = ctx.search.DB("query", { + models: ["articles"], + wheres: [{ column: "status", value: "published" }], + orders: [{ column: "created_at", option: "desc" }], + select: ["id", "title", "content"], + limit: 20, + rerank: { top_n: 10 } +}); +``` + +### Parallel Search + +```typescript +// Wait for all +const results = ctx.search.All([ + { type: "web", query: "topic" }, + { type: "kb", query: "topic", collections: ["docs"] }, + { type: "db", query: "topic", models: ["articles"] } +]); + +// First success with results +const results = ctx.search.Any([ + { type: "web", query: "topic" }, + { type: "kb", query: "topic" } +]); + +// First to complete +const results = ctx.search.Race([ + { type: "web", query: "topic" }, + { type: "kb", query: "topic" } +]); +``` + +## Result Structure + +```typescript +interface SearchResult { + type: "web" | "kb" | "db"; + query: string; + source: "hook" | "auto" | "user"; + items: SearchItem[]; + error?: string; +} + +interface SearchItem { + citation_id: string; // "1", "2", etc. + title: string; + url: string; + content: string; + score: number; +} +``` + +## Custom Search in Hooks + +```typescript +function Create(ctx: agent.Context, messages: agent.Message[]): agent.Create { + const query = messages[messages.length - 1]?.content || ""; + + // Custom KB search + const kbResult = ctx.search.KB(query, { + collections: ["internal_docs"], + threshold: 0.8 + }); + + if (kbResult.items?.length > 0) { + // Format results as context + const context = kbResult.items + .map((item, i) => `[${i + 1}] ${item.title}\n${item.content}`) + .join("\n\n"); + + // Inject as system message + const contextMsg = { + role: "system", + content: `Reference information:\n${context}` + }; + + return { + messages: [contextMsg, ...messages], + search: false // Skip auto search + }; + } + + return { messages }; +} +``` + +## Authorization + +### KB Collections + +Collections are filtered by user authorization: + +```typescript +// Only collections user has access to are searched +const result = ctx.search.KB("query", { + collections: ["public", "internal", "secret"] + // User without "secret" access won't search that collection +}); +``` + +### DB Models + +Database queries include permission filters: + +```typescript +// Auth filters are automatically added +// e.g., { column: "__yao_created_by", value: user_id } +const result = ctx.search.DB("query", { + models: ["user_documents"] +}); +``` + +## Web Search Providers + +### Tavily + +```yaml +web: + provider: tavily + api_key_env: TAVILY_API_KEY +``` + +### Serper + +```yaml +web: + provider: serper + api_key_env: SERPER_API_KEY +``` + +### SerpAPI + +```yaml +web: + provider: serpapi + api_key_env: SERPAPI_API_KEY +``` + +## Citation + +LLM responses can include citations: + +``` +Based on the documentation [1], the feature works by... [2] + +References: +[1] Getting Started Guide - https://docs.example.com/start +[2] API Reference - https://docs.example.com/api +``` + +### Citation Format + +```yaml +citation: + format: "[{index}]" # or "({index})" or "[^{index}]" + auto_inject_prompt: true + custom_prompt: | + When citing sources, use the format [N] where N is the reference number. +``` diff --git a/agent/docs/testing.md b/agent/docs/testing.md new file mode 100644 index 00000000..532fd34d --- /dev/null +++ b/agent/docs/testing.md @@ -0,0 +1,367 @@ +# Agent Testing + +A comprehensive testing framework for Yao AI agents with support for standard testing, dynamic (simulator-driven) testing, agent-driven assertions, and CI integration. + +## Quick Start + +```bash +# Test with direct message (auto-detect agent from current directory) +cd assistants/my-assistant +yao agent test -i "Hello, how are you?" + +# Test with JSONL file +yao agent test -i tests/inputs.jsonl + +# Generate HTML report +yao agent test -i tests/inputs.jsonl -o report.html + +# Stability analysis (run each test 5 times) +yao agent test -i tests/inputs.jsonl --runs 5 +``` + +## Input Modes + +The `-i` flag supports multiple input modes: + +| Mode | Example | Description | +|------|---------|-------------| +| Direct message | `-i "Hello"` | Single message test | +| JSONL file | `-i tests/inputs.jsonl` | Multiple test cases | +| Agent-driven | `-i "agents:tests.generator?count=10"` | Generate tests with agent | +| Script test | `-i scripts.expense.setup` | Test handler scripts | +| Script-generated | `-i "scripts:tests.gen.Generate"` | Generate tests from script | + +## Test Case Format (JSONL) + +### Basic Test + +```jsonl +{"id": "greeting", "input": "Hello", "assert": {"type": "contains", "value": "Hi"}} +``` + +### With Conversation History + +```jsonl +{ + "id": "multi-turn", + "input": [ + {"role": "user", "content": "What's 2+2?"}, + {"role": "assistant", "content": "4"}, + {"role": "user", "content": "Multiply by 3"} + ], + "assert": {"type": "contains", "value": "12"} +} +``` + +### With File Attachments + +```jsonl +{ + "id": "image-test", + "input": { + "role": "user", + "content": [ + {"type": "text", "text": "Describe this image"}, + {"type": "image", "source": "file://fixtures/test.jpg"} + ] + } +} +``` + +## Assertions + +### Static Assertions + +| Type | Description | Example | +|------|-------------|---------| +| `equals` | Exact match | `{"type": "equals", "value": {"key": "val"}}` | +| `contains` | Output contains value | `{"type": "contains", "value": "keyword"}` | +| `not_contains` | Output does not contain | `{"type": "not_contains", "value": "error"}` | +| `regex` | Match regex pattern | `{"type": "regex", "value": "\\d+"}` | +| `json_path` | Extract and compare | `{"type": "json_path", "path": "$.field", "value": true}` | +| `type` | Check output type | `{"type": "type", "value": "object"}` | +| `tool_called` | Check tool was called | `{"type": "tool_called", "value": "setup"}` | +| `tool_result` | Check tool result | `{"type": "tool_result", "value": {"tool": "setup", "result": {"success": true}}}` | + +### Agent-Driven Assertions + +Use LLM to validate response semantics: + +```jsonl +{ + "id": "helpful-response", + "input": "How do I reset my password?", + "assert": { + "type": "agent", + "use": "agents:tests.validator-agent", + "value": "Response should provide clear step-by-step instructions" + } +} +``` + +### Multiple Assertions + +All assertions must pass: + +```jsonl +{ + "id": "complete-check", + "input": "Submit expense", + "assert": [ + {"type": "contains", "value": "expense"}, + {"type": "not_contains", "value": "error"}, + {"type": "regex", "value": "(?i)(submitted|created)"} + ] +} +``` + +## Dynamic Mode (Simulator) + +For testing complex conversation flows with a user simulator: + +```jsonl +{ + "id": "order-flow", + "input": "I want to order coffee", + "simulator": { + "use": "tests.simulator-agent", + "options": { + "metadata": { + "persona": "Customer", + "goal": "Order a medium latte" + } + } + }, + "checkpoints": [ + { + "id": "greeting", + "assert": {"type": "regex", "value": "(?i)(hello|hi)"} + }, + { + "id": "ask-size", + "after": ["greeting"], + "assert": {"type": "regex", "value": "(?i)size"} + }, + { + "id": "confirm", + "after": ["ask-size"], + "assert": {"type": "regex", "value": "(?i)confirm"} + } + ], + "max_turns": 10 +} +``` + +Run with: + +```bash +yao agent test -i tests/dynamic.jsonl --simulator tests.simulator-agent -v +``` + +## Script Testing + +Test agent handler scripts with the `t.assert` API: + +```typescript +// assistants/my-assistant/src/setup_test.ts +import { SystemReady } from "./setup"; + +export function TestSystemReady(t: TestingT, ctx: Context) { + const result = SystemReady(ctx); + + t.assert.True(result.success, "Should succeed"); + t.assert.Equal(result.status, "ready", "Status should be ready"); + t.assert.NotNil(result.data, "Data should not be nil"); +} + +export function TestWithAgentAssertion(t: TestingT, ctx: Context) { + const response = Process("agents.my-assistant.Stream", ctx, messages); + + // Static assertion + t.assert.Contains(response.content, "confirm"); + + // Agent-driven assertion + t.assert.Agent(response.content, "tests.validator-agent", { + criteria: "Response should ask for confirmation" + }); +} +``` + +Run with: + +```bash +yao agent test -i scripts.my-assistant.setup -v +``` + +### Available Assertions + +| Method | Description | +|--------|-------------| +| `t.assert.True(value, msg)` | Assert value is true | +| `t.assert.False(value, msg)` | Assert value is false | +| `t.assert.Equal(a, b, msg)` | Assert a equals b | +| `t.assert.NotEqual(a, b, msg)` | Assert a not equals b | +| `t.assert.Nil(value, msg)` | Assert value is null/undefined | +| `t.assert.NotNil(value, msg)` | Assert value is not nil | +| `t.assert.Contains(s, sub, msg)` | Assert string contains substr | +| `t.assert.Len(arr, n, msg)` | Assert array/string length | +| `t.assert.Agent(resp, id, opts)` | Agent-driven assertion | + +## Before/After Hooks + +### Per-Test Hooks + +```jsonl +{ + "id": "with-setup", + "input": "Show my data", + "before": "env_test.Before", + "after": "env_test.After" +} +``` + +### Global Hooks + +```bash +yao agent test -i tests/inputs.jsonl --before env_test.BeforeAll --after env_test.AfterAll +``` + +### Hook Implementation + +```typescript +// assistants/my-assistant/src/env_test.ts + +export function Before(ctx: Context, testCase: TestCase): any { + const userId = Process("models.user.Create", { name: "Test User" }); + return { userId }; // Passed to After +} + +export function After(ctx: Context, testCase: TestCase, result: TestResult, beforeData: any) { + if (beforeData?.userId) { + Process("models.user.Delete", beforeData.userId); + } +} + +export function BeforeAll(ctx: Context, testCases: TestCase[]): any { + Process("models.migrate"); + return { initialized: true }; +} + +export function AfterAll(ctx: Context, results: TestResult[], beforeData: any) { + const passed = results.filter(r => r.status === "passed").length; + console.log(`Tests completed: ${passed}/${results.length} passed`); +} +``` + +## Custom Context + +Create a JSON file for custom authorization: + +```json +{ + "chat_id": "test-chat-001", + "authorized": { + "user_id": "test-user-123", + "team_id": "test-team-456", + "constraints": { + "owner_only": true, + "extra": { "department": "engineering" } + } + } +} +``` + +Use with `--ctx`: + +```bash +yao agent test -i scripts.my-assistant.setup --ctx tests/context.json -v +``` + +## Command Line Options + +| Flag | Description | Default | +|------|-------------|---------| +| `-i` | Input: JSONL file, message, `agents:xxx`, or `scripts:xxx` | (required) | +| `-o` | Output file path | `output-{timestamp}.jsonl` | +| `-n` | Agent ID (optional, auto-detected) | auto-detect | +| `-a` | Application directory | auto-detect | +| `-e` | Environment file | - | +| `-c` | Override connector | agent default | +| `-u` | Test user ID | `test-user` | +| `-t` | Test team ID | `test-team` | +| `-r` | Reporter agent ID | built-in | +| `-v` | Verbose output | false | +| `--ctx` | Path to context JSON file | - | +| `--simulator` | Default simulator agent ID | - | +| `--before` | Global BeforeAll hook | - | +| `--after` | Global AfterAll hook | - | +| `--runs` | Runs per test (stability analysis) | 1 | +| `--run` | Regex pattern to filter tests | - | +| `--timeout` | Timeout per test | 2m | +| `--parallel` | Parallel test cases | 1 | +| `--fail-fast` | Stop on first failure | false | +| `--dry-run` | Generate tests without running | false | + +## Output Formats + +Determined by `-o` file extension: + +| Extension | Format | Description | +|-----------|--------|-------------| +| `.jsonl` | JSONL | Streaming (default) | +| `.json` | JSON | Complete structured | +| `.md` | Markdown | Human-readable | +| `.html` | HTML | Interactive web report | + +## Stability Analysis + +Run each test multiple times to measure consistency: + +```bash +yao agent test -i tests/inputs.jsonl --runs 5 -o stability.json +``` + +| Pass Rate | Classification | +|-----------|----------------| +| 100% | Stable | +| 80-99% | Mostly Stable | +| 50-79% | Unstable | +| < 50% | Highly Unstable | + +## CI Integration + +```bash +# Exit code: 0 = all passed, 1 = failures +yao agent test -i tests/inputs.jsonl --fail-fast + +# Run with parallel execution +yao agent test -i tests/inputs.jsonl --parallel 4 +``` + +### GitHub Actions Example + +```yaml +- name: Run Agent Tests + run: | + yao agent test -i assistants/my-assistant/tests/inputs.jsonl \ + -u ci-user -t ci-team \ + --runs 3 \ + -o report.json + +- name: Run Dynamic Tests + run: | + yao agent test -i assistants/my-assistant/tests/dynamic.jsonl \ + --simulator tests.simulator-agent \ + -v + +- name: Run Script Tests + run: | + yao agent test -i scripts.my-assistant.setup -v +``` + +## Exit Codes + +| Code | Description | +|------|-------------| +| 0 | All tests passed | +| 1 | Tests failed, configuration error, or runtime error | diff --git a/sui/README.md b/sui/README.md index 69620729..373dafff 100644 --- a/sui/README.md +++ b/sui/README.md @@ -100,4 +100,4 @@ See [Agent SUI Documentation](docs/agent-sui.md) for details. ## License -Apache-2.0 +This project is part of the Yao App Engine and follows the [Yao Open Source License](../LICENSE).