Compare commits

...

80 commits

Author SHA1 Message Date
Max
cb95bb7871
Merge pull request #1534 from trheyi/main
feat(assistant): add hot-reload functionality for assistants and enhance gRPC metadata handling
2026-05-13 21:28:16 +08:00
Max
da4a803c11 feat(locale): enhance locale handling and agent functionalities
- Added locale support in various components, including the Assistant's execution context and agent tools, to improve internationalization.
- Updated the `pickNodeByFilter` function to include a fallback mechanism for host_exec nodes when no container nodes are available.
- Introduced a new `agent_reference` tool for downloading agent source code for read-only study, alongside updates to the `agent_download` tool to restrict downloads to the 'smith' namespace.
- Enhanced documentation and tests to reflect the new functionalities and ensure robust locale extraction and handling.
2026-05-13 21:25:49 +08:00
Max
639f0c59fc feat(assistant): add hot-reload functionality for assistants and enhance gRPC metadata handling
- Implemented AssistantReloadFunc to enable hot-reloading of assistants after deployment, improving deployment flexibility.
- Enhanced gRPC authProvider to include workspace and sandbox IDs from incoming context metadata, enriching the authentication context.
- Updated tools to support new agent-related functionalities, including listing, downloading, deploying, and connecting agents.
- Expanded system tools documentation to include new agent commands, ensuring comprehensive guidance for users.
2026-05-13 14:52:32 +08:00
Max
a0c4c543f3
Merge pull request #1533 from trheyi/main
refactor(robot_prompt): update prompt generation for robot configuration
2026-05-08 17:46:27 +08:00
Max
6af83149ef refactor(robot_prompt): update prompt generation for robot configuration
- Revised the system prompt to focus on generating and refining robot configurations rather than solely crafting prompts for autonomous robots.
- Enhanced input context details to include specific fields relevant to robot configuration, improving clarity and usability.
- Streamlined output rules to ensure a pure JSON object is returned, aligning with best practices for API responses.
- Removed outdated examples and sections to simplify the prompt structure and improve overall coherence.
2026-05-08 17:45:22 +08:00
Max
59bf6ddc8a
Merge pull request #1532 from trheyi/main
feat(workspace): enhance attachment handling and execution context
2026-05-08 11:27:56 +08:00
Max
421d946971 feat(buffer): refine step retrieval for resume functionality
- Updated the GetStepsForResume method to return only steps with valid resume statuses (failed or interrupted), improving recovery context handling.
- Enhanced error handling in message processing across various integrations, including friendly error messages for users in different locales.
- Modified file handling in message integrations (Dingtalk, Discord, Feishu, Telegram, Weixin) to differentiate between image and file types, ensuring appropriate content delivery.
2026-05-08 10:14:26 +08:00
Max
4376ac9dad feat(weixin): add workspace file handling and MIME type detection
- Implemented support for reading files from workspace URIs in the Weixin reply process, enhancing attachment management.
- Introduced a new `mimeFromExt` function to determine content types based on file extensions, improving file handling capabilities.
- Updated the `sendMediaFromURL` method to accommodate workspace file retrieval, ensuring robust error handling and logging for file operations.
2026-05-07 20:28:28 +08:00
Max
f230f1e90c feat(workspace): enhance attachment handling and execution context
- Added support for reading files from workspace URIs in the delivery process, allowing for more flexible attachment management.
- Introduced a new `convertWorkspaceAttachment` function to handle workspace-based file retrieval and integration into messenger attachments.
- Updated the `AgentCaller` to include execution mode in the context, improving task execution tracking.
- Enhanced the `RunDelivery` method to utilize workspace manifests for delivery input, reducing token usage and improving efficiency.
- Implemented locale handling in various request structures to support multi-language capabilities in user interfaces.
2026-05-07 19:32:48 +08:00
Max
a01eb869ad
Merge pull request #1531 from trheyi/main
feat(claude): implement user ID hashing and enhance environment setup
2026-05-06 18:37:56 +08:00
Max
cc078b953a feat(claude): implement user ID hashing and enhance environment setup
- Added a `hashUserID` function to generate a collision-safe hash for user IDs, improving metadata handling.
- Updated the environment setup in `buildEnv` to override `metadata.user_id` with a sanitized value, ensuring compatibility with third-party APIs.
- Enhanced logging for the environment variables to aid in debugging and monitoring.
2026-05-06 18:36:47 +08:00
Max
a80fc9a7ff
Merge pull request #1530 from trheyi/main
feat: LLM audio/connector/key enhancements, Agent role/tool/sandbox improvements, Image tools, and various fixes
2026-05-06 11:09:51 +08:00
Max
af0d4edd74 feat(image): add image generation and enhanced reading capabilities
- Introduced `image_generate` tool for generating images from text prompts, with options for specifying output file paths and image dimensions.
- Updated `image_read` functionality to allow optional provider specification for enhanced image analysis.
- Implemented new `GenerateImage` method in the LLM API for seamless integration of image generation capabilities.
- Enhanced documentation to include detailed usage examples for both image reading and generation tools.
- Updated tests to validate new image generation features and ensure robust functionality across image tools.
2026-05-06 10:59:35 +08:00
Max
20fc9c24df feat(image): introduce image reading and analysis tools
- Added `image_read` tool for reading and analyzing images using vision-capable models, with a detailed input schema for image path, prompt, max size, and provider.
- Implemented `ReadImage` function to handle image processing and integration with vision models, returning descriptive text.
- Created comprehensive tests for image resizing, data URI handling, and various input scenarios to ensure robust functionality.
- Updated documentation to include usage examples and parameter descriptions for the new image tools.
2026-05-06 10:58:52 +08:00
Max
d6626b2a7e refactor(setup): update preference retrieval to user-scope only
- Modified the handleSetupStatus function to read user-specific preferences instead of merged settings, ensuring that personal preferences do not inherit from system or team scopes.
- This change enhances the accuracy of user preference handling in the setup status process.
2026-05-05 08:56:13 +08:00
Max
01daa783e0 feat(chat): improve error handling for chat operations
- Added centralized error handling for "not found" scenarios in GetChat, UpdateChat, DeleteChat, and GetMessages functions, enhancing user feedback for missing resources.
- Introduced isNotFoundError and respondNotFound utility functions to streamline error responses across chat-related operations.
- Updated the setting package to include new endpoints for setup status and user preferences, improving API functionality and user experience.
2026-05-04 21:53:26 +08:00
Max
cf0ebd5601 chore(license): remove deprecated commercial license documents
- Deleted the outdated COMMERCIAL_LICENSE.md and COMMERCIAL_LICENSE.zh-CN.md files, as they are no longer in effect.
- Removed the COMMERCIAL.md and COMMERCIAL.zh-CN.md files related to Yao OpenAPI commercial usage, streamlining licensing documentation.
- Updated LICENSE file to reflect changes in commercial usage terms and organization.
2026-05-04 18:43:07 +08:00
Max
2993b0b946 feat(messenger): enhance dynamic SMTP resolution and identity handling
- Implemented dynamic SMTP provider resolution based on user/team context, improving email sending capabilities.
- Updated the Send and SendT methods to utilize identity information from the context for dynamic provider selection.
- Refactored cloud encryption and decryption methods to delegate to the setting package, streamlining cryptographic operations.
- Enhanced team invitation email sending to include identity context, ensuring accurate user/team information is used.
- Removed obsolete crypto helper functions, simplifying the codebase and improving maintainability.
2026-05-04 18:15:21 +08:00
Max
54607e26b7 feat(agent): enhance image content handling and vision support
- Added support for processing image content in the agent, providing placeholder text when image parsing fails.
- Implemented a fallback mechanism to utilize vision tools for image analysis, improving the handling of image content.
- Updated tests to validate the new image handling behavior, ensuring correct responses when vision support is unavailable.
- Enhanced system prompts to guide users on using the new `image_read` tool for image analysis, improving user experience.
2026-05-04 13:46:30 +08:00
Max
222daaf645 fix(tests): update provider key handling in TestOwnerPrefixedIDs
- Modified the TestOwnerPrefixedIDs test to use ScopedKey for provider key generation, ensuring unique keys based on owner context.
- This change enhances the accuracy of tests related to provider management by preventing key collisions during test execution.
2026-05-03 21:46:55 +08:00
Max
960f47c238 feat(grpc): enhance MCP handler with gRPC authorization support
- Introduced grpcAuthProvider to adapt gRPC AuthorizedInfo for use in MCP process calls, enabling better authorization handling.
- Updated MCPListTools and MCPCallTool methods to include authorization information in tool calls, improving security and context awareness.
- Refactored fetch functions in webfetch to support Brightdata API endpoint configuration, enhancing flexibility in fetching HTML content.
- Improved fetchHTML and fetchRawHTML methods to prioritize Brightdata when configured, streamlining content retrieval processes.
- Added isHTMLContent utility function to determine response content type, enhancing the robustness of content handling in fetch operations.
2026-05-03 19:57:26 +08:00
Max
194faac9b7 feat(agent): enhance sandbox V2 initialization and role management
- Refactored the initSandboxV2 function to return a structured result, consolidating the runner, computer, configuration, cleanup function, loading message ID, and roles into a single return type.
- Updated the Stream method to utilize the new sandboxV2InitResult structure, improving clarity and reducing complexity in handling sandbox initialization.
- Introduced role management enhancements, allowing for pre-resolved role connectors to be passed through the request, streamlining connector resolution during execution.
- Adjusted various components to support the new roles structure, ensuring consistent handling across the agent's sandbox operations.
- Added logging for connector resolution and role management, improving diagnostics and traceability during sandbox execution.
- Updated .gitignore to include tools/TOOL-REGISTRATION.md for better project organization.
2026-05-03 19:11:29 +08:00
Max
7877797549 fix(tests): temporarily disable KB/DB search in tests
- Skipped tests related to KB/DB search functionality due to temporary unavailability.
- Updated test cases in chat_test.go, search_auth_integration_test.go, search_auto_full_test.go, and others to reflect this change.
- Adjusted search handling in search.go to limit search types to "web" only until KB/DB search is re-enabled.
2026-05-03 10:26:06 +08:00
Max
fb01a1c141 feat(agent): implement tool loop processing and enhance error handling
- Added support for tool loop processing when tool call responses are present and sandbox mode is disabled, improving the assistant's ability to handle complex tool interactions.
- Implemented fallback delegation to a loop fallback mechanism in case of tool loop execution failure, enhancing robustness in error scenarios.
- Updated the assistant message structure to include reasoning content, providing better context for generated responses.
- Enhanced error logging in tool call execution to include detailed content, improving diagnostics for tool call failures.
- Updated system configuration to include a new loop fallback agent, expanding the assistant's capabilities.
2026-05-02 21:42:55 +08:00
Max
7da06a4ae1 feat(search): refactor builtinSearch to utilize websearch tool and enhance context handling
- Updated the builtinSearch function to delegate search operations to the websearch tool, improving modularity and maintainability.
- Enhanced context handling by passing the agent context to builtinSearch, allowing for user and team identification during searches.
- Added a new DecryptValue function in cloud.go to streamline value decryption, delegating to the existing config.DecryptValue method.
- Updated .gitignore to include tools/README.md for better project organization.
2026-05-02 19:58:03 +08:00
Max
a5ca482d7c feat(llm): implement scoped key management for providers
- Introduced ScopedKey function to generate unique provider keys based on owner type (user/team) to prevent key collisions.
- Updated provider creation and management tests to utilize scoped keys, ensuring consistent key handling across different contexts.
- Refactored existing tests to validate the correct generation and retrieval of scoped keys, enhancing overall provider management reliability.
- Adjusted OpenAPI settings to reflect the use of scoped keys in provider operations, improving API consistency.
2026-05-02 15:30:39 +08:00
Max
756ff95d3f feat(llm): enhance capabilities management with embedding and image generation support
- Added support for embedding and image generation capabilities in the capabilitiesFromMap function.
- Updated the ToMap method to delegate to the canonical Capabilities.ToMap() method, simplifying the conversion process.
- Enhanced the capabilities handling in various components, including filters for non-chat models in OpenAPI settings.
- Introduced utility functions for filter checking and non-chat model identification, improving overall capabilities management.
2026-05-02 14:45:31 +08:00
Max
c5da1c1ba1 feat(agent): add heavy role support in system configuration and connector management
- Introduced a new "Heavy" role in the system configuration, allowing for complex reasoning tasks.
- Updated the initialization and environment resolution functions to handle the new Heavy role appropriately.
- Enhanced tests to validate the integration of the Heavy role, ensuring proper connector resolution and model handling.
- Adjusted OpenAPI settings to reflect the addition of the Heavy role, improving overall system capabilities.
2026-05-02 13:59:14 +08:00
Max
8c34aa12ef feat(llm): enhance connector handling and model configuration
- Integrated logging for connector resolution failures, providing clearer diagnostics during fallback scenarios.
- Simplified connector ID extraction in various components, ensuring accurate handling of model identifiers.
- Updated model configuration to support new parameters and capabilities, improving overall provider management.
- Enhanced OpenAPI settings to reflect changes in model options and connector behavior, ensuring better alignment with upstream API requirements.
2026-05-02 10:52:10 +08:00
Max
fa30898dee fix(tests): reduce timeout in TestProcessCall_Timeout_Short to 1 second
- Updated the timeout parameter in the TestProcessCall_Timeout_Short test to 1 second to ensure it accurately reflects expected behavior during LLM round-trip execution.
- This change aims to improve the reliability of the test by aligning the timeout with realistic processing times.
2026-05-01 15:37:17 +08:00
Max
1583e988f1 fix(llm): update connector ID handling for built-in providers
- Enhanced the extractConnectorID function to correctly return the connector ID for built-in providers without appending a model, preventing the creation of non-existent composite IDs.
- Improved logic for handling connector IDs based on provider source, ensuring accurate retrieval in various scenarios.
2026-05-01 14:19:02 +08:00
Max
9eef569e4b feat(agent): enhance system configuration and connector management
- Updated the system configuration to include new role-level defaults for Light, Vision, and Audio connectors.
- Refactored the resolveSystemConnector function to prioritize per-agent overrides, improving connector resolution logic.
- Enhanced LLMConnector integration across various components to streamline settings retrieval and capabilities management.
- Improved error handling and logging for connector-related operations, ensuring better diagnostics and user feedback.
2026-05-01 10:31:58 +08:00
Max
aa9632d67c fix(llm): update LLM provider retrieval to include optional parameter
- Modified the LLM provider retrieval functions to include an optional parameter for enhanced functionality.
- Updated the handleLLMProviderUpdate and handleLLMProviderTest functions to reflect this change, improving error handling and provider management.
2026-04-30 15:16:09 +08:00
Max
6f78f6066b feat(llm): refactor audio capabilities and enhance role management
- Replaced 'Voice' with 'Audio' in the system configuration and related tests to better reflect functionality.
- Introduced new methods for role management in the llmprovider, allowing for dynamic retrieval of roles based on user and team context.
- Updated the OpenAPI settings to support new role management endpoints and capabilities.
- Enhanced the handling of API keys in provider management, allowing for optional plain-text retrieval.
2026-04-30 12:33:05 +08:00
Max
d4553056ef
Merge pull request #1529 from trheyi/main
feat: add Settings, LLM, Search, SMTP, MCP, Sandbox module management endpoints
2026-04-29 19:26:25 +08:00
Max
a58cac6d5c fix(smtp): update address formatting in SMTP connection functions
- Replaced fmt.Sprintf with net.JoinHostPort for constructing the SMTP address in smtpValidateConnection and smtpSendTestEmail functions, improving address handling and ensuring proper formatting.
2026-04-29 18:57:36 +08:00
Max
bde4442ff6 feat(sandbox): add sandbox management endpoints and data structures
- Introduced new endpoints for managing sandbox operations, including GET, PUT, and POST methods for registry and image management.
- Added data structures for sandbox configuration and image details, enhancing the OpenAPI settings to support sandbox functionality.
- Updated Makefile to skip sandbox-related tests in CI, ensuring local execution only.
2026-04-29 18:47:38 +08:00
Max
3b642fea78 feat(mcp): add MCP server management endpoints
- Introduced new endpoints for managing MCP servers, including GET, POST, PUT, and DELETE methods for server operations.
- Organized routing under a new /mcp group to enhance endpoint management within the OpenAPI settings.
2026-04-29 14:53:00 +08:00
Max
13e16c7099 feat(smtp): add SMTP configuration and management endpoints
- Introduced new endpoints for managing SMTP settings, including GET, PUT, and POST methods for retrieving, updating, and testing SMTP configurations.
- Added data structures for SMTP presets and configuration, enhancing the OpenAPI settings to support email functionality.
- Organized routing under a new /smtp group for better endpoint management.
2026-04-29 12:21:19 +08:00
Max
c3040559e6 feat(search): add search provider endpoints and data structures
- Introduced new endpoints for managing search providers, including GET, PUT, and POST methods for testing and updating providers.
- Added data structures for search provider configuration, presets, and tool assignments to support the new functionality.
- Enhanced the OpenAPI settings to accommodate the new search-related features.
2026-04-29 11:29:03 +08:00
Max
1efa87e50a feat(llm): add Yao Agents preset and LLM management endpoints
- Introduced a new preset for Yao Agents in presets.yml, including configuration details such as API URL and capabilities.
- Added new LLM management endpoints in the OpenAPI settings, allowing for CRUD operations on LLM providers and roles.
- Enhanced data structures to support the new LLM functionality, including LLMPageData for aggregated responses.
2026-04-29 10:28:07 +08:00
Max
bba43c369a feat(setting): implement cloud settings endpoints and owner verification
- Added new endpoints for managing cloud settings under /setting/cloud, including GET, PUT, and POST methods.
- Introduced owner verification logic to ensure only team owners can modify settings, utilizing caching for efficiency.
- Enhanced data structures for cloud settings responses, including CloudRegion and CloudPageData types.
- Refactored existing functions to integrate new owner verification and error handling mechanisms.
2026-04-28 21:20:43 +08:00
Max
3bc15b1039 feat(openapi): add unified /setting/* endpoints for OpenAPI
- Introduced handlers for the new /setting/* endpoints in the OpenAPI module.
- Updated import statements to include the new setting package for better organization.
- Enhanced routing capabilities to support the new settings functionality.
2026-04-28 19:33:27 +08:00
Max
934424f9ea feat(load): add initialization and reloading for Setting Registry
- Implemented initialization for the Setting Registry during the Load process.
- Added reload functionality to refresh the Setting Registry as needed.
- Enhanced error handling to capture and report issues during initialization and reloading.
2026-04-28 15:08:22 +08:00
Max
654e7ee567 feat(load): initialize and reload LLM Provider and MCP Client Registries
- Added initialization for LLM Provider and MCP Client Registries during the Load process.
- Implemented reload functionality for both registries to ensure they are properly refreshed when needed.
- Enhanced error handling to capture and report issues during initialization and reloading of the registries.
2026-04-28 14:33:10 +08:00
Max
11317c0f86
Merge pull request #1528 from trheyi/main
feat(logging): enhance logger output control and add trace diagnostics
2026-04-28 10:38:03 +08:00
Max
9326f4b747 feat(logging): enhance logger output control and add trace diagnostics
- Updated logger methods to respect the new `config.Silent` flag, allowing suppression of console output in development mode.
- Refactored `buildTurnResponse` to utilize a shared `buildTrace` function for improved tool call association.
- Introduced JSON output capabilities in `OutputWriter` for detailed trace and duration reporting.
- Enhanced reporting mechanisms to include trace details in JSON and Markdown formats for better diagnostics.
- Added support for a new `Trace` structure to capture detailed execution information, including tool calls and completion data.
2026-04-28 10:37:04 +08:00
Max
72d7ed6f80
Merge pull request #1527 from trheyi/main
feat(sandbox): add opencode runner and enhance attachment processing
2026-04-25 11:12:35 +08:00
Max
04c3114344 feat(opencode): enhance platform-specific command handling and environment variable syntax
- Introduced `shellQuoteForPlatform` to build shell-safe command strings tailored for Windows and POSIX environments.
- Updated `buildSandboxEnvPrompt` to display the correct environment variable syntax based on the operating system.
- Refactored the command copying logic in the runner to use platform-aware commands for copying custom OpenCode tools.
- Removed outdated test scenarios related to the vision connector, streamlining the test suite.
2026-04-25 11:09:49 +08:00
Max
4b97a890fd feat(opencode): add tests for read.ts tool copying based on vision connector presence
- Implemented two new test scenarios: one to verify that read.ts is not copied when no vision connector is present, and another to confirm that it is copied when a vision connector is configured.
- Updated the runner logic to conditionally copy custom tools into the OpenCode global config directory based on the presence of a vision connector in the configuration.
2026-04-25 09:48:06 +08:00
Max
12e88943e9 feat(sandbox): add opencode runner and enhance attachment processing
- Registered new "opencode" and "opencode/cli" runners in the sandbox initialization.
- Updated the `runExecStep` function to use the computer's working directory and set the HOME environment variable.
- Refactored attachment processing in the Claude package to delegate to shared functionality, improving code maintainability.
2026-04-24 23:26:55 +08:00
Max
142c53889e
Merge pull request #1526 from trheyi/main
feat: sui backend script test and fix streamline release process
2026-04-24 09:13:34 +08:00
Max
105c3aae5b fix(workflows): streamline release process for Linux and macOS
- Removed the workflow_dispatch trigger from both release workflows to enforce tag-based releases.
- Added a mechanism to wait for draft releases before uploading assets, ensuring all necessary files are present.
- Implemented asset upload to GitHub releases and conditional publishing based on asset completeness.
- Enhanced SHA256 checksum generation for both production and development binaries.
2026-04-24 09:11:29 +08:00
Max
a1e745ec90 chore(tests): update model versions in Anthropic tests and add Test command to SUI
- Updated testConnectorID comment and model version in TestAnthropicStreamRetry to reflect the latest model (Claude Haiku 4.5).
- Added Test command to the SUI command set with associated flags for improved testing capabilities.
2026-04-24 08:45:19 +08:00
Max
9ff123433a
Merge pull request #1525 from trheyi/main
feat(doc): add yao doc CLI commands and YAML documentation
2026-04-23 22:30:02 +08:00
Max
c47d593c58 chore(deps): update spdystream dependency to v0.5.1 and adjust model version in tests
- Updated the spdystream dependency version from v0.5.0 to v0.5.1 in go.mod and go.sum files.
- Modified testConnectorID comment to reflect the updated model version (Claude Haiku 3.5).
- Updated model version in TestAnthropicStreamRetry to use "claude-3-5-haiku-20241022".
2026-04-23 22:27:39 +08:00
Max
2c11a647ba refactor(logs): remove unnecessary comment in query order for log listing
- Cleaned up comments in ListLogs and ListExecutionLogs functions to enhance code clarity without altering functionality.
2026-04-23 21:39:08 +08:00
Max
efc84fbb97 feat(doc): add yao doc CLI commands and YAML documentation for all packages
Implement yao doc process list/inspect/validate and yao doc runtime
list/inspect/validate commands. Validate uses engine addressing logic
(process.Of) and checks dynamic-ID group registries (model, store, fs,
task, schedule) to verify resources actually exist.

- cmd/doc/: CLI command tree with process and runtime subcommands
- cmd/root.go: wire docCmd into rootCmd
- 27 process doc.yml + doc.go pairs across yao packages
- cmd/doc/doc_test.go: integration tests

Made-with: Cursor
2026-04-23 21:37:43 +08:00
Max
936d104d31
Merge pull request #1524 from trheyi/main
feat(user): add token-based login endpoint for automation and testing
2026-04-22 14:13:26 +08:00
Max
4d22a9a655 feat(user): add token-based login endpoint for automation and testing
- Implemented GinTokenLogin function to handle POST requests for token-based login.
- Allows users to authenticate using a pre-signed access token, returning session cookies and user info.
- Updated user routes to include the new /token/login endpoint for public access.
2026-04-22 14:12:53 +08:00
Max
0578a068a9
Merge pull request #1523 from trheyi/main
update README
2026-04-21 00:16:05 +08:00
Max
e7bb997e2a docs: update README files to reflect new branding for Yao Desktop
- Changed the label for the desktop app in both English and Chinese README files to "Yao Desktop" for consistency and improved branding.
- Ensured that the visual elements align with the updated terminology across both language versions.
2026-04-21 00:15:12 +08:00
Max
f61dbe7390 docs: update README files for improved formatting and clarity
- Removed unnecessary separators in the English and Chinese README files for a cleaner presentation.
- Ensured consistent formatting across both language versions to enhance readability.
2026-04-21 00:10:20 +08:00
Max
04501a1274 docs: update README and Chinese translation for clarity and consistency
- Revised the English README to reflect the new branding and features of Yao as an AI runtime for building applications.
- Enhanced the structure and content of the README, including quick links, how it works, and features sections.
- Updated the Chinese README to align with the changes made in the English version, ensuring consistency across languages.
- Improved visual elements and descriptions to better convey the functionality and purpose of Yao.
2026-04-21 00:06:18 +08:00
Max
ee9ca0a132
Merge pull request #1522 from trheyi/main
fix(workflows): refine release and notarization processes for macOS a…
2026-04-19 18:49:06 +08:00
Max
aabd875bdf fix(workflows): refine release and notarization processes for macOS and Linux
- Updated the notarization workflow to correctly reference the "Release macOS" workflow.
- Enhanced version and run ID resolution logic to ensure accurate detection from the latest release or inputs.
- Improved asset verification process on R2 with a retry mechanism to handle potential delays in asset availability.
- Removed the deprecated unified release workflow, streamlining the release process for macOS and Linux.
- Adjusted CDN update prerequisites to reflect the new workflow structure.
2026-04-19 18:48:06 +08:00
Max
4e645d1ed9
Merge pull request #1521 from trheyi/main
fix(workflows): improve version and run ID resolution in notarization…
2026-04-19 18:20:20 +08:00
Max
0ffd04c428 fix(workflows): improve version and run ID resolution in notarization and release workflows
- Simplified version detection logic in the notarization workflow to prioritize the latest GitHub release.
- Enhanced error handling for cases where version or run ID cannot be determined.
- Updated the release workflow to ensure consistent version extraction from the upstream branch or fallback to the latest release.
2026-04-19 18:18:07 +08:00
Max
41969759a8
Merge pull request #1520 from trheyi/main
fix(workflows): update R2 bucket and asset paths for consistency acro…
2026-04-19 17:55:55 +08:00
Max
b210758370 fix(workflows): update R2 bucket and asset paths for consistency across workflows
- Changed default R2_BUCKET from 'get-yaoapps' to 'releases' for better alignment with asset management.
- Updated asset prefix paths in multiple workflows to remove 'releases/' prefix, ensuring uniformity in asset retrieval.
- Enhanced localization in command outputs to reflect updated CDN URLs for better user experience.
2026-04-19 17:55:14 +08:00
Max
ccedc28828
Merge pull request #1519 from trheyi/main
feat(workflows): enhance notarization and release processes for macOS…
2026-04-19 17:18:49 +08:00
Max
f7ee21372f feat(workflows): enhance notarization and release processes for macOS and Linux
- Updated the notarization workflow to automatically resolve version and run ID from the latest release or inputs.
- Added a finalize job to verify assets on R2 and trigger CDN updates after notarization.
- Improved the release workflow to upload binaries to R2 with enhanced error handling and asset verification.
- Refactored upgrade command to support CDN fallback for version resolution, improving update reliability.
- Enhanced localization in the command output for better user experience.
2026-04-19 17:18:24 +08:00
Max
735a5efab7
Merge pull request #1516 from trheyi/main
Feat: Attachment Metadata + Device Auth Support
2026-04-14 18:54:58 +08:00
Max
dd9c81068b feat(user): implement Device Flow support in OAuth provider configuration
- Added DeviceClientID and DeviceClientSecret fields to the Provider struct for Device Flow (RFC 8628) support.
- Introduced new types and methods for handling Device Authorization and Token requests/responses.
- Updated API routes to include endpoints for initiating Device Flow and polling for tokens.
- Enhanced loadProviders function to process new device-related environment variables.
2026-04-14 18:23:30 +08:00
Max
1deef3cabf feat(robot): enhance DeliveryAttachment structure with size and content type fields
- Added Size field to represent the file size in bytes.
- Introduced ContentType field to specify the MIME type of the attachment.
- Updated API endpoint to support bundling multiple files into a ZIP archive.
2026-04-14 15:01:05 +08:00
Max
36d92f1901
Merge pull request #1515 from trheyi/main
feat(workspace): add workspace ID retrieval and integration into environment configuration
2026-04-14 09:04:02 +08:00
Max
c976a3e398 chore(dependencies): update Go module dependencies to latest versions
- Upgraded several dependencies in go.mod and go.sum, including golang.org/x/crypto, golang.org/x/net, golang.org/x/sys, and google.golang.org/grpc to their latest versions for improved security and performance.
- Updated opentelemetry packages to v1.43.0 and added new indirect dependencies for better observability support.
- Adjusted the handling of execute messages in stream.go to ensure proper persistence based on message status.
- Enhanced command handling in command.go to maintain system prompt constraints during continuation turns.
2026-04-13 23:43:17 +08:00
Max
0cbc7d961d refactor(parser): streamline event handling and tool state management
- Replaced chunkRecord with recordedEvent to simplify event recording.
- Introduced activeToolID to manage the currently streaming tool, allowing for better state handling during parsing.
- Enhanced closeStreamingTool and suspendStreamingTool methods for improved tool message management.
- Updated parsing logic to handle multiple concurrent tool calls more effectively.
- Added utility functions for extracting message groups and properties from recorded events.
2026-04-13 22:24:40 +08:00
Max
c987bd687f fix(sandbox): pass explicit user to docker exec for UID-mapped containers
After tai sandbox containers start as root with dynamic UID mapping,
docker exec defaults to root. Add User field to ExecOptions and
hardcode "sandbox" in Box.Exec/Stream to ensure commands run as the
correct user.

Made-with: Cursor
2026-04-13 18:26:41 +08:00
Max
2084e20479 feat(workspace): add workspace ID retrieval and integration into environment configuration
- Implemented GetWorkspaceID method in the context package to retrieve the workspace ID.
- Updated buildContextVariables to include WORKSPACE_ID in the context variables if available.
- Enhanced buildEnv function to set CTX_WORKSPACE_ID in the environment configuration based on the workspace ID.
- Refactored role connector handling to support multiple connectors in the environment setup.
- Added tests for new role connector functionality and workspace ID integration.
2026-04-13 14:47:08 +08:00
368 changed files with 42402 additions and 2895 deletions

25
.github/workflows/create-release.yml vendored Normal file
View file

@ -0,0 +1,25 @@
name: Create Release
on:
push:
tags:
- "v*"
permissions:
contents: write
jobs:
create:
runs-on: ubuntu-latest
steps:
- name: Create Draft Release
env:
GH_TOKEN: ${{ github.token }}
run: |
TAG="${GITHUB_REF#refs/tags/}"
VERSION="${TAG#v}"
gh release create "$TAG" \
--repo "$GITHUB_REPOSITORY" \
--title "Yao v${VERSION}" \
--generate-notes \
--draft

View file

@ -1,23 +1,77 @@
name: Notarize macOS
on:
workflow_run:
workflows: ["Release macOS"]
types: [completed]
workflow_dispatch:
inputs:
run_id:
description: "Release macOS workflow run ID (to download artifacts from)"
required: true
version:
description: "Version used in the release build (e.g. 1.0.0 or 1.0.0-alpha)"
required: true
description: "Version (auto-detected from latest release if empty)"
required: false
run_id:
description: "Release macOS workflow run ID (auto-detected if empty)"
required: false
permissions:
contents: write
actions: write
concurrency:
group: notarize-${{ github.event.workflow_run.head_branch || github.run_id }}
cancel-in-progress: true
jobs:
# ===================================================================
# Resolve version + macOS build run_id automatically
# ===================================================================
resolve:
runs-on: ubuntu-latest
if: >
github.event_name == 'workflow_dispatch' ||
(github.event.workflow_run.conclusion == 'success' &&
startsWith(github.event.workflow_run.head_branch, 'v'))
outputs:
version: ${{ steps.resolve.outputs.version }}
run_id: ${{ steps.resolve.outputs.run_id }}
steps:
- name: Resolve version and run_id
id: resolve
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
VERSION="${{ github.event.inputs.version }}"
RUN_ID="${{ github.event.inputs.run_id }}"
if [ -z "$VERSION" ]; then
TAG=$(gh release view --repo "$GITHUB_REPOSITORY" --json tagName -q '.tagName')
VERSION="${TAG#v}"
fi
if [ -z "$RUN_ID" ]; then
RUN_ID=$(gh run list --repo "$GITHUB_REPOSITORY" \
--workflow="Release macOS" --branch="v${VERSION}" --limit=1 \
--json databaseId,conclusion --jq '.[] | select(.conclusion=="success") | .databaseId')
fi
else
TAG="${{ github.event.workflow_run.head_branch }}"
VERSION="${TAG#v}"
RUN_ID="${{ github.event.workflow_run.id }}"
fi
if [ -z "$VERSION" ] || [ -z "$RUN_ID" ]; then
echo "::error::Failed to resolve version='${VERSION}' run_id='${RUN_ID}'"
exit 1
fi
echo "version=${VERSION}" >> $GITHUB_OUTPUT
echo "run_id=${RUN_ID}" >> $GITHUB_OUTPUT
echo "Resolved: version=${VERSION} run_id=${RUN_ID}"
# ===================================================================
# Notarize Yao binaries (arm64 + amd64)
# ===================================================================
notarize:
needs: resolve
runs-on: macos-latest
strategy:
matrix:
@ -28,7 +82,7 @@ jobs:
with:
name: yao-darwin-${{ matrix.arch }}
path: bin
run-id: ${{ github.event.inputs.run_id }}
run-id: ${{ needs.resolve.outputs.run_id }}
github-token: ${{ secrets.GITHUB_TOKEN }}
- name: Install Certificates
@ -86,3 +140,72 @@ jobs:
exit 1
fi
echo "Yao ${{ matrix.arch }} notarization accepted."
# ===================================================================
# After both architectures finish: wait for Linux R2, then trigger CDN
# ===================================================================
finalize:
needs: [resolve, notarize]
runs-on: ubuntu-latest
if: success()
env:
AWS_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
R2_ENDPOINTS: ${{ secrets.R2_ENDPOINTS }}
R2_BUCKET: ${{ secrets.R2_BUCKET || 'releases' }}
steps:
- name: Checkout (for gh CLI context)
uses: actions/checkout@v4
with:
sparse-checkout: .github
- name: Configure AWS CLI
run: |
aws configure set default.region us-east-1
aws configure set default.s3.signature_version s3v4
- name: Wait for all platform assets on R2
run: |
VERSION="${{ needs.resolve.outputs.version }}"
PREFIX="yao/${VERSION}"
PLATFORMS=(
"darwin-arm64"
"darwin-amd64"
"linux-amd64"
"linux-arm64"
)
for ATTEMPT in $(seq 1 30); do
MISSING=0
for P in "${PLATFORMS[@]}"; do
KEY="${PREFIX}/yao-${VERSION}-${P}"
if ! aws s3 ls "s3://${R2_BUCKET}/${KEY}" --endpoint-url "$R2_ENDPOINTS" >/dev/null 2>&1; then
MISSING=$((MISSING+1))
fi
if ! aws s3 ls "s3://${R2_BUCKET}/${KEY}.sha256" --endpoint-url "$R2_ENDPOINTS" >/dev/null 2>&1; then
MISSING=$((MISSING+1))
fi
done
if [ "$MISSING" -eq 0 ]; then
echo "All 4 platform assets verified on R2."
exit 0
fi
echo "Attempt $ATTEMPT: $MISSING asset(s) still missing, waiting 30s..."
sleep 30
done
echo "::error::Timed out waiting for all platform assets on R2."
exit 1
- name: Trigger CDN latest.json update
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
VERSION="${{ needs.resolve.outputs.version }}"
gh workflow run update-cdn-latest.yml \
-f version="${VERSION}" \
-f mark_latest="true"
echo "Triggered update-cdn-latest.yml for ${VERSION}"

View file

@ -1,7 +1,6 @@
name: Release Linux
on:
workflow_dispatch:
push:
tags:
- "v*"
@ -20,20 +19,7 @@ jobs:
runs-on: ubuntu-latest
container:
image: yaoapp/yao-build:1.0.0
env:
CF_ACCESS_KEY_ID: ${{ secrets.CF_ACCESS_KEY_ID }}
CF_SECRET_ACCESS_KEY: ${{ secrets.CF_SECRET_ACCESS_KEY }}
R2_BUCKET: ${{ secrets.R2_BUCKET }}
R2_ACCOUNT_ID: ${{ secrets.R2_ACCOUNT_ID }}
steps:
- name: Configure R2 For Cloudflare
run: |
aws configure set aws_access_key_id $CF_ACCESS_KEY_ID
aws configure set aws_secret_access_key $CF_SECRET_ACCESS_KEY
aws configure set default.region us-east-1
aws configure set default.s3.signature_version s3v4
aws configure set default.s3.endpoint_url https://$R2_ACCOUNT_ID.r2.cloudflarestorage.com
- name: Build
run: |
export PATH=$PATH:/github/home/go/bin
@ -72,13 +58,6 @@ jobs:
mv /app/yao/dist/release/* /data/
ls -l /data
- name: Push To R2
run: |
for file in /data/*; do
aws s3 cp "$file" s3://$R2_BUCKET/archives/ \
--endpoint-url https://$R2_ACCOUNT_ID.r2.cloudflarestorage.com
done
- name: Upload Artifact
uses: actions/upload-artifact@v4
with:
@ -106,6 +85,27 @@ jobs:
echo "version=${VERSION}" >> $GITHUB_OUTPUT
echo "VERSION=${VERSION}"
- name: Download Linux Artifacts
uses: actions/download-artifact@v4
with:
name: yao-linux
path: artifacts
- name: Prepare Docker Contexts
run: |
VERSION="${{ steps.version.outputs.version }}"
ls -la artifacts/
# Development image uses dev (unstripped) binaries
cp "artifacts/yao-${VERSION}-linux-amd64" docker/development/yao-amd64
cp "artifacts/yao-${VERSION}-linux-arm64" docker/development/yao-arm64
chmod +x docker/development/yao-*
# Production image uses prod (stripped) binaries
cp "artifacts/yao-${VERSION}-linux-amd64-prod" docker/production/yao-amd64
cp "artifacts/yao-${VERSION}-linux-arm64-prod" docker/production/yao-arm64
chmod +x docker/production/yao-*
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
@ -123,8 +123,6 @@ jobs:
with:
context: ./docker/development
platforms: linux/amd64,linux/arm64
build-args: |
VERSION=${{ steps.version.outputs.version }}
push: true
tags: |
${{ env.IMAGE_NAME }}:${{ steps.version.outputs.version }}-dev
@ -135,9 +133,113 @@ jobs:
with:
context: ./docker/production
platforms: linux/amd64,linux/arm64
build-args: |
VERSION=${{ steps.version.outputs.version }}
push: true
tags: |
${{ env.IMAGE_NAME }}:${{ steps.version.outputs.version }}
${{ env.IMAGE_NAME }}:latest
# ===================================================================
# GitHub Release + R2 Upload (Linux binaries)
# ===================================================================
release:
needs: build
if: startsWith(github.ref, 'refs/tags/v')
runs-on: ubuntu-latest
steps:
- name: Get Version
id: version
run: |
if [[ "$GITHUB_REF" != refs/tags/v* ]]; then
echo "::error::This workflow requires a tag. Got: $GITHUB_REF"
exit 1
fi
VERSION="${GITHUB_REF#refs/tags/v}"
TAG="${GITHUB_REF#refs/tags/}"
echo "version=${VERSION}" >> $GITHUB_OUTPUT
echo "tag=${TAG}" >> $GITHUB_OUTPUT
- name: Download Linux Artifacts
uses: actions/download-artifact@v4
with:
name: yao-linux
path: artifacts
- name: Prepare Release Files
run: |
VERSION="${{ steps.version.outputs.version }}"
mkdir -p release
cp "artifacts/yao-${VERSION}-linux-amd64-prod" "release/yao-${VERSION}-linux-amd64"
cp "artifacts/yao-${VERSION}-linux-arm64-prod" "release/yao-${VERSION}-linux-arm64"
cp "artifacts/yao-${VERSION}-linux-amd64" "release/yao-${VERSION}-linux-amd64-dev"
cp "artifacts/yao-${VERSION}-linux-arm64" "release/yao-${VERSION}-linux-arm64-dev"
chmod +x release/yao-*
for ARCH in amd64 arm64; do
sha256sum "release/yao-${VERSION}-linux-${ARCH}" | awk '{print $1}' > "release/yao-linux-${ARCH}-prod.sha256"
sha256sum "release/yao-${VERSION}-linux-${ARCH}-dev" | awk '{print $1}' > "release/yao-linux-${ARCH}-dev.sha256"
done
ls -lh release/
- name: Wait for Draft Release
env:
GH_TOKEN: ${{ github.token }}
run: |
TAG="${{ steps.version.outputs.tag }}"
for i in $(seq 1 30); do
if gh release view "$TAG" --repo "$GITHUB_REPOSITORY" &>/dev/null; then
echo "Draft release found for $TAG."
exit 0
fi
echo "Waiting for draft release... ($i/30)"
sleep 10
done
echo "::error::Timed out waiting for draft release $TAG"
exit 1
- name: Upload Assets to GitHub Release
env:
GH_TOKEN: ${{ github.token }}
run: |
TAG="${{ steps.version.outputs.tag }}"
gh release upload "$TAG" release/* --repo "$GITHUB_REPOSITORY" --clobber
- name: Publish Release if Complete
env:
GH_TOKEN: ${{ github.token }}
run: |
TAG="${{ steps.version.outputs.tag }}"
ASSET_COUNT=$(gh release view "$TAG" --repo "$GITHUB_REPOSITORY" --json assets --jq '.assets | length')
echo "Current assets: $ASSET_COUNT / 16"
if [ "$ASSET_COUNT" -ge 16 ]; then
echo "All assets present, publishing release..."
gh release edit "$TAG" --repo "$GITHUB_REPOSITORY" --draft=false --latest
else
echo "Assets incomplete ($ASSET_COUNT/16), waiting for other workflow to publish."
fi
- name: Upload Linux binaries to R2
env:
AWS_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
R2_ENDPOINTS: ${{ secrets.R2_ENDPOINTS }}
R2_BUCKET: ${{ secrets.R2_BUCKET || 'releases' }}
run: |
aws configure set default.region us-east-1
aws configure set default.s3.signature_version s3v4
VERSION="${{ steps.version.outputs.version }}"
PREFIX="yao/${VERSION}"
for PLATFORM in linux-amd64 linux-arm64; do
FILE="release/yao-${VERSION}-${PLATFORM}"
NAME="yao-${VERSION}-${PLATFORM}"
sha256sum "$FILE" | awk '{print $1}' > "/tmp/${NAME}.sha256"
aws s3 cp "$FILE" "s3://${R2_BUCKET}/${PREFIX}/${NAME}" \
--endpoint-url "$R2_ENDPOINTS" \
--content-type "application/octet-stream"
aws s3 cp "/tmp/${NAME}.sha256" "s3://${R2_BUCKET}/${PREFIX}/${NAME}.sha256" \
--endpoint-url "$R2_ENDPOINTS" \
--content-type "text/plain"
echo "Uploaded: ${NAME} + ${NAME}.sha256"
done

View file

@ -1,7 +1,6 @@
name: Release macOS
on:
workflow_dispatch:
push:
tags:
- "v*"
@ -217,3 +216,129 @@ jobs:
with:
name: yao-darwin-checksums
path: /tmp/checksums/*.sha256
# ===================================================================
# GitHub Release + R2 Upload (macOS binaries)
# ===================================================================
release:
needs: build
if: startsWith(github.ref, 'refs/tags/v')
runs-on: ubuntu-latest
steps:
- name: Get Version
id: version
run: |
if [[ "$GITHUB_REF" != refs/tags/v* ]]; then
echo "::error::This workflow requires a tag. Got: $GITHUB_REF"
exit 1
fi
VERSION="${GITHUB_REF#refs/tags/v}"
TAG="${GITHUB_REF#refs/tags/}"
echo "version=${VERSION}" >> $GITHUB_OUTPUT
echo "tag=${TAG}" >> $GITHUB_OUTPUT
- name: Download macOS Artifacts
uses: actions/download-artifact@v4
with:
name: yao-darwin-arm64
path: artifacts/arm64-prod
- name: Download arm64 Dev
uses: actions/download-artifact@v4
with:
name: yao-darwin-arm64-dev
path: artifacts/arm64-dev
- name: Download amd64 Prod
uses: actions/download-artifact@v4
with:
name: yao-darwin-amd64
path: artifacts/amd64-prod
- name: Download amd64 Dev
uses: actions/download-artifact@v4
with:
name: yao-darwin-amd64-dev
path: artifacts/amd64-dev
- name: Download Checksums
uses: actions/download-artifact@v4
with:
name: yao-darwin-checksums
path: artifacts/checksums
- name: Prepare Release Files
run: |
VERSION="${{ steps.version.outputs.version }}"
mkdir -p release
cp artifacts/arm64-prod/yao "release/yao-${VERSION}-darwin-arm64"
cp artifacts/amd64-prod/yao "release/yao-${VERSION}-darwin-amd64"
cp artifacts/arm64-dev/yao "release/yao-${VERSION}-darwin-arm64-dev"
cp artifacts/amd64-dev/yao "release/yao-${VERSION}-darwin-amd64-dev"
cp artifacts/checksums/*.sha256 release/ 2>/dev/null || true
chmod +x release/yao-*
ls -lh release/
- name: Wait for Draft Release
env:
GH_TOKEN: ${{ github.token }}
run: |
TAG="${{ steps.version.outputs.tag }}"
for i in $(seq 1 30); do
if gh release view "$TAG" --repo "$GITHUB_REPOSITORY" &>/dev/null; then
echo "Draft release found for $TAG."
exit 0
fi
echo "Waiting for draft release... ($i/30)"
sleep 10
done
echo "::error::Timed out waiting for draft release $TAG"
exit 1
- name: Upload Assets to GitHub Release
env:
GH_TOKEN: ${{ github.token }}
run: |
TAG="${{ steps.version.outputs.tag }}"
gh release upload "$TAG" release/* --repo "$GITHUB_REPOSITORY" --clobber
- name: Publish Release if Complete
env:
GH_TOKEN: ${{ github.token }}
run: |
TAG="${{ steps.version.outputs.tag }}"
ASSET_COUNT=$(gh release view "$TAG" --repo "$GITHUB_REPOSITORY" --json assets --jq '.assets | length')
echo "Current assets: $ASSET_COUNT / 16"
if [ "$ASSET_COUNT" -ge 16 ]; then
echo "All assets present, publishing release..."
gh release edit "$TAG" --repo "$GITHUB_REPOSITORY" --draft=false --latest
else
echo "Assets incomplete ($ASSET_COUNT/16), waiting for other workflow to publish."
fi
- name: Upload macOS binaries to R2
env:
AWS_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
R2_ENDPOINTS: ${{ secrets.R2_ENDPOINTS }}
R2_BUCKET: ${{ secrets.R2_BUCKET || 'releases' }}
run: |
aws configure set default.region us-east-1
aws configure set default.s3.signature_version s3v4
VERSION="${{ steps.version.outputs.version }}"
PREFIX="yao/${VERSION}"
for PLATFORM in darwin-arm64 darwin-amd64; do
FILE="release/yao-${VERSION}-${PLATFORM}"
NAME="yao-${VERSION}-${PLATFORM}"
sha256sum "$FILE" | awk '{print $1}' > "/tmp/${NAME}.sha256"
aws s3 cp "$FILE" "s3://${R2_BUCKET}/${PREFIX}/${NAME}" \
--endpoint-url "$R2_ENDPOINTS" \
--content-type "application/octet-stream"
aws s3 cp "/tmp/${NAME}.sha256" "s3://${R2_BUCKET}/${PREFIX}/${NAME}.sha256" \
--endpoint-url "$R2_ENDPOINTS" \
--content-type "text/plain"
echo "Uploaded: ${NAME} + ${NAME}.sha256"
done

View file

@ -1,114 +0,0 @@
name: Release
on:
workflow_run:
workflows: ["Release Linux", "Release macOS"]
types:
- completed
permissions:
contents: write
jobs:
# ===================================================================
# Wait for both workflows to succeed, then create a unified release
# ===================================================================
release:
runs-on: ubuntu-latest
if: >
github.event.workflow_run.conclusion == 'success' &&
startsWith(github.event.workflow_run.head_branch, 'v')
steps:
- name: Checkout Code
uses: actions/checkout@v4
- name: Get Version
id: version
run: |
TAG="${{ github.event.workflow_run.head_branch }}"
VERSION="${TAG#v}"
echo "version=${VERSION}" >> $GITHUB_OUTPUT
echo "tag=${TAG}" >> $GITHUB_OUTPUT
echo "TAG=${TAG} VERSION=${VERSION}"
- name: Wait for Both Workflows
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
TAG="${{ steps.version.outputs.tag }}"
echo "Waiting for both Release Linux and Release macOS to complete for $TAG..."
for i in $(seq 1 60); do
LINUX_STATUS=$(gh run list --workflow="Release Linux" --branch="$TAG" --limit=1 --json conclusion --jq '.[0].conclusion // "pending"')
MACOS_STATUS=$(gh run list --workflow="Release macOS" --branch="$TAG" --limit=1 --json conclusion --jq '.[0].conclusion // "pending"')
echo "Attempt $i: Linux=$LINUX_STATUS macOS=$MACOS_STATUS"
if [ "$LINUX_STATUS" = "success" ] && [ "$MACOS_STATUS" = "success" ]; then
echo "Both workflows completed successfully."
exit 0
fi
if [ "$LINUX_STATUS" = "failure" ] || [ "$MACOS_STATUS" = "failure" ]; then
echo "::error::One or both workflows failed (Linux=$LINUX_STATUS macOS=$MACOS_STATUS)"
exit 1
fi
sleep 60
done
echo "::error::Timed out waiting for workflows"
exit 1
- name: Download Linux Artifacts
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
TAG="${{ steps.version.outputs.tag }}"
LINUX_RUN_ID=$(gh run list --workflow="Release Linux" --branch="$TAG" --limit=1 --json databaseId --jq '.[0].databaseId')
mkdir -p dist/linux
gh run download "$LINUX_RUN_ID" --name yao-linux --dir dist/linux
- name: Download macOS Artifacts
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
TAG="${{ steps.version.outputs.tag }}"
MACOS_RUN_ID=$(gh run list --workflow="Release macOS" --branch="$TAG" --limit=1 --json databaseId --jq '.[0].databaseId')
mkdir -p dist/macos
gh run download "$MACOS_RUN_ID" --name yao-darwin-arm64 --dir dist/macos/arm64-prod
gh run download "$MACOS_RUN_ID" --name yao-darwin-arm64-dev --dir dist/macos/arm64-dev
gh run download "$MACOS_RUN_ID" --name yao-darwin-amd64 --dir dist/macos/amd64-prod
gh run download "$MACOS_RUN_ID" --name yao-darwin-amd64-dev --dir dist/macos/amd64-dev
gh run download "$MACOS_RUN_ID" --name yao-darwin-checksums --dir dist/macos/checksums
- name: Prepare Release Files
run: |
VERSION="${{ steps.version.outputs.version }}"
mkdir -p release
# Linux artifacts (already named correctly from build.sh)
cp dist/linux/* release/ 2>/dev/null || true
# macOS prod binaries
cp dist/macos/arm64-prod/yao "release/yao-${VERSION}-darwin-arm64"
cp dist/macos/amd64-prod/yao "release/yao-${VERSION}-darwin-amd64"
# macOS dev binaries
cp dist/macos/arm64-dev/yao "release/yao-${VERSION}-darwin-arm64-dev"
cp dist/macos/amd64-dev/yao "release/yao-${VERSION}-darwin-amd64-dev"
# Checksums
cp dist/macos/checksums/*.sha256 release/ 2>/dev/null || true
chmod +x release/yao-* 2>/dev/null || true
echo "=== Release files ==="
ls -lh release/
- name: Create GitHub Release
uses: softprops/action-gh-release@v2
with:
tag_name: ${{ steps.version.outputs.tag }}
name: Yao v${{ steps.version.outputs.version }}
files: release/*
generate_release_notes: true

121
.github/workflows/update-cdn-latest.yml vendored Normal file
View file

@ -0,0 +1,121 @@
name: Update CDN latest.json
# Assembles yao/latest.json after all platform binaries are on R2.
#
# Normally triggered automatically by notarize-macos.yml's finalize job after
# notarization completes. Can also be triggered manually as a fallback.
#
# Prerequisites: release-linux.yml and release-macos.yml must have uploaded
# all 4 platform binaries to R2.
on:
workflow_dispatch:
inputs:
version:
description: "Engine version to publish (e.g. 1.0.0 or 1.0.0-alpha)"
required: true
mark_latest:
description: "Also update yao/latest.json (set false for pre-releases you want on CDN but not as latest)"
required: false
default: "true"
permissions:
contents: read
jobs:
publish-latest:
runs-on: ubuntu-latest
env:
AWS_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
R2_ENDPOINTS: ${{ secrets.R2_ENDPOINTS }}
R2_BUCKET: ${{ secrets.R2_BUCKET || 'releases' }}
CDN_BASE: https://get.yaoapps.com
steps:
- name: Configure AWS CLI
run: |
aws configure set default.region us-east-1
aws configure set default.s3.signature_version s3v4
- name: Verify platform assets exist
run: |
VERSION="${{ github.event.inputs.version }}"
PREFIX="yao/${VERSION}"
PLATFORMS=(
"darwin-arm64"
"darwin-amd64"
"linux-amd64"
"linux-arm64"
)
MISSING=0
for P in "${PLATFORMS[@]}"; do
KEY="${PREFIX}/yao-${VERSION}-${P}"
echo "Checking s3://${R2_BUCKET}/${KEY}"
if ! aws s3 ls "s3://${R2_BUCKET}/${KEY}" --endpoint-url "$R2_ENDPOINTS" >/dev/null 2>&1; then
echo "::warning::Missing asset: ${KEY}"
MISSING=$((MISSING+1))
fi
if ! aws s3 ls "s3://${R2_BUCKET}/${KEY}.sha256" --endpoint-url "$R2_ENDPOINTS" >/dev/null 2>&1; then
echo "::warning::Missing sha256: ${KEY}.sha256"
MISSING=$((MISSING+1))
fi
done
if [ "$MISSING" -gt 0 ]; then
echo "::error::$MISSING required asset(s) are missing on R2. Run platform CI workflows first."
exit 1
fi
echo "All platform assets verified."
- name: Build latest.json
run: |
VERSION="${{ github.event.inputs.version }}"
RELEASED_AT="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
python3 <<PY > /tmp/latest.json
import json
version = "${VERSION}"
base = "${CDN_BASE}/yao/${VERSION}"
assets = {
"darwin-arm64": f"{base}/yao-{version}-darwin-arm64",
"darwin-amd64": f"{base}/yao-{version}-darwin-amd64",
"linux-amd64": f"{base}/yao-{version}-linux-amd64",
"linux-arm64": f"{base}/yao-{version}-linux-arm64",
}
sha256 = {
"darwin-arm64": f"{base}/yao-{version}-darwin-arm64.sha256",
"darwin-amd64": f"{base}/yao-{version}-darwin-amd64.sha256",
"linux-amd64": f"{base}/yao-{version}-linux-amd64.sha256",
"linux-arm64": f"{base}/yao-{version}-linux-arm64.sha256",
}
data = {
"version": version,
"released_at": "${RELEASED_AT}",
"assets": assets,
"sha256": sha256,
}
print(json.dumps(data, indent=2, ensure_ascii=False))
PY
cat /tmp/latest.json
- name: Upload versioned latest.json
run: |
VERSION="${{ github.event.inputs.version }}"
aws s3 cp /tmp/latest.json \
"s3://${R2_BUCKET}/yao/${VERSION}/latest.json" \
--endpoint-url "$R2_ENDPOINTS" \
--content-type "application/json" \
--cache-control "public, max-age=60"
- name: Promote to yao/latest.json
if: ${{ github.event.inputs.mark_latest != 'false' }}
run: |
aws s3 cp /tmp/latest.json \
"s3://${R2_BUCKET}/yao/latest.json" \
--endpoint-url "$R2_ENDPOINTS" \
--content-type "application/json" \
--cache-control "public, max-age=60"
echo "Promoted to yao/latest.json"

4
.gitignore vendored
View file

@ -83,3 +83,7 @@ agent/robot/ROBOT-CACHE-IMPROVEMENT.md
sandbox/v2/PID-KILL-UPGRADE.md
sandbox/v2/*.md
POSTGRESQL_COMPAT.md
openapi/setting/*.md
agent/docs/design/*.md
tools/README.md
tools/TOOL-REGISTRATION.md

View file

@ -1,61 +0,0 @@
> **DEPRECATED**: This license is no longer in effect. Please refer to the [LICENSE](LICENSE) file for current licensing terms.
# Commercial License for Yao
This document outlines the terms for the commercial license of the **Yao** project. While the Yao project is primarily licensed under the **Apache License, Version 2.0**, certain commercial use cases require a separate commercial license.
## 1. Commercial License Requirements
The following use cases require a commercial license:
1. **Application Hosting Services**
If you use Yao, or any derivative product (such as a forked or modified version of Yao), to provide Yao-based application hosting services (e.g., Software-as-a-Service (SaaS) or Platform-as-a-Service (PaaS)) to users, you must obtain a commercial license. This restriction applies regardless of whether the original Yao code or a modified version is used to host and manage applications on behalf of third-party users for commercial purposes.
**In addition**, if you provide hosting services for applications that are built using Yao (even if they are customized or modified versions of Yao), a commercial license is required.
### Definition: Application Hosting Services
"Application Hosting Services" refers to any service that involves hosting Yao-based applications or web applications created with Yao (including modified versions of Yao) for third-party users. This includes, but is not limited to:
- **Hosting platforms** providing software or services built on top of Yao for third-party users.
- **SaaS or PaaS offerings** where you manage and host applications that are based on or utilize Yao, either in their original or modified form.
- **Managed hosting services** where Yao is used as the underlying technology for applications deployed for external clients.
In these cases, a commercial license is required, whether you are using the original Yao code or a fork/modified version.
2. **AI Web Application Generation Services**
If you provide services that generate AI-driven web applications using Yao, or any derivative product (such as a fork or modified version of Yao), to third-party users, you are required to purchase a commercial license.
### Definition: AI Web Application Generation Services
"AI Web Application Generation Services" refers to any service or functionality that utilizes Yao (or any forked or modified version of Yao) to automate the creation of web applications with AI capabilities. This includes, but is not limited to, providing third-party users with:
- **Automated web application development** driven by AI, where the service generates complete or partial web applications.
- **Customizable web solutions** that are powered by AI and built using Yao as the core technology.
- **On-demand application generation** for specific client needs, using Yao to dynamically build, configure, or deploy applications for users.
In these cases, whether Yao is directly used, forked, or modified, a commercial license is required to operate legally.
## 2. Use Under Apache License 2.0
For all other uses, the **Apache License, Version 2.0** applies. You are free to use, modify, and distribute the Yao project under the terms of Apache 2.0 as long as your usage does not fall within the restricted scenarios outlined above.
## 3. Obtaining a Commercial License
To inquire about or obtain a commercial license, please contact us at:
- **Email**: [friends@iqka.com]
- **Website**: [https://moapi.ai/contact]
Pricing and terms for commercial licenses vary based on usage scenarios, user scale, and other factors.
## 4. Compliance and Auditing
If you have any questions about whether your use case requires a commercial license, please contact us for clarification. We reserve the right to audit usage for compliance and enforce commercial licensing terms where necessary.
## 5. Disclaimer
Failure to comply with these licensing terms may result in a violation of the Yao licensing agreement and could lead to legal action.
---
**Note:** This commercial license is supplementary to the Apache 2.0 license and only applies in specific commercial scenarios outlined above.

View file

@ -1,61 +0,0 @@
> **已废弃**: 本许可证已不再生效。请参考 [LICENSE](LICENSE) 文件获取当前的许可条款。
# Yao 商业许可证
本文件概述了 **Yao** 项目的商业许可证条款。虽然 Yao 项目主要使用 **Apache 许可证 2.0 版** 授权,但某些商业使用场景需要单独的商业许可证。
## 1. 商业许可证要求
以下使用场景需要商业许可证:
1. **应用托管服务**
如果您使用 Yao 或其衍生产品(如 Yao 的分支版本或修改版本)为用户提供基于 Yao 的应用托管服务例如软件即服务SaaS或平台即服务PaaS您必须获得商业许可证。此限制适用于无论是否使用原始 Yao 代码或修改版 Yao 代码,托管和管理应用程序的行为只要是为第三方用户提供的商业目的。
**此外**,如果您提供的托管服务是为使用 Yao 构建的应用程序提供托管服务(即使它们是定制或修改版的 Yao也需要获得商业许可证。
### 定义:应用托管服务
"应用托管服务"指任何涉及托管基于 Yao 的应用程序或使用 Yao 创建的 WEB 应用程序(包括 Yao 的修改版本)的服务,服务对象为第三方用户。包括但不限于:
- **托管平台** 提供基于 Yao 的软件或服务给第三方用户。
- **SaaS 或 PaaS 服务**,在这些服务中,您管理并托管基于或利用 Yao 的应用程序,可能是原版或修改版。
- **托管服务**,其中 Yao 被用作为客户外部部署应用程序的基础技术。
在这些情况下,无论是使用原始 Yao 代码还是修改版 Yao都需要获得商业许可证。
2. **AI WEB 应用生成服务**
如果您提供利用 Yao 或其衍生产品(如 Yao 的分支版本或修改版本)为第三方用户生成 AI 驱动的 WEB 应用程序的服务,您需要购买商业许可证。
### 定义AI WEB 应用生成服务
"AI WEB 应用生成服务"指任何利用 Yao或任何分支版本或修改版本的 Yao自动化创建具有 AI 功能的 WEB 应用程序的服务或功能。包括但不限于,为第三方用户提供以下服务:
- **AI 驱动的自动化 WEB 应用开发**,该服务生成完整或部分 WEB 应用程序。
- **可定制的 WEB 解决方案**,这些解决方案由 AI 提供支持,并以 Yao 作为核心技术构建。
- **按需应用生成**,根据特定客户需求,使用 Yao 动态构建、配置或部署应用程序。
在这些情况下,无论是直接使用 Yao还是使用其分支或修改版均需要获得商业许可证。
## 2. 使用 Apache 许可证 2.0
对于所有其他用途,**Apache 许可证 2.0 版** 适用。只要您的使用不属于上述限制的商业场景,您可以自由地根据 Apache 2.0 许可证使用、修改和分发 Yao 项目。
## 3. 获取商业许可证
如需咨询或获取商业许可证,请通过以下方式联系我们:
- **电子邮件**[friends@iqka.com]
- **网站**[https://moapi.ai/contact](https://moapi.ai/contact)
商业许可证的定价和条款会根据使用场景、用户规模及其他因素有所不同。
## 4. 合规与审计
如果您对您的使用场景是否需要商业许可证有任何疑问,请联系我们以获取澄清。我们保留审核使用情况以确保合规,并在必要时执行商业许可条款的权利。
## 5. 免责声明
未遵守这些许可条款可能会导致违反 Yao 许可证协议,并可能导致法律诉讼。
---
**注意:** 此商业许可证是 Apache 2.0 许可证的补充,仅适用于上述特定的商业场景。

18
LICENSE
View file

@ -1,13 +1,13 @@
# Open Source License
Yao App Engine is licensed under a modified version of the Apache License 2.0, with the following additional conditions:
Yao Engine is licensed under a modified version of the Apache License 2.0, with the following additional conditions:
1. Commercial Usage Terms:
Yao App Engine may be utilized commercially, A commercial license from the producer is required if:
Yao Engine may be utilized commercially, A commercial license from the producer is required if:
a. Trademark and Branding Requirements
- The Yao App Engine console/application logo and copyright information must not be removed or modified
- The Yao Engine / Yao Agents / Tai / Tai Link console/application logo and copyright information must not be removed or modified
- Logo and copyright information can only be changed with an authorization certificate issued through Yao Developer Certificate
b. Authorization Verification Requirements
@ -15,10 +15,16 @@ Yao App Engine is licensed under a modified version of the Apache License 2.0, w
- The Yao certificate verification logic, processes, and related pages (marked in code comments) must be preserved
- The complete Yao certificate verification system must be maintained regardless of usage purpose
c. Enterprise Scale Requirements
- Organizations with 50 or more employees, or with annual revenue exceeding USD 1,000,000, must obtain a commercial license from Infinite Wisdom Software.
- To obtain a commercial license, please contact us at https://yaoagents.com/enterprise
2. Contributor Agreement:
- The producer reserves the right to modify the open-source agreement terms
- Contributed code may be used for commercial purposes, including cloud business operations
As a contributor, you should agree that:
a. Infinite Wisdom Software can adjust the open-source agreement to be more strict or relaxed as deemed necessary.
b. Your contributed code may be used for commercial purposes, including but not limited to its cloud business operations.
All other rights and restrictions follow the Apache License 2.0 (http://www.apache.org/licenses/LICENSE-2.0).
© 2025 Infinite Wisdom Software.
© 2026 Infinite Wisdom Software.

30
LICENSE.zh-CN Normal file
View file

@ -0,0 +1,30 @@
# 开源许可证
Yao Engine 基于修改版 Apache License 2.0 授权,并附加以下额外条款:
1. 商业使用条款:
Yao Engine 可用于商业用途,但在以下情况下须向 Infinite Wisdom Software 获取商业授权许可:
a. 商标与品牌要求
- 不得删除或修改 Yao Engine / Yao Agents / Tai / Tai Link 控制台/应用程序的徽标及版权信息
- 徽标和版权信息仅可在持有通过 Yao 开发者证书颁发的授权证书时方可更改
b. 授权验证要求
- 必须保留 Yao 证书验证逻辑、流程及相关页面(已在代码注释中标注)
- 无论使用目的如何,必须维持完整的 Yao 证书验证系统
c. 企业规模要求
- 员工人数达到 50 人及以上,或年收入超过 100 万美元的企业,须向 Infinite Wisdom Software 购买商业授权许可。
- 如需获取商业授权,请访问 https://yaoagents.com/enterprise 联系我们。
2. 贡献者协议:
作为贡献者,您需同意以下条款:
a. Infinite Wisdom Software 可视需要对本开源协议进行更严格或更宽松的调整。
b. 您贡献的代码可被用于商业用途,包括但不限于云服务业务运营。
其他所有权利与限制遵循 Apache License 2.0http://www.apache.org/licenses/LICENSE-2.0)。
© 2026 Infinite Wisdom Software.

View file

@ -11,6 +11,7 @@ OS := $(shell uname)
# ROOT_DIR := $(shell dirname $(realpath $(firstword $(MAKEFILE_LIST))))
TESTFOLDER := $(shell $(GO) list ./... | grep -vE 'examples|openai|aigc|neo|twilio|share*|registry|agent/sandbox/v2' | awk '!/\/tests\// || /openapi\/tests/' | grep -vE 'openapi/tests/(nodes|sandbox|workspace)')
# Sandbox setting tests (openapi/tests/setting/sandbox_test.go) require Docker + Tai — skipped in CI, run locally only
# Core tests (exclude AI-related: agent, aigc, openai, KB, sandbox, registry, grpc, and integrations which require external services)
TESTFOLDER_CORE := $(shell $(GO) list ./... | grep -vE 'examples|openai|aigc|neo|twilio|share*|agent|kb|sandbox|integrations|registry|tai|grpc' | awk '!/\/tests\// || /openapi\/tests/' | grep -vE 'openapi/tests/(nodes|sandbox|workspace)')
# Agent tests (agent, aigc) - exclude agent/search/handlers/web (requires external API keys), robot packages (tested in robot job), and agent/sandbox/v2 (WIP, has its own job)
@ -36,7 +37,7 @@ TESTTAGS ?= ""
unit-test:
echo "mode: count" > coverage.out
for d in $(TESTFOLDER); do \
$(GO) test -tags $(TESTTAGS) -v -covermode=count -coverprofile=profile.out -coverpkg=$$(echo $$d | sed "s/\/test$$//g") -skip='TestMemoryLeak|TestIsolateDisposal|TestLeak_|TestScenario_' $$d > tmp.out; \
$(GO) test -tags $(TESTTAGS) -v -covermode=count -coverprofile=profile.out -coverpkg=$$(echo $$d | sed "s/\/test$$//g") -skip='TestMemoryLeak|TestIsolateDisposal|TestLeak_|TestScenario_|TestSandbox' $$d > tmp.out; \
cat tmp.out; \
if grep -q "^--- FAIL" tmp.out; then \
rm tmp.out; \
@ -68,7 +69,7 @@ unit-test:
unit-test-core:
echo "mode: count" > coverage.out
for d in $(TESTFOLDER_CORE); do \
$(GO) test -tags $(TESTTAGS) -v -covermode=count -coverprofile=profile.out -coverpkg=$$(echo $$d | sed "s/\/test$$//g") -skip='TestMemoryLeak|TestIsolateDisposal|TestLeak_|TestScenario_' $$d > tmp.out; \
$(GO) test -tags $(TESTTAGS) -v -covermode=count -coverprofile=profile.out -coverpkg=$$(echo $$d | sed "s/\/test$$//g") -skip='TestMemoryLeak|TestIsolateDisposal|TestLeak_|TestScenario_|TestSandbox' $$d > tmp.out; \
cat tmp.out; \
if grep -q "^--- FAIL" tmp.out; then \
rm tmp.out; \

View file

@ -1,65 +1,71 @@
# Yao — Build Autonomous Agents. Just Define the Role.
# Yao — App Runtime for the AI Era
Yao is an open-source engine for autonomous agents — event-driven, proactive, and self-scheduling.
Yao is an open-source runtime for building AI agents and web applications — shipped as a single binary.
![Mission Control](docs/mission-control.png)
**Quick Links:**
**🏠 Homepage:** [https://yaoagents.com](https://yaoagents.com)
**🏠 Homepage:** [https://yaoapps.com](https://yaoapps.com)
**📚 Docs:** [https://yaoagents.com/docs](https://yaoagents.com/docs)
**🚀 Quick Start:** [https://yaoapps.com/docs/documentation/en-us/getting-started](https://yaoapps.com/docs/documentation/en-us/getting-started#quickstart)
**📚 Documentation:** [https://yaoapps.com/docs](https://yaoapps.com/docs)
**✨ Why Yao?** [https://yaoapps.com/docs/why-yao](https://yaoapps.com/docs/documentation/en-us/getting-started/why-yao)
**🤖 Yao Agents:** [https://github.com/YaoAgents/awesome](https://github.com/YaoAgents/awesome) ( Preview )
**🖥️ Yao Desktop:** [https://yaoagents.com/download](https://yaoagents.com/download)
---
## What Makes Yao Different?
## How It Works
| Traditional AI Assistants | Yao Autonomous Agents |
| ----------------------------- | ------------------------------------- |
| Entry point: Chatbox | Entry point: Email, Events, Schedules |
| Passive: You ask, they answer | Proactive: They work autonomously |
| Role: Tool | Role: Team member |
Think of Yao Agent as a **cage, not an animal**. What you put inside determines the behavior; the cage keeps it controlled.
> The entry point is not a chatbox — it's email, events, and scheduled tasks.
Every request flows through the same pipeline:
![Pipeline](docs/pipeline.png)
`Create Hook` runs before the executor — inject context, enforce constraints, route requests.
`Next Hook` runs after — validate output, trigger downstream actions, drive multi-step loops.
**The AI does the heavy lifting. You define the boundaries.**
### Three Modes
| Mode | Executor | When to use |
|------|----------|-------------|
| **LLM** | OpenAI, Anthropic, etc. | Conversational assistants, Q&A, content generation |
| **CLI Agent** | OpenCode, Claude Code, Codex in a container | Computer use, sandbox isolation, SKILL ecosystem |
| **Pure Hook** | Your own TypeScript code | Deterministic logic, routing, menu flows — no AI needed |
All three share the same Hook interface. You can mix them freely — route some requests through the LLM, handle others with pure code, all inside a single `Create Hook`.
---
## Features
### Autonomous Agent Framework
### Agent Framework
Build agents that work like real team members:
- **Three Trigger Modes** — Clock (scheduled), Human (email/message), Event (webhook/database)
- **Six-Phase Execution** — Inspiration → Goals → Tasks → Run → Deliver → Learn
- **Multi-Agent Orchestration** — Agents delegate, collaborate, and compose dynamically
- **Continuous Learning** — Agents accumulate experience in private knowledge bases
### Native MCP Support
Integrate tools without writing adapters:
- **Process Transport** — Map Yao processes directly to MCP tools
- **External Servers** — Connect via SSE or STDIO
- **Schema Mapping** — Declarative input/output schemas
### Built-in GraphRAG
- **Vector Search** — Embeddings with OpenAI/FastEmbed
- **Knowledge Graph** — Entity-relationship retrieval
- **Hybrid Search** — Combine vector similarity with graph traversal
- **TypeScript Hooks**`Create` and `Next` hooks intercept every request; built-in V8 engine
- **Native MCP Support** — Connect tools via process, SSE, or STDIO transport
- **Memory API** — Four scopes: request-level, session, user, team
- **Multi-Agent** — Delegate to specialist agents or call agents in parallel
- **CLI Agent / Sandbox** — Run Claude Code (or other CLI runners) in an isolated container with VNC desktop support
- **Skills Ecosystem** — Drop reusable capability packs (`SKILL.md`) into any CLI Agent
### Full-Stack Runtime
Everything in a single executable:
- **All-in-One** — Data, API, Agent, UI in one engine
- **TypeScript Support** — Built-in V8 engine
- **Single Binary** — No Node.js, Python, or containers required
- **Edge-Ready** — Runs on ARM64/x64 devices
- **Data Models** — Define database tables and relations in JSON/YAML
- **REST APIs** — Map routes to model queries or TypeScript processors
- **SUI Pages** — Component-based web UI with server-side rendering
- **Chat UI (CUI)** — Built-in conversation interface for agents
- **TypeScript** — Built-in V8 engine; no Node.js required
- **Single Binary** — Runs on ARM64/x64; no Python, Node, or containers needed on the host
### Built-in Search
- **Vector Search** — Embeddings with OpenAI or FastEmbed
- **Knowledge Graph** — Entity-relationship retrieval
- **GraphRAG** — Hybrid vector + graph search
---
## About the Name
Yao (爻, yáo) is the fundamental symbol in the I Ching — the building block of the eight trigrams. Like a binary digit, it has two states. Their combinations describe the patterns of everything.

View file

@ -1,83 +1,73 @@
# Yao
# Yao — AI 时代的应用运行时
[![UnitTest](https://github.com/YaoApp/yao/actions/workflows/unit-test.yml/badge.svg)](https://github.com/YaoApp/yao/actions/workflows/unit-test.yml)
[![codecov](https://codecov.io/gh/YaoApp/yao/branch/main/graph/badge.svg?token=294Y05U71J)](https://codecov.io/gh/YaoApp/yao)
Yao 是一个开源的 AI Agent 和 Web 应用运行时,以单一二进制的形式发布,下载即用。
https://github.com/YaoApp/yao/assets/1842210/6b23ac89-ef6e-4c24-874f-753a98370dec
![Mission Control](docs/mission-control.png)
**🏠 官网:** [https://yaoagents.com](https://yaoagents.com)
**📚 文档:** [https://yaoagents.com/docs](https://yaoagents.com/docs)
**🖥️ Yao Desktop** [https://yaoagents.com/download](https://yaoagents.com/download)
[English](README.md)
YAO 是一款开源应用引擎,使用 Golang 编写,以一个命令行工具的形式存在, 下载即用。适合用于开发业务系统、网站/APP API 接口、管理后台、自建低代码平台等。
---
YAO 采用 flow-based 的编程模式,通过编写 YAO DSL (JSON 格式逻辑描述) 或使用 JavaScript 编写处理器,实现各种功能。 YAO DSL 可以有多种编写方式:
## 工作原理
1. 纯手工编写
Yao Agent 本质上是一个**笼子,而不是动物**。放进去的东西决定行为,笼子保证可控。
2. 使用自动化脚本,根据上下文逻辑生成
每个请求都经过同一套管道:
3. 使用可视化编辑器,通过“拖拉拽”制作
![Pipeline](docs/pipeline.png)
官网: [https://yaoapps.com](https://yaoapps.com)
`Create Hook` 在执行器前运行 —— 注入上下文、施加约束、路由请求。
`Next Hook` 在执行器后运行 —— 校验输出、触发下游动作、驱动多步循环。
**AI 负责干活,你来划定边界。**
文档: [https://yaoapps.com/doc](https://yaoapps.com/doc)
### 三种模式
## 最新版本下载安装 (推荐)
| 模式 | 执行器 | 适用场景 |
|------|--------|---------|
| **LLM** | OpenAI、Anthropic 等 | 对话助手、问答、内容生成 |
| **CLI Agent** | 容器中的 OpenCode、Claude Code、Codex | Computer Use、沙箱隔离、SKILL 生态 |
| **纯 Hook** | 你自己的 TypeScript 代码 | 确定性逻辑、菜单路由、无需 AI 的业务流程 |
https://github.com/YaoApp/xgen-dev-app
三种模式共享同一套 Hook 接口,可以自由混合 —— 在一个 `Create Hook` 里,部分请求走 LLM部分用纯代码处理。
## 演示
---
![界面](docs/yao-setup-demo.jpg)
## 功能特性
使用 YAO 开发的应用
### Agent 框架
| 应用 | 简介 | 代码仓库 |
| -------------------- | ---------------------------- | --------------------------------------- |
| yaoapp/yao-examples | Yao 应用示例 | https://github.com/YaoApp/yao-examples |
| yaoapp/yao-knowledge | ChatGPT 驱动的知识管理库应用 | https://github.com/YaoApp/yao-knowledge |
| yaoapp/xgen-dev-app | 演示应用 (演示) | https://github.com/YaoApp/xgen-dev-app |
| yaoapp/demo-project | 工程项目管理演示应用(演示) | https://github.com/yaoapp/demo-project |
| yaoapp/demo-finance | 财务管理演示应用(演示) | https://github.com/yaoapp/demo-finance |
| yaoapp/demo-plm | 生产项目管理演示应用(演示) | https://github.com/yaoapp/demo-plm |
- **TypeScript Hook**`Create``Next` 两个钩子拦截每一次请求;内置 V8 引擎
- **原生 MCP 支持** — 通过 process、SSE 或 STDIO 传输协议接入工具
- **Memory API** — 四个作用域:请求级、会话级、用户级、团队级
- **多 Agent 协作** — 委派给专属 Agent 或并行调用多个 Agent
- **CLI Agent / 沙箱** — 在隔离容器中运行 Claude Code 等 CLI 程序,支持 VNC 桌面
- **Skills 生态** — 将可复用的能力包(`SKILL.md`)挂载到任意 CLI Agent
## 介绍
### 全栈运行时
Yao 是一个只需使用 JSON 即可创建数据库模型、编写 API 接口、描述管理后台界面的应用引擎,使用 Yao 构建的应用可运行在云端或物联网设备上。 开发者不需要写一行代码,就可以拥有 10 倍生产力。
一个二进制文件包含所有能力:
Yao 基于 **flow-based** 编程思想,采用 **Go** 语言开发,支持多种方式扩展数据流处理器。这使得 Yao 具有极好的**通用性**,大部分场景下可以代替编程语言, 在复用性和编码效率上是传统编程语言的 **10 倍**;应用性能和资源占比上优于 **PHP**, **JAVA** 等语言。
- **数据模型** — 用 JSON/YAML 定义数据库表和关联关系
- **REST API** — 将路由映射到模型查询或 TypeScript 处理器
- **SUI 页面** — 组件化 Web UI支持服务端渲染
- **Chat UICUI** — 内置对话界面,开箱即用
- **TypeScript** — 内置 V8 引擎,不依赖 Node.js
- **单一二进制** — 支持 ARM64/x64宿主机无需 Python、Node 或容器
Yao 内置了一套数据管理系统,通过编写 **JSON** 描述界面布局,即可实现 90% 常见界面交互功能特别适合快速制作各类管理后台、CRM、ERP 等企业内部系统。对于特殊交互功能亦可通过编写扩展组件或 HTML 页面的方式实现。内置管理系统与 Yao 并不耦合,亦可采用 **VUE**, **React** 等任意前端技术实现管理界面。
### 内置搜索
## 安装
- **向量搜索** — 支持 OpenAI 或 FastEmbed 嵌入模型
- **知识图谱** — 实体关系检索
- **GraphRAG** — 向量 + 图谱混合搜索
Yao v0.10.4 使用说明
---
https://github.com/YaoApp/xgen-dev-app/blob/main/README.zh-CN.md
## 关于名字
## 入门指南
详细说明请看[文档](https://yaoapps.com/doc/%E4%BB%8B%E7%BB%8D/%E5%85%A5%E9%97%A8%E6%8C%87%E5%8D%97)
### 创建应用
#### 新建一个空白应用
新建一个应用目录,进入应用目录,运行 `yao start` 命令, 启动安装界面。
```bash
mkdir -p /data/app # 创建应用目录
cd /data/app # 进入应用目录
yao start # 启动安装界面
```
**默认账号**
- 用户名: **xiang@iqka.com**
- 密码: **A123456p+**
![安装界面](docs/yao-setup-step2.jpg)
## 关于 Yao
Yao 的名字源于汉字**爻(yáo)**,是构成八卦的基本符号。八卦,是上古大神伏羲观测总结自然规律后,创造的一个可以指代万事万物的符号体系。爻,有阴阳两种状态,就像 0 和 1。爻的阴阳转换驱动八卦更替以此来总结记录事物的发展规律。
Yao 的名字源于汉字**爻yáo**,是构成八卦的基本符号。八卦,是上古大神伏羲观测自然规律后创造的符号体系。爻有阴阳两种状态,就像 0 和 1。爻的阴阳转换驱动八卦更替记录事物的发展规律。

View file

@ -2,20 +2,19 @@ package assistant
import (
"fmt"
"log"
"time"
jsoniter "github.com/json-iterator/go"
"github.com/yaoapp/gou/connector"
goullm "github.com/yaoapp/gou/llm"
"github.com/yaoapp/kun/log"
"github.com/yaoapp/yao/agent/assistant/handlers"
"github.com/yaoapp/yao/agent/context"
"github.com/yaoapp/yao/agent/i18n"
"github.com/yaoapp/yao/agent/llm"
"github.com/yaoapp/yao/agent/output/message"
agentsandbox "github.com/yaoapp/yao/agent/sandbox"
sandboxTypes "github.com/yaoapp/yao/agent/sandbox/v2/types"
infraV2 "github.com/yaoapp/yao/sandbox/v2"
"github.com/yaoapp/yao/llmprovider"
)
// Stream stream the agent
@ -167,30 +166,25 @@ func (ast *Assistant) Stream(ctx *context.Context, inputMessages []context.Messa
var sandboxLoadingMsgID string
// V2 sandbox state
var v2Runner sandboxTypes.Runner
var v2Computer infraV2.Computer
var v2LoadingMsgID string
var v2Cfg *sandboxTypes.SandboxConfig
var v2Init *sandboxV2InitResult
if ast.HasSandboxV2() {
ctx.Logger.Phase("Sandbox V2")
var err error
var v2Cleanup func()
v2Runner, v2Computer, v2Cfg, v2Cleanup, v2LoadingMsgID, err = ast.initSandboxV2(ctx, opts)
v2Init, err = ast.initSandboxV2(ctx, opts)
if err != nil {
ast.traceAgentFail(agentNode, err)
ast.sendStreamEndOnError(ctx, streamHandler, streamStartTime, err)
return nil, err
}
sandboxCleanup = v2Cleanup
sandboxCleanup = v2Init.Cleanup
ctx.Logger.PhaseComplete("Sandbox V2")
if v2Computer != nil {
ci := v2Computer.ComputerInfo()
if v2Init.Computer != nil {
ci := v2Init.Computer.ComputerInfo()
ctx.Logger.Trace("Node: %s (%s)", ci.NodeID, ci.Kind)
if ci.BoxID != "" {
ctx.Logger.Trace("Computer: %s", ci.BoxID)
}
ctx.Logger.Trace("Workspace: %s", v2Cfg.WorkspaceID)
ctx.Logger.Trace("Workspace: %s", v2Init.Config.WorkspaceID)
if conn, _, err := ast.GetConnector(ctx, opts); err == nil && conn != nil {
ctx.Logger.Trace("Connector: %s", conn.ID())
}
@ -330,22 +324,23 @@ func (ast *Assistant) Stream(ctx *context.Context, inputMessages []context.Messa
// Execute the LLM streaming call
// Choose between sandbox execution or direct LLM execution
if ast.HasSandboxV2() && v2Runner != nil && v2Computer != nil && v2Runner.Name() != "yao" {
if ast.HasSandboxV2() && v2Init != nil && v2Init.Runner != nil && v2Init.Computer != nil && v2Init.Runner.Name() != "yao" {
// V2 Sandbox execution path (non-yao runners replace LLM.Stream)
completionResponse, err = ast.executeSandboxV2Stream(ctx, &sandboxV2StreamParams{
Messages: completionMessages,
AgentNode: agentNode,
Handler: streamHandler,
Runner: v2Runner,
Computer: v2Computer,
Config: v2Cfg,
LoadingMsgID: v2LoadingMsgID,
Runner: v2Init.Runner,
Computer: v2Init.Computer,
Config: v2Init.Config,
LoadingMsgID: v2Init.LoadingMsgID,
Options: opts,
Roles: v2Init.Roles,
})
} else if ast.HasSandboxV2() && v2Runner != nil && v2Runner.Name() == "yao" {
} else if ast.HasSandboxV2() && v2Init != nil && v2Init.Runner != nil && v2Init.Runner.Name() == "yao" {
// V2 yao runner: Prepare is done, close loading, fall through to LLM
if v2LoadingMsgID != "" {
closeLoadingV2(ctx, v2LoadingMsgID, "")
if v2Init.LoadingMsgID != "" {
closeLoadingV2(ctx, v2Init.LoadingMsgID, "")
}
completionResponse, err = ast.executeLLMStream(ctx, completionMessages, completionOptions, agentNode, streamHandler, opts)
} else if ast.HasSandbox() {
@ -570,11 +565,47 @@ func (ast *Assistant) Stream(ctx *context.Context, inputMessages []context.Messa
ast.sendStreamEndOnError(ctx, streamHandler, streamStartTime, err)
return nil, err
}
} else if len(toolCallResponses) > 0 && !ast.HasSandbox() && !ast.isToolLoopDisabled() {
// No Next hook + has tool results + not sandbox → tool loop
ctx.Logger.Debug("Entering tool loop for tool result processing")
loopResponse, loopCompletion, loopTools, err := ast.executeToolLoop(ctx, &ToolLoopParams{
CompletionMessages: completionMessages,
CompletionOptions: completionOptions,
CompletionResponse: completionResponse,
ToolCallResponses: toolCallResponses,
FullMessages: fullMessages,
AgentNode: agentNode,
StreamHandler: streamHandler,
CreateResponse: createResponse,
Opts: opts,
})
if err != nil {
// Fallback to __yao.loop_fallback delegation
ctx.Logger.Warn("Tool loop failed: %v, falling back to loop_fallback", err)
fallbackDelegate := ast.buildLoopFallbackDelegate(ctx, fullMessages, completionResponse, toolCallResponses)
delegateResponse, delegateErr := ast.handleDelegation(ctx, fallbackDelegate, streamHandler)
if delegateErr != nil {
ctx.Logger.Warn("loop_fallback also failed: %v, using standard response", delegateErr)
finalResponse = ast.buildStandardResponse(&NextProcessContext{
Context: ctx,
CompletionResponse: completionResponse,
FullMessages: fullMessages,
ToolCallResponses: toolCallResponses,
StreamHandler: streamHandler,
CreateResponse: createResponse,
})
} else {
finalResponse = delegateResponse
}
} else {
completionResponse = loopCompletion
toolCallResponses = loopTools
finalResponse = loopResponse
}
} else {
// No Next hook: use standard response
// No tool calls, sandbox mode, or loop disabled: standard response
finalResponse = ast.buildStandardResponse(&NextProcessContext{
Context: ctx,
NextResponse: nil,
CompletionResponse: completionResponse,
FullMessages: fullMessages,
ToolCallResponses: toolCallResponses,
@ -627,35 +658,41 @@ func (ast *Assistant) Stream(ctx *context.Context, inputMessages []context.Messa
return finalResponse, nil
}
// GetConnector get the connector object, capabilities, and error with priority:
// opts.Connector > ast.Connector > defaultConnector (fallback)
// GetConnector get the connector object, capabilities, and error.
// Priority: opts.Connector > ast.Connector (may be "use::<role>") > "default" role > legacy fallback
// Note: opts.Connector may be set by Create hook's applyOptionsAdjustments
// Returns: (connector, capabilities, error)
func (ast *Assistant) GetConnector(ctx *context.Context, opts ...*context.Options) (connector.Connector, *goullm.Capabilities, error) {
connectorID := ast.Connector
cid := ast.Connector
if len(opts) > 0 && opts[0] != nil && opts[0].Connector != "" {
connectorID = opts[0].Connector
cid = opts[0].Connector
}
if connectorID == "" {
connectorID = defaultConnector
// Extract identity for role-based resolution
var identity llmprovider.Identity
if ctx != nil && ctx.Authorized != nil {
identity = ctx.Authorized
}
if connectorID == "" {
return nil, nil, fmt.Errorf("connector not specified")
// Unified resolution: explicit connector / use:: prefix / empty → all handled
conn, caps, err := llm.ResolveConnector(cid, identity)
if err == nil {
return conn, caps, nil
}
// Legacy fallback
if defaultConnector != "" {
if conn, err := connector.Select(defaultConnector); err == nil {
log.Warn("[LLM] Connector %s resolve failed, fallback to %s", cid, defaultConnector)
return conn, llm.GetCapabilitiesFromConn(conn), nil
}
}
if fallback := findCapableConnector(); fallback != "" {
if conn, err := connector.Select(fallback); err == nil {
log.Warn("[LLM] Connector %s resolve failed, fallback to %s (auto-detected)", cid, fallback)
return conn, llm.GetCapabilitiesFromConn(conn), nil
}
}
conn, err := connector.Select(connectorID)
if err != nil && connectorID != defaultConnector && defaultConnector != "" {
log.Printf("[Assistant] connector %q not found, falling back to default %q", connectorID, defaultConnector)
conn, err = connector.Select(defaultConnector)
}
if err != nil {
return nil, nil, err
}
capabilities := llm.GetCapabilitiesFromConn(conn)
return conn, capabilities, nil
return nil, nil, fmt.Errorf("connector not specified")
}
// Info get the assistant information
@ -794,9 +831,10 @@ func (ast *Assistant) buildToolRetryMessages(
// Add assistant message with tool calls
assistantMsg := context.Message{
Role: context.RoleAssistant,
Content: completionResponse.Content,
ToolCalls: completionResponse.ToolCalls,
Role: context.RoleAssistant,
Content: completionResponse.Content,
ReasoningContent: completionResponse.ReasoningContent,
ToolCalls: completionResponse.ToolCalls,
}
retryMessages = append(retryMessages, assistantMsg)

View file

@ -3,6 +3,7 @@ package assistant
import (
"fmt"
"path"
"strings"
"github.com/yaoapp/gou/fs"
"github.com/yaoapp/yao/agent/caller"
@ -27,6 +28,28 @@ func init() {
return &agentCallerWrapper{ast: ast}, nil
}
// Initialize AssistantReloadFunc for hot-reload after deploy
caller.AssistantReloadFunc = func(id string) error {
p := "/assistants/" + strings.Replace(id, ".", "/", 1)
ast, err := LoadPath(p)
if err != nil {
return err
}
ast.BuiltIn = true
ast.Readonly = true
if ast.Tags == nil {
ast.Tags = []string{}
}
if err := ast.Save(); err != nil {
return err
}
if err := ast.initialize(); err != nil {
return err
}
loaded.Put(ast)
return nil
}
// Initialize Agent JSAPI factory for ctx.agent.* methods
caller.SetJSAPIFactory()
@ -161,9 +184,6 @@ func (ast *Assistant) Validate() error {
if ast.Name == "" {
return fmt.Errorf("name is required")
}
if ast.Connector == "" {
return fmt.Errorf("connector is required")
}
return nil
}

View file

@ -203,6 +203,11 @@ func (ast *Assistant) buildContextVariables(ctx *context.Context) map[string]str
}
}
// Workspace info
if workspaceID, err := ctx.GetWorkspaceID(); err == nil {
vars["WORKSPACE_ID"] = workspaceID
}
if ctx == nil {
return vars
}

View file

@ -18,6 +18,7 @@ import (
)
func TestGetChatKBID(t *testing.T) {
t.Skip("KB/DB search temporarily disabled")
t.Run("WithTeamAndUser", func(t *testing.T) {
teamID := "5659-5504-2879"
userID := "4287-9400-2030-0504"
@ -81,6 +82,7 @@ func TestGetChatKBID(t *testing.T) {
}
func TestPrepareKBCollection(t *testing.T) {
t.Skip("KB/DB search temporarily disabled")
testutils.Prepare(t)
defer testutils.Clean(t)

View file

@ -408,12 +408,15 @@ func (s *streamState) handleMessageEnd(data []byte) int {
shouldSkipHistory := s.ctx.Stack != nil && s.ctx.Stack.Options != nil &&
s.ctx.Stack.Options.Skip != nil && s.ctx.Stack.Options.Skip.History
// Execute messages have two phases sharing the same message_id:
// 1. running — streamed for UI display only, NOT persisted
// Execute messages have two (or more) phases sharing the same message_id:
// 1. running / suspended / resumed — streamed for UI display only, NOT persisted
// 2. completed / error — the final state, persisted to the buffer
isExecuteRunning := msgType == message.TypeExecute && s.lastExecStatus == "running"
// Only persist when we have an explicit terminal status.
isExecuteFinal := msgType == message.TypeExecute &&
(s.lastExecStatus == "completed" || s.lastExecStatus == "error")
skipExecute := msgType == message.TypeExecute && !isExecuteFinal
if s.ctx.Buffer != nil && len(s.buffer) > 0 && !shouldSkipHistory && !isExecuteRunning {
if s.ctx.Buffer != nil && len(s.buffer) > 0 && !shouldSkipHistory && !skipExecute {
assistantID := ""
if s.ctx.Stack != nil {
assistantID = s.ctx.Stack.AssistantID

View file

@ -881,20 +881,11 @@ func loadMap(data map[string]interface{}) (*Assistant, error) {
// Init init the assistant
// Choose the connector and initialize the assistant
func (ast *Assistant) initialize() error {
conn := defaultConnector
if ast.Connector != "" {
conn = ast.Connector
}
ast.Connector = conn
// Register scripts as process handlers
if len(ast.Scripts) > 0 {
if err := ast.RegisterScripts(); err != nil {
return fmt.Errorf("failed to register scripts: %w", err)
}
}
return nil
}

View file

@ -29,21 +29,28 @@ var systemAgents = []string{
"entity",
"vision",
"fetch",
"loop_fallback",
}
// SystemConfig holds the system agents connector configuration
// This is set from agent.yml system block
type SystemConfig struct {
Default string // Default connector for all system agents
Keyword string // Connector for __yao.keyword agent
QueryDSL string // Connector for __yao.querydsl agent
Title string // Connector for __yao.title agent
Prompt string // Connector for __yao.prompt agent
RobotPrompt string // Connector for __yao.robot_prompt agent
NeedSearch string // Connector for __yao.needsearch agent
Entity string // Connector for __yao.entity agent
Vision string // Connector for vision capabilities
Voice string // Connector for voice/STT capabilities
// Role-level defaults (consumed by buildSystemRoles → SetDefaults)
Default string // Default connector for the "default" role
Light string // Default connector for the "light" role
Vision string // Default connector for the "vision" role
Audio string // Default connector for the "audio" role
Heavy string // Default connector for the "heavy" role (complex reasoning)
// Per-agent overrides (consumed by resolveSystemConnector → ast.Connector)
Keyword string // Connector for __yao.keyword agent
QueryDSL string // Connector for __yao.querydsl agent
Title string // Connector for __yao.title agent
Prompt string // Connector for __yao.prompt agent
RobotPrompt string // Connector for __yao.robot_prompt agent
NeedSearch string // Connector for __yao.needsearch agent
Entity string // Connector for __yao.entity agent
LoopFallback string // Connector for __yao.loop_fallback agent
}
// systemConfig holds the system agents configuration (global variable like others in load.go)
@ -160,10 +167,9 @@ func loadSystemAgent(id, pathPrefix string) (*Assistant, error) {
pkgData["type"] = "assistant"
}
// Resolve connector for this system agent
connectorID := resolveSystemConnector(id)
if connectorID != "" {
pkgData["connector"] = connectorID
// Override connector only if agent.yml has an explicit per-agent setting
if override := resolveSystemConnector(id); override != "" {
pkgData["connector"] = override
}
// Read prompts.yml from bindata (default prompts)
@ -207,97 +213,36 @@ func loadSystemAgent(id, pathPrefix string) (*Assistant, error) {
return loadMap(pkgData)
}
// resolveSystemConnector resolves the connector for a system agent
// Priority: specific agent config > system.default > defaultConnector > fallback to first capable connector
// resolveSystemConnector returns an explicit per-agent connector override from agent.yml.
// Returns empty string if no override exists, so the connector declared in package.yao
// (e.g. "use::light") is preserved as-is.
func resolveSystemConnector(agentID string) string {
// Try specific agent config first
if systemConfig != nil {
switch agentID {
case "__yao.keyword":
if systemConfig.Keyword != "" {
return systemConfig.Keyword
}
case "__yao.querydsl":
if systemConfig.QueryDSL != "" {
return systemConfig.QueryDSL
}
case "__yao.title":
if systemConfig.Title != "" {
return systemConfig.Title
}
case "__yao.prompt":
if systemConfig.Prompt != "" {
return systemConfig.Prompt
}
case "__yao.robot_prompt":
if systemConfig.RobotPrompt != "" {
return systemConfig.RobotPrompt
}
case "__yao.needsearch":
if systemConfig.NeedSearch != "" {
return systemConfig.NeedSearch
}
case "__yao.entity":
if systemConfig.Entity != "" {
return systemConfig.Entity
}
case "__yao.vision":
if systemConfig.Vision != "" {
return systemConfig.Vision
}
case "__yao.voice":
if systemConfig.Voice != "" {
return systemConfig.Voice
}
}
// Try system default
if systemConfig.Default != "" {
return systemConfig.Default
}
if systemConfig == nil {
return ""
}
// Try global default connector
if defaultConnector != "" {
return defaultConnector
switch agentID {
case "__yao.keyword":
return systemConfig.Keyword
case "__yao.querydsl":
return systemConfig.QueryDSL
case "__yao.title":
return systemConfig.Title
case "__yao.prompt":
return systemConfig.Prompt
case "__yao.robot_prompt":
return systemConfig.RobotPrompt
case "__yao.needsearch":
return systemConfig.NeedSearch
case "__yao.entity":
return systemConfig.Entity
case "__yao.vision":
return systemConfig.Vision
case "__yao.audio":
return systemConfig.Audio
case "__yao.loop_fallback":
return systemConfig.LoopFallback
}
// Fallback: find first connector that supports tool calling
return findCapableConnector()
}
// GetVisionConnector returns the connector for vision capabilities.
// Priority: system.vision > system.default > defaultConnector > findCapableConnector
func GetVisionConnector() string {
if systemConfig != nil {
if systemConfig.Vision != "" {
return systemConfig.Vision
}
if systemConfig.Default != "" {
return systemConfig.Default
}
}
if defaultConnector != "" {
return defaultConnector
}
return findCapableConnector()
}
// GetVoiceConnector returns the connector for voice/STT capabilities.
// Priority: system.voice > system.default > defaultConnector > findCapableConnector
func GetVoiceConnector() string {
if systemConfig != nil {
if systemConfig.Voice != "" {
return systemConfig.Voice
}
if systemConfig.Default != "" {
return systemConfig.Default
}
}
if defaultConnector != "" {
return defaultConnector
}
return findCapableConnector()
return ""
}
// findCapableConnector finds the first connector that supports tool calling

View file

@ -0,0 +1,58 @@
package assistant
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestResolveSystemConnector_NoConfig(t *testing.T) {
saved := systemConfig
systemConfig = nil
defer func() { systemConfig = saved }()
assert.Equal(t, "", resolveSystemConnector("__yao.title"))
assert.Equal(t, "", resolveSystemConnector("__yao.keyword"))
assert.Equal(t, "", resolveSystemConnector("__yao.querydsl"))
assert.Equal(t, "", resolveSystemConnector("__yao.vision"))
}
func TestResolveSystemConnector_PerAgentOverride(t *testing.T) {
saved := systemConfig
systemConfig = &SystemConfig{
Title: "openai.gpt-4o",
}
defer func() { systemConfig = saved }()
assert.Equal(t, "openai.gpt-4o", resolveSystemConnector("__yao.title"))
assert.Equal(t, "", resolveSystemConnector("__yao.keyword"))
assert.Equal(t, "", resolveSystemConnector("__yao.querydsl"))
assert.Equal(t, "", resolveSystemConnector("__yao.vision"))
}
func TestResolveSystemConnector_RoleLevelOnly(t *testing.T) {
saved := systemConfig
systemConfig = &SystemConfig{
Default: "openai.gpt-4o",
Light: "openai.gpt-4o-mini",
}
defer func() { systemConfig = saved }()
// Role-level keys don't produce per-agent overrides
assert.Equal(t, "", resolveSystemConnector("__yao.title"))
assert.Equal(t, "", resolveSystemConnector("__yao.keyword"))
assert.Equal(t, "", resolveSystemConnector("__yao.querydsl"))
assert.Equal(t, "", resolveSystemConnector("__yao.vision"))
}
func TestResolveSystemConnector_UnknownAgent(t *testing.T) {
saved := systemConfig
systemConfig = &SystemConfig{
Default: "openai.gpt-4o",
Title: "openai.gpt-4o",
}
defer func() { systemConfig = saved }()
assert.Equal(t, "", resolveSystemConnector("__yao.nonexistent"))
assert.Equal(t, "", resolveSystemConnector("custom.agent"))
}

295
agent/assistant/loop.go Normal file
View file

@ -0,0 +1,295 @@
package assistant
import (
"fmt"
"strings"
jsoniter "github.com/json-iterator/go"
"github.com/yaoapp/yao/agent/context"
"github.com/yaoapp/yao/agent/output/message"
"github.com/yaoapp/yao/trace/types"
)
// ToolLoopParams holds all parameters needed by executeToolLoop.
type ToolLoopParams struct {
CompletionMessages []context.Message
CompletionOptions *context.CompletionOptions
CompletionResponse *context.CompletionResponse
ToolCallResponses []context.ToolCallResponse
FullMessages []context.Message
AgentNode types.Node
StreamHandler message.StreamFunc
CreateResponse *context.HookCreateResponse
Opts *context.Options
}
// executeToolLoop feeds tool results back to the LLM in a loop until
// the LLM produces a final text response (no more tool_calls) or
// the maximum number of turns is reached.
//
// Returns the final Response, the last CompletionResponse (for tracing),
// accumulated ToolCallResponses, and any error.
func (ast *Assistant) executeToolLoop(
ctx *context.Context,
params *ToolLoopParams,
) (*context.Response, *context.CompletionResponse, []context.ToolCallResponse, error) {
maxTurns := ast.getMaxToolLoopTurns()
currentMessages := params.CompletionMessages
currentCompletion := params.CompletionResponse
allToolResponses := make([]context.ToolCallResponse, 0, len(params.ToolCallResponses))
allToolResponses = append(allToolResponses, params.ToolCallResponses...)
for turn := 0; turn < maxTurns; turn++ {
ctx.Logger.Debug("Tool loop turn %d/%d", turn+1, maxTurns)
// Build messages: previous messages + assistant(tool_calls) + tool results
loopMessages := buildToolLoopMessages(currentMessages, currentCompletion, allToolResponses[len(allToolResponses)-len(params.ToolCallResponses):])
// Step tracking: LLM call
ast.BeginStep(ctx, context.StepTypeLLM, map[string]interface{}{
"messages": loopMessages,
"loop_turn": turn + 1,
})
// Call LLM with tool results included
newCompletion, err := ast.executeLLMStream(ctx, loopMessages, params.CompletionOptions, params.AgentNode, params.StreamHandler, params.Opts)
if err != nil {
return nil, nil, nil, fmt.Errorf("tool loop LLM call failed (turn %d): %w", turn+1, err)
}
ast.CompleteStep(ctx, map[string]interface{}{
"content": newCompletion.Content,
"tool_calls": newCompletion.ToolCalls,
})
// No tool_calls → LLM gave final text response
if newCompletion.ToolCalls == nil || len(newCompletion.ToolCalls) == 0 {
finalResponse := ast.buildStandardResponse(&NextProcessContext{
Context: ctx,
CompletionResponse: newCompletion,
FullMessages: params.FullMessages,
ToolCallResponses: allToolResponses,
StreamHandler: params.StreamHandler,
CreateResponse: params.CreateResponse,
})
return finalResponse, newCompletion, allToolResponses, nil
}
// Has tool_calls → execute them
ast.BeginStep(ctx, context.StepTypeTool, map[string]interface{}{
"tool_calls": newCompletion.ToolCalls,
"loop_turn": turn + 1,
})
toolResults, _ := ast.executeToolCalls(ctx, newCompletion.ToolCalls, 0)
// Convert ToolCallResult → ToolCallResponse
toolCallArgsMap := make(map[string]interface{})
for _, tc := range newCompletion.ToolCalls {
toolCallArgsMap[tc.ID] = tc.Function.Arguments
}
turnResponses := make([]context.ToolCallResponse, len(toolResults))
for i, result := range toolResults {
parsedContent, _ := result.ParsedContent()
turnResponses[i] = context.ToolCallResponse{
ToolCallID: result.ToolCallID,
Server: result.Server(),
Tool: result.Tool(),
Arguments: toolCallArgsMap[result.ToolCallID],
Result: parsedContent,
Error: "",
}
if result.Error != nil {
turnResponses[i].Error = result.Error.Error()
}
}
ast.CompleteStep(ctx, map[string]interface{}{
"results": turnResponses,
"loop_turn": turn + 1,
})
// Accumulate and prepare next iteration
allToolResponses = append(allToolResponses, turnResponses...)
currentMessages = loopMessages
currentCompletion = newCompletion
params.ToolCallResponses = turnResponses
}
return nil, nil, allToolResponses, fmt.Errorf("tool loop reached max turns (%d)", maxTurns)
}
// buildToolLoopMessages constructs the message sequence for the next LLM call:
// previous messages + assistant message (with tool_calls) + tool result messages.
// Unlike buildToolRetryMessages, this does NOT append a retry system prompt.
func buildToolLoopMessages(
previousMessages []context.Message,
completion *context.CompletionResponse,
toolResponses []context.ToolCallResponse,
) []context.Message {
messages := make([]context.Message, 0, len(previousMessages)+len(toolResponses)+2)
messages = append(messages, previousMessages...)
// Assistant message with tool_calls
messages = append(messages, context.Message{
Role: context.RoleAssistant,
Content: completion.Content,
ReasoningContent: completion.ReasoningContent,
ToolCalls: completion.ToolCalls,
})
// One tool-role message per tool call result
for _, tr := range toolResponses {
var content string
if tr.Error != "" {
content = fmt.Sprintf("Error: %s", tr.Error)
} else if tr.Result != nil {
raw, _ := jsoniter.MarshalToString(tr.Result)
content = raw
}
toolCallID := tr.ToolCallID
messages = append(messages, context.Message{
Role: context.RoleTool,
Content: content,
ToolCallID: &toolCallID,
})
}
return messages
}
// isToolLoopDisabled checks mcp.options.tool_loop.
// Default is enabled (returns false). Only disabled when explicitly set to false.
func (ast *Assistant) isToolLoopDisabled() bool {
if ast.MCP == nil || ast.MCP.Options == nil {
return false
}
if v, ok := ast.MCP.Options["tool_loop"]; ok {
if enabled, ok := v.(bool); ok {
return !enabled
}
}
return false
}
// getMaxToolLoopTurns reads mcp.options.max_turn. Default is 5.
func (ast *Assistant) getMaxToolLoopTurns() int {
const defaultMaxTurns = 5
if ast.MCP == nil || ast.MCP.Options == nil {
return defaultMaxTurns
}
if v, ok := ast.MCP.Options["max_turn"]; ok {
switch n := v.(type) {
case float64:
if n > 0 {
return int(n)
}
case int:
if n > 0 {
return n
}
}
}
return defaultMaxTurns
}
// ---------------------------------------------------------------------------
// Fallback: __yao.loop_fallback delegation (used when tool loop fails/maxes out)
// ---------------------------------------------------------------------------
// buildLoopFallbackDelegate constructs a DelegateConfig for __yao.loop_fallback.
// It packages conversation context and tool results into a Markdown user message.
func (ast *Assistant) buildLoopFallbackDelegate(
ctx *context.Context,
fullMessages []context.Message,
completion *context.CompletionResponse,
toolResults []context.ToolCallResponse,
) *context.DelegateConfig {
content := buildLoopFallbackMarkdown(fullMessages, toolResults)
return &context.DelegateConfig{
AgentID: "__yao.loop_fallback",
Messages: []context.Message{
{Role: context.RoleUser, Content: content},
},
}
}
// buildLoopFallbackMarkdown formats context into a Markdown string for the fallback agent.
func buildLoopFallbackMarkdown(
fullMessages []context.Message,
toolResults []context.ToolCallResponse,
) string {
var sb strings.Builder
sb.WriteString("## Assistant Context\n\n")
for _, msg := range fullMessages {
if msg.Role == context.RoleSystem {
if text := messageText(msg); text != "" {
sb.WriteString(text)
sb.WriteString("\n\n")
}
}
}
sb.WriteString("## Conversation\n\n")
for _, msg := range fullMessages {
text := messageText(msg)
switch msg.Role {
case context.RoleUser:
if text != "" {
sb.WriteString(fmt.Sprintf("**User**: %s\n\n", text))
}
case context.RoleAssistant:
if text != "" {
sb.WriteString(fmt.Sprintf("**Assistant**: %s\n\n", text))
}
}
}
sb.WriteString("## Tool Results\n\n")
for _, tr := range toolResults {
toolName := tr.Tool
if tr.Server != "" {
toolName = tr.Server + "." + tr.Tool
}
sb.WriteString(fmt.Sprintf("### %s\n", toolName))
if tr.Error != "" {
sb.WriteString(fmt.Sprintf("Error: %s\n\n", tr.Error))
} else {
raw, _ := jsoniter.MarshalToString(tr.Result)
sb.WriteString(fmt.Sprintf("```json\n%s\n```\n\n", raw))
}
}
sb.WriteString("---\nPlease answer the user's question based on the above context and tool results.\n")
sb.WriteString("Respond in the same language as the user.\n")
return sb.String()
}
// messageText extracts text content from a message's Content field.
// Content can be a string or an array of content parts (multimodal).
func messageText(msg context.Message) string {
if msg.Content == nil {
return ""
}
if str, ok := msg.Content.(string); ok {
return str
}
if parts, ok := msg.Content.([]interface{}); ok {
var texts []string
for _, part := range parts {
if partMap, ok := part.(map[string]interface{}); ok {
if partMap["type"] == "text" {
if text, ok := partMap["text"].(string); ok {
texts = append(texts, text)
}
}
}
}
return strings.Join(texts, "\n")
}
return fmt.Sprintf("%v", msg.Content)
}

View file

@ -375,12 +375,6 @@ func (ast *Assistant) executeSingleToolCall(ctx *agentContext.Context, toolCall
return []ToolCallResult{result}, true
}
// Check if result is an error
if callResult.IsError {
result.Error = fmt.Errorf("MCP tool error")
result.IsRetryableError = false // MCP internal error is not retryable
}
// Serialize the Content field only ([]ToolContent)
contentBytes, err := jsoniter.Marshal(callResult.Content)
if err != nil {
@ -396,6 +390,19 @@ func (ast *Assistant) executeSingleToolCall(ctx *agentContext.Context, toolCall
}
result.Content = string(contentBytes)
// Check if result is an error — include actual content so LLM can see the details
if callResult.IsError {
result.Error = fmt.Errorf("tool call error: %s", result.Content)
result.IsRetryableError = isRetryableToolError(result.Error)
ctx.Logger.Error("Tool call failed: %s - %s (retryable: %v)", toolCall.Function.Name, result.Content, result.IsRetryableError)
ctx.Logger.ToolComplete(toolCall.Function.Name, false)
if toolNode != nil {
toolNode.Fail(result.Error)
}
return []ToolCallResult{result}, true
}
ctx.Logger.ToolComplete(toolCall.Function.Name, true)
if toolNode != nil {
@ -545,7 +552,7 @@ func (ast *Assistant) executeServerToolsParallelWithTrace(mcpCtx context.Context
// Prepare parallel trace inputs
var parallelInputs []types.TraceParallelInput
mcpCalls := make([]mcpTypes.ToolCall, 0, len(toolCalls))
callMap := make(map[string]agentContext.ToolCall)
orderedCalls := make([]agentContext.ToolCall, 0, len(toolCalls))
for _, tc := range toolCalls {
_, toolName, ok := ParseMCPToolName(tc.Function.Name)
@ -565,7 +572,7 @@ func (ast *Assistant) executeServerToolsParallelWithTrace(mcpCtx context.Context
Name: toolName,
Arguments: args,
})
callMap[toolName] = tc
orderedCalls = append(orderedCalls, tc)
ctx.Logger.ToolStart(tc.Function.Name)
// Add trace input for this tool
@ -606,10 +613,8 @@ func (ast *Assistant) executeServerToolsParallelWithTrace(mcpCtx context.Context
if node != nil {
node.Fail(err)
}
if i < len(mcpCalls) {
if tc, ok := callMap[mcpCalls[i].Name]; ok {
ctx.Logger.ToolComplete(tc.Function.Name, false)
}
if i < len(orderedCalls) {
ctx.Logger.ToolComplete(orderedCalls[i].Function.Name, false)
}
}
return nil, true
@ -621,7 +626,7 @@ func (ast *Assistant) executeServerToolsParallelWithTrace(mcpCtx context.Context
for i, mcpResult := range mcpResponse.Results {
toolName := mcpCalls[i].Name
originalCall := callMap[toolName]
originalCall := orderedCalls[i]
var toolNode types.Node
if i < len(toolNodes) {
toolNode = toolNodes[i]
@ -809,19 +814,12 @@ func (ast *Assistant) executeServerToolsSequentialWithTrace(mcpCtx context.Conte
toolNode.Fail(err)
}
} else {
// Check if result is an error
if mcpResult.IsError {
result.Error = fmt.Errorf("MCP tool error")
result.IsRetryableError = false // MCP internal error is not retryable
hasErrors = true
}
// Serialize the Content field only ([]ToolContent)
contentBytes, err := jsoniter.Marshal(mcpResult.Content)
if err != nil {
result.Error = err
result.Content = fmt.Sprintf("Failed to serialize result: %v", err)
result.IsRetryableError = false // Serialization error is not retryable
result.IsRetryableError = false
hasErrors = true
ctx.Logger.ToolComplete(tc.Function.Name, false)
if toolNode != nil {
@ -829,11 +827,24 @@ func (ast *Assistant) executeServerToolsSequentialWithTrace(mcpCtx context.Conte
}
} else {
result.Content = string(contentBytes)
ctx.Logger.ToolComplete(tc.Function.Name, !mcpResult.IsError)
if toolNode != nil {
toolNode.Complete(map[string]any{
"result": mcpResult.Content,
})
// Check if result is an error — include actual content so LLM can see the details
if mcpResult.IsError {
result.Error = fmt.Errorf("tool call error: %s", result.Content)
result.IsRetryableError = isRetryableToolError(result.Error)
hasErrors = true
ctx.Logger.Error("Tool call failed: %s - %s (retryable: %v)", toolName, result.Content, result.IsRetryableError)
ctx.Logger.ToolComplete(tc.Function.Name, false)
if toolNode != nil {
toolNode.Fail(result.Error)
}
} else {
ctx.Logger.ToolComplete(tc.Function.Name, true)
if toolNode != nil {
toolNode.Complete(map[string]any{
"result": mcpResult.Content,
})
}
}
}
}

View file

@ -10,6 +10,7 @@ import (
"time"
"github.com/yaoapp/gou/connector"
goullm "github.com/yaoapp/gou/llm"
gouMCP "github.com/yaoapp/gou/mcp"
mcpProcess "github.com/yaoapp/gou/mcp/process"
"github.com/yaoapp/yao/agent/context"
@ -266,30 +267,26 @@ func (ast *Assistant) buildSandboxOptions(ctx *context.Context, opts *context.Op
execOpts.ConnectorType = "openai"
}
// Extract standard fields via LLMConnector when available, fallback to Setting()
setting := conn.Setting()
if host, ok := setting["host"].(string); ok {
execOpts.ConnectorHost = host
}
if key, ok := setting["key"].(string); ok {
execOpts.ConnectorKey = key
}
if model, ok := setting["model"].(string); ok {
execOpts.Model = model
}
// Extract extra connector options (thinking, max_tokens, temperature, etc.)
// These are backend-specific parameters that need to be passed through to the proxy
connectorOptions := make(map[string]interface{})
for k, v := range setting {
// Skip standard fields that are already handled
switch k {
case "host", "key", "model", "azure", "capabilities":
continue
default:
// Include all other fields as extra options
connectorOptions[k] = v
if lc, ok := conn.(goullm.LLMConnector); ok {
execOpts.ConnectorHost = lc.GetURL()
execOpts.ConnectorKey = lc.GetKey()
execOpts.Model = lc.GetModel()
} else {
if host, ok := setting["host"].(string); ok {
execOpts.ConnectorHost = host
}
if key, ok := setting["key"].(string); ok {
execOpts.ConnectorKey = key
}
if model, ok := setting["model"].(string); ok {
execOpts.Model = model
}
}
// Whitelist-filter remaining settings for sandbox proxy options
connectorOptions := connector.FilterRequestBodyParams(setting, conn)
if len(connectorOptions) > 0 {
execOpts.ConnectorOptions = connectorOptions
ctx.Logger.Debug("Connector options extracted: %v", connectorOptions)

View file

@ -8,6 +8,7 @@ import (
"path/filepath"
"time"
"github.com/yaoapp/gou/connector"
"github.com/yaoapp/yao/agent/context"
"github.com/yaoapp/yao/agent/i18n"
"github.com/yaoapp/yao/agent/output/message"
@ -15,6 +16,7 @@ import (
sandboxTypes "github.com/yaoapp/yao/agent/sandbox/v2/types"
store "github.com/yaoapp/yao/agent/store/types"
"github.com/yaoapp/yao/config"
"github.com/yaoapp/yao/llmprovider"
infraV2 "github.com/yaoapp/yao/sandbox/v2"
traceTypes "github.com/yaoapp/yao/trace/types"
"github.com/yaoapp/yao/workspace"
@ -25,15 +27,22 @@ func (ast *Assistant) HasSandboxV2() bool {
return ast.SandboxV2 != nil
}
// sandboxV2InitResult bundles everything returned by initSandboxV2.
type sandboxV2InitResult struct {
Runner sandboxTypes.Runner
Computer infraV2.Computer
Config *sandboxTypes.SandboxConfig
Cleanup func()
LoadingMsgID string
Roles map[string]connector.Connector
}
// initSandboxV2 initializes the V2 sandbox: obtains a Computer, gets a Runner,
// runs Prepare, and returns the runner, computer, a per-request copy of the
// SandboxConfig, cleanup closure, loading message ID, and any error.
// resolves the role matrix, runs Prepare, and returns the result.
//
// A shallow copy of ast.SandboxV2 is made so that concurrent requests to the
// same assistant each get their own mutable config (Owner, ID, NodeID, etc.).
func (ast *Assistant) initSandboxV2(ctx *context.Context, opts *context.Options) (
sandboxTypes.Runner, infraV2.Computer, *sandboxTypes.SandboxConfig, func(), string, error,
) {
func (ast *Assistant) initSandboxV2(ctx *context.Context, opts *context.Options) (*sandboxV2InitResult, error) {
cfgCopy := *ast.SandboxV2
cfg := &cfgCopy
manager := infraV2.M()
@ -52,9 +61,12 @@ func (ast *Assistant) initSandboxV2(ctx *context.Context, opts *context.Options)
conn, _, err := ast.GetConnector(ctx, opts)
if err != nil && cfg.Runner.Name != "yao" {
closeLoadingV2(ctx, loadingMsgID, "sandbox.failed")
return nil, nil, nil, nil, "", fmt.Errorf("get connector: %w", err)
return nil, fmt.Errorf("get connector: %w", err)
}
// 1b. Resolve role matrix once; passed to both Prepare and Stream.
roles := resolveRoles(conn, ctx.Authorized)
// 2. Build human-readable DisplayName from real Agent name + Workspace name.
cfg.DisplayName = buildBoxDisplayName(ctx, ast.ID, ast.Name)
@ -89,7 +101,7 @@ func (ast *Assistant) initSandboxV2(ctx *context.Context, opts *context.Options)
computer, identifier, err := sandboxv2.GetComputer(ctx, cfg, manager)
if err != nil {
closeLoadingV2(ctx, loadingMsgID, "sandbox.failed")
return nil, nil, nil, nil, "", fmt.Errorf("getComputer failed: %w", err)
return nil, fmt.Errorf("getComputer failed: %w", err)
}
_ = identifier
@ -98,7 +110,7 @@ func (ast *Assistant) initSandboxV2(ctx *context.Context, opts *context.Options)
if err != nil {
sandboxv2.LifecycleAction(stdCtx, cfg, computer, manager)
closeLoadingV2(ctx, loadingMsgID, "sandbox.failed")
return nil, nil, nil, nil, "", fmt.Errorf("get runner %q: %w", cfg.Runner.Name, err)
return nil, fmt.Errorf("get runner %q: %w", cfg.Runner.Name, err)
}
// 5. Resolve assistant directory and skills subdirectory.
@ -129,6 +141,7 @@ func (ast *Assistant) initSandboxV2(ctx *context.Context, opts *context.Options)
Computer: computer,
Config: cfg,
Connector: conn,
Roles: roles,
AssistantID: ast.ID,
SkillsDir: skillsDir,
AssistantDir: assistantDir,
@ -140,11 +153,9 @@ func (ast *Assistant) initSandboxV2(ctx *context.Context, opts *context.Options)
runner.Cleanup(stdCtx, computer)
sandboxv2.LifecycleAction(stdCtx, cfg, computer, manager)
closeLoadingV2(ctx, loadingMsgID, "sandbox.failed")
return nil, nil, nil, nil, "", fmt.Errorf("runner.Prepare: %w", err)
return nil, fmt.Errorf("runner.Prepare: %w", err)
}
// Inject computer + workspace into context so Create/Next hooks
// can access ctx.computer and ctx.workspace.
ctx.SetComputer(computer)
cleanup := func() {
@ -154,7 +165,14 @@ func (ast *Assistant) initSandboxV2(ctx *context.Context, opts *context.Options)
sandboxv2.LifecycleAction(cleanCtx, cfg, computer, manager)
}
return runner, computer, cfg, cleanup, loadingMsgID, nil
return &sandboxV2InitResult{
Runner: runner,
Computer: computer,
Config: cfg,
Cleanup: cleanup,
LoadingMsgID: loadingMsgID,
Roles: roles,
}, nil
}
// sandboxV2StreamParams groups arguments for executeSandboxV2Stream.
@ -167,6 +185,7 @@ type sandboxV2StreamParams struct {
Config *sandboxTypes.SandboxConfig
LoadingMsgID string
Options *context.Options
Roles map[string]connector.Connector
}
// executeSandboxV2Stream calls the V2 Runner.Stream and wraps it in the
@ -208,12 +227,15 @@ func (ast *Assistant) executeSandboxV2Stream(
Computer: p.Computer,
Config: cfg,
Connector: conn,
Roles: p.Roles,
AssistantID: ast.ID,
Messages: p.Messages,
SystemPrompt: systemPrompt,
ChatID: ctx.ChatID,
Token: tok,
Logger: ctx.Logger,
UserExplicit: p.Options != nil && p.Options.Connector != "",
Locale: ctx.Locale,
}
execReq := &sandboxv2.ExecuteRequest{
@ -228,6 +250,25 @@ func (ast *Assistant) executeSandboxV2Stream(
return sandboxv2.ExecuteSandboxStream(ctx, execReq, p.Handler)
}
// resolveRoles builds the role → connector map using the llmprovider role system.
// The primary connector (user-selected or system default) becomes "default";
// other roles (heavy, light, vision) are fetched from llmprovider settings.
func resolveRoles(conn connector.Connector, identity llmprovider.Identity) map[string]connector.Connector {
roles := map[string]connector.Connector{}
if conn != nil {
roles["default"] = conn
}
if llmprovider.Global == nil || identity == nil {
return roles
}
for _, role := range []string{"heavy", "light", "vision"} {
if c, err := llmprovider.Global.GetRoleModelBy(role, identity); err == nil {
roles[role] = c
}
}
return roles
}
// initStandaloneWorkspace loads the workspace FS into context when no sandbox
// is configured but the user selected a workspace (metadata["workspace_id"]).
func (ast *Assistant) initStandaloneWorkspace(ctx *context.Context) {

View file

@ -91,7 +91,7 @@ func parseSearchField(search any) *SearchIntent {
if v {
return &SearchIntent{
NeedSearch: true,
SearchTypes: []string{"web", "kb", "db"},
SearchTypes: []string{"web"}, // TODO: 恢复 KB/DB 搜索时改回 []string{"web", "kb", "db"}
Confidence: 1.0,
Reason: "enabled by hook",
}

View file

@ -62,6 +62,7 @@ func (c *authTestCollections) cleanup(ctx context.Context, t *testing.T) {
// FilterKBCollectionsByAuth filters collections based on user authorization.
func TestKBCollectionAuthFilter(t *testing.T) {
t.Skip("KB/DB search temporarily disabled")
testutils.Prepare(t)
defer testutils.Clean(t)
@ -155,6 +156,7 @@ func TestKBCollectionAuthFilter(t *testing.T) {
// ========== DB Auth Wheres Tests ==========
func TestDBAuthWheresFilter(t *testing.T) {
t.Skip("KB/DB search temporarily disabled")
// Note: This test doesn't need KB, just tests the BuildDBAuthWheres function
t.Run("TeamOnlyGeneratesCorrectWheres", func(t *testing.T) {
ctx := createAuthContext(TestUserA, TestTeam1, true, false)
@ -273,6 +275,7 @@ func TestDBAuthWheresFilter(t *testing.T) {
// ========== KB Search Integration Tests ==========
func TestKBSearchIntegration(t *testing.T) {
t.Skip("KB/DB search temporarily disabled")
testutils.Prepare(t)
defer testutils.Clean(t)

View file

@ -52,23 +52,28 @@ func TestSearchAutoFull(t *testing.T) {
assert.Equal(t, 3, ast.Search.Web.MaxResults)
})
// KB/DB search temporarily disabled
t.Run("ShouldHaveKBSearchConfig", func(t *testing.T) {
t.Skip("KB/DB search temporarily disabled")
assert.NotNil(t, ast.Search.KB, "kb search config should be set")
assert.Equal(t, 0.7, ast.Search.KB.Threshold)
assert.False(t, ast.Search.KB.Graph)
})
t.Run("ShouldHaveDBSearchConfig", func(t *testing.T) {
t.Skip("KB/DB search temporarily disabled")
assert.NotNil(t, ast.Search.DB, "db search config should be set")
assert.Equal(t, 10, ast.Search.DB.MaxResults)
})
t.Run("ShouldHaveKBCollections", func(t *testing.T) {
t.Skip("KB/DB search temporarily disabled")
assert.NotNil(t, ast.KB, "kb config should be set")
assert.Contains(t, ast.KB.Collections, "test-collection")
})
t.Run("ShouldHaveDBModels", func(t *testing.T) {
t.Skip("KB/DB search temporarily disabled")
assert.NotNil(t, ast.DB, "db config should be set")
assert.Contains(t, ast.DB.Models, "user")
assert.Contains(t, ast.DB.Models, "article")
@ -87,6 +92,7 @@ func TestSearchAutoFull(t *testing.T) {
})
t.Run("StreamShouldExecuteMultipleSearchTypes", func(t *testing.T) {
t.Skip("KB/DB search temporarily disabled")
// Get agent via assistant.Get (required for Stream)
agent, err := assistant.Get("tests.search-auto-full")
require.NoError(t, err)

View file

@ -15,3 +15,7 @@ type AgentCaller interface {
// AgentGetterFunc is a function type that gets an agent by ID
// This should be set by the assistant package during initialization
var AgentGetterFunc func(agentID string) (AgentCaller, error)
// AssistantReloadFunc reloads a single assistant from disk after deploy.
// Set by the assistant package during initialization.
var AssistantReloadFunc func(id string) error

11
agent/caller/doc.go Normal file
View file

@ -0,0 +1,11 @@
package caller
import (
_ "embed"
"github.com/yaoapp/gou/doc"
)
//go:embed doc.yml
var docYAML []byte
func init() { doc.LoadYAML(docYAML) }

13
agent/caller/doc.yml Normal file
View file

@ -0,0 +1,13 @@
group: agent
type: process
entries:
- name: Call
desc: Call an agent from contexts without agent.Context, enabling agent-to-agent communication
args:
- name: request
type: object
required: true
desc: "Request object with fields: assistant_id (string, required), messages (array of message objects, required), model (string, connector override), skip (object, skip config), metadata (object, passed to hooks), locale (string), route (string), chat_id (string, auto-generated if empty), timeout (number, seconds, default 600)"
return:
type: object
desc: "Result object: { agent_id (string), response (object, full agent response), content (string, extracted text), error (string, error message if failed) }"

View file

@ -273,14 +273,14 @@ func TestProcessCall_Timeout_Short(t *testing.T) {
testutils.Prepare(t)
defer testutils.Clean(t)
// Set timeout=2 seconds — LLM round-trip will certainly exceed this.
// Set timeout=1 second — LLM round-trip will certainly exceed this.
// Verifies that the timeout parameter is respected and produces an error.
proc := newLLMProcess(t, "agent.call", map[string]interface{}{
"assistant_id": "tests.simple-greeting",
"messages": []interface{}{
map[string]interface{}{"role": "user", "content": "Tell me a very long story about the history of computing."},
},
"timeout": 2,
"timeout": 1,
})
err := proc.Execute()

View file

@ -77,7 +77,15 @@ func parseContentParts(ctx *agentContext.Context, message agentContext.Message,
for _, part := range content {
parsedPart, refs, err := parseContentPart(ctx, part, options)
if err != nil {
parts = append(parts, part)
if part.Type == agentContext.ContentImageURL {
parts = append(parts, agentContext.ContentPart{
Type: agentContext.ContentText,
Text: "[Image content could not be processed]",
})
} else {
parts = append(parts, part)
}
log.Error("Failed to parse content part type=%s: %v", part.Type, err)
continue
}
parts = append(parts, parsedPart)

View file

@ -12,6 +12,7 @@ import (
"github.com/yaoapp/yao/agent/output/message"
searchTypes "github.com/yaoapp/yao/agent/search/types"
"github.com/yaoapp/yao/attachment"
toolsImage "github.com/yaoapp/yao/tools/image"
)
// Image handles image content
@ -65,24 +66,27 @@ func (h *Image) Parse(ctx *agentContext.Context, content agentContext.ContentPar
return h.base64(ctx, content, visionFormat)
}
// Model doesn't support vision - check cache first, then use vision agent/MCP
// Try to get cached text (from attachment's content_preview)
// Model doesn't support vision - fallback chain:
// 1. Cache -> 2. Uses.Vision (explicit config) -> 3. tools/vision (auto) -> 4. Placeholder text
cachedText, found, err := h.readFromCache(ctx, content.ImageURL.URL)
if err == nil && found {
// Cache hit! Return as text content
return agentContext.ContentPart{
Type: agentContext.ContentText,
Text: cachedText,
}, nil, nil
}
// No cache, try to use vision agent/MCP
if h.options.CompletionOptions != nil && h.options.CompletionOptions.Uses != nil && h.options.CompletionOptions.Uses.Vision != "" {
return h.agent(ctx, content)
}
// No vision support and no vision tool specified, return error
return content, nil, fmt.Errorf("model doesn't support vision and no vision tool specified in uses.Vision")
if text, err := h.readImageWithTools(ctx, content); err == nil {
h.saveToCache(ctx, content.ImageURL.URL, text)
return agentContext.ContentPart{Type: agentContext.ContentText, Text: text}, nil, nil
}
return agentContext.ContentPart{Type: agentContext.ContentText, Text: "[Image content - vision model not available]"}, nil, nil
}
// base64 encodes image content to base64 (for vision support)
@ -360,6 +364,37 @@ func (h *Image) callMCPVisionTool(ctx *agentContext.Context, serverID string, co
return result, err
}
// readImageWithTools calls tools/vision.ReadImage to convert image to text
// using a vision-capable model resolved via llmprovider.
func (h *Image) readImageWithTools(ctx *agentContext.Context, content agentContext.ContentPart) (string, error) {
if ctx.Authorized == nil {
return "", fmt.Errorf("no auth info available for vision model resolution")
}
src := wrapperToAttachURI(content.ImageURL.URL)
loadingID := h.sendLoading(ctx, i18n.T(ctx.Locale, "content.image.analyzing"))
resp, err := toolsImage.ReadImage(ctx.Context, src, "Please describe this image in detail.", 1080, ctx.Authorized, "")
h.sendLoadingDone(ctx, loadingID)
if err != nil {
return "", err
}
return resp.Content, nil
}
// wrapperToAttachURI converts __uploader://fileID to attach://uploader/fileID
// format expected by tools/vision.readBytes.
func wrapperToAttachURI(url string) string {
uploaderName, fileID, ok := attachment.Parse(url)
if !ok {
return url
}
return "attach://" + uploaderName + "/" + fileID
}
// sendLoading sends a loading message and returns the message ID
// Returns empty string if SilentLoading is enabled
func (h *Image) sendLoading(ctx *agentContext.Context, msg string) string {

View file

@ -117,11 +117,12 @@ func TestParseWithoutVisionSupport(t *testing.T) {
}
handler := image.New(options)
_, _, err := handler.Parse(ctx, content)
result, _, err := handler.Parse(ctx, content)
// Should return error because no vision support and no vision tool specified
assert.Error(t, err)
assert.Contains(t, err.Error(), "no vision tool specified")
// Should return placeholder text (no error) when no vision support
assert.NoError(t, err)
assert.Equal(t, agentContext.ContentText, result.Type)
assert.Contains(t, result.Text, "Image content")
}
// TestParseWithEmptyURL tests parsing image with empty URL

View file

@ -382,9 +382,14 @@ func (b *ChatBuffer) GetStepsForResume(finalStatus string) []*BufferedStep {
b.currentStep.Status = finalStatus
}
// Return all steps (they will all have the context for recovery)
result := make([]*BufferedStep, len(b.steps))
copy(result, b.steps)
// Only return steps with valid resume status (failed or interrupted)
result := make([]*BufferedStep, 0, len(b.steps))
for _, step := range b.steps {
if step.Status != ResumeStatusFailed && step.Status != ResumeStatusInterrupted {
continue
}
result = append(result, step)
}
return result
}

View file

@ -498,10 +498,10 @@ func TestBufferGetStepsForResume(t *testing.T) {
steps := buffer.GetStepsForResume(context.ResumeStatusFailed)
require.NotNil(t, steps)
assert.Len(t, steps, 2)
assert.Len(t, steps, 1)
// Current step should be marked as failed
assert.Equal(t, context.ResumeStatusFailed, steps[1].Status)
// Only the failed step should be returned
assert.Equal(t, context.ResumeStatusFailed, steps[0].Status)
})
t.Run("InterruptedRequest", func(t *testing.T) {
@ -516,8 +516,8 @@ func TestBufferGetStepsForResume(t *testing.T) {
steps := buffer.GetStepsForResume(context.ResumeStatusInterrupted)
require.NotNil(t, steps)
assert.Len(t, steps, 3)
assert.Equal(t, context.ResumeStatusInterrupted, steps[2].Status)
assert.Len(t, steps, 1)
assert.Equal(t, context.ResumeStatusInterrupted, steps[0].Status)
})
}
@ -1014,10 +1014,10 @@ func TestBufferCompleteWorkflow(t *testing.T) {
// Get steps for resume
steps := buffer.GetStepsForResume(context.ResumeStatusInterrupted)
require.NotNil(t, steps)
assert.Len(t, steps, 2)
assert.Len(t, steps, 1)
// Last step should be interrupted with space snapshot
lastStep := steps[len(steps)-1]
// Only the interrupted step should be returned
lastStep := steps[0]
assert.Equal(t, context.ResumeStatusInterrupted, lastStep.Status)
assert.NotNil(t, lastStep.SpaceSnapshot)
assert.Equal(t, "previous conversation", lastStep.SpaceSnapshot["user_context"])

View file

@ -559,3 +559,11 @@ func (ctx *Context) MergeMetadata(metadata map[string]interface{}) {
ctx.Metadata[k] = v
}
}
// GetWorkspaceID returns the ID of the workspace
func (ctx *Context) GetWorkspaceID() (string, error) {
if ctx.workspace == nil {
return "", nil
}
return ctx.workspace.GetID()
}

View file

@ -6,7 +6,6 @@ import (
"fmt"
"net/http"
"github.com/yaoapp/gou/connector"
"github.com/yaoapp/gou/store"
"github.com/yaoapp/yao/openapi/oauth/types"
)
@ -63,9 +62,7 @@ func GetGRPCAgentRequest(parent context.Context, input GRPCAgentInput) ([]Messag
}
if connectorID := getStringOpt(rawOpts, "connector"); connectorID != "" {
if _, err := connector.Select(connectorID); err == nil {
opts.Connector = connectorID
}
opts.Connector = connectorID
}
ctx.Interrupt = NewInterruptController()

View file

@ -14,6 +14,9 @@ type LlmAPI interface {
// Returns *llm.Result or error information
Stream(connector string, messages []interface{}, opts map[string]interface{}) interface{}
// GenerateImage generates an image from a text prompt using an image generation model
GenerateImage(connector string, prompt string, opts map[string]interface{}) interface{}
// Parallel LLM call methods - inspired by JavaScript Promise
// All waits for all LLM calls to complete (like Promise.all)
All(requests []interface{}) []interface{}
@ -68,6 +71,9 @@ func (ctx *Context) newLlmObject(iso *v8go.Isolate) *v8go.ObjectTemplate {
// Single LLM call method
llmObj.Set("Stream", ctx.llmStreamMethod(iso))
// Image generation method
llmObj.Set("GenerateImage", ctx.llmGenerateImageMethod(iso))
// Parallel LLM call methods - inspired by JavaScript Promise
llmObj.Set("All", ctx.llmAllMethod(iso))
llmObj.Set("Any", ctx.llmAnyMethod(iso))
@ -163,6 +169,54 @@ func (ctx *Context) llmStreamMethod(iso *v8go.Isolate) *v8go.FunctionTemplate {
})
}
// llmGenerateImageMethod implements ctx.llm.GenerateImage(connector, prompt, options?)
// Usage: const result = ctx.llm.GenerateImage("dall-e-3", "A sunset over mountains", { size: "1024x1024" })
// Returns: { connector, image (base64), format, error }
func (ctx *Context) llmGenerateImageMethod(iso *v8go.Isolate) *v8go.FunctionTemplate {
return v8go.NewFunctionTemplate(iso, func(info *v8go.FunctionCallbackInfo) *v8go.Value {
v8ctx := info.Context()
args := info.Args()
if len(args) < 2 {
return bridge.JsException(v8ctx, "GenerateImage requires connector and prompt parameters")
}
if !args[0].IsString() {
return bridge.JsException(v8ctx, "connector must be a string")
}
connectorID := args[0].String()
if !args[1].IsString() {
return bridge.JsException(v8ctx, "prompt must be a string")
}
prompt := args[1].String()
var opts map[string]interface{}
if len(args) >= 3 && !args[2].IsUndefined() && !args[2].IsNull() {
goVal, err := bridge.GoValue(args[2], v8ctx)
if err == nil {
if optsMap, ok := goVal.(map[string]interface{}); ok {
opts = optsMap
}
}
}
llmAPI := ctx.Llm()
if llmAPI == nil {
return bridge.JsException(v8ctx, "LLM API not available")
}
result := llmAPI.GenerateImage(connectorID, prompt, opts)
jsVal, err := bridge.JsValue(v8ctx, result)
if err != nil {
return bridge.JsException(v8ctx, "failed to convert result: "+err.Error())
}
return jsVal
})
}
// llmAllMethod implements ctx.llm.All(requests, options?)
// Usage: const results = ctx.llm.All([
//

View file

@ -9,7 +9,6 @@ import (
"strings"
"github.com/gin-gonic/gin"
"github.com/yaoapp/gou/connector"
"github.com/yaoapp/gou/store"
"github.com/yaoapp/yao/openapi/oauth/authorized"
)
@ -70,19 +69,9 @@ func GetCompletionRequest(c *gin.Context, cache store.Store) (*CompletionRequest
Mode: GetMode(c, completionReq),
}
// Try to extract custom connector from model field
// If model is a valid connector ID, set it to opts.Connector
// Otherwise, keep the standard OpenAI-compatible behavior (model as assistant ID)
if completionReq != nil && completionReq.Model != "" {
// Check if model is a valid connector (not containing "-yao_" which indicates assistant ID format)
if !strings.Contains(completionReq.Model, "-yao_") {
// Try to validate if it's a real connector
if _, err := connector.Select(completionReq.Model); err == nil {
// It's a valid connector, use it
opts.Connector = completionReq.Model
}
// If not a valid connector, ignore it (keep opts.Connector empty to use assistant's default)
}
// Pass model as connector ID; downstream ResolveConnector handles validation + lazy loading
if completionReq != nil && completionReq.Model != "" && !strings.Contains(completionReq.Model, "-yao_") {
opts.Connector = completionReq.Model
}
// Initialize interrupt controller

View file

@ -566,8 +566,9 @@ type Message struct {
ToolCallID *string `json:"tool_call_id,omitempty"` // Required for tool messages: tool call that this message is responding to
// Assistant message specific fields
ToolCalls []ToolCall `json:"tool_calls,omitempty"` // Optional for assistant: tool calls generated by the model
Refusal *string `json:"refusal,omitempty"` // Optional for assistant: refusal message (null when not refusing)
ReasoningContent string `json:"reasoning_content,omitempty"` // Optional for assistant: reasoning/thinking content (DeepSeek, OpenAI o-series)
ToolCalls []ToolCall `json:"tool_calls,omitempty"` // Optional for assistant: tool calls generated by the model
Refusal *string `json:"refusal,omitempty"` // Optional for assistant: refusal message (null when not refusing)
}
// ContentPartType represents the type of content part

View file

@ -53,19 +53,28 @@ func (a *VisionAdapter) removeImageContent(messages []context.Message) []context
for _, msg := range messages {
processedMsg := msg
// Handle multimodal content (array of map)
if contentParts, ok := msg.Content.([]map[string]interface{}); ok {
if contentParts, ok := msg.Content.([]context.ContentPart); ok {
filtered := make([]context.ContentPart, 0)
for _, part := range contentParts {
if part.Type != context.ContentImageURL {
filtered = append(filtered, part)
}
}
if len(filtered) == 0 {
processedMsg.Content = "[Image content not supported by this model]"
} else {
processedMsg.Content = filtered
}
} else if contentParts, ok := msg.Content.([]map[string]interface{}); ok {
filteredParts := make([]map[string]interface{}, 0)
for _, part := range contentParts {
partType, _ := part["type"].(string)
// Skip image content
if partType != "image_url" && partType != "image" {
filteredParts = append(filteredParts, part)
}
}
// If all parts were filtered out, add placeholder text
if len(filteredParts) == 0 {
processedMsg.Content = "[Image content not supported by this model]"
} else if len(filteredParts) == 1 {

View file

@ -20,12 +20,21 @@ func GetCapabilities(connectorID string) *goullm.Capabilities {
return GetCapabilitiesFromConn(conn)
}
// GetCapabilitiesFromConn get the capabilities from a connector instance
// GetCapabilitiesFromConn get the capabilities from a connector instance.
// Prefers LLMConnector.GetCapabilities() when available, falls back to Setting() parsing.
func GetCapabilitiesFromConn(conn connector.Connector) *goullm.Capabilities {
if conn == nil {
return getDefaultCapabilities()
}
// Prefer typed LLMConnector interface
if lc, ok := conn.(goullm.LLMConnector); ok {
if caps := lc.GetCapabilities(); caps != nil {
return caps
}
}
// Fallback to Setting() parsing for non-LLMConnector or nil capabilities
settings := conn.Setting()
if settings != nil {
if caps, ok := settings["capabilities"]; ok {
@ -35,12 +44,54 @@ func GetCapabilitiesFromConn(conn connector.Connector) *goullm.Capabilities {
if capabilities, ok := caps.(goullm.Capabilities); ok {
return &capabilities
}
if capsMap, ok := caps.(map[string]interface{}); ok {
return capabilitiesFromMap(capsMap)
}
}
}
return getDefaultCapabilities()
}
// capabilitiesFromMap converts a JSON-deserialized map into goullm.Capabilities.
func capabilitiesFromMap(m map[string]interface{}) *goullm.Capabilities {
caps := getDefaultCapabilities()
if v, ok := m["streaming"].(bool); ok {
caps.Streaming = v
}
if v, ok := m["tool_calls"].(bool); ok {
caps.ToolCalls = v
}
if v, ok := m["vision"]; ok {
caps.Vision = v
}
if v, ok := m["audio"].(bool); ok {
caps.Audio = v
}
if v, ok := m["stt"].(bool); ok {
caps.STT = v
}
if v, ok := m["reasoning"].(bool); ok {
caps.Reasoning = v
}
if v, ok := m["json"].(bool); ok {
caps.JSON = v
}
if v, ok := m["multimodal"].(bool); ok {
caps.Multimodal = v
}
if v, ok := m["temperature_adjustable"].(bool); ok {
caps.TemperatureAdjustable = v
}
if v, ok := m["embedding"].(bool); ok {
caps.Embedding = v
}
if v, ok := m["image_generation"].(bool); ok {
caps.ImageGeneration = v
}
return caps
}
// getDefaultCapabilities returns minimal default capabilities
func getDefaultCapabilities() *goullm.Capabilities {
return &goullm.Capabilities{
@ -65,26 +116,8 @@ func GetCapabilitiesMap(connectorID string) map[string]interface{} {
return ToMap(caps)
}
// ToMap converts Capabilities to map[string]interface{}
// ToMap converts Capabilities to map[string]interface{}.
// Delegates to the canonical Capabilities.ToMap() method in gou/llm.
func ToMap(caps *goullm.Capabilities) map[string]interface{} {
if caps == nil {
return nil
}
result := make(map[string]interface{})
if caps.Vision != nil {
result["vision"] = caps.Vision
}
result["audio"] = caps.Audio
result["stt"] = caps.STT
result["tool_calls"] = caps.ToolCalls
result["reasoning"] = caps.Reasoning
result["streaming"] = caps.Streaming
result["json"] = caps.JSON
result["multimodal"] = caps.Multimodal
result["temperature_adjustable"] = caps.TemperatureAdjustable
return result
return caps.ToMap()
}

11
agent/llm/doc.go Normal file
View file

@ -0,0 +1,11 @@
package llm
import (
_ "embed"
"github.com/yaoapp/gou/doc"
)
//go:embed doc.yml
var docYAML []byte
func init() { doc.LoadYAML(docYAML) }

44
agent/llm/doc.yml Normal file
View file

@ -0,0 +1,44 @@
group: llm
type: process
entries:
- name: ChatCompletions
desc: Universal LLM chat completions that auto-detects connector type and routes accordingly
args:
- name: connector
type: string
required: true
desc: Connector ID (supports any type, e.g. openai, anthropic)
- name: messages
type: array
required: true
desc: "Message array in OpenAI format: each element is an object with role, content (string or multimodal array), and optional name, tool_call_id, tool_calls"
- name: opts
type: object
required: false
desc: "Completion options: temperature, max_tokens, and other model parameters"
- name: callback
type: function
required: false
desc: "Streaming callback function that receives data chunks; signature: func(data []byte) int"
return:
type: object
desc: "OpenAI-compatible response: { id, object, created, model, choices: [{ index, message: { role, content, tool_calls? }, finish_reason }], usage? }"
- name: ImageGeneration
desc: Generate an image from a text prompt using an image generation model
args:
- name: connector
type: string
required: true
desc: Connector ID for an image generation model (e.g. dall-e-3)
- name: prompt
type: string
required: true
desc: Text description of the image to generate
- name: opts
type: object
required: false
desc: "Generation options: size (1024x1024), quality, style, n, etc."
return:
type: object
desc: "Image generation result: { image (base64), format (png) }"

183
agent/llm/image.go Normal file
View file

@ -0,0 +1,183 @@
package llm
import (
"encoding/base64"
"encoding/json"
"fmt"
"io"
"net/http"
"strings"
"time"
"github.com/yaoapp/gou/connector"
gouhttp "github.com/yaoapp/gou/http"
goullm "github.com/yaoapp/gou/llm"
)
// ImageGenResponse holds the result of an image generation call.
// Image is always base64 encoded; if the provider returns a URL, it is downloaded and converted.
type ImageGenResponse struct {
Image string `json:"image"` // base64 encoded image data
Format string `json:"format"` // image format, e.g. "png", "jpeg"
}
// GenerateImage calls the /images/generations endpoint through the connector.
// options may include: size, n, quality, style, model, etc.
func GenerateImage(conn connector.Connector, prompt string, options map[string]interface{}) (*ImageGenResponse, error) {
host, key, authMode := resolveConnSettings(conn)
if host == "" {
return nil, fmt.Errorf("no host found in connector settings")
}
if key == "" {
return nil, fmt.Errorf("API key is not set")
}
if options == nil {
options = map[string]interface{}{}
}
options["prompt"] = prompt
if _, ok := options["model"]; !ok {
if lc, ok := conn.(goullm.LLMConnector); ok {
if m := lc.GetModel(); m != "" {
options["model"] = m
}
}
}
url := connector.BuildAPIURL(host, "/images/generations")
req := gouhttp.New(url)
req.SetHeader("Content-Type", "application/json")
setImageAuthHeaders(req, authMode, key)
resp := req.Post(options)
if resp.Status != 200 {
errMsg := extractAPIError(resp.Data)
return nil, fmt.Errorf("image generation failed (status %d, url %s): %s", resp.Status, url, errMsg)
}
return extractImageFromResponse(resp.Data)
}
func resolveConnSettings(conn connector.Connector) (host, key string, authMode goullm.AuthMode) {
authMode = goullm.AuthBearer
if lc, ok := conn.(goullm.LLMConnector); ok {
host = lc.GetURL()
key = lc.GetKey()
authMode = lc.GetAuthMode()
}
if host == "" || key == "" {
setting := conn.Setting()
if host == "" {
host, _ = setting["host"].(string)
}
if key == "" {
key, _ = setting["key"].(string)
}
}
return
}
func setImageAuthHeaders(req *gouhttp.Request, authMode goullm.AuthMode, key string) {
switch authMode {
case goullm.AuthAPIKey:
req.SetHeader("api-key", key)
case goullm.AuthXAPIKey:
req.SetHeader("x-api-key", key)
default:
req.SetHeader("Authorization", fmt.Sprintf("Bearer %s", key))
}
}
func extractImageFromResponse(data interface{}) (*ImageGenResponse, error) {
raw, err := json.Marshal(data)
if err != nil {
return nil, fmt.Errorf("marshal response: %w", err)
}
var parsed struct {
Data []struct {
B64JSON *string `json:"b64_json"`
URL *string `json:"url"`
} `json:"data"`
}
if err := json.Unmarshal(raw, &parsed); err != nil {
return nil, fmt.Errorf("unmarshal response: %w", err)
}
if len(parsed.Data) == 0 {
return nil, fmt.Errorf("provider returned empty data array, no image was generated")
}
item := parsed.Data[0]
if item.B64JSON != nil && *item.B64JSON != "" {
return &ImageGenResponse{Image: *item.B64JSON, Format: "png"}, nil
}
if item.URL != nil && *item.URL != "" {
b64, format, err := downloadImageAsBase64(*item.URL)
if err != nil {
return nil, fmt.Errorf("provider returned url but download failed: %w", err)
}
return &ImageGenResponse{Image: b64, Format: format}, nil
}
return nil, fmt.Errorf("provider returned data but neither b64_json nor url field is present, the model may not support image generation")
}
func downloadImageAsBase64(imageURL string) (b64 string, format string, err error) {
client := &http.Client{Timeout: 30 * time.Second}
resp, err := client.Get(imageURL)
if err != nil {
return "", "", fmt.Errorf("http get: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
return "", "", fmt.Errorf("download returned status %d", resp.StatusCode)
}
body, err := io.ReadAll(resp.Body)
if err != nil {
return "", "", fmt.Errorf("read body: %w", err)
}
if len(body) == 0 {
return "", "", fmt.Errorf("downloaded image is empty")
}
format = "png"
ct := resp.Header.Get("Content-Type")
switch {
case strings.Contains(ct, "jpeg") || strings.Contains(ct, "jpg"):
format = "jpeg"
case strings.Contains(ct, "webp"):
format = "webp"
case strings.Contains(ct, "gif"):
format = "gif"
default:
if strings.Contains(imageURL, ".jpeg") || strings.Contains(imageURL, ".jpg") {
format = "jpeg"
} else if strings.Contains(imageURL, ".webp") {
format = "webp"
}
}
b64 = base64.StdEncoding.EncodeToString(body)
return b64, format, nil
}
func extractAPIError(data interface{}) string {
raw, err := json.Marshal(data)
if err != nil {
return fmt.Sprintf("%v", data)
}
var parsed struct {
Error struct {
Message string `json:"message"`
} `json:"error"`
}
if err := json.Unmarshal(raw, &parsed); err == nil && parsed.Error.Message != "" {
return parsed.Error.Message
}
return string(raw)
}

197
agent/llm/image_test.go Normal file
View file

@ -0,0 +1,197 @@
package llm
import (
"encoding/base64"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
)
func TestExtractImageFromResponse_B64(t *testing.T) {
data := map[string]interface{}{
"data": []interface{}{
map[string]interface{}{
"b64_json": "iVBORw0KGgoAAAANS...",
},
},
}
resp, err := extractImageFromResponse(data)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if resp.Image != "iVBORw0KGgoAAAANS..." {
t.Errorf("got Image=%q, want %q", resp.Image, "iVBORw0KGgoAAAANS...")
}
if resp.Format != "png" {
t.Errorf("got Format=%q, want %q", resp.Format, "png")
}
}
func TestExtractImageFromResponse_URL(t *testing.T) {
fakeImage := []byte{0xFF, 0xD8, 0xFF, 0xE0, 0x00, 0x10} // fake JPEG header bytes
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "image/jpeg")
w.Write(fakeImage)
}))
defer srv.Close()
data := map[string]interface{}{
"data": []interface{}{
map[string]interface{}{
"b64_json": nil,
"url": srv.URL + "/image_0.jpeg",
},
},
}
resp, err := extractImageFromResponse(data)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
expected := base64.StdEncoding.EncodeToString(fakeImage)
if resp.Image != expected {
t.Errorf("got Image=%q, want %q", resp.Image, expected)
}
if resp.Format != "jpeg" {
t.Errorf("got Format=%q, want %q", resp.Format, "jpeg")
}
}
func TestExtractImageFromResponse_URLPng(t *testing.T) {
fakeImage := []byte{0x89, 0x50, 0x4E, 0x47} // PNG magic bytes
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "image/png")
w.Write(fakeImage)
}))
defer srv.Close()
data := map[string]interface{}{
"data": []interface{}{
map[string]interface{}{
"url": srv.URL + "/output.png",
},
},
}
resp, err := extractImageFromResponse(data)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if resp.Format != "png" {
t.Errorf("got Format=%q, want %q", resp.Format, "png")
}
if resp.Image == "" {
t.Error("expected non-empty base64 Image")
}
}
func TestExtractImageFromResponse_URLDownloadFail(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusNotFound)
}))
defer srv.Close()
data := map[string]interface{}{
"data": []interface{}{
map[string]interface{}{
"url": srv.URL + "/missing.png",
},
},
}
_, err := extractImageFromResponse(data)
if err == nil {
t.Error("expected error for failed download")
}
}
func TestExtractImageFromResponse_Empty(t *testing.T) {
data := map[string]interface{}{
"data": []interface{}{},
}
_, err := extractImageFromResponse(data)
if err == nil {
t.Error("expected error for empty data array")
}
}
func TestExtractImageFromResponse_NoData(t *testing.T) {
data := map[string]interface{}{}
_, err := extractImageFromResponse(data)
if err == nil {
t.Error("expected error for missing data field")
}
}
func TestExtractImageFromResponse_NullBoth(t *testing.T) {
data := map[string]interface{}{
"data": []interface{}{
map[string]interface{}{
"b64_json": nil,
"url": nil,
},
},
}
_, err := extractImageFromResponse(data)
if err == nil {
t.Error("expected error when both b64_json and url are null")
}
}
func TestDownloadImageAsBase64(t *testing.T) {
payload := []byte("fake-png-data")
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "image/png")
w.Write(payload)
}))
defer srv.Close()
b64, format, err := downloadImageAsBase64(srv.URL + "/test.png")
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if format != "png" {
t.Errorf("got format=%q, want %q", format, "png")
}
decoded, _ := base64.StdEncoding.DecodeString(b64)
if string(decoded) != string(payload) {
t.Errorf("decoded content mismatch")
}
}
func TestDownloadImageAsBase64_FormatFromURL(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/octet-stream")
w.Write([]byte("data"))
}))
defer srv.Close()
_, format, err := downloadImageAsBase64(srv.URL + "/image.webp")
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if format != "webp" {
t.Errorf("got format=%q, want %q (from URL fallback)", format, "webp")
}
}
func TestExtractAPIError_WithMessage(t *testing.T) {
data := map[string]interface{}{
"error": map[string]interface{}{
"message": "insufficient quota",
},
}
msg := extractAPIError(data)
if msg != "insufficient quota" {
t.Errorf("got %q, want %q", msg, "insufficient quota")
}
}
func TestExtractAPIError_NoMessage(t *testing.T) {
data := map[string]interface{}{
"something": "else",
}
msg := extractAPIError(data)
raw, _ := json.Marshal(data)
if msg != string(raw) {
t.Errorf("got %q, want raw JSON fallback", msg)
}
}

View file

@ -31,6 +31,37 @@ func SetJSAPIFactory() {
}
}
// GenerateImage implements LlmAPI.GenerateImage - generates an image from a text prompt
func (api *JSAPI) GenerateImage(connectorID string, prompt string, opts map[string]interface{}) interface{} {
result := &ImageGenResult{
Connector: connectorID,
}
conn, err := connector.Select(connectorID)
if err != nil {
result.Error = fmt.Sprintf("failed to select connector %s: %v", connectorID, err)
return result
}
resp, err := GenerateImage(conn, prompt, opts)
if err != nil {
result.Error = fmt.Sprintf("image generation failed: %v", err)
return result
}
result.Image = resp.Image
result.Format = resp.Format
return result
}
// ImageGenResult is the return type for GenerateImage JSAPI
type ImageGenResult struct {
Connector string `json:"connector"`
Image string `json:"image,omitempty"`
Format string `json:"format,omitempty"`
Error string `json:"error,omitempty"`
}
// Stream implements LlmAPI.Stream - calls LLM with streaming output to ctx.Writer
func (api *JSAPI) Stream(connectorID string, messages []interface{}, opts map[string]interface{}) interface{} {
return api.StreamWithHandler(connectorID, messages, opts, nil)

View file

@ -16,6 +16,7 @@ import (
func init() {
process.Register("llm.ChatCompletions", ProcessChatCompletions)
process.Register("llm.ImageGeneration", ProcessImageGeneration)
}
// ProcessChatCompletions implements the llm.ChatCompletions Process.
@ -155,6 +156,55 @@ func ProcessChatCompletions(p *process.Process) interface{} {
return toOpenAIFormat(response)
}
// ProcessImageGeneration implements the llm.ImageGeneration Process.
//
// Usage:
//
// Process("llm.ImageGeneration", connectorID, prompt)
// Process("llm.ImageGeneration", connectorID, prompt, opts)
//
// Args:
// - connectorID (string): Connector ID for an image generation model
// - prompt (string): Text description of the image to generate
// - opts (map): Optional. size, quality, style, n, etc.
//
// Returns: { image (base64), format (png) }
func ProcessImageGeneration(p *process.Process) interface{} {
p.ValidateArgNums(2)
connectorID := p.ArgsString(0)
if connectorID == "" {
return newErrorResponse("llm.ImageGeneration: connector is required")
}
prompt := p.ArgsString(1)
if prompt == "" {
return newErrorResponse("llm.ImageGeneration: prompt is required")
}
var opts map[string]interface{}
if p.NumOfArgs() > 2 && p.Args[2] != nil {
if o, ok := p.Args[2].(map[string]interface{}); ok {
opts = o
}
}
conn, _, err := selectWithCapabilities(connectorID)
if err != nil {
return newErrorResponse(fmt.Sprintf("llm.ImageGeneration: connector %s not found: %v", connectorID, err))
}
resp, err := GenerateImage(conn, prompt, opts)
if err != nil {
return newErrorResponse(fmt.Sprintf("llm.ImageGeneration: %v", err))
}
return map[string]interface{}{
"image": resp.Image,
"format": resp.Format,
}
}
// toOpenAIFormat converts CompletionResponse to OpenAI chat.completions format
// for backward compatibility with code that consumed openai.chat.Completions.
func toOpenAIFormat(resp *agentContext.CompletionResponse) map[string]interface{} {

View file

@ -17,6 +17,7 @@ import (
"github.com/yaoapp/yao/agent/llm/adapters"
"github.com/yaoapp/yao/agent/llm/providers/base"
"github.com/yaoapp/yao/agent/output/message"
"github.com/yaoapp/yao/share"
)
// Provider Anthropic Messages API provider
@ -44,11 +45,11 @@ func buildAdapters(cap *goullm.Capabilities) []adapters.CapabilityAdapter {
// Tool call adapter
result = append(result, adapters.NewToolCallAdapter(cap.ToolCalls))
// Vision adapter
// Vision adapter (always registered to strip unsupported image content)
visionSupport, visionFormat := context.GetVisionSupport(cap)
if visionSupport {
result = append(result, adapters.NewVisionAdapter(true, visionFormat))
} else if cap.Vision != nil {
} else {
result = append(result, adapters.NewVisionAdapter(false, context.VisionFormatNone))
}
@ -201,21 +202,10 @@ func (p *Provider) streamWithRetry(ctx *context.Context, messages []context.Mess
return nil, fmt.Errorf("failed to build request body: %w", err)
}
// Get connector settings
setting := p.Connector.Setting()
host, ok := setting["host"].(string)
if !ok || host == "" {
return nil, fmt.Errorf("no host found in connector settings")
}
key, ok := setting["key"].(string)
if !ok || key == "" {
return nil, fmt.Errorf("API key is not set")
}
version := "2023-06-01"
if v, ok := setting["version"].(string); ok && v != "" {
version = v
// Get connector settings via LLMConnector or fallback
host, key, version, err := p.resolveHostKeyVersion()
if err != nil {
return nil, err
}
// Build URL: host/v1/messages
@ -227,13 +217,13 @@ func (p *Provider) streamWithRetry(ctx *context.Context, messages []context.Mess
})
}
// Create HTTP request with Anthropic auth headers
// Create HTTP request with auth headers
req := http.New(url).
SetHeader("Content-Type", "application/json").
SetHeader("x-api-key", key).
SetHeader("anthropic-version", version).
SetHeader("Accept", "text/event-stream").
SetHeader("User-Agent", "YaoAgent/1.0 (+https://yaoagents.com)")
SetHeader("User-Agent", "YaoEngine/"+share.VERSION)
setAnthropicAuthHeaders(req, p.Connector, key)
// Accumulate response data
accumulator := &streamAccumulator{
@ -678,31 +668,20 @@ func (p *Provider) postWithRetry(ctx *context.Context, messages []context.Messag
return nil, fmt.Errorf("failed to build request body: %w", err)
}
// Get connector settings
setting := p.Connector.Setting()
host, ok := setting["host"].(string)
if !ok || host == "" {
return nil, fmt.Errorf("no host found in connector settings")
}
key, ok := setting["key"].(string)
if !ok || key == "" {
return nil, fmt.Errorf("API key is not set")
}
version := "2023-06-01"
if v, ok := setting["version"].(string); ok && v != "" {
version = v
// Get connector settings via LLMConnector or fallback
host, key, version, err := p.resolveHostKeyVersion()
if err != nil {
return nil, err
}
url := buildAPIURL(host, "/messages")
// Create HTTP request
// Create HTTP request with auth headers
req := http.New(url).
SetHeader("Content-Type", "application/json").
SetHeader("x-api-key", key).
SetHeader("anthropic-version", version).
SetHeader("User-Agent", "YaoAgent/1.0 (+https://yaoagents.com)")
SetHeader("User-Agent", "YaoEngine/"+share.VERSION)
setAnthropicAuthHeaders(req, p.Connector, key)
resp := req.Post(requestBody)
if resp.Code != 200 {
@ -915,6 +894,11 @@ func (p *Provider) buildRequestBody(messages []context.Message, options *context
} else if mt, ok := setting["max_tokens"].(int); ok && mt > 0 {
maxTokens = mt
}
if lc, ok := p.Connector.(goullm.LLMConnector); ok {
if caps := lc.GetCapabilities(); caps != nil && caps.MaxOutputTokens > 0 && maxTokens > caps.MaxOutputTokens {
maxTokens = caps.MaxOutputTokens
}
}
body["max_tokens"] = maxTokens
// Temperature
@ -942,9 +926,13 @@ func (p *Provider) buildRequestBody(messages []context.Message, options *context
body["tool_choice"] = convertToolChoice(options.ToolChoice)
}
// Thinking configuration from connector settings
if thinking, exists := setting["thinking"]; exists && thinking != nil {
body["thinking"] = thinking
// Merge connector-level body params (thinking, etc.)
// filtered through the SupportedParams / default whitelist.
connParams := connector.FilterRequestBodyParams(setting, p.Connector)
for k, v := range connParams {
if _, exists := body[k]; !exists {
body[k] = v
}
}
return body, nil
@ -1176,3 +1164,47 @@ func isRetryableError(err error) bool {
return false
}
// resolveHostKeyVersion extracts host, key, and version via LLMConnector or Setting().
// Setting() is called at most once, and only when needed.
func (p *Provider) resolveHostKeyVersion() (host, key, version string, err error) {
setting := p.Connector.Setting()
if lc, ok := p.Connector.(goullm.LLMConnector); ok {
host = lc.GetURL()
key = lc.GetKey()
} else {
host, _ = setting["host"].(string)
key, _ = setting["key"].(string)
}
// Version is Anthropic-specific, not on LLMConnector interface
version = "2023-06-01"
if v, ok := setting["version"].(string); ok && v != "" {
version = v
}
if host == "" {
return "", "", "", fmt.Errorf("no host found in connector settings")
}
if key == "" {
return "", "", "", fmt.Errorf("API key is not set")
}
return host, key, version, nil
}
// setAnthropicAuthHeaders sets auth headers based on LLMConnector.GetAuthMode().
func setAnthropicAuthHeaders(req *http.Request, conn connector.Connector, key string) {
if lc, ok := conn.(goullm.LLMConnector); ok {
switch lc.GetAuthMode() {
case goullm.AuthAPIKey:
req.SetHeader("api-key", key)
return
case goullm.AuthBearer:
req.SetHeader("Authorization", fmt.Sprintf("Bearer %s", key))
return
}
}
// Default for Anthropic: x-api-key
req.SetHeader("x-api-key", key)
}

View file

@ -16,7 +16,7 @@ import (
"github.com/yaoapp/yao/test"
)
// testConnectorID uses the cheapest model (Claude Haiku 3) to save tokens
// testConnectorID uses the cheapest model (Claude Haiku 4.5) to save tokens
const testConnectorID = "claude.haiku-3_0"
// TestAnthropicStreamBasic tests basic streaming completion with Anthropic API
@ -214,7 +214,7 @@ func TestAnthropicStreamRetry(t *testing.T) {
connDSL := `{
"type": "anthropic",
"options": {
"model": "claude-3-haiku-20240307",
"model": "claude-haiku-4-5-20251001",
"key": "sk-ant-invalid-key-should-fail"
}
}`

View file

@ -140,15 +140,30 @@ func (p *Provider) GetConnectorStringSetting(key string) (string, error) {
// GetModel gets the model name from connector settings
func (p *Provider) GetModel() (string, error) {
if lc, ok := p.Connector.(llm.LLMConnector); ok {
if m := lc.GetModel(); m != "" {
return m, nil
}
}
return p.GetConnectorStringSetting("model")
}
// GetAPIKey gets the API key from connector settings
func (p *Provider) GetAPIKey() (string, error) {
if lc, ok := p.Connector.(llm.LLMConnector); ok {
if k := lc.GetKey(); k != "" {
return k, nil
}
}
return p.GetConnectorStringSetting("key")
}
// GetHost gets the host URL from connector settings
func (p *Provider) GetHost() (string, error) {
if lc, ok := p.Connector.(llm.LLMConnector); ok {
if u := lc.GetURL(); u != "" {
return u, nil
}
}
return p.GetConnectorStringSetting("host")
}

View file

@ -4,6 +4,7 @@ import (
"fmt"
"github.com/yaoapp/gou/connector"
goullm "github.com/yaoapp/gou/llm"
"github.com/yaoapp/yao/agent/context"
"github.com/yaoapp/yao/agent/llm/providers/anthropic"
"github.com/yaoapp/yao/agent/llm/providers/openai"
@ -61,16 +62,23 @@ func DetectAPIFormat(conn connector.Connector) string {
return "openai"
}
// Check connector settings for host URL patterns as fallback
settings := conn.Setting()
if settings != nil {
if host, ok := settings["host"].(string); ok {
if contains(host, "anthropic.com") || contains(host, "api.kimi.com/coding") {
return "anthropic"
}
if contains(host, "deepseek.com") {
return "openai"
}
// Try LLMConnector for typed URL access, fall back to Setting() map
var host string
if lc, ok := conn.(goullm.LLMConnector); ok {
host = lc.GetURL()
}
if host == "" {
if settings := conn.Setting(); settings != nil {
host, _ = settings["host"].(string)
}
}
if host != "" {
if contains(host, "anthropic.com") || contains(host, "api.kimi.com/coding") {
return "anthropic"
}
if contains(host, "deepseek.com") {
return "openai"
}
}

View file

@ -16,6 +16,7 @@ import (
"github.com/yaoapp/yao/agent/llm/adapters"
"github.com/yaoapp/yao/agent/llm/providers/base"
"github.com/yaoapp/yao/agent/output/message"
"github.com/yaoapp/yao/share"
"github.com/yaoapp/yao/utils/jsonschema"
)
@ -155,12 +156,11 @@ func buildAdapters(cap *goullm.Capabilities) []adapters.CapabilityAdapter {
// Tool call adapter
result = append(result, adapters.NewToolCallAdapter(cap.ToolCalls))
// Vision adapter
// Vision adapter (always registered to strip unsupported image content)
visionSupport, visionFormat := context.GetVisionSupport(cap)
if visionSupport {
result = append(result, adapters.NewVisionAdapter(true, visionFormat))
} else if cap.Vision != nil {
// Vision explicitly disabled, add adapter to remove image content
} else {
result = append(result, adapters.NewVisionAdapter(false, context.VisionFormatNone))
}
@ -385,16 +385,10 @@ func (p *Provider) streamWithRetry(ctx *context.Context, messages []context.Mess
return nil, fmt.Errorf("failed to build request body: %w", err)
}
// Get connector settings
setting := p.Connector.Setting()
host, ok := setting["host"].(string)
if !ok || host == "" {
return nil, fmt.Errorf("no host found in connector settings")
}
key, ok := setting["key"].(string)
if !ok || key == "" {
return nil, fmt.Errorf("API key is not set")
// Get connector settings via LLMConnector or fallback
host, key, err := p.resolveHostKey()
if err != nil {
return nil, err
}
// Build URL
@ -409,9 +403,9 @@ func (p *Provider) streamWithRetry(ctx *context.Context, messages []context.Mess
// Create HTTP request with proxy support
req := http.New(url).
SetHeader("Content-Type", "application/json").
SetHeader("Authorization", fmt.Sprintf("Bearer %s", key)).
SetHeader("Accept", "text/event-stream").
SetHeader("User-Agent", "YaoAgent/1.0 (+https://yaoagents.com)")
SetHeader("User-Agent", "YaoEngine/"+share.VERSION)
setAuthHeaders(req, p.Connector, key)
// Accumulate response data
accumulator := &streamAccumulator{
@ -498,16 +492,18 @@ func (p *Provider) streamWithRetry(ctx *context.Context, messages []context.Mess
accumulator.role = delta.Role
}
// Handle reasoning content (DeepSeek R1)
if delta.ReasoningContent != "" {
// Start thinking message if not active
reasoningText := delta.ReasoningContent
if reasoningText == "" {
reasoningText = delta.Reasoning
}
if reasoningText != "" {
if !messageTracker.active || messageTracker.messageType != message.ChunkThinking {
messageTracker.startMessage(message.ChunkThinking, handler)
}
accumulator.reasoningContent += delta.ReasoningContent
accumulator.reasoningContent += reasoningText
if handler != nil {
handler(message.ChunkThinking, []byte(delta.ReasoningContent))
handler(message.ChunkThinking, []byte(reasoningText))
messageTracker.incrementChunk()
}
}
@ -922,16 +918,10 @@ func (p *Provider) postWithRetry(ctx *context.Context, messages []context.Messag
return nil, fmt.Errorf("failed to build request body: %w", err)
}
// Get connector settings
setting := p.Connector.Setting()
host, ok := setting["host"].(string)
if !ok || host == "" {
return nil, fmt.Errorf("no host found in connector settings")
}
key, ok := setting["key"].(string)
if !ok || key == "" {
return nil, fmt.Errorf("API key is not set")
// Get connector settings via LLMConnector or fallback
host, key, err := p.resolveHostKey()
if err != nil {
return nil, err
}
// Build URL
@ -940,8 +930,8 @@ func (p *Provider) postWithRetry(ctx *context.Context, messages []context.Messag
// Create HTTP request with proxy support
req := http.New(url).
SetHeader("Content-Type", "application/json").
SetHeader("Authorization", fmt.Sprintf("Bearer %s", key)).
SetHeader("User-Agent", "YaoAgent/1.0 (+https://yaoagents.com)")
SetHeader("User-Agent", "YaoEngine/"+share.VERSION)
setAuthHeaders(req, p.Connector, key)
// Make request
resp := req.Post(requestBody)
@ -1006,7 +996,7 @@ func (p *Provider) postWithRetry(ctx *context.Context, messages []context.Messag
Model: fullResp.Model,
Role: string(choice.Message.Role),
Content: content,
ReasoningContent: choice.Message.ReasoningContent,
ReasoningContent: reasoningOrFallback(choice.Message.ReasoningContent, choice.Message.Reasoning),
ToolCalls: choice.Message.ToolCalls,
FinishReason: choice.FinishReason,
Usage: fullResp.Usage,
@ -1040,12 +1030,6 @@ func (p *Provider) buildRequestBody(messages []context.Message, options *context
return nil, fmt.Errorf("model is not set in connector")
}
// Get thinking setting from connector (for models that support reasoning/thinking mode)
var thinkingSetting interface{}
if thinking, exists := setting["thinking"]; exists {
thinkingSetting = thinking
}
// Convert messages to API format
apiMessages := make([]map[string]interface{}, 0, len(messages))
for _, msg := range messages {
@ -1099,6 +1083,10 @@ func (p *Provider) buildRequestBody(messages []context.Message, options *context
apiMsg["tool_calls"] = msg.ToolCalls
}
if msg.ReasoningContent != "" {
apiMsg["reasoning_content"] = msg.ReasoningContent
}
if msg.Refusal != nil {
apiMsg["refusal"] = *msg.Refusal
}
@ -1120,11 +1108,19 @@ func (p *Provider) buildRequestBody(messages []context.Message, options *context
// Use max_completion_tokens (modern API parameter for GPT-5+)
// GPT-5 models only support max_completion_tokens (not max_tokens)
if options.MaxCompletionTokens != nil {
body["max_completion_tokens"] = *options.MaxCompletionTokens
} else if options.MaxTokens != nil {
// Fallback: convert MaxTokens to max_completion_tokens for compatibility
body["max_completion_tokens"] = *options.MaxTokens
if options.MaxCompletionTokens != nil || options.MaxTokens != nil {
maxTokens := 0
if options.MaxCompletionTokens != nil {
maxTokens = *options.MaxCompletionTokens
} else {
maxTokens = *options.MaxTokens
}
if lc, ok := p.Connector.(goullm.LLMConnector); ok {
if caps := lc.GetCapabilities(); caps != nil && caps.MaxOutputTokens > 0 && maxTokens > caps.MaxOutputTokens {
maxTokens = caps.MaxOutputTokens
}
}
body["max_completion_tokens"] = maxTokens
}
if options.TopP != nil {
@ -1202,9 +1198,14 @@ func (p *Provider) buildRequestBody(messages []context.Message, options *context
body["audio"] = options.Audio
}
// Add thinking parameter for models that support reasoning/thinking mode
if thinkingSetting != nil {
body["thinking"] = thinkingSetting
// Merge connector-level body params (thinking, reasoning, enable_thinking, etc.)
// filtered through the SupportedParams / default whitelist.
// CompletionOptions (per-call) take precedence over connector defaults.
connParams := connector.FilterRequestBodyParams(setting, p.Connector)
for k, v := range connParams {
if _, exists := body[k]; !exists {
body[k] = v
}
}
return body, nil
@ -1289,3 +1290,44 @@ func isRetryableError(err error) bool {
return false
}
// resolveHostKey extracts host and key via LLMConnector or Setting() fallback.
func (p *Provider) resolveHostKey() (host, key string, err error) {
if lc, ok := p.Connector.(goullm.LLMConnector); ok {
host = lc.GetURL()
key = lc.GetKey()
} else {
setting := p.Connector.Setting()
host, _ = setting["host"].(string)
key, _ = setting["key"].(string)
}
if host == "" {
return "", "", fmt.Errorf("no host found in connector settings")
}
if key == "" {
return "", "", fmt.Errorf("API key is not set")
}
return host, key, nil
}
// setAuthHeaders sets authentication headers based on LLMConnector.GetAuthMode().
func setAuthHeaders(req *http.Request, conn connector.Connector, key string) {
if lc, ok := conn.(goullm.LLMConnector); ok {
switch lc.GetAuthMode() {
case goullm.AuthAPIKey:
req.SetHeader("api-key", key)
return
case goullm.AuthXAPIKey:
req.SetHeader("x-api-key", key)
return
}
}
req.SetHeader("Authorization", fmt.Sprintf("Bearer %s", key))
}
func reasoningOrFallback(primary, fallback string) string {
if primary != "" {
return primary
}
return fallback
}

View file

@ -30,7 +30,8 @@ type Delta struct {
type DeltaContent struct {
Role string `json:"role,omitempty"`
Content string `json:"content,omitempty"`
ReasoningContent string `json:"reasoning_content,omitempty"` // DeepSeek R1 reasoning
ReasoningContent string `json:"reasoning_content,omitempty"` // DeepSeek direct API
Reasoning string `json:"reasoning,omitempty"` // OpenRouter
ToolCalls []ToolCallDelta `json:"tool_calls,omitempty"`
Refusal string `json:"refusal,omitempty"`
}
@ -60,7 +61,8 @@ type CompletionResponseFull struct {
Message struct {
Role context.MessageRole `json:"role"`
Content interface{} `json:"content,omitempty"` // string or array
ReasoningContent string `json:"reasoning_content,omitempty"` // DeepSeek R1 reasoning
ReasoningContent string `json:"reasoning_content,omitempty"` // DeepSeek direct API
Reasoning string `json:"reasoning,omitempty"` // OpenRouter
ToolCalls []context.ToolCall `json:"tool_calls,omitempty"`
Refusal *string `json:"refusal,omitempty"`
} `json:"message"`

94
agent/llm/resolve.go Normal file
View file

@ -0,0 +1,94 @@
package llm
import (
"fmt"
"strings"
"github.com/yaoapp/gou/connector"
goullm "github.com/yaoapp/gou/llm"
"github.com/yaoapp/yao/llmprovider"
)
// RolePrefix marks a Connector field value as a role reference (e.g. "use::light").
const RolePrefix = "use::"
// ResolveConnector resolves an LLM connector using a unified priority chain.
//
// connectorID may be:
// - explicit connector ID (e.g. "openai.gpt-4o") — resolved directly
// - role reference with prefix (e.g. "use::light") — resolved via llmprovider roles
// - empty string — falls back to the "default" role
//
// Priority for role-based resolution:
// 1. GetRoleBy(role, identity) — user/team scoped setting
// 2. GetRole(role) — system-level default for that role
// 3. GetRoleBy("default", identity) — fallback to "default" role (user/team)
// 4. GetRole("default") — fallback to "default" role (system)
// 5. error — caller decides whether to apply legacy fallback
func ResolveConnector(connectorID string, identity llmprovider.Identity) (connector.Connector, *goullm.Capabilities, error) {
// Parse use:: prefix to extract role
role := ""
if strings.HasPrefix(connectorID, RolePrefix) {
role = strings.TrimPrefix(connectorID, RolePrefix)
connectorID = ""
}
// Explicit connector ID takes highest priority
if connectorID != "" {
return selectWithCapabilities(connectorID)
}
// Empty connector with no role → treat as "default"
if role == "" {
role = "default"
}
if llmprovider.Global == nil {
return nil, nil, fmt.Errorf("llmprovider not initialized and no explicit connector specified")
}
// Resolve by the specified role (e.g. "light", "vision")
if role != "default" {
if identity != nil {
if cid, err := llmprovider.Global.GetRoleBy(role, identity); err == nil && cid != "" {
if conn, caps, err := selectWithCapabilities(cid); err == nil {
return conn, caps, nil
}
}
}
if cid, err := llmprovider.Global.GetRole(role); err == nil && cid != "" {
if conn, caps, err := selectWithCapabilities(cid); err == nil {
return conn, caps, nil
}
}
}
// Fallback to "default" role
if identity != nil {
if cid, err := llmprovider.Global.GetRoleBy("default", identity); err == nil && cid != "" {
if conn, caps, err := selectWithCapabilities(cid); err == nil {
return conn, caps, nil
}
}
}
if cid, err := llmprovider.Global.GetRole("default"); err == nil && cid != "" {
if conn, caps, err := selectWithCapabilities(cid); err == nil {
return conn, caps, nil
}
}
return nil, nil, fmt.Errorf("no connector resolved for role %q", role)
}
func selectWithCapabilities(connectorID string) (connector.Connector, *goullm.Capabilities, error) {
conn, err := connector.Select(connectorID)
if err != nil && llmprovider.Global != nil {
conn, err = llmprovider.Global.GetModel(connectorID)
}
if err != nil {
return nil, nil, err
}
caps := GetCapabilitiesFromConn(conn)
return conn, caps, nil
}

171
agent/llm/resolve_test.go Normal file
View file

@ -0,0 +1,171 @@
package llm_test
import (
"os"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/yaoapp/gou/connector"
"github.com/yaoapp/gou/store"
"github.com/yaoapp/yao/agent/llm"
"github.com/yaoapp/yao/config"
"github.com/yaoapp/yao/llmprovider"
"github.com/yaoapp/yao/setting"
"github.com/yaoapp/yao/test"
)
func TestMain(m *testing.M) {
test.Prepare(nil, config.Conf)
defer test.Clean()
os.Exit(m.Run())
}
type mockIdentity struct {
UserID string
TeamID string
}
func (m *mockIdentity) GetUserID() string { return m.UserID }
func (m *mockIdentity) GetTeamID() string { return m.TeamID }
func setupResolveTest(t *testing.T) string {
t.Helper()
test.Prepare(t, config.Conf)
err := setting.Init()
require.NoError(t, err)
err = llmprovider.Init()
require.NoError(t, err)
connIDs := connector.AIConnectors
if len(connIDs) == 0 {
t.Skip("no AI connectors available in test env")
}
cid := connIDs[0].Value
t.Cleanup(func() {
s, _ := store.Get("__yao.store")
if s != nil {
s.Del("llmprovider:*")
}
c, _ := store.Get("__yao.cache")
if c != nil {
c.Del("llmprovider:*")
}
test.Clean()
})
return cid
}
// --- use:: prefix tests ---
func TestResolveConnector_UseLight(t *testing.T) {
cid := setupResolveTest(t)
err := llmprovider.Global.SetDefaults(map[string]string{
"default": cid,
"light": cid,
})
require.NoError(t, err)
conn, caps, err := llm.ResolveConnector("use::light", nil)
require.NoError(t, err)
assert.NotNil(t, conn)
assert.NotNil(t, caps)
}
func TestResolveConnector_UseDefault(t *testing.T) {
cid := setupResolveTest(t)
err := llmprovider.Global.SetDefaults(map[string]string{
"default": cid,
})
require.NoError(t, err)
conn, caps, err := llm.ResolveConnector("use::default", nil)
require.NoError(t, err)
assert.NotNil(t, conn)
assert.NotNil(t, caps)
}
func TestResolveConnector_UseLightWithIdentity(t *testing.T) {
cid := setupResolveTest(t)
err := llmprovider.Global.SetDefaults(map[string]string{
"default": cid,
"light": cid,
})
require.NoError(t, err)
conn, caps, err := llm.ResolveConnector("use::light", &mockIdentity{UserID: "u1", TeamID: "t1"})
require.NoError(t, err)
assert.NotNil(t, conn)
assert.NotNil(t, caps)
}
func TestResolveConnector_UseLightNoProvider(t *testing.T) {
test.Prepare(t, config.Conf)
defer test.Clean()
saved := llmprovider.Global
llmprovider.Global = nil
defer func() { llmprovider.Global = saved }()
_, _, err := llm.ResolveConnector("use::light", nil)
assert.Error(t, err)
}
// --- Explicit connector tests ---
func TestResolveConnector_ExplicitID(t *testing.T) {
cid := setupResolveTest(t)
conn, caps, err := llm.ResolveConnector(cid, nil)
require.NoError(t, err)
assert.NotNil(t, conn)
assert.NotNil(t, caps)
}
func TestResolveConnector_ExplicitIDPriority(t *testing.T) {
cid := setupResolveTest(t)
err := llmprovider.Global.SetDefaults(map[string]string{
"default": cid,
"light": cid,
})
require.NoError(t, err)
// Explicit connector ID is NOT a use:: prefix, so it takes priority
conn, caps, err := llm.ResolveConnector(cid, &mockIdentity{UserID: "u1"})
require.NoError(t, err)
assert.NotNil(t, conn)
assert.NotNil(t, caps)
}
func TestResolveConnector_InvalidID(t *testing.T) {
setupResolveTest(t)
_, _, err := llm.ResolveConnector("nonexistent-connector-xyz", nil)
assert.Error(t, err)
}
// --- Empty connector fallback ---
func TestResolveConnector_EmptyFallbackDefault(t *testing.T) {
cid := setupResolveTest(t)
err := llmprovider.Global.SetDefaults(map[string]string{
"default": cid,
})
require.NoError(t, err)
// Empty string → treated as use::default
conn, caps, err := llm.ResolveConnector("", nil)
require.NoError(t, err)
assert.NotNil(t, conn)
assert.NotNil(t, caps)
}

View file

@ -19,6 +19,7 @@ import (
"github.com/yaoapp/yao/agent/store/xun"
"github.com/yaoapp/yao/agent/types"
"github.com/yaoapp/yao/config"
"github.com/yaoapp/yao/llmprovider"
)
var agentDSL *types.DSL
@ -226,15 +227,18 @@ func initAssistant() error {
// Set system agents configuration
if agentDSL.System != nil {
assistant.SetSystemConfig(&assistant.SystemConfig{
Default: agentDSL.System.Default,
Keyword: agentDSL.System.Keyword,
QueryDSL: agentDSL.System.QueryDSL,
Title: agentDSL.System.Title,
Prompt: agentDSL.System.Prompt,
NeedSearch: agentDSL.System.NeedSearch,
Entity: agentDSL.System.Entity,
Vision: agentDSL.System.Vision,
Voice: agentDSL.System.Voice,
Default: agentDSL.System.Default,
Light: agentDSL.System.Light,
Vision: agentDSL.System.Vision,
Audio: agentDSL.System.Audio,
Heavy: agentDSL.System.Heavy,
Keyword: agentDSL.System.Keyword,
QueryDSL: agentDSL.System.QueryDSL,
Title: agentDSL.System.Title,
Prompt: agentDSL.System.Prompt,
RobotPrompt: agentDSL.System.RobotPrompt,
NeedSearch: agentDSL.System.NeedSearch,
Entity: agentDSL.System.Entity,
})
}
@ -453,6 +457,22 @@ func GetSearchConfig() *searchTypes.Config {
return agentDSL.Search
}
// SyncLLMDefaults writes the agent.yml system role defaults into setting.Global.
// Must be called after both llmprovider.Init() and setting.Init() have completed.
func SyncLLMDefaults() error {
if agentDSL == nil || agentDSL.System == nil {
return nil
}
if llmprovider.Global == nil {
return fmt.Errorf("llmprovider.Global not initialized")
}
roles := buildSystemRoles(agentDSL.System)
if len(roles) == 0 {
return nil
}
return llmprovider.Global.SetDefaults(roles)
}
// defaultAssistant get the default assistant
func defaultAssistant() (*assistant.Assistant, error) {
if agentDSL.Uses == nil || agentDSL.Uses.Default == "" {
@ -461,12 +481,34 @@ func defaultAssistant() (*assistant.Assistant, error) {
return assistant.Get(agentDSL.Uses.Default)
}
// buildSystemRoles converts the System config block into a role→connectorID map
// for llmprovider.SetDefaults. Only role-level keys are written here; per-agent
// overrides (keyword, title, querydsl, etc.) are consumed by resolveSystemConnector.
func buildSystemRoles(sys *types.System) map[string]string {
roles := make(map[string]string)
add := func(role, cid string) {
if cid != "" {
roles[role] = cid
}
}
add("default", sys.Default)
add("light", sys.Light)
add("vision", sys.Vision)
add("audio", sys.Audio)
add("heavy", sys.Heavy)
return roles
}
// resolveEnvStrings resolves $ENV.XXX references in agent.yml string fields.
// agent.yml is parsed via yaml.Unmarshal which does not handle $ENV substitution,
// unlike connector files which call helper.EnvString explicitly during Register.
func resolveEnvStrings(setting *types.DSL) {
if setting.System != nil {
setting.System.Default = helper.EnvString(setting.System.Default)
setting.System.Light = helper.EnvString(setting.System.Light)
setting.System.Vision = helper.EnvString(setting.System.Vision)
setting.System.Audio = helper.EnvString(setting.System.Audio)
setting.System.Heavy = helper.EnvString(setting.System.Heavy)
setting.System.Keyword = helper.EnvString(setting.System.Keyword)
setting.System.QueryDSL = helper.EnvString(setting.System.QueryDSL)
setting.System.Title = helper.EnvString(setting.System.Title)
@ -474,8 +516,6 @@ func resolveEnvStrings(setting *types.DSL) {
setting.System.RobotPrompt = helper.EnvString(setting.System.RobotPrompt)
setting.System.NeedSearch = helper.EnvString(setting.System.NeedSearch)
setting.System.Entity = helper.EnvString(setting.System.Entity)
setting.System.Vision = helper.EnvString(setting.System.Vision)
setting.System.Voice = helper.EnvString(setting.System.Voice)
}
if setting.Uses != nil {

View file

@ -229,7 +229,7 @@ func TestResolveEnvStrings(t *testing.T) {
NeedSearch: "$ENV.TEST_CONNECTOR",
Entity: "$ENV.TEST_CONNECTOR",
Vision: "$ENV.TEST_CONNECTOR",
Voice: "$ENV.TEST_CONNECTOR",
Audio: "$ENV.TEST_CONNECTOR",
},
}
resolveEnvStrings(setting)
@ -243,24 +243,24 @@ func TestResolveEnvStrings(t *testing.T) {
assert.Equal(t, "openai.gpt-5", setting.System.NeedSearch)
assert.Equal(t, "openai.gpt-5", setting.System.Entity)
assert.Equal(t, "openai.gpt-5", setting.System.Vision)
assert.Equal(t, "openai.gpt-5", setting.System.Voice)
assert.Equal(t, "openai.gpt-5", setting.System.Audio)
})
t.Run("SystemVisionVoiceSeparateEnv", func(t *testing.T) {
t.Run("SystemVisionAudioSeparateEnv", func(t *testing.T) {
t.Setenv("TEST_VISION_CONN", "openai.gpt-4o")
t.Setenv("TEST_VOICE_CONN", "whisper-1")
t.Setenv("TEST_AUDIO_CONN", "whisper-1")
setting := &types.DSL{
System: &types.System{
Default: "$ENV.TEST_CONNECTOR",
Vision: "$ENV.TEST_VISION_CONN",
Voice: "$ENV.TEST_VOICE_CONN",
Audio: "$ENV.TEST_AUDIO_CONN",
},
}
resolveEnvStrings(setting)
assert.Equal(t, "openai.gpt-5", setting.System.Default)
assert.Equal(t, "openai.gpt-4o", setting.System.Vision)
assert.Equal(t, "whisper-1", setting.System.Voice)
assert.Equal(t, "whisper-1", setting.System.Audio)
})
t.Run("UsesFields", func(t *testing.T) {

View file

@ -111,6 +111,7 @@ func triggerHuman(ctx *types.Context, mgr managerInterface, memberID string, req
Messages: req.Messages,
PlanTime: req.PlanAt,
ExecutorMode: req.ExecutorMode,
Locale: req.Locale,
}
// Call manager's Intervene

11
agent/robot/doc.go Normal file
View file

@ -0,0 +1,11 @@
package robot
import (
_ "embed"
"github.com/yaoapp/gou/doc"
)
//go:embed doc.yml
var docYAML []byte
func init() { doc.LoadYAML(docYAML) }

80
agent/robot/doc.yml Normal file
View file

@ -0,0 +1,80 @@
group: robot
type: process
entries:
- name: get
desc: Get a robot's details by member ID
args:
- name: memberID
type: string
required: true
desc: The member ID of the robot to retrieve
return:
type: object
desc: Robot detail object
- name: list
desc: List all robots with optional filtering and pagination
args:
- name: filter
type: object
required: false
desc: "Filter options: page (number), pagesize (number), status (string), search (string, keywords), team_id (string)"
return:
type: object
desc: Paginated list of robots
- name: status
desc: Get the current status of a robot by member ID
args:
- name: memberID
type: string
required: true
desc: The member ID of the robot
return:
type: object
desc: Robot status object
- name: executions
desc: List executions for a robot with optional filtering and pagination
args:
- name: memberID
type: string
required: true
desc: The member ID of the robot
- name: filter
type: object
required: false
desc: "Filter options: page (number), pagesize (number), status (string, execution status), trigger (string, trigger type)"
return:
type: object
desc: Paginated list of execution records
- name: execution
desc: Get a specific execution record by member ID and execution ID
args:
- name: memberID
type: string
required: true
desc: The member ID of the robot (reserved for permission scoping)
- name: executionID
type: string
required: true
desc: The execution ID to retrieve
return:
type: object
desc: Execution status and details
- name: updateChatTitle
desc: Update the title of a chat session
args:
- name: chatID
type: string
required: true
desc: The chat session ID to update
- name: title
type: string
required: true
desc: The new title for the chat session
return:
type: "null"
desc: Returns null on success

View file

@ -13,6 +13,8 @@ import (
"strings"
"time"
"path/filepath"
"github.com/yaoapp/gou/process"
"github.com/yaoapp/gou/text"
agentcontext "github.com/yaoapp/yao/agent/context"
@ -21,6 +23,7 @@ import (
eventtypes "github.com/yaoapp/yao/event/types"
"github.com/yaoapp/yao/messenger"
messengerTypes "github.com/yaoapp/yao/messenger/types"
"github.com/yaoapp/yao/workspace"
)
// handleDelivery routes delivery content to configured channels (email, webhook, process).
@ -52,6 +55,10 @@ func (h *robotHandler) handleDelivery(ctx context.Context, ev *eventtypes.Event,
return
}
if ev.Auth != nil {
ctx = context.WithValue(ctx, "identity", ev.Auth)
}
deliveryCtx := &robottypes.DeliveryContext{
MemberID: payload.MemberID,
ExecutionID: payload.ExecutionID,
@ -428,6 +435,15 @@ func convertAttachments(ctx context.Context, attachments []robottypes.DeliveryAt
result := make([]messengerTypes.Attachment, 0, len(attachments))
for _, att := range attachments {
// Handle workspace:// URIs — read file content from workspace FS
if strings.HasPrefix(att.File, "workspace://") {
wsAtt := convertWorkspaceAttachment(ctx, att)
if wsAtt != nil {
result = append(result, *wsAtt)
}
continue
}
uploader, fileID, isWrapper := attachment.Parse(att.File)
if !isWrapper {
log.Warn("convertAttachments: skipping non-wrapper file value=%q title=%q", att.File, att.Title)
@ -477,6 +493,87 @@ func convertAttachments(ctx context.Context, attachments []robottypes.DeliveryAt
return result
}
// convertWorkspaceAttachment reads a file from workspace:// URI and returns a messenger attachment.
// URI format: workspace://<wsID>/<path>
func convertWorkspaceAttachment(ctx context.Context, att robottypes.DeliveryAttachment) *messengerTypes.Attachment {
uri := att.File
// Strip "workspace://" prefix
rest := strings.TrimPrefix(uri, "workspace://")
slashIdx := strings.Index(rest, "/")
if slashIdx < 0 {
log.Warn("convertWorkspaceAttachment: invalid URI %q — no path after wsID", uri)
return nil
}
wsID := rest[:slashIdx]
filePath := rest[slashIdx+1:]
if wsID == "" || filePath == "" {
log.Warn("convertWorkspaceAttachment: empty wsID or path in URI %q", uri)
return nil
}
wsm := workspace.M()
if wsm == nil {
log.Warn("convertWorkspaceAttachment: workspace manager not available for URI %q", uri)
return nil
}
wsFS, err := wsm.FS(ctx, wsID)
if err != nil {
log.Warn("convertWorkspaceAttachment: cannot get FS for workspace %q: %v", wsID, err)
return nil
}
content, err := wsFS.ReadFile(filePath)
if err != nil {
log.Warn("convertWorkspaceAttachment: failed to read %q from workspace %q: %v", filePath, wsID, err)
return nil
}
filename := filepath.Base(filePath)
if att.Title != "" {
filename = att.Title
}
contentType := mimeFromExtDelivery(filepath.Ext(filename))
log.Info("convertWorkspaceAttachment: added workspace attachment filename=%q contentType=%q size=%d uri=%q",
filename, contentType, len(content), uri)
return &messengerTypes.Attachment{
Filename: filename,
ContentType: contentType,
Content: content,
}
}
func mimeFromExtDelivery(ext string) string {
switch strings.ToLower(ext) {
case ".pdf":
return "application/pdf"
case ".html", ".htm":
return "text/html"
case ".png":
return "image/png"
case ".jpg", ".jpeg":
return "image/jpeg"
case ".gif":
return "image/gif"
case ".csv":
return "text/csv"
case ".json":
return "application/json"
case ".md":
return "text/markdown"
case ".txt":
return "text/plain"
case ".xlsx":
return "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
case ".pptx":
return "application/vnd.openxmlformats-officedocument.presentationml.presentation"
default:
return "application/octet-stream"
}
}
func attachmentManagerKeys() []string {
keys := make([]string, 0, len(attachment.Managers))
for k := range attachment.Managers {

View file

@ -91,12 +91,23 @@ func buildContentParts(cm *dtapi.ConvertedMessage) []interface{} {
if url == "" {
url = mi.URL
}
parts = append(parts, map[string]interface{}{
"type": "file",
"file_url": url,
"mime_type": mi.MimeType,
"file_name": mi.FileName,
})
if strings.HasPrefix(mi.MimeType, "image/") {
parts = append(parts, map[string]interface{}{
"type": "image_url",
"image_url": map[string]interface{}{
"url": url,
"detail": "auto",
},
})
} else {
parts = append(parts, map[string]interface{}{
"type": "file",
"file": map[string]interface{}{
"url": url,
"filename": mi.FileName,
},
})
}
}
return parts

View file

@ -96,12 +96,23 @@ func buildContentParts(cm *dcapi.ConvertedMessage) []interface{} {
if url == "" {
continue
}
parts = append(parts, map[string]interface{}{
"type": "file",
"file_url": url,
"mime_type": mi.ContentType,
"file_name": mi.FileName,
})
if strings.HasPrefix(mi.ContentType, "image/") {
parts = append(parts, map[string]interface{}{
"type": "image_url",
"image_url": map[string]interface{}{
"url": url,
"detail": "auto",
},
})
} else {
parts = append(parts, map[string]interface{}{
"type": "file",
"file": map[string]interface{}{
"url": url,
"filename": mi.FileName,
},
})
}
}
return parts

View file

@ -85,12 +85,23 @@ func buildContentParts(cm *fsapi.ConvertedMessage) []interface{} {
if mi.Wrapper == "" {
continue
}
parts = append(parts, map[string]interface{}{
"type": "file",
"file_url": mi.Wrapper,
"mime_type": mi.MimeType,
"file_name": mi.FileName,
})
if strings.HasPrefix(mi.MimeType, "image/") {
parts = append(parts, map[string]interface{}{
"type": "image_url",
"image_url": map[string]interface{}{
"url": mi.Wrapper,
"detail": "auto",
},
})
} else {
parts = append(parts, map[string]interface{}{
"type": "file",
"file": map[string]interface{}{
"url": mi.Wrapper,
"filename": mi.FileName,
},
})
}
}
return parts

View file

@ -93,12 +93,23 @@ func buildContentParts(cm *tgapi.ConvertedMessage) []interface{} {
if mi.Wrapper == "" {
continue
}
parts = append(parts, map[string]interface{}{
"type": "file",
"file_url": mi.Wrapper,
"mime_type": mi.MimeType,
"file_name": mi.FileName,
})
if strings.HasPrefix(mi.MimeType, "image/") {
parts = append(parts, map[string]interface{}{
"type": "image_url",
"image_url": map[string]interface{}{
"url": mi.Wrapper,
"detail": "auto",
},
})
} else {
parts = append(parts, map[string]interface{}{
"type": "file",
"file": map[string]interface{}{
"url": mi.Wrapper,
"filename": mi.FileName,
},
})
}
}
return parts

View file

@ -3,6 +3,7 @@ package weixin
import (
"context"
"fmt"
"strings"
"time"
agentcontext "github.com/yaoapp/yao/agent/context"
@ -113,12 +114,23 @@ func (a *Adapter) handleMessage(ctx context.Context, entry *botEntry, msg *weixi
parts = append(parts, map[string]interface{}{"type": "text", "text": content})
}
for _, m := range mediaItems {
parts = append(parts, map[string]interface{}{
"type": "file",
"file_url": m.Wrapper,
"mime_type": m.MimeType,
"file_name": m.FileName,
})
if strings.HasPrefix(m.MimeType, "image/") {
parts = append(parts, map[string]interface{}{
"type": "image_url",
"image_url": map[string]interface{}{
"url": m.Wrapper,
"detail": "auto",
},
})
} else {
parts = append(parts, map[string]interface{}{
"type": "file",
"file": map[string]interface{}{
"url": m.Wrapper,
"filename": m.FileName,
},
})
}
}
msgContent = parts
}

View file

@ -5,12 +5,14 @@ import (
"fmt"
"io"
"net/http"
"path/filepath"
"strings"
agentcontext "github.com/yaoapp/yao/agent/context"
events "github.com/yaoapp/yao/agent/robot/events"
"github.com/yaoapp/yao/attachment"
weixinapi "github.com/yaoapp/yao/integrations/weixin"
"github.com/yaoapp/yao/workspace"
)
func (a *Adapter) Reply(ctx context.Context, msg *agentcontext.Message, metadata *events.MessageMetadata) error {
@ -168,7 +170,37 @@ func (a *Adapter) sendMediaFromURL(ctx context.Context, entry *botEntry, toUserI
var plaintext []byte
var contentType string
if isWrapper(fileURL) {
if strings.HasPrefix(fileURL, "workspace://") {
rest := strings.TrimPrefix(fileURL, "workspace://")
slashIdx := strings.Index(rest, "/")
if slashIdx < 0 {
return fmt.Errorf("invalid workspace URL: %s", fileURL)
}
wsID := rest[:slashIdx]
filePath := rest[slashIdx+1:]
wsm := workspace.M()
if wsm == nil {
return fmt.Errorf("workspace manager not initialized")
}
wsFS, err := wsm.FS(ctx, wsID)
if err != nil {
return fmt.Errorf("open workspace %s: %w", wsID, err)
}
defer wsFS.Close()
plaintext, err = wsFS.ReadFile(filePath)
if err != nil {
return fmt.Errorf("read workspace file %s/%s: %w", wsID, filePath, err)
}
contentType = mimeFromExt(filepath.Ext(filePath))
if fileName == "" {
fileName = filepath.Base(filePath)
}
log.Info("weixin sendMedia: workspace read bytes=%d contentType=%q fileName=%q", len(plaintext), contentType, fileName)
} else if isWrapper(fileURL) {
managerName, fileID, err := parseWrapper(fileURL)
if err != nil {
return err
@ -291,6 +323,37 @@ func toContentParts(content interface{}) ([]agentcontext.ContentPart, bool) {
return parts, ok
}
func mimeFromExt(ext string) string {
switch strings.ToLower(ext) {
case ".pdf":
return "application/pdf"
case ".html", ".htm":
return "text/html"
case ".png":
return "image/png"
case ".jpg", ".jpeg":
return "image/jpeg"
case ".gif":
return "image/gif"
case ".webp":
return "image/webp"
case ".md":
return "text/markdown"
case ".txt":
return "text/plain"
case ".csv":
return "text/csv"
case ".json":
return "application/json"
case ".xlsx":
return "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
case ".pptx":
return "application/vnd.openxmlformats-officedocument.presentationml.presentation"
default:
return "application/octet-stream"
}
}
func (a *Adapter) resolveByAccountID(accountID string) *botEntry {
if accountID == "" {
return nil

View file

@ -33,6 +33,15 @@ func (h *robotHandler) handleMessage(ctx context.Context, ev *eventtypes.Event,
result, err := callHostAgent(ctx, &payload)
if err != nil {
log.Error("message handler: host agent call failed robot=%s: %v", payload.RobotID, err)
if reply := getReplyFunc(); reply != nil {
errMsg := friendlyErrorMessage(payload.Metadata.Locale)
_ = reply(ctx, &agentcontext.Message{
Role: agentcontext.RoleAssistant,
Content: errMsg,
}, payload.Metadata)
}
if ev.IsCall {
resp <- eventtypes.Result{Err: err}
}
@ -73,6 +82,7 @@ func callHostAgent(ctx context.Context, payload *MessagePayload) (*MessageResult
authorized := &oauthtypes.AuthorizedInfo{
UserID: payload.Metadata.SenderID,
TeamID: record.TeamID,
}
chatID := fmt.Sprintf("%s:%s", payload.Metadata.Channel, payload.Metadata.ChatID)
agentCtx := agentcontext.New(ctx, authorized, chatID)
@ -232,6 +242,13 @@ func resolveHostAssistantID(ctx context.Context, memberID string) (string, *robo
return hostID, record, nil
}
func friendlyErrorMessage(locale string) string {
if strings.HasPrefix(locale, "zh") {
return "抱歉,处理您的消息时出现了问题,请稍后重试。"
}
return "Sorry, there was a problem processing your message. Please try again later."
}
func taskDeployedMessage(execID string, locale string) string {
if strings.HasPrefix(locale, "zh") {
return fmt.Sprintf("任务已部署(执行编号: %s完成后会将结果发送给你。", execID)

View file

@ -53,6 +53,11 @@ type AgentCaller struct {
// When non-empty, injected into agentCtx.Metadata["workspace_id"] for sandbox node resolution.
Workspace string
// Mode is the agent execution mode (e.g., "task" for robot P3 execution).
// When non-empty, injected into agentCtx.Metadata["mode"] (exposed as $CTX.MODE
// in prompt templates) and into opts.Mode for framework-level buffer/chat recording.
Mode string
// log is an optional structured logger; when set, Call emits agent-call logs.
log *execLogger
}
@ -194,6 +199,7 @@ func (c *AgentCaller) Call(ctx *robottypes.Context, assistantID string, messages
Search: c.SkipSearch,
},
Connector: c.Connector,
Mode: c.Mode,
}
agentCtx := c.buildAgentContext(ctx, assistantID)
@ -228,7 +234,7 @@ func (c *AgentCaller) Call(ctx *robottypes.Context, assistantID string, messages
}
if c.log != nil {
c.log.logAgentCall(assistantID, result)
c.log.logAgentCall(assistantID, c.Connector, result)
}
return result, nil
@ -276,6 +282,7 @@ func (c *AgentCaller) CallStream(ctx *robottypes.Context, assistantID string, me
Search: c.SkipSearch,
},
Connector: c.Connector,
Mode: c.Mode,
}
// Hook OnMessage to intercept streaming chunks and forward to callback
@ -332,7 +339,7 @@ func (c *AgentCaller) CallStream(ctx *robottypes.Context, assistantID string, me
}
if c.log != nil {
c.log.logAgentCall(assistantID, result)
c.log.logAgentCall(assistantID, c.Connector, result)
}
return result, nil
@ -366,6 +373,7 @@ func (c *AgentCaller) CallStreamRaw(ctx *robottypes.Context, assistantID string,
Search: c.SkipSearch,
},
Connector: c.Connector,
Mode: c.Mode,
}
if onMessage != nil {
@ -400,7 +408,7 @@ func (c *AgentCaller) CallStreamRaw(ctx *robottypes.Context, assistantID string,
}
if c.log != nil {
c.log.logAgentCall(assistantID, result)
c.log.logAgentCall(assistantID, c.Connector, result)
}
return result, nil
@ -448,11 +456,16 @@ func (c *AgentCaller) buildAgentContext(ctx *robottypes.Context, assistantID str
}
agentCtx.Logger = agentcontext.Noop()
if c.Workspace != "" {
if c.Workspace != "" || c.Mode != "" {
if agentCtx.Metadata == nil {
agentCtx.Metadata = map[string]interface{}{}
}
agentCtx.Metadata["workspace_id"] = c.Workspace
if c.Workspace != "" {
agentCtx.Metadata["workspace_id"] = c.Workspace
}
if c.Mode != "" {
agentCtx.Metadata["MODE"] = c.Mode
}
}
kunlog.Trace("[robot-agent] context built: assistantID=%s chatID=%s contextID=%s", assistantID, c.ChatID, agentCtx.ID)

View file

@ -3,14 +3,17 @@ package standard
import (
"encoding/json"
"fmt"
"path"
"strings"
"time"
"github.com/yaoapp/gou/model"
"github.com/yaoapp/gou/process"
kunlog "github.com/yaoapp/kun/log"
robotevents "github.com/yaoapp/yao/agent/robot/events"
robottypes "github.com/yaoapp/yao/agent/robot/types"
"github.com/yaoapp/yao/event"
"github.com/yaoapp/yao/workspace"
)
// RunDelivery executes P4: Delivery phase
@ -35,14 +38,36 @@ func (e *Executor) RunDelivery(ctx *robottypes.Context, exec *robottypes.Executi
}
formatter := NewInputFormatter()
userContent := formatter.FormatDeliveryInput(exec, robot)
// Try workspace-based delivery input (manifest summaries instead of raw output inline)
var userContent string
if robot.Workspace != "" {
wsm := workspace.M()
if wsm != nil {
wsFS, err := wsm.FS(ctx, robot.Workspace)
if err == nil {
execDir := path.Join("robots", robot.MemberID, exec.ID)
data, err := wsFS.ReadFile(path.Join(execDir, "manifest.json"))
if err == nil {
var manifest Manifest
if json.Unmarshal(data, &manifest) == nil {
userContent = formatter.FormatDeliveryInputWithManifest(exec, robot, &manifest, robot.Workspace, execDir, locale)
}
}
}
}
}
// Fallback to legacy full-output inline
if userContent == "" {
userContent = formatter.FormatDeliveryInput(exec, robot)
}
if userContent == "" {
return fmt.Errorf("no content available for delivery generation")
}
caller := NewAgentCaller()
caller.Connector = robot.LanguageModel
caller.Workspace = robot.Workspace
result, err := caller.CallWithMessages(ctx, agentID, userContent)
if err != nil {
@ -98,7 +123,16 @@ func (e *Executor) pushDeliveryEvent(ctx *robottypes.Context, exec *robottypes.E
}
}
_, err := event.Push(ctx.Context, robotevents.Delivery, robotevents.DeliveryPayload{
eventCtx := ctx.Context
if ctx.Auth != nil {
eventCtx = event.WithAuth(eventCtx, &process.AuthorizedInfo{
UserID: ctx.Auth.UserID,
TeamID: ctx.Auth.TeamID,
Subject: ctx.Auth.Subject,
})
}
_, err := event.Push(eventCtx, robotevents.Delivery, robotevents.DeliveryPayload{
ExecutionID: exec.ID,
MemberID: exec.MemberID,
TeamID: exec.TeamID,
@ -403,3 +437,103 @@ func (f *InputFormatter) FormatDeliveryInput(exec *robottypes.Execution, robot *
return sb.String()
}
// FormatDeliveryInputWithManifest formats delivery input using workspace manifest summaries
// instead of inlining full task outputs. This drastically reduces token usage.
func (f *InputFormatter) FormatDeliveryInputWithManifest(
exec *robottypes.Execution,
robot *robottypes.Robot,
manifest *Manifest,
wsID string,
execDir string,
locale string,
) string {
if exec == nil || manifest == nil {
return ""
}
var sb strings.Builder
// Robot Identity (same as legacy)
if robot != nil && robot.Config != nil && robot.Config.Identity != nil {
sb.WriteString("## Robot Identity\n\n")
sb.WriteString(fmt.Sprintf("- **Role**: %s\n", robot.Config.Identity.Role))
if len(robot.Config.Identity.Duties) > 0 {
sb.WriteString("- **Duties**: ")
sb.WriteString(strings.Join(robot.Config.Identity.Duties, ", "))
sb.WriteString("\n")
}
sb.WriteString("\n")
}
sb.WriteString("## Execution Context\n\n")
sb.WriteString(fmt.Sprintf("- **Trigger**: %s\n", exec.TriggerType))
sb.WriteString(fmt.Sprintf("- **Status**: %s\n", exec.Status))
if locale != "" {
sb.WriteString(fmt.Sprintf("- **Language**: %s\n", locale))
}
sb.WriteString(fmt.Sprintf("- **Start Time**: %s\n", exec.StartTime.Format("2006-01-02 15:04:05")))
if exec.EndTime != nil {
duration := exec.EndTime.Sub(exec.StartTime)
sb.WriteString(fmt.Sprintf("- **Duration**: %s\n", duration.String()))
}
sb.WriteString("\n")
sb.WriteString("## Goals\n\n")
sb.WriteString(manifest.Goals)
sb.WriteString("\n\n")
// Results from manifest summaries
sb.WriteString("## Results (P3)\n\n")
successCount := 0
failCount := 0
for _, t := range manifest.Tasks {
if t.Status == "completed" {
successCount++
sb.WriteString(fmt.Sprintf("### ✓ Task: %s\n\n", t.ID))
} else if t.Status == "failed" {
failCount++
sb.WriteString(fmt.Sprintf("### ✗ Task: %s\n\n", t.ID))
} else {
continue
}
sb.WriteString(fmt.Sprintf("- **Description**: %s\n", t.Description))
sb.WriteString(fmt.Sprintf("- **Executor**: %s (%s)\n", t.Executor, t.ExecutorType))
if t.Summary != "" {
sb.WriteString(fmt.Sprintf("- **Summary**: %s\n", t.Summary))
}
if len(t.KeyOutputs) > 0 {
sb.WriteString(fmt.Sprintf("- **Key Outputs**: %s\n", strings.Join(t.KeyOutputs, ", ")))
}
// File references — use existing URI if present, otherwise build from execDir
if len(t.Files) > 0 {
sb.WriteString("- **Artifacts**:\n")
for _, file := range t.Files {
uri := file.URI
if uri == "" {
uri = fmt.Sprintf("workspace://%s/%s/%s/%s", wsID, execDir, t.ID, file.Name)
}
sb.WriteString(fmt.Sprintf(" - [%s](%s)", file.Name, uri))
if file.Desc != "" {
sb.WriteString(fmt.Sprintf(" — %s", file.Desc))
}
sb.WriteString("\n")
}
}
// Full output file reference
outputURI := fmt.Sprintf("workspace://%s/%s/%s.output.md", wsID, execDir, t.ID)
sb.WriteString(fmt.Sprintf("- **Full Output**: [%s.output.md](%s)\n", t.ID, outputURI))
sb.WriteString("\n")
}
sb.WriteString(fmt.Sprintf("### Summary\n\n- **Total Tasks**: %d\n- **Succeeded**: %d\n- **Failed**: %d\n\n",
successCount+failCount, successCount, failCount))
return sb.String()
}

View file

@ -84,7 +84,6 @@ func (e *Executor) RunGoals(ctx *robottypes.Context, exec *robottypes.Execution,
// Call agent
caller := NewAgentCaller()
caller.Connector = robot.LanguageModel
caller.Workspace = robot.Workspace
result, err := caller.CallWithMessages(ctx, agentID, userContent)
if err != nil {

View file

@ -32,7 +32,6 @@ func (e *Executor) CallHostAgent(ctx *robottypes.Context, robot *robottypes.Robo
kunlog.Info("calling Host Agent %s for scenario=%s chatID=%s", agentID, input.Scenario, chatID)
caller := NewConversationCaller(chatID)
caller.Connector = robot.LanguageModel
caller.Workspace = robot.Workspace
result, err := caller.CallWithMessages(ctx, agentID, string(inputJSON))
if err != nil {

View file

@ -183,6 +183,7 @@ func (f *InputFormatter) FormatAvailableResourcesWithLocale(robot *robottypes.Ro
capabilities := i18n.Translate(agentID, locale, ast.Capabilities).(string)
sb.WriteString(fmt.Sprintf(" - **Capabilities**: %s\n", capabilities))
}
}
sb.WriteString("\n")
}

View file

@ -53,7 +53,6 @@ func (e *Executor) RunInspiration(ctx *robottypes.Context, exec *robottypes.Exec
// Call agent
caller := NewAgentCaller()
caller.Connector = robot.LanguageModel
caller.Workspace = robot.Workspace
result, err := caller.CallWithMessages(ctx, agentID, userContent)
if err != nil {

View file

@ -103,9 +103,9 @@ func (l *execLogger) devTaskOverview(tasks []robottypes.Task) {
// P3: Task Input
// ---------------------------------------------------------------------------
func (l *execLogger) logTaskInput(task *robottypes.Task, prompt string) {
func (l *execLogger) logTaskInput(task *robottypes.Task, prompt string, actualConnector string) {
if config.IsDevelopment() {
l.devTaskInput(task, prompt)
l.devTaskInput(task, prompt, actualConnector)
}
kunlog.With(kunlog.F{
"robot_id": l.robotID(),
@ -115,17 +115,23 @@ func (l *execLogger) logTaskInput(task *robottypes.Task, prompt string) {
"executor_id": task.ExecutorID,
"prompt_len": len(prompt),
"language_model": l.connector(),
"connector": actualConnector,
}).Info("Task input: %s [%s]", task.ID, task.ExecutorID)
}
func (l *execLogger) devTaskInput(task *robottypes.Task, prompt string) {
func (l *execLogger) devTaskInput(task *robottypes.Task, prompt string, actualConnector string) {
w := logger.Gray
v := logger.White
r := logger.Reset
connLabel := actualConnector
if connLabel == "" {
connLabel = "agent-default"
}
var sb strings.Builder
sb.WriteString(fmt.Sprintf("%s ▶ Task %s%s%s [%s:%s] Prompt: %d chars%s\n",
w, v, task.ID, w, task.ExecutorType, task.ExecutorID, len(prompt), r))
sb.WriteString(fmt.Sprintf("%s ▶ Task %s%s%s [%s:%s] Connector: %s%s%s Prompt: %d chars%s\n",
w, v, task.ID, w, task.ExecutorType, task.ExecutorID, v, connLabel, w, len(prompt), r))
logger.Raw(sb.String())
}
@ -190,18 +196,19 @@ func (l *execLogger) devTaskOutput(task *robottypes.Task, result *robottypes.Tas
// Agent Call
// ---------------------------------------------------------------------------
func (l *execLogger) logAgentCall(agentID string, result *CallResult) {
func (l *execLogger) logAgentCall(agentID string, connector string, result *CallResult) {
if result == nil {
return
}
if config.IsDevelopment() {
l.devAgentCall(agentID, result)
l.devAgentCall(agentID, connector, result)
}
fields := kunlog.F{
"robot_id": l.robotID(),
"execution_id": l.execID,
"agent_id": agentID,
"connector": connector,
"content_len": len(result.Content),
"language_model": l.connector(),
}
@ -209,23 +216,28 @@ func (l *execLogger) logAgentCall(agentID string, result *CallResult) {
fields["next_type"] = fmt.Sprintf("%T", result.Next)
fields["next_len"] = outputLen(result.Next)
}
kunlog.With(fields).Info("Agent call: %s (content=%d, next=%T)", agentID, len(result.Content), result.Next)
kunlog.With(fields).Info("Agent call: %s (connector=%s, content=%d, next=%T)", agentID, connector, len(result.Content), result.Next)
}
func (l *execLogger) devAgentCall(agentID string, result *CallResult) {
func (l *execLogger) devAgentCall(agentID string, connector string, result *CallResult) {
w := logger.Gray
v := logger.White
c := logger.Cyan
r := logger.Reset
displayConn := connector
if displayConn == "" {
displayConn = "agent-default"
}
nextInfo := "—"
if result.Next != nil {
nextInfo = fmt.Sprintf("%T (len=%d)", result.Next, outputLen(result.Next))
}
var sb strings.Builder
sb.WriteString(fmt.Sprintf("%s → Agent(%s%s%s) Content: %s%d%s chars Next: %s%s%s\n",
c, v, agentID, c, v, len(result.Content), w, v, nextInfo, r))
sb.WriteString(fmt.Sprintf("%s → Agent(%s%s%s) Connector: %s%s%s Content: %s%d%s chars Next: %s%s%s\n",
c, v, agentID, c, v, displayConn, c, v, len(result.Content), w, v, nextInfo, r))
logger.Raw(sb.String())
}

View file

@ -0,0 +1,60 @@
workspace: |
## Workspace
Execution directory: {{.ExecDir}}
### Available Files
{{range .Files}}
- {{.Path}} — {{.Desc}}
{{- end}}
### Your Task Directory
Write any output files to: {{.TaskDir}}
context: |
## Execution Context
Goals: {{.Goals}}
{{- if .Locale}}
Language: {{.Locale}}
{{- end}}
Mode: automated-pipeline
{{- if .FailureWarning}}
⚠ {{.FailureWarning}}
{{- end}}
{{if .CompletedTasks}}
### Completed Tasks
{{range .CompletedTasks}}
{{.Seq}}. [{{.ID}}] {{.Description}} ({{.ExecutorType}}: {{.Executor}}) ✓
Summary: {{.Summary}}
{{- if .KeyOutputs}}
Key outputs: {{.KeyOutputs}}
{{- end}}
{{- if .HasFiles}}
Files: {{.Files}}
{{- end}}
{{end}}
{{- end}}
{{- if .FailedTasks}}
### Failed Tasks
{{range .FailedTasks}}
{{.Seq}}. [{{.ID}}] {{.Description}} ✗
Error: {{.Error}}
{{end}}
{{- end}}
### Current Task
{{.CurrentOrder}}. [{{.CurrentID}}] {{.CurrentDesc}} ({{.CurrentType}}: {{.CurrentExec}})
instructions: |
## Task Instructions
{{.TaskInstructions}}
{{- if .ExpectedOutput}}
### Expected Output
{{.ExpectedOutput}}
{{- end}}

View file

@ -2,8 +2,10 @@ package standard
import (
"fmt"
"path"
"time"
kunlog "github.com/yaoapp/kun/log"
robotevents "github.com/yaoapp/yao/agent/robot/events"
robottypes "github.com/yaoapp/yao/agent/robot/types"
"github.com/yaoapp/yao/event"
@ -53,9 +55,6 @@ func (e *Executor) RunExecution(ctx *robottypes.Context, exec *robottypes.Execut
config = DefaultRunConfig()
}
// Determine locale for UI messages
locale := getEffectiveLocale(robot, exec.Input)
// Determine start index and restore results from resume context
startIndex := 0
if exec.ResumeContext != nil {
@ -67,6 +66,26 @@ func (e *Executor) RunExecution(ctx *robottypes.Context, exec *robottypes.Execut
// Create task runner with execution-level chatID (§8.4)
runner := NewRunner(ctx, robot, config, exec.ChatID, exec.ID)
if ctx.Locale != "" {
runner.locale = ctx.Locale
} else {
runner.locale = getEffectiveLocale(robot, exec.Input)
}
// Initialize workspace for file-based context
wsFS, err := ensureRobotWorkspace(ctx, robot)
if err != nil {
kunlog.Warn("[robot-run] workspace unavailable, falling back to in-memory context: %v", err)
} else {
execDir := path.Join("robots", robot.MemberID, exec.ID)
if mkErr := wsFS.MkdirAll(execDir, 0755); mkErr != nil {
kunlog.Warn("[robot-run] mkdir %s: %v", execDir, mkErr)
} else {
runner.wsFS = wsFS
runner.execDir = execDir
runner.initManifest(exec)
}
}
// Execute tasks sequentially from startIndex
for i := startIndex; i < len(exec.Tasks); i++ {
@ -80,7 +99,7 @@ func (e *Executor) RunExecution(ctx *robottypes.Context, exec *robottypes.Execut
}
// Update UI field with current task description (i18n)
taskName := formatTaskProgressName(task, i, len(exec.Tasks), locale)
taskName := formatTaskProgressName(task, i, len(exec.Tasks), runner.locale)
e.updateUIFields(ctx, exec, "", taskName)
// Mark task as running
@ -127,7 +146,10 @@ func (e *Executor) RunExecution(ctx *robottypes.Context, exec *robottypes.Execut
})
}
// Store result
// Write task files to workspace (non-blocking, errors logged)
runner.writeTaskOutput(task, result, runner.lastPromptSnapshot)
// Store result (in-memory, for persistence + resume)
exec.Results = append(exec.Results, *result)
// Persist completed/failed state to database

View file

@ -10,16 +10,24 @@ import (
"github.com/yaoapp/gou/process"
kunlog "github.com/yaoapp/kun/log"
agentcontext "github.com/yaoapp/yao/agent/context"
"github.com/yaoapp/yao/agent/llm"
robottypes "github.com/yaoapp/yao/agent/robot/types"
taiworkspace "github.com/yaoapp/yao/tai/workspace"
)
// Runner handles execution of individual tasks
type Runner struct {
ctx *robottypes.Context
robot *robottypes.Robot
config *RunConfig
chatID string // execution-level chatID for conversation persistence (§8.4)
log *execLogger
ctx *robottypes.Context
robot *robottypes.Robot
config *RunConfig
chatID string // execution-level chatID for conversation persistence (§8.4)
log *execLogger
wsFS taiworkspace.FS // workspace file system (nil if unavailable)
execDir string // workspace-relative execution directory (e.g. "robots/<id>/<exec_id>")
lastPromptSnapshot string // captured prompt text for workspace .input.md
currentTaskIndex int // current task index for workspace prompt building
currentExec *robottypes.Execution
locale string // effective locale for this execution (e.g. "zh", "en")
}
// NewRunner creates a new task runner
@ -47,12 +55,15 @@ type RunnerContext struct {
// BuildTaskContext builds context for a task including previous results
func (r *Runner) BuildTaskContext(exec *robottypes.Execution, taskIndex int) *RunnerContext {
r.currentTaskIndex = taskIndex
r.currentExec = exec
ctx := &RunnerContext{
Goals: exec.Goals,
SystemPrompt: r.robot.SystemPrompt,
}
// Include results from previous tasks (with bounds check)
// Include results from previous tasks (with bounds check) — kept for fallback
if taskIndex > 0 && len(exec.Results) > 0 {
endIndex := taskIndex
if endIndex > len(exec.Results) {
@ -132,26 +143,56 @@ func (r *Runner) executeNonAssistantTask(task *robottypes.Task, taskCtx *RunnerC
// Returns the extracted output, the raw CallResult (for need_input detection), and any error.
func (r *Runner) executeAssistantTask(task *robottypes.Task, taskCtx *RunnerContext) (interface{}, *CallResult, error) {
caller := NewAgentCaller()
caller.Mode = "task"
caller.log = r.log
caller.Connector = r.robot.LanguageModel
if r.robot.LanguageModel != "" {
if _, _, err := llm.ResolveConnector(r.robot.LanguageModel, nil); err == nil {
caller.Connector = r.robot.LanguageModel
} else {
kunlog.Warn("[robot-runner] connector %s invalid, using agent default: %v",
r.robot.LanguageModel, err)
}
}
caller.Workspace = r.robot.Workspace
caller.ChatID = r.chatID
messages := r.BuildAssistantMessages(task, taskCtx)
input := r.FormatMessagesAsText(messages)
var input string
workspacePromptUsed := false
// Use workspace-based prompt when available
if r.wsFS != nil {
manifest, err := r.readManifest()
if err == nil {
taskInstructions := r.FormatMessagesAsText(task.Messages)
input = r.buildWorkspacePrompt(manifest, r.currentTaskIndex, task, taskInstructions)
workspacePromptUsed = (input != "")
}
}
// Fallback to legacy in-memory context
if input == "" {
messages := r.BuildAssistantMessages(task, taskCtx)
input = r.FormatMessagesAsText(messages)
}
if strings.TrimSpace(input) == "" {
return nil, nil, fmt.Errorf("no valid input messages for task %s", task.ID)
}
if taskCtx.SystemPrompt != "" {
// Only inject robot system prompt for legacy (non-workspace) path.
// In workspace mode the agent's own prompts.yml defines its role;
// injecting the robot's dispatcher prompt would confuse the executor.
if taskCtx.SystemPrompt != "" && !workspacePromptUsed {
input = "## Context\n\n" + taskCtx.SystemPrompt + "\n\n## Task\n\n" + input
}
// Capture prompt snapshot for workspace .input.md
r.lastPromptSnapshot = input
kunlog.Trace("[robot-runner] executeAssistantTask: task=%s assistant=%s promptLen=%d prevResults=%d",
task.ID, task.ExecutorID, len(input), len(taskCtx.PreviousResults))
r.log.logTaskInput(task, input)
r.log.logTaskInput(task, input, caller.Connector)
result, err := caller.CallWithMessages(r.ctx, task.ExecutorID, input)
if err != nil {

View file

@ -2,7 +2,9 @@ package standard
import (
"fmt"
"strings"
kunlog "github.com/yaoapp/kun/log"
agentcontext "github.com/yaoapp/yao/agent/context"
robottypes "github.com/yaoapp/yao/agent/robot/types"
)
@ -54,7 +56,6 @@ func (e *Executor) RunTasks(ctx *robottypes.Context, exec *robottypes.Execution,
// Call agent
caller := NewAgentCaller()
caller.log = newExecLogger(robot, exec.ID)
caller.Connector = robot.LanguageModel
caller.Workspace = robot.Workspace
result, err := caller.CallWithMessages(ctx, agentID, userContent)
if err != nil {
@ -80,6 +81,9 @@ func (e *Executor) RunTasks(ctx *robottypes.Context, exec *robottypes.Execution,
return fmt.Errorf("tasks agent (%s) returned invalid task structure: %w", agentID, err)
}
// Normalize executor IDs and types against available resources
NormalizeTaskExecutors(tasks, robot)
// Validate tasks
if err := ValidateTasks(tasks); err != nil {
return fmt.Errorf("tasks validation failed: %w", err)
@ -395,3 +399,81 @@ func ValidateMCPTask(task *robottypes.Task) error {
}
return nil
}
// NormalizeTaskExecutors fixes LLM-generated executor_id and executor_type
// against the robot's actual resource lists. It handles two common LLM errors:
// 1. Partial executor_id (e.g. "report-writer" instead of "yao.report-writer")
// 2. Wrong executor_type (e.g. classifying an assistant as "mcp")
func NormalizeTaskExecutors(tasks []robottypes.Task, robot *robottypes.Robot) {
if robot == nil || robot.Config == nil || robot.Config.Resources == nil {
return
}
agentSet := make(map[string]bool, len(robot.Config.Resources.Agents))
for _, id := range robot.Config.Resources.Agents {
agentSet[id] = true
}
mcpSet := make(map[string]bool, len(robot.Config.Resources.MCP))
for _, m := range robot.Config.Resources.MCP {
mcpSet[m.ID] = true
}
for i := range tasks {
task := &tasks[i]
origID := task.ExecutorID
origType := task.ExecutorType
// Skip process tasks — they are not in resource lists
if task.ExecutorType == robottypes.ExecutorProcess {
continue
}
// Step 1: Try exact match first
if agentSet[task.ExecutorID] {
task.ExecutorType = robottypes.ExecutorAssistant
if origType != task.ExecutorType {
kunlog.Trace("[normalize] task %s: executor_type %s -> %s (crosscheck)", task.ID, origType, task.ExecutorType)
}
continue
}
if mcpSet[task.ExecutorID] {
task.ExecutorType = robottypes.ExecutorMCP
if origType != task.ExecutorType {
kunlog.Trace("[normalize] task %s: executor_type %s -> %s (crosscheck)", task.ID, origType, task.ExecutorType)
}
continue
}
// Step 2: Suffix match — LLM may omit namespace prefix
if match := suffixMatch(task.ExecutorID, robot.Config.Resources.Agents); match != "" {
task.ExecutorID = match
task.ExecutorType = robottypes.ExecutorAssistant
kunlog.Trace("[normalize] task %s: executor_id %s -> %s (suffix match)", task.ID, origID, match)
continue
}
mcpIDs := make([]string, 0, len(robot.Config.Resources.MCP))
for _, m := range robot.Config.Resources.MCP {
mcpIDs = append(mcpIDs, m.ID)
}
if match := suffixMatch(task.ExecutorID, mcpIDs); match != "" {
task.ExecutorID = match
task.ExecutorType = robottypes.ExecutorMCP
kunlog.Trace("[normalize] task %s: executor_id %s -> %s (suffix match)", task.ID, origID, match)
continue
}
}
}
// suffixMatch finds the first entry in candidates that ends with "."+partial
// (or equals partial exactly, which is already handled by the caller).
func suffixMatch(partial string, candidates []string) string {
suffix := "." + partial
for _, c := range candidates {
if strings.HasSuffix(c, suffix) {
return c
}
}
return ""
}

View file

@ -433,7 +433,6 @@ func (v *Validator) validateSemantic(task *robottypes.Task, output interface{})
// Call validation agent
caller := NewAgentCaller()
caller.Connector = v.robot.LanguageModel
caller.Workspace = v.robot.Workspace
result, err := caller.CallWithMessages(v.ctx, validationAgentID, validationPrompt)
if err != nil {
@ -641,7 +640,6 @@ func (av *robotAgentValidator) Validate(agentID string, output, input, criteria
// Call agent
caller := NewAgentCaller()
caller.Connector = av.v.robot.LanguageModel
caller.Workspace = av.v.robot.Workspace
callResult, err := caller.CallWithMessages(av.v.ctx, agentID, string(inputJSON))
if err != nil {

View file

@ -0,0 +1,850 @@
package standard
import (
"bytes"
_ "embed"
"encoding/json"
"fmt"
"path"
"path/filepath"
"regexp"
"strings"
"text/template"
kunlog "github.com/yaoapp/kun/log"
agentcontext "github.com/yaoapp/yao/agent/context"
"github.com/yaoapp/yao/agent/llm"
robottypes "github.com/yaoapp/yao/agent/robot/types"
taiworkspace "github.com/yaoapp/yao/tai/workspace"
"github.com/yaoapp/yao/workspace"
"gopkg.in/yaml.v3"
)
//go:embed prompts/workspace.yml
var workspacePromptYAML []byte
type workspacePrompts struct {
Workspace string `yaml:"workspace"`
Context string `yaml:"context"`
Instructions string `yaml:"instructions"`
}
var wsPromptTpls struct {
Workspace *template.Template
Context *template.Template
Instructions *template.Template
}
func init() {
var p workspacePrompts
if err := yaml.Unmarshal(workspacePromptYAML, &p); err != nil {
return
}
wsPromptTpls.Workspace, _ = template.New("ws").Parse(p.Workspace)
wsPromptTpls.Context, _ = template.New("ctx").Parse(p.Context)
wsPromptTpls.Instructions, _ = template.New("inst").Parse(p.Instructions)
}
// Manifest is the shared context hub for an execution, written as manifest.json.
type Manifest struct {
ExecID string `json:"exec_id"`
RobotID string `json:"robot_id"`
Goals string `json:"goals"`
Tasks []ManifestTask `json:"tasks"`
}
// ManifestTask represents a single task entry in the manifest.
type ManifestTask struct {
ID string `json:"id"`
Order int `json:"order"`
Description string `json:"description"`
Executor string `json:"executor"`
ExecutorType string `json:"executor_type"`
Status string `json:"status"`
Summary string `json:"summary,omitempty"`
KeyOutputs []string `json:"key_outputs,omitempty"`
Files []ManifestFile `json:"files,omitempty"`
Error string `json:"error,omitempty"`
}
// ManifestFile represents a produced artifact.
type ManifestFile struct {
Name string `json:"name"`
Type string `json:"type,omitempty"`
Desc string `json:"desc,omitempty"`
URI string `json:"uri,omitempty"`
}
// ensureRobotWorkspace guarantees a workspace FS exists for the robot.
// If robot.Workspace is empty, it derives a deterministic ID and auto-creates.
// It also writes back robot.Workspace so subsequent callers get the correct ID.
func ensureRobotWorkspace(ctx *robottypes.Context, robot *robottypes.Robot) (taiworkspace.FS, error) {
wsm := workspace.M()
if wsm == nil {
return nil, fmt.Errorf("workspace manager not available")
}
wsID := robot.Workspace
if wsID == "" {
nodes := wsm.Nodes()
nodeID := ""
for _, n := range nodes {
if n.Online {
nodeID = n.Name
break
}
}
if nodeID == "" {
return nil, fmt.Errorf("no available node for workspace")
}
wsID = workspace.DefaultWorkspaceID(robot.TeamID, nodeID)
if _, err := wsm.Get(ctx, wsID); err != nil {
if _, err := wsm.Create(ctx, workspace.CreateOptions{
ID: wsID,
Name: "Robot Workspace",
Owner: robot.TeamID,
Node: nodeID,
}); err != nil {
return nil, fmt.Errorf("workspace create failed: %w", err)
}
}
robot.Workspace = wsID
}
return wsm.FS(ctx, wsID)
}
// initManifest creates the initial manifest.json with goals and pending task list.
// Uses slice index (not P2's order field) so manifests always have sequential numbering.
func (r *Runner) initManifest(exec *robottypes.Execution) {
if r.wsFS == nil {
return
}
goalsContent := ""
if exec.Goals != nil {
goalsContent = exec.Goals.Content
}
m := &Manifest{
ExecID: exec.ID,
RobotID: r.robot.MemberID,
Goals: goalsContent,
Tasks: make([]ManifestTask, 0, len(exec.Tasks)),
}
for i, t := range exec.Tasks {
m.Tasks = append(m.Tasks, ManifestTask{
ID: t.ID,
Order: i,
Description: t.Description,
Executor: t.ExecutorID,
ExecutorType: string(t.ExecutorType),
Status: string(t.Status),
})
}
r.writeManifest(m)
}
// writeManifest serializes and writes manifest.json.
func (r *Runner) writeManifest(m *Manifest) {
data, err := json.MarshalIndent(m, "", " ")
if err != nil {
kunlog.Warn("[robot-workspace] marshal manifest: %v", err)
return
}
p := path.Join(r.execDir, "manifest.json")
if err := r.wsFS.WriteFile(p, data, 0644); err != nil {
kunlog.Warn("[robot-workspace] write manifest: %v", err)
}
}
// readManifest reads and parses manifest.json from workspace.
func (r *Runner) readManifest() (*Manifest, error) {
if r.wsFS == nil {
return nil, fmt.Errorf("wsFS not available")
}
data, err := r.wsFS.ReadFile(path.Join(r.execDir, "manifest.json"))
if err != nil {
return nil, err
}
var m Manifest
if err := json.Unmarshal(data, &m); err != nil {
return nil, err
}
return &m, nil
}
// writeTaskOutput writes the three task files and updates manifest after task completion.
func (r *Runner) writeTaskOutput(task *robottypes.Task, result *robottypes.TaskResult, promptSnapshot string) {
if r.wsFS == nil {
return
}
taskID := task.ID
// Write task-NNN.input.md (prompt snapshot for debug)
if promptSnapshot != "" {
inputPath := path.Join(r.execDir, taskID+".input.md")
if err := r.wsFS.WriteFile(inputPath, []byte(promptSnapshot), 0644); err != nil {
kunlog.Warn("[robot-workspace] write %s.input.md: %v", taskID, err)
}
}
// Write task-NNN.output.md (full output)
outputText := formatOutputAsText(result.Output)
outputPath := path.Join(r.execDir, taskID+".output.md")
if err := r.wsFS.WriteFile(outputPath, []byte(outputText), 0644); err != nil {
kunlog.Warn("[robot-workspace] write %s.output.md: %v", taskID, err)
}
// Write task-NNN.json (metadata)
meta := map[string]interface{}{
"id": taskID,
"executor": task.ExecutorID,
"executor_type": string(task.ExecutorType),
"status": string(task.Status),
"duration_ms": result.Duration,
"success": result.Success,
}
if result.Error != "" {
meta["error"] = result.Error
}
metaJSON, _ := json.MarshalIndent(meta, "", " ")
metaPath := path.Join(r.execDir, taskID+".json")
if err := r.wsFS.WriteFile(metaPath, metaJSON, 0644); err != nil {
kunlog.Warn("[robot-workspace] write %s.json: %v", taskID, err)
}
r.updateManifestForTask(task, result)
}
// updateManifestForTask reads manifest, updates the matching task entry, and writes back.
func (r *Runner) updateManifestForTask(task *robottypes.Task, result *robottypes.TaskResult) {
m, err := r.readManifest()
if err != nil {
kunlog.Warn("[robot-workspace] read manifest for update: %v", err)
return
}
// Scan files first so LLM summary can reference them
files := r.scanTaskArtifacts(task.ID)
outputURIs := r.extractAndVerifyFiles(result.Output)
files = mergeManifestFiles(files, outputURIs)
for i := range m.Tasks {
if m.Tasks[i].ID == task.ID {
if result.Success {
m.Tasks[i].Status = "completed"
summary := r.llmSummarize(task, result.Output, files)
if summary == "" {
summary = generateSummary(result.Output)
}
m.Tasks[i].Summary = summary
m.Tasks[i].KeyOutputs = extractKeyOutputs(result.Output)
} else {
m.Tasks[i].Status = "failed"
m.Tasks[i].Error = result.Error
if result.Error != "" {
m.Tasks[i].Summary = "Failed: " + result.Error
}
}
m.Tasks[i].Files = files
break
}
}
r.writeManifest(m)
}
// scanTaskArtifacts scans the task-id/ directory for produced files.
func (r *Runner) scanTaskArtifacts(taskID string) []ManifestFile {
if r.wsFS == nil {
return nil
}
dirPath := path.Join(r.execDir, taskID)
entries, err := r.wsFS.ReadDir(dirPath)
if err != nil {
return nil
}
var files []ManifestFile
for _, e := range entries {
if e.IsDir() {
continue
}
files = append(files, ManifestFile{
Name: e.Name(),
Type: mimeFromExt(filepath.Ext(e.Name())),
})
}
return files
}
// llmSummarize uses a lightweight LLM to generate a concise task summary
// from the full task context (description, output, produced files).
// Returns empty string on any failure, allowing caller to fall back to static extraction.
func (r *Runner) llmSummarize(task *robottypes.Task, output interface{}, files []ManifestFile) string {
text := flattenOutput(output)
if text == "" {
return ""
}
conn, _, err := llm.ResolveConnector("use::light", r.ctx.Auth)
if err != nil {
kunlog.Warn("[robot-workspace] llmSummarize resolve connector: %v", err)
return ""
}
opts := llm.BuildCompletionOptions(conn, nil)
instance, err := llm.New(conn, opts)
if err != nil {
kunlog.Warn("[robot-workspace] llmSummarize create LLM: %v", err)
return ""
}
var sb strings.Builder
sb.WriteString("Task: " + task.Description + "\n\n")
if len(files) > 0 {
sb.WriteString("Produced files:\n")
for _, f := range files {
sb.WriteString("- " + f.Name + " (" + f.Type + ")\n")
}
sb.WriteString("\n")
}
sb.WriteString("Output:\n")
outputText := text
if len(outputText) > 4000 {
outputText = outputText[:4000]
}
sb.WriteString(outputText)
messages := []agentcontext.Message{
{Role: agentcontext.RoleSystem, Content: "Summarize the task execution result in 1-2 concise sentences. " +
"Focus on what was actually produced or accomplished, not the process. " +
"If files were produced, mention them. Reply in the same language as the task description."},
{Role: agentcontext.RoleUser, Content: sb.String()},
}
agentCtx := agentcontext.New(r.ctx.Context, r.ctx.Auth, "")
defer agentCtx.Release()
resp, err := instance.Post(agentCtx, messages, opts)
if err != nil {
kunlog.Warn("[robot-workspace] llmSummarize Post: %v", err)
return ""
}
return extractLLMContent(resp)
}
// extractLLMContent extracts the text content from a CompletionResponse.
func extractLLMContent(resp *agentcontext.CompletionResponse) string {
if resp == nil {
return ""
}
if s, ok := resp.Content.(string); ok {
return strings.TrimSpace(s)
}
return ""
}
// workspaceURIRegex matches workspace://wsID/path patterns in markdown links and plain text.
// Excludes trailing backticks, quotes, and brackets that are markdown formatting artifacts.
var workspaceURIRegex = regexp.MustCompile("workspace://([^/\\s)]+)/([^\\s)\\]`\"']+)")
// extractAndVerifyFiles extracts workspace:// URIs from output and verifies each
// file exists via wsFS.Stat, eliminating false positives from regex artifacts.
func (r *Runner) extractAndVerifyFiles(output interface{}) []ManifestFile {
text := flattenOutput(output)
if text == "" || r.wsFS == nil {
return nil
}
wsID, err := r.wsFS.GetID()
if err != nil {
return nil
}
prefix := "workspace://" + wsID + "/"
matches := workspaceURIRegex.FindAllStringSubmatch(text, -1)
seen := make(map[string]bool)
var files []ManifestFile
for _, m := range matches {
uri := "workspace://" + m[1] + "/" + m[2]
if seen[uri] {
continue
}
seen[uri] = true
if !strings.HasPrefix(uri, prefix) {
continue
}
relPath := strings.TrimPrefix(uri, prefix)
info, err := r.wsFS.Stat(relPath)
if err != nil || info.IsDir() {
continue
}
name := filepath.Base(relPath)
files = append(files, ManifestFile{
Name: name,
Type: mimeFromExt(filepath.Ext(name)),
URI: uri,
})
}
return files
}
// mergeManifestFiles deduplicates files from scanTaskArtifacts and extractWorkspaceURIs.
// If a URI-bearing entry has the same Name as a scan entry, the URI is merged onto the
// existing entry instead of creating a duplicate.
func mergeManifestFiles(scanned []ManifestFile, fromURIs []ManifestFile) []ManifestFile {
if len(fromURIs) == 0 {
return scanned
}
nameIndex := make(map[string]int, len(scanned))
for i, f := range scanned {
nameIndex[f.Name] = i
}
for _, uf := range fromURIs {
if idx, exists := nameIndex[uf.Name]; exists {
if scanned[idx].URI == "" {
scanned[idx].URI = uf.URI
}
} else {
nameIndex[uf.Name] = len(scanned)
scanned = append(scanned, uf)
}
}
return scanned
}
// --- Summary & key_outputs extraction ---
// llmPrefixPatterns are common LLM filler prefixes that carry no information.
var llmPrefixPatterns = []string{
"It seems ", "It appears ", "Here is ", "Here's ",
"Based on ", "I encountered ", "I wasn't able ",
"I'm currently unable ", "Let me ", "I recommend ",
}
// generateSummary produces a concise summary of the task's actual output/result.
// It prioritizes conclusion/summary sections over the beginning of the output,
// because agent responses typically start with planning/thinking text.
func generateSummary(output interface{}) string {
text := flattenOutput(output)
if text == "" {
return ""
}
const maxLen = 200
// Try to find an explicit summary/conclusion section
for _, heading := range []string{"## Summary", "## Conclusion", "## Result", "## 总结", "## 结论", "## 结果"} {
if idx := strings.Index(text, heading); idx >= 0 {
section := strings.TrimSpace(text[idx+len(heading):])
section = strings.TrimPrefix(section, "\n")
if nextH := strings.Index(section, "\n## "); nextH > 0 {
section = section[:nextH]
}
section = strings.TrimSpace(section)
if section != "" {
if len(section) > maxLen {
return section[:maxLen] + "..."
}
return section
}
}
}
// No explicit section — use last substantive paragraph as it's
// more likely to contain the actual result than the beginning
paragraphs := strings.Split(text, "\n\n")
for i := len(paragraphs) - 1; i >= 0; i-- {
p := strings.TrimSpace(paragraphs[i])
if p == "" || len(p) < 10 {
continue
}
isFiller := false
for _, prefix := range llmPrefixPatterns {
if strings.HasPrefix(p, prefix) {
isFiller = true
break
}
}
if isFiller {
continue
}
if len(p) > maxLen {
return p[:maxLen] + "..."
}
return p
}
// Fallback: skip filler and take from the beginning
text = skipFillerPrefixes(text)
if len(text) > maxLen {
return text[:maxLen] + "..."
}
return text
}
// skipFillerPrefixes skips past common LLM opening phrases to find substantive content.
func skipFillerPrefixes(text string) string {
lines := strings.SplitN(text, "\n", 20)
for i, line := range lines {
trimmed := strings.TrimSpace(line)
if trimmed == "" {
continue
}
isFiller := false
for _, prefix := range llmPrefixPatterns {
if strings.HasPrefix(trimmed, prefix) {
isFiller = true
break
}
}
if !isFiller {
return strings.Join(lines[i:], "\n")
}
}
return text
}
// boldPatternRegex matches **bold text** in markdown
var boldPatternRegex = regexp.MustCompile(`\*\*([^*]+)\*\*`)
// extractKeyOutputs tries to extract structured key_outputs from the output.
func extractKeyOutputs(output interface{}) []string {
if output == nil {
return nil
}
// If output is a map with key_outputs/outputs/results, extract directly
if m, ok := output.(map[string]interface{}); ok {
for _, key := range []string{"key_outputs", "outputs", "results"} {
if arr, ok := m[key].([]interface{}); ok {
result := make([]string, 0, len(arr))
for _, v := range arr {
if s, ok := v.(string); ok {
result = append(result, s)
}
}
if len(result) > 0 {
return capSlice(result, 5)
}
}
}
}
text := flattenOutput(output)
// Try ## headings
if strings.Contains(text, "\n## ") {
var headings []string
for _, line := range strings.Split(text, "\n") {
if strings.HasPrefix(line, "## ") {
headings = append(headings, strings.TrimPrefix(line, "## "))
}
}
if len(headings) > 0 {
return capSlice(headings, 5)
}
}
// Try **bold** items from numbered lists or bullet lists
// e.g. "1. **Model-Driven Architecture:**" or "- **Low-Code Engine**"
var boldItems []string
for _, line := range strings.Split(text, "\n") {
trimmed := strings.TrimSpace(line)
if !strings.Contains(trimmed, "**") {
continue
}
// Only extract from list-like lines
if !(strings.HasPrefix(trimmed, "- ") || strings.HasPrefix(trimmed, "* ") ||
(len(trimmed) > 2 && trimmed[0] >= '0' && trimmed[0] <= '9' && trimmed[1] == '.')) {
continue
}
matches := boldPatternRegex.FindStringSubmatch(trimmed)
if len(matches) >= 2 {
item := strings.TrimRight(matches[1], ":")
if len(item) > 0 && len(item) < 80 {
boldItems = append(boldItems, item)
}
}
}
if len(boldItems) > 0 {
return capSlice(boldItems, 5)
}
return nil
}
func capSlice(s []string, max int) []string {
if len(s) > max {
return s[:max]
}
return s
}
// flattenOutput converts any output value to a plain text string.
func flattenOutput(output interface{}) string {
if output == nil {
return ""
}
switch v := output.(type) {
case string:
return v
case map[string]interface{}:
if text, ok := v["text"].(string); ok {
return text
}
if content, ok := v["content"].(string); ok {
return content
}
b, _ := json.Marshal(v)
return string(b)
default:
b, err := json.Marshal(v)
if err != nil {
return fmt.Sprintf("%v", v)
}
return string(b)
}
}
// formatOutputAsText converts task output to markdown text for .output.md files.
func formatOutputAsText(output interface{}) string {
if output == nil {
return ""
}
switch v := output.(type) {
case string:
return v
default:
b, err := json.MarshalIndent(v, "", " ")
if err != nil {
return fmt.Sprintf("%v", v)
}
return string(b)
}
}
// --- Prompt template rendering ---
// wsTemplateData holds template variables for the workspace section.
type wsTemplateData struct {
ExecDir string
Files []wsFileEntry
TaskDir string
}
type wsFileEntry struct {
Path string
Desc string
}
// ctxTemplateData holds template variables for the execution context section.
type ctxTemplateData struct {
Goals string
Locale string
CompletedTasks []ctxTaskEntry
FailedTasks []ctxFailedEntry
CurrentOrder int
CurrentID string
CurrentDesc string
CurrentType string
CurrentExec string
FailureWarning string
}
type ctxTaskEntry struct {
Seq int
ID string
Description string
ExecutorType string
Executor string
Summary string
KeyOutputs string
Files string
HasFiles bool
}
type ctxFailedEntry struct {
Seq int
ID string
Description string
Error string
}
// instTemplateData holds template variables for the instructions section.
type instTemplateData struct {
TaskInstructions string
ExpectedOutput string
}
// buildWorkspacePrompt renders the full prompt for a task from manifest + template.
func (r *Runner) buildWorkspacePrompt(manifest *Manifest, taskIndex int, task *robottypes.Task, taskInstructions string) string {
if manifest == nil || taskIndex >= len(manifest.Tasks) {
return taskInstructions
}
var sb strings.Builder
// Section 1: Workspace
if wsPromptTpls.Workspace != nil {
wd := wsTemplateData{
ExecDir: r.execDir + "/",
TaskDir: r.execDir + "/" + manifest.Tasks[taskIndex].ID + "/",
}
wd.Files = append(wd.Files, wsFileEntry{
Path: "manifest.json",
Desc: "Execution context: goals, completed task summaries, progress",
})
for i := 0; i < taskIndex; i++ {
t := manifest.Tasks[i]
if t.Status == "completed" {
wd.Files = append(wd.Files, wsFileEntry{
Path: t.ID + ".output.md",
Desc: "Full output: " + t.Description,
})
for _, f := range t.Files {
desc := "Artifact: " + f.Name
if f.URI != "" {
desc = "Artifact: " + f.Name + " (" + f.URI + ")"
}
filePath := t.ID + "/" + f.Name
if f.URI != "" {
filePath = f.URI
}
wd.Files = append(wd.Files, wsFileEntry{
Path: filePath,
Desc: desc,
})
}
}
}
var buf bytes.Buffer
if err := wsPromptTpls.Workspace.Execute(&buf, wd); err == nil {
sb.WriteString(buf.String())
sb.WriteString("\n\n")
}
}
// Section 2: Execution Context
if wsPromptTpls.Context != nil {
ct := manifest.Tasks[taskIndex]
cd := ctxTemplateData{
Goals: manifest.Goals,
Locale: r.locale,
CurrentID: ct.ID,
CurrentOrder: taskIndex + 1,
CurrentDesc: ct.Description,
CurrentType: ct.ExecutorType,
CurrentExec: ct.Executor,
}
seq := 0
failedCount := 0
for i := 0; i < taskIndex; i++ {
t := manifest.Tasks[i]
seq++
if t.Status == "completed" {
entry := ctxTaskEntry{
Seq: seq,
ID: t.ID,
Description: t.Description,
ExecutorType: t.ExecutorType,
Executor: t.Executor,
Summary: t.Summary,
KeyOutputs: strings.Join(t.KeyOutputs, ", "),
}
if len(t.Files) > 0 {
entry.HasFiles = true
names := make([]string, 0, len(t.Files))
for _, f := range t.Files {
names = append(names, f.Name)
}
entry.Files = strings.Join(names, ", ")
}
cd.CompletedTasks = append(cd.CompletedTasks, entry)
} else if t.Status == "failed" {
failedCount++
errMsg := t.Error
if errMsg == "" {
errMsg = t.Summary
}
cd.FailedTasks = append(cd.FailedTasks, ctxFailedEntry{
Seq: seq,
ID: t.ID,
Description: t.Description,
Error: errMsg,
})
}
}
// P8: failure cascade warning
if failedCount > 0 && len(cd.CompletedTasks) == 0 {
cd.FailureWarning = "WARNING: All previous tasks failed. You may lack necessary input data. Do your best with available information or report the limitation."
} else if failedCount > 0 {
cd.FailureWarning = fmt.Sprintf("Note: %d previous task(s) failed. Some expected input may be missing.", failedCount)
}
var buf bytes.Buffer
if err := wsPromptTpls.Context.Execute(&buf, cd); err == nil {
sb.WriteString(buf.String())
sb.WriteString("\n\n")
}
}
// Section 3: Task Instructions (enriched with expected_output from P2)
if wsPromptTpls.Instructions != nil {
instData := instTemplateData{TaskInstructions: taskInstructions}
if task != nil && task.ExpectedOutput != "" {
instData.ExpectedOutput = task.ExpectedOutput
}
var buf bytes.Buffer
if err := wsPromptTpls.Instructions.Execute(&buf, instData); err == nil {
sb.WriteString(buf.String())
}
} else {
sb.WriteString("## Task Instructions\n\n")
sb.WriteString(taskInstructions)
}
return sb.String()
}
func mimeFromExt(ext string) string {
switch strings.ToLower(ext) {
case ".md":
return "text/markdown"
case ".html", ".htm":
return "text/html"
case ".json":
return "application/json"
case ".pdf":
return "application/pdf"
case ".png":
return "image/png"
case ".jpg", ".jpeg":
return "image/jpeg"
case ".csv":
return "text/csv"
case ".txt":
return "text/plain"
case ".xlsx":
return "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
case ".pptx":
return "application/vnd.openxmlformats-officedocument.presentationml.presentation"
default:
return "application/octet-stream"
}
}

View file

@ -0,0 +1,371 @@
package standard
import (
"context"
"path"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/yaoapp/gou/connector"
"github.com/yaoapp/gou/store"
"github.com/yaoapp/yao/agent"
agentcontext "github.com/yaoapp/yao/agent/context"
robottypes "github.com/yaoapp/yao/agent/robot/types"
"github.com/yaoapp/yao/agent/testutils"
"github.com/yaoapp/yao/llmprovider"
oauthtypes "github.com/yaoapp/yao/openapi/oauth/types"
"github.com/yaoapp/yao/setting"
"github.com/yaoapp/yao/tai/volume"
taiworkspace "github.com/yaoapp/yao/tai/workspace"
)
// ============================================================================
// extractLLMContent — pure unit tests (no external dependencies)
// ============================================================================
func TestExtractLLMContent(t *testing.T) {
t.Run("string content", func(t *testing.T) {
resp := &agentcontext.CompletionResponse{Content: " hello world "}
assert.Equal(t, "hello world", extractLLMContent(resp))
})
t.Run("nil response", func(t *testing.T) {
assert.Equal(t, "", extractLLMContent(nil))
})
t.Run("non-string content", func(t *testing.T) {
resp := &agentcontext.CompletionResponse{Content: []interface{}{"a", "b"}}
assert.Equal(t, "", extractLLMContent(resp))
})
t.Run("empty string content", func(t *testing.T) {
resp := &agentcontext.CompletionResponse{Content: " "}
assert.Equal(t, "", extractLLMContent(resp))
})
t.Run("multiline content trimmed", func(t *testing.T) {
resp := &agentcontext.CompletionResponse{Content: "\n summary line\n"}
assert.Equal(t, "summary line", extractLLMContent(resp))
})
t.Run("int content", func(t *testing.T) {
resp := &agentcontext.CompletionResponse{Content: 42}
assert.Equal(t, "", extractLLMContent(resp))
})
}
// ============================================================================
// extractAndVerifyFiles — unit tests with local volume-backed FS
// ============================================================================
func newTestWorkspaceFS(t *testing.T) taiworkspace.FS {
t.Helper()
dir := t.TempDir()
vol := volume.NewLocal(dir)
t.Cleanup(func() { vol.Close() })
wfs := taiworkspace.New(vol, "ws-test")
t.Cleanup(func() { wfs.Close() })
return wfs
}
func TestExtractAndVerifyFiles(t *testing.T) {
t.Run("valid URI with existing file", func(t *testing.T) {
wfs := newTestWorkspaceFS(t)
require.NoError(t, wfs.MkdirAll("robots/r1/exec1/task-001", 0755))
require.NoError(t, wfs.WriteFile("robots/r1/exec1/task-001/notes.md", []byte("hello"), 0644))
r := &Runner{
wsFS: wfs,
execDir: "robots/r1/exec1",
}
output := "I wrote the file to workspace://ws-test/robots/r1/exec1/task-001/notes.md for you."
files := r.extractAndVerifyFiles(output)
require.Len(t, files, 1)
assert.Equal(t, "notes.md", files[0].Name)
assert.Equal(t, "text/markdown", files[0].Type)
assert.Equal(t, "workspace://ws-test/robots/r1/exec1/task-001/notes.md", files[0].URI)
})
t.Run("URI with non-existent file filtered out", func(t *testing.T) {
wfs := newTestWorkspaceFS(t)
r := &Runner{
wsFS: wfs,
execDir: "robots/r1/exec1",
}
output := "See workspace://ws-test/robots/r1/exec1/task-001/missing.pdf"
files := r.extractAndVerifyFiles(output)
assert.Empty(t, files)
})
t.Run("URI with trailing backtick excluded by regex", func(t *testing.T) {
wfs := newTestWorkspaceFS(t)
require.NoError(t, wfs.MkdirAll("robots/r1/exec1/task-001", 0755))
require.NoError(t, wfs.WriteFile("robots/r1/exec1/task-001/data.json", []byte("{}"), 0644))
r := &Runner{
wsFS: wfs,
execDir: "robots/r1/exec1",
}
// Backtick-wrapped URI — the regex excludes the backtick from the captured path
output := "`workspace://ws-test/robots/r1/exec1/task-001/data.json`"
files := r.extractAndVerifyFiles(output)
require.Len(t, files, 1)
assert.Equal(t, "data.json", files[0].Name)
assert.Equal(t, "application/json", files[0].Type)
})
t.Run("duplicate URIs deduplicated", func(t *testing.T) {
wfs := newTestWorkspaceFS(t)
require.NoError(t, wfs.MkdirAll("robots/r1/exec1/task-001", 0755))
require.NoError(t, wfs.WriteFile("robots/r1/exec1/task-001/report.html", []byte("<h1>hi</h1>"), 0644))
r := &Runner{
wsFS: wfs,
execDir: "robots/r1/exec1",
}
output := "workspace://ws-test/robots/r1/exec1/task-001/report.html and again workspace://ws-test/robots/r1/exec1/task-001/report.html"
files := r.extractAndVerifyFiles(output)
require.Len(t, files, 1)
assert.Equal(t, "report.html", files[0].Name)
})
t.Run("empty output returns nil", func(t *testing.T) {
wfs := newTestWorkspaceFS(t)
r := &Runner{wsFS: wfs, execDir: "robots/r1/exec1"}
assert.Nil(t, r.extractAndVerifyFiles(""))
})
t.Run("nil wsFS returns nil", func(t *testing.T) {
r := &Runner{wsFS: nil, execDir: "robots/r1/exec1"}
assert.Nil(t, r.extractAndVerifyFiles("some text with workspace://ws-test/foo/bar"))
})
t.Run("URI from different workspace filtered", func(t *testing.T) {
wfs := newTestWorkspaceFS(t)
r := &Runner{wsFS: wfs, execDir: "robots/r1/exec1"}
output := "See workspace://other-ws/robots/r1/exec1/task-001/file.txt"
files := r.extractAndVerifyFiles(output)
assert.Empty(t, files)
})
t.Run("directory URI filtered out", func(t *testing.T) {
wfs := newTestWorkspaceFS(t)
require.NoError(t, wfs.MkdirAll("robots/r1/exec1/task-001", 0755))
r := &Runner{wsFS: wfs, execDir: "robots/r1/exec1"}
output := "workspace://ws-test/robots/r1/exec1/task-001"
files := r.extractAndVerifyFiles(output)
assert.Empty(t, files)
})
t.Run("multiple valid files extracted", func(t *testing.T) {
wfs := newTestWorkspaceFS(t)
require.NoError(t, wfs.MkdirAll("robots/r1/exec1/task-002", 0755))
require.NoError(t, wfs.WriteFile("robots/r1/exec1/task-002/slides.html", []byte("<html>"), 0644))
require.NoError(t, wfs.WriteFile("robots/r1/exec1/task-002/slides.pdf", []byte("%PDF"), 0644))
r := &Runner{wsFS: wfs, execDir: "robots/r1/exec1"}
output := "Generated workspace://ws-test/robots/r1/exec1/task-002/slides.html and exported workspace://ws-test/robots/r1/exec1/task-002/slides.pdf"
files := r.extractAndVerifyFiles(output)
require.Len(t, files, 2)
names := []string{files[0].Name, files[1].Name}
assert.Contains(t, names, "slides.html")
assert.Contains(t, names, "slides.pdf")
})
}
// ============================================================================
// mergeManifestFiles — pure unit tests
// ============================================================================
func TestMergeManifestFiles(t *testing.T) {
t.Run("empty fromURIs returns scanned unchanged", func(t *testing.T) {
scanned := []ManifestFile{{Name: "a.md", Type: "text/markdown"}}
result := mergeManifestFiles(scanned, nil)
assert.Equal(t, scanned, result)
})
t.Run("merge URI onto matching scanned entry", func(t *testing.T) {
scanned := []ManifestFile{{Name: "notes.md", Type: "text/markdown"}}
fromURIs := []ManifestFile{{Name: "notes.md", Type: "text/markdown", URI: "workspace://ws/path/notes.md"}}
result := mergeManifestFiles(scanned, fromURIs)
require.Len(t, result, 1)
assert.Equal(t, "workspace://ws/path/notes.md", result[0].URI)
})
t.Run("add new URI entry when no scan match", func(t *testing.T) {
scanned := []ManifestFile{{Name: "a.md", Type: "text/markdown"}}
fromURIs := []ManifestFile{{Name: "b.pdf", Type: "application/pdf", URI: "workspace://ws/b.pdf"}}
result := mergeManifestFiles(scanned, fromURIs)
require.Len(t, result, 2)
assert.Equal(t, "b.pdf", result[1].Name)
})
}
// ============================================================================
// llmSummarize — integration test (real LLM call)
// ============================================================================
func setupLLMProvider(t *testing.T) {
t.Helper()
if err := setting.Init(); err != nil {
t.Skipf("setting.Init failed (store not available): %v", err)
}
if err := llmprovider.Init(); err != nil {
t.Skipf("llmprovider.Init failed: %v", err)
}
if err := agent.SyncLLMDefaults(); err != nil {
t.Skipf("SyncLLMDefaults failed: %v", err)
}
connIDs := connector.AIConnectors
if len(connIDs) == 0 {
t.Skip("no AI connectors available in test env")
}
t.Cleanup(func() {
s, _ := store.Get("__yao.store")
if s != nil {
s.Del("llmprovider:*")
}
c, _ := store.Get("__yao.cache")
if c != nil {
c.Del("llmprovider:*")
}
})
}
func TestLLMSummarize(t *testing.T) {
if testing.Short() {
t.Skip("skipping integration test (requires real LLM)")
}
testutils.Prepare(t)
defer testutils.Clean(t)
setupLLMProvider(t)
auth := &oauthtypes.AuthorizedInfo{UserID: "test-user", TeamID: "test-team"}
ctx := robottypes.NewContext(context.Background(), auth)
r := &Runner{
ctx: ctx,
}
task := &robottypes.Task{
ID: "task-001",
Description: "Research Yao Agents platform and compile structured notes",
}
output := `## Research Findings
Yao Agents is an AI-powered platform that enables developers to build intelligent agents.
### Key Features
- **Model-Driven Architecture**: Define data models in YAML/JSON
- **Low-Code Engine**: Visual workflow builder
- **Multi-Agent Orchestration**: Coordinate multiple AI agents
### Conclusion
Yao Agents provides a comprehensive toolkit for building production-ready AI applications with minimal boilerplate code.`
files := []ManifestFile{
{Name: "research-notes.md", Type: "text/markdown"},
}
summary := r.llmSummarize(task, output, files)
t.Logf("LLM Summary: %s", summary)
assert.NotEmpty(t, summary, "summary should not be empty")
assert.Less(t, len(summary), 500, "summary should be concise (< 500 chars)")
}
// ============================================================================
// updateManifestForTask end-to-end — integration test
// ============================================================================
func TestUpdateManifestForTaskWithLLMSummary(t *testing.T) {
if testing.Short() {
t.Skip("skipping integration test (requires real LLM)")
}
testutils.Prepare(t)
defer testutils.Clean(t)
setupLLMProvider(t)
wfs := newTestWorkspaceFS(t)
auth := &oauthtypes.AuthorizedInfo{UserID: "test-user", TeamID: "test-team"}
ctx := robottypes.NewContext(context.Background(), auth)
execDir := "robots/r1/exec1"
r := &Runner{
ctx: ctx,
wsFS: wfs,
execDir: execDir,
robot: &robottypes.Robot{MemberID: "r1"},
}
task := robottypes.Task{
ID: "task-001",
Description: "Research Yao Agents platform",
ExecutorID: "yao.general",
ExecutorType: robottypes.ExecutorAssistant,
Status: robottypes.TaskPending,
}
exec := &robottypes.Execution{
ID: "exec1",
Tasks: []robottypes.Task{task},
}
r.initManifest(exec)
// Verify manifest was created with pending status
m, err := r.readManifest()
require.NoError(t, err)
require.Len(t, m.Tasks, 1)
assert.Equal(t, "pending", m.Tasks[0].Status)
// Write an artifact file that scanTaskArtifacts can discover
require.NoError(t, wfs.MkdirAll(path.Join(execDir, "task-001"), 0755))
require.NoError(t, wfs.WriteFile(path.Join(execDir, "task-001", "notes.md"), []byte("research content"), 0644))
// Simulate task completion with output referencing the artifact
wsID, _ := wfs.GetID()
result := &robottypes.TaskResult{
Success: true,
Duration: 5000,
Output: "Completed research. Notes saved to workspace://" + wsID + "/" + execDir + "/task-001/notes.md",
}
r.updateManifestForTask(&task, result)
// Re-read and verify
m, err = r.readManifest()
require.NoError(t, err)
require.Len(t, m.Tasks, 1)
mt := m.Tasks[0]
assert.Equal(t, "completed", mt.Status)
assert.NotEmpty(t, mt.Summary, "summary should be generated (LLM or fallback)")
t.Logf("Summary: %s", mt.Summary)
// Files should include the scanned artifact, potentially with URI merged
assert.NotEmpty(t, mt.Files, "files should be populated")
found := false
for _, f := range mt.Files {
if f.Name == "notes.md" {
found = true
assert.Equal(t, "text/markdown", f.Type)
}
}
assert.True(t, found, "notes.md should be in files list")
}

View file

@ -47,7 +47,7 @@ func (l *Logger) prefix() string {
func (l *Logger) Trace(format string, args ...interface{}) {
msg := fmt.Sprintf(format, args...)
if config.IsDevelopment() {
if config.IsDevelopment() && !config.Silent {
fmt.Printf("%s → %s %s%s\n", gray, l.prefix(), msg, reset)
}
kunlog.Trace("%s %s", l.prefix(), msg)
@ -55,7 +55,7 @@ func (l *Logger) Trace(format string, args ...interface{}) {
func (l *Logger) Debug(format string, args ...interface{}) {
msg := fmt.Sprintf(format, args...)
if config.IsDevelopment() {
if config.IsDevelopment() && !config.Silent {
fmt.Printf("%s • %s %s%s\n", gray, l.prefix(), msg, reset)
}
kunlog.Debug("%s %s", l.prefix(), msg)
@ -63,7 +63,7 @@ func (l *Logger) Debug(format string, args ...interface{}) {
func (l *Logger) Info(format string, args ...interface{}) {
msg := fmt.Sprintf(format, args...)
if config.IsDevelopment() {
if config.IsDevelopment() && !config.Silent {
fmt.Printf("%s %s %s%s\n", cyan, l.prefix(), msg, reset)
}
kunlog.Info("%s %s", l.prefix(), msg)
@ -71,7 +71,7 @@ func (l *Logger) Info(format string, args ...interface{}) {
func (l *Logger) Warn(format string, args ...interface{}) {
msg := fmt.Sprintf(format, args...)
if config.IsDevelopment() {
if config.IsDevelopment() && !config.Silent {
fmt.Printf("%s ⚠ %s %s%s\n", yellow, l.prefix(), msg, reset)
}
kunlog.Warn("%s %s", l.prefix(), msg)
@ -79,7 +79,7 @@ func (l *Logger) Warn(format string, args ...interface{}) {
func (l *Logger) Error(format string, args ...interface{}) {
msg := fmt.Sprintf(format, args...)
if config.IsDevelopment() {
if config.IsDevelopment() && !config.Silent {
fmt.Printf("%s ✗ %s %s%s\n", red, l.prefix(), msg, reset)
}
kunlog.Error("%s %s", l.prefix(), msg)
@ -94,7 +94,7 @@ func IsDev() bool {
// Use for rich multi-line output (box-style logs, tables, etc.)
// that should bypass the standard single-line prefix format.
func Raw(s string) {
if config.IsDevelopment() {
if config.IsDevelopment() && !config.Silent {
fmt.Print(s)
}
}

View file

@ -337,7 +337,6 @@ func (m *Manager) callHostAgent(ctx *types.Context, agentID string, input *types
}
caller := standard.NewConversationCaller(chatID)
caller.Connector = robot.LanguageModel
caller.Workspace = robot.Workspace
result, err := caller.CallWithMessages(ctx, agentID, string(inputJSON))
if err != nil {
@ -740,7 +739,6 @@ func (m *Manager) callHostAgentStream(ctx *types.Context, agentID string, input
}
caller := standard.NewConversationCaller(chatID)
caller.Connector = robot.LanguageModel
caller.Workspace = robot.Workspace
result, err := caller.CallWithMessagesStream(ctx, agentID, string(inputJSON), streamFn)
if err != nil {
@ -960,7 +958,6 @@ func (m *Manager) callHostAgentStreamRaw(ctx *types.Context, agentID string, inp
}
caller := standard.NewConversationCaller(chatID)
caller.Connector = robot.LanguageModel
caller.Workspace = robot.Workspace
result, err := caller.CallWithMessagesStreamRaw(ctx, agentID, string(inputJSON), wrappedOnMessage)
if err != nil {

View file

@ -524,6 +524,7 @@ func (m *Manager) Intervene(ctx *types.Context, req *types.InterveneRequest) (*t
Action: req.Action,
Messages: req.Messages,
UserID: ctx.UserID(),
Locale: req.Locale,
}
// Handle plan.add action - schedule for later

View file

@ -14,6 +14,7 @@ type InterveneRequest struct {
Messages []agentcontext.Message `json:"messages"` // user input (text, images, files)
PlanTime *time.Time `json:"plan_time,omitempty"` // for action=plan
ExecutorMode ExecutorMode `json:"executor_mode,omitempty"` // optional: override robot config
Locale string `json:"locale,omitempty"` // language for UI display (e.g., "en", "zh")
}
// EventRequest - event trigger request

View file

@ -404,10 +404,12 @@ type DeliveryContent struct {
// DeliveryAttachment - Task output attachment with metadata
type DeliveryAttachment struct {
Title string `json:"title"` // Human-readable title
Description string `json:"description,omitempty"` // What this artifact is
TaskID string `json:"task_id,omitempty"` // Which task produced this
File string `json:"file"` // Wrapper: __<uploader>://<fileID>
Title string `json:"title"` // Human-readable title
Description string `json:"description,omitempty"` // What this artifact is
TaskID string `json:"task_id,omitempty"` // Which task produced this
File string `json:"file"` // Wrapper: __<uploader>://<fileID>, workspace://, or URL
Size int64 `json:"size,omitempty"` // File size in bytes
ContentType string `json:"content_type,omitempty"` // MIME type
}
// DeliveryRequest - pushed to Delivery Center (no channels - center decides based on preferences)

View file

@ -372,9 +372,12 @@ func buildEnvironment(opts *Options, systemPrompt string) map[string]string {
}
}
// Note: System prompt and max_turns are passed via CLI flags in BuildCommand
// CLAUDE_SYSTEM_PROMPT environment variable is NOT supported by Claude CLI
// --append-system-prompt or --system-prompt flags must be used instead
// Prevent Claude CLI from using an excessive max_tokens that the backend
// API will reject. In OpenAI-proxy mode the hardcoded model is
// claude-sonnet-4-6 whose limit is 16384.
if opts.ConnectorType != "anthropic" {
env["CLAUDE_CODE_MAX_OUTPUT_TOKENS"] = "16384"
}
return env
}

View file

@ -2,224 +2,19 @@ package claude
import (
"context"
"fmt"
"path/filepath"
"strings"
agentContext "github.com/yaoapp/yao/agent/context"
"github.com/yaoapp/yao/attachment"
"github.com/yaoapp/yao/agent/sandbox/v2/shared"
workspace "github.com/yaoapp/yao/tai/workspace"
)
// prepareAttachments resolves __yao.attachment:// URLs in messages,
// copies actual files into the workspace .attachments/{chatID}/ directory via ws.Copy,
// and replaces multimodal content parts with text references.
//
// Delegates to shared.PrepareAttachments; the returned text-replaced messages
// are used directly by the Claude CLI (which reads local files via text refs).
func prepareAttachments(ctx context.Context, messages []agentContext.Message, chatID string, ws workspace.FS) ([]agentContext.Message, error) {
usedNames := make(map[string]int)
attachDir := ".attachments/" + chatID
result := make([]agentContext.Message, len(messages))
copy(result, messages)
for i, msg := range result {
if msg.Role != "user" {
continue
}
parts, ok := msg.Content.([]interface{})
if !ok {
if typedParts, ok := msg.Content.([]agentContext.ContentPart); ok {
iparts := make([]interface{}, len(typedParts))
for j, p := range typedParts {
m := map[string]interface{}{"type": string(p.Type)}
if p.Text != "" {
m["text"] = p.Text
}
if p.ImageURL != nil {
m["image_url"] = map[string]interface{}{
"url": p.ImageURL.URL,
"detail": string(p.ImageURL.Detail),
}
}
if p.File != nil {
m["file"] = map[string]interface{}{
"url": p.File.URL,
"filename": p.File.Filename,
}
}
iparts[j] = m
}
parts = iparts
} else {
continue
}
}
if len(parts) == 0 {
continue
}
var textParts []string
for _, item := range parts {
m, ok := item.(map[string]interface{})
if !ok {
continue
}
partType, _ := m["type"].(string)
switch partType {
case "text":
if text, ok := m["text"].(string); ok && text != "" {
textParts = append(textParts, text)
}
case "image_url":
imgData, _ := m["image_url"].(map[string]interface{})
if imgData == nil {
continue
}
url, _ := imgData["url"].(string)
if url == "" {
continue
}
uploaderName, fileID, isWrapper := attachment.Parse(url)
if !isWrapper {
textParts = append(textParts, fmt.Sprintf("[Image: %s]", url))
continue
}
ref, err := resolveAttachment(ctx, uploaderName, fileID, "", attachDir, usedNames, ws)
if err != nil {
textParts = append(textParts, "[Attached image: failed to load]")
continue
}
textParts = append(textParts, ref)
case "file":
fileData, _ := m["file"].(map[string]interface{})
if fileData == nil {
continue
}
url, _ := fileData["url"].(string)
hintName, _ := fileData["filename"].(string)
if url == "" {
continue
}
uploaderName, fileID, isWrapper := attachment.Parse(url)
if !isWrapper {
textParts = append(textParts, fmt.Sprintf("[File: %s]", url))
continue
}
ref, err := resolveAttachment(ctx, uploaderName, fileID, hintName, attachDir, usedNames, ws)
if err != nil {
textParts = append(textParts, "[Attached file: failed to load]")
continue
}
textParts = append(textParts, ref)
}
}
if len(textParts) > 0 {
newMsg := result[i]
newMsg.Content = strings.Join(textParts, "\n\n")
result[i] = newMsg
}
}
return result, nil
}
// resolveAttachment gets the local path of an attachment and copies it into
// the workspace via ws.Copy("local:///abs/path", ".attachments/{chatID}/filename").
func resolveAttachment(
ctx context.Context,
uploaderName, fileID, hintName, attachDir string,
usedNames map[string]int,
ws workspace.FS,
) (string, error) {
manager, exists := attachment.Managers[uploaderName]
if !exists {
return "", fmt.Errorf("attachment manager not found: %s", uploaderName)
}
fileInfo, err := manager.Info(ctx, fileID)
if err != nil {
return "", fmt.Errorf("failed to get file info: %w", err)
}
absPath, _, err := manager.LocalPath(ctx, fileID)
if err != nil {
return "", fmt.Errorf("failed to get local path: %w", err)
}
filename := fileInfo.Filename
if filename == "" && hintName != "" {
filename = hintName
}
if filename == "" {
ext := extensionFromContentType(fileInfo.ContentType)
filename = fileID + ext
}
baseName := filename
if count, exists := usedNames[baseName]; exists {
ext := filepath.Ext(filename)
name := strings.TrimSuffix(filename, ext)
filename = fmt.Sprintf("%s_%d%s", name, count+1, ext)
usedNames[baseName] = count + 1
} else {
usedNames[baseName] = 0
}
dstPath := attachDir + "/" + filename
src := "local:///" + absPath
if _, err := ws.Copy(src, dstPath); err != nil {
return "", fmt.Errorf("failed to copy attachment to workspace: %w", err)
}
sizeStr := formatFileSize(fileInfo.Bytes)
return fmt.Sprintf("[Attached file: %s (%s, %s)]", dstPath, fileInfo.ContentType, sizeStr), nil
}
func extensionFromContentType(contentType string) string {
switch contentType {
case "image/png":
return ".png"
case "image/jpeg":
return ".jpg"
case "image/gif":
return ".gif"
case "image/webp":
return ".webp"
case "image/svg+xml":
return ".svg"
case "application/pdf":
return ".pdf"
case "text/plain":
return ".txt"
case "text/html":
return ".html"
case "text/css":
return ".css"
case "text/javascript", "application/javascript":
return ".js"
case "application/json":
return ".json"
case "application/zip":
return ".zip"
default:
return ""
}
}
func formatFileSize(bytes int) string {
switch {
case bytes >= 1024*1024:
return fmt.Sprintf("%.1fMB", float64(bytes)/(1024*1024))
case bytes >= 1024:
return fmt.Sprintf("%.1fKB", float64(bytes)/1024)
default:
return fmt.Sprintf("%dB", bytes)
}
processed, _, err := shared.PrepareAttachments(ctx, messages, chatID, ws)
return processed, err
}

View file

@ -2,6 +2,8 @@ package claude
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"regexp"
@ -10,7 +12,9 @@ import (
"github.com/google/uuid"
"github.com/yaoapp/gou/connector"
goullm "github.com/yaoapp/gou/llm"
"github.com/yaoapp/gou/store"
"github.com/yaoapp/kun/log"
"github.com/yaoapp/kun/str"
agentContext "github.com/yaoapp/yao/agent/context"
"github.com/yaoapp/yao/agent/sandbox/v2/types"
@ -18,11 +22,17 @@ import (
)
const defaultA2OPort = 3099
const defaultA2OMaxOutputTokens = 16384
var yaoSessionNS = uuid.MustParse("f47ac10b-58cc-4372-a567-0e02b2c3d479")
var safeNameRe = regexp.MustCompile(`[^a-zA-Z0-9_\-.]`)
func hashUserID(raw string) string {
h := sha256.Sum256([]byte(raw))
return hex.EncodeToString(h[:8]) // 16 hex chars — short yet collision-safe
}
func chatIDToSessionUUID(assistantID, chatID string) string {
return uuid.NewSHA1(yaoSessionNS, []byte(assistantID+":"+chatID)).String()
}
@ -74,6 +84,9 @@ func (r *Runner) buildCommand(ctx context.Context, req *types.StreamRequest, p p
var systemPrompt string
envPrompt := buildSandboxEnvPrompt(p, workDir)
if capPrompt := buildModelCapabilityPrompt(req); capPrompt != "" {
envPrompt += "\n\n" + capPrompt
}
if !isContinuation && req.SystemPrompt != "" {
systemPrompt = req.SystemPrompt + "\n\n" + envPrompt
} else if !isContinuation {
@ -85,6 +98,14 @@ func (r *Runner) buildCommand(ctx context.Context, req *types.StreamRequest, p p
promptFile = p.PathJoin(workDir, ".yao", ".system-prompt.txt")
}
// On continuation turns the system prompt is not re-sent, but the previously
// written prompt file is still on disk. Pass --append-system-prompt-file so
// Claude keeps the same constraints (e.g. workspace path rules) across the
// entire session without injecting a duplicate system turn.
if isContinuation {
args = append(args, "--append-system-prompt-file", promptFile)
}
script, stdin := p.BuildScript(scriptInput{
args: args,
systemPrompt: systemPrompt,
@ -105,11 +126,24 @@ func buildEnv(req *types.StreamRequest, p platform) map[string]string {
env := make(map[string]string)
workDir := req.Computer.GetWorkDir()
// Workspace ID
workspace := req.Computer.Workplace()
if workspace != nil {
workspaceID, err := workspace.GetID()
if err == nil {
env["CTX_WORKSPACE_ID"] = workspaceID
}
}
for k, v := range p.HomeEnv(workDir) {
env[k] = v
}
env["WORKDIR"] = workDir
if req.Locale != "" {
env["CTX_LOCALE"] = req.Locale
}
assistantID := req.AssistantID
if assistantID != "" {
configDir := p.PathJoin(workDir, ".yao", "assistants", assistantID)
@ -123,32 +157,45 @@ func buildEnv(req *types.StreamRequest, p platform) map[string]string {
if req.Connector != nil {
setting := req.Connector.Setting()
host, _ := setting["host"].(string)
key, _ := setting["key"].(string)
model, _ := setting["model"].(string)
if req.Connector.Is(connector.ANTHROPIC) {
env["ANTHROPIC_BASE_URL"] = host
env["ANTHROPIC_API_KEY"] = key
if model != "" {
env["ANTHROPIC_MODEL"] = model
env["ANTHROPIC_DEFAULT_OPUS_MODEL"] = model
env["ANTHROPIC_DEFAULT_SONNET_MODEL"] = model
env["ANTHROPIC_DEFAULT_HAIKU_MODEL"] = model
env["CLAUDE_CODE_SUBAGENT_MODEL"] = model
}
var host, key, model string
if lc, ok := req.Connector.(goullm.LLMConnector); ok {
host = lc.GetURL()
key = lc.GetKey()
model = lc.GetModel()
}
if host == "" {
host, _ = setting["host"].(string)
}
if key == "" {
key, _ = setting["key"].(string)
}
if model == "" {
model, _ = setting["model"].(string)
}
isAnthropic := req.Connector.Is(connector.ANTHROPIC)
if isAnthropic {
setAnthropicModelEnv(env, host, key, model, req.Connector)
applyAnthropicRoleOverrides(env, host, req.Roles)
} else {
connectorID := req.Connector.ID()
env["ANTHROPIC_BASE_URL"] = fmt.Sprintf("http://127.0.0.1:%d/c/%s", defaultA2OPort, connectorID)
env["ANTHROPIC_API_KEY"] = "dummy"
// Use a valid Anthropic model name to pass Claude CLI's local
// validation. The a2o proxy ignores this and substitutes the
// real backend model from its connector config.
env["ANTHROPIC_MODEL"] = "claude-sonnet-4-6"
env["ANTHROPIC_DEFAULT_OPUS_MODEL"] = "claude-sonnet-4-6"
env["ANTHROPIC_DEFAULT_SONNET_MODEL"] = "claude-sonnet-4-6"
env["ANTHROPIC_DEFAULT_HAIKU_MODEL"] = "claude-sonnet-4-6"
env["CLAUDE_CODE_SUBAGENT_MODEL"] = "claude-sonnet-4-6"
setA2OModelEnv(env, req.Connector.ID(), model, req.Connector)
applyA2ORoleOverrides(env, req.Roles)
}
if lc, ok := req.Connector.(goullm.LLMConnector); ok {
if caps := lc.GetCapabilities(); caps != nil {
if caps.MaxOutputTokens > 0 {
env["CLAUDE_CODE_MAX_OUTPUT_TOKENS"] = fmt.Sprintf("%d", caps.MaxOutputTokens)
}
if caps.MaxInputTokens > 0 {
env["CLAUDE_CODE_AUTO_COMPACT_WINDOW"] = fmt.Sprintf("%d", caps.MaxInputTokens)
}
}
}
if _, ok := env["CLAUDE_CODE_MAX_OUTPUT_TOKENS"]; !ok && !req.Connector.Is(connector.ANTHROPIC) {
env["CLAUDE_CODE_MAX_OUTPUT_TOKENS"] = fmt.Sprintf("%d", defaultA2OMaxOutputTokens)
}
if thinking, ok := setting["thinking"].(map[string]interface{}); ok {
@ -179,6 +226,40 @@ func buildEnv(req *types.StreamRequest, p platform) map[string]string {
}
}
logger := req.Logger
if logger == nil {
logger = agentContext.NoopLogger()
}
connectorID := ""
if req.Connector != nil {
connectorID = req.Connector.ID()
}
logger.Debug("claude-env: connector=%s isAnthropic=%v", connectorID, req.Connector != nil && req.Connector.Is(connector.ANTHROPIC))
logger.Debug("claude-env: ANTHROPIC_MODEL=%s", env["ANTHROPIC_MODEL"])
logger.Debug("claude-env: OPUS_MODEL=%s SONNET_MODEL=%s HAIKU_MODEL=%s",
env["ANTHROPIC_DEFAULT_OPUS_MODEL"],
env["ANTHROPIC_DEFAULT_SONNET_MODEL"],
env["ANTHROPIC_DEFAULT_HAIKU_MODEL"])
logger.Debug("claude-env: CUSTOM_MODEL_OPTION=%s CAPABILITIES=%s",
env["ANTHROPIC_CUSTOM_MODEL_OPTION"],
env["ANTHROPIC_CUSTOM_MODEL_OPTION_SUPPORTED_CAPABILITIES"])
logger.Debug("claude-env: MAX_THINKING_TOKENS=%s", env["MAX_THINKING_TOKENS"])
// Override metadata.user_id with a sanitized value.
// Claude CLI sets metadata.user_id to a JSON object that third-party
// Anthropic-compatible APIs (e.g. DeepSeek) reject because the value
// doesn't match ^[a-zA-Z0-9_-]+$.
if _, ok := env["CLAUDE_CODE_EXTRA_BODY"]; !ok {
uid := "yao-sandbox"
if req.Config != nil && req.Config.Owner != "" {
uid = hashUserID(req.Config.Owner)
} else if assistantID != "" {
uid = hashUserID(assistantID)
}
env["CLAUDE_CODE_EXTRA_BODY"] = fmt.Sprintf(`{"metadata":{"user_id":"%s"}}`, uid)
}
logger.Debug("claude-env: EXTRA_BODY=%s", env["CLAUDE_CODE_EXTRA_BODY"])
return env
}
@ -249,26 +330,149 @@ func buildSandboxEnvPrompt(p platform, workDir string) string {
shellNote := p.EnvPromptNote()
envVarSyntax := "$VAR_NAME"
if osName == "windows" {
envVarSyntax = "$env:VAR_NAME"
}
return fmt.Sprintf(`## Sandbox Environment
- **Operating System**: %[2]s
- **Shell**: %[3]s
- **Working Directory**: %[1]s
- **File Access**: You have full read/write access to %[1]s
- **Environment variable syntax**: `+"`%[5]s`"+` (e.g. `+"`$CTX_SKILLS_DIR`"+` on POSIX, `+"`$env:CTX_SKILLS_DIR`"+` on Windows)%[4]s
%[4]s`, workDir, osName, shell, shellNote)
}
## User Attachments
func buildModelCapabilityPrompt(req *types.StreamRequest) string {
if req.Connector == nil {
return ""
}
User-uploaded files (images, documents, code files, etc.) are placed in %[1]s/.attachments/{chatID}/
Each chat session has its own subdirectory to avoid conflicts.
When the user references an attached file, read it from this directory using the Read or Bash tool.
For image files, you can view them directly as Claude supports vision on local files.
`, workDir, osName, shell, shellNote, envVarSyntax)
lc, ok := req.Connector.(goullm.LLMConnector)
if !ok {
return ""
}
primaryModel := lc.GetModel()
if primaryModel == "" {
return ""
}
primaryCaps := lc.GetCapabilities()
type tierInfo struct {
tier string
alias string
model string
caps *goullm.Capabilities
conn connector.Connector
}
tiers := []tierInfo{
{tier: "Default", alias: "sonnet", model: primaryModel, caps: primaryCaps, conn: req.Connector},
}
hasDifferentTier := false
if rc, exists := req.Roles["heavy"]; exists && rc != nil {
m := connectorModel(rc)
if m != "" {
caps := connectorCaps(rc)
tiers = append(tiers, tierInfo{tier: "Heavy", alias: "opus", model: m, caps: caps, conn: rc})
if m != primaryModel {
hasDifferentTier = true
}
}
}
if rc, exists := req.Roles["light"]; exists && rc != nil {
m := connectorModel(rc)
if m != "" {
caps := connectorCaps(rc)
tiers = append(tiers, tierInfo{tier: "Light", alias: "haiku", model: m, caps: caps, conn: rc})
if m != primaryModel {
hasDifferentTier = true
}
}
}
var sb strings.Builder
sb.WriteString("## Model Capabilities\n\n")
sb.WriteString(fmt.Sprintf("Your current model: `%s`\n", primaryModel))
if hasDifferentTier {
sb.WriteString("\n### Available Model Tiers\n\n")
sb.WriteString("| Tier | Alias | Model | Capabilities |\n")
sb.WriteString("| ---- | ----- | ----- | ------------ |\n")
for _, t := range tiers {
capList := formatCapabilities(t.caps, t.conn)
sb.WriteString(fmt.Sprintf("| %s | %s | %s | %s |\n", t.tier, t.alias, t.model, capList))
}
}
var guidance []string
if hasDifferentTier {
guidance = append(guidance,
"For complex reasoning, multi-step analysis, or tasks requiring deep thought, delegate to a sub-agent with `model: \"opus\"`",
"For simple tasks (formatting, translation, summarization), use `model: \"haiku\"` for faster responses",
)
}
primaryHasVision := primaryCaps.HasVision()
if !primaryHasVision {
if _, hasVisionRole := req.Roles["vision"]; hasVisionRole {
guidance = append(guidance,
"**Image/Vision**: Your current model cannot process images directly. Use the `image_read` system tool (`tai tool image_read`) to analyze images — see the yao-image skill for details",
)
}
}
if len(guidance) > 0 {
sb.WriteString("\n### Usage Guidance\n\n")
for _, g := range guidance {
sb.WriteString("- " + g + "\n")
}
}
if !hasDifferentTier && len(guidance) == 0 {
return ""
}
return sb.String()
}
func connectorModel(c connector.Connector) string {
if lc, ok := c.(goullm.LLMConnector); ok {
if m := lc.GetModel(); m != "" {
return m
}
}
m, _ := c.Setting()["model"].(string)
return m
}
func connectorCaps(c connector.Connector) *goullm.Capabilities {
if lc, ok := c.(goullm.LLMConnector); ok {
return lc.GetCapabilities()
}
return nil
}
func formatCapabilities(caps *goullm.Capabilities, conn connector.Connector) string {
var parts []string
hasThinking := caps.HasReasoning()
if !hasThinking && conn != nil {
if thinking, ok := conn.Setting()["thinking"].(map[string]interface{}); ok {
if t, _ := thinking["type"].(string); t == "enabled" {
hasThinking = true
}
}
}
if hasThinking {
parts = append(parts, "thinking")
}
if caps.HasVision() {
parts = append(parts, "vision")
}
if caps.HasToolCalls() {
parts = append(parts, "tool_calls")
}
if len(parts) == 0 {
return "-"
}
return strings.Join(parts, ", ")
}
func hasExistingSession(ctx context.Context, computer infra.Computer, p platform, assistantID string) bool {
@ -344,8 +548,194 @@ func buildLastUserMessageJSONL(messages []agentContext.Message) string {
return ""
}
// claudeRoleEnvMap maps abstract Yao model roles to Claude CLI environment
// variables. Only roles with matching Claude CLI env vars are listed here.
// ANTHROPIC_DEFAULT_SONNET_MODEL is set to the primary model in buildEnv.
var claudeRoleEnvMap = map[string]struct{ EnvVar string }{
"default": {EnvVar: "ANTHROPIC_MODEL"},
"heavy": {EnvVar: "ANTHROPIC_DEFAULT_OPUS_MODEL"},
"light": {EnvVar: "ANTHROPIC_DEFAULT_HAIKU_MODEL"},
}
func connectorHost(c connector.Connector) string {
if c == nil {
return ""
}
if lc, ok := c.(goullm.LLMConnector); ok {
if u := lc.GetURL(); u != "" {
return u
}
}
host, _ := c.Setting()["host"].(string)
return host
}
func connectorProtocols(c connector.Connector) []string {
if c == nil {
return nil
}
setting := c.Setting()
if ps, ok := setting["protocols"].([]string); ok && len(ps) > 0 {
return ps
}
if c.Is(connector.ANTHROPIC) {
return []string{"anthropic"}
}
return []string{"openai"}
}
func supportsProtocol(c connector.Connector, proto string) bool {
for _, p := range connectorProtocols(c) {
if p == proto {
return true
}
}
return false
}
var claudeArgWhitelist = map[string]string{
"max_turns": "--max-turns",
"disallowed_tools": "--disallowed-tools",
"allowed_tools": "--allowedTools",
}
func isStandardAnthropicModel(model string) bool {
return strings.HasPrefix(model, "claude-") || strings.HasPrefix(model, "anthropic.")
}
func buildClaudeCodeCapabilities(conn connector.Connector) string {
if conn == nil {
return ""
}
setting := conn.Setting()
if setting == nil {
return ""
}
var caps []string
if thinking, ok := setting["thinking"].(map[string]interface{}); ok {
if thinkType, _ := thinking["type"].(string); thinkType == "enabled" {
caps = append(caps, "thinking")
}
}
return strings.Join(caps, ",")
}
func setAnthropicModelEnv(env map[string]string, host, key, model string, conn connector.Connector) {
env["ANTHROPIC_BASE_URL"] = host
env["ANTHROPIC_API_KEY"] = key
if model == "" {
return
}
env["ANTHROPIC_MODEL"] = model
env["ANTHROPIC_DEFAULT_OPUS_MODEL"] = model
env["ANTHROPIC_DEFAULT_SONNET_MODEL"] = model
env["ANTHROPIC_DEFAULT_HAIKU_MODEL"] = model
if isStandardAnthropicModel(model) {
return
}
caps := buildClaudeCodeCapabilities(conn)
env["ANTHROPIC_CUSTOM_MODEL_OPTION"] = model
env["ANTHROPIC_CUSTOM_MODEL_OPTION_NAME"] = model
env["ANTHROPIC_DEFAULT_OPUS_MODEL_NAME"] = model
env["ANTHROPIC_DEFAULT_SONNET_MODEL_NAME"] = model
env["ANTHROPIC_DEFAULT_HAIKU_MODEL_NAME"] = model
if caps != "" {
env["ANTHROPIC_CUSTOM_MODEL_OPTION_SUPPORTED_CAPABILITIES"] = caps
env["ANTHROPIC_DEFAULT_OPUS_MODEL_SUPPORTED_CAPABILITIES"] = caps
env["ANTHROPIC_DEFAULT_SONNET_MODEL_SUPPORTED_CAPABILITIES"] = caps
env["ANTHROPIC_DEFAULT_HAIKU_MODEL_SUPPORTED_CAPABILITIES"] = caps
}
}
func applyAnthropicRoleOverrides(
env map[string]string,
primaryHost string,
roles map[string]connector.Connector,
) {
for role, rm := range claudeRoleEnvMap {
if role == "default" {
continue
}
rc, ok := roles[role]
if !ok || rc == nil {
continue
}
roleHost := connectorHost(rc)
if roleHost != primaryHost {
log.Warn("[claude] role %s: host mismatch (%s != %s), falling back to primary", role, roleHost, primaryHost)
continue
}
if !supportsProtocol(rc, "anthropic") {
log.Warn("[claude] role %s: not anthropic protocol, falling back to primary", role)
continue
}
rcModel, _ := rc.Setting()["model"].(string)
if rcModel == "" {
continue
}
env[rm.EnvVar] = rcModel
if isStandardAnthropicModel(rcModel) {
continue
}
env[rm.EnvVar+"_NAME"] = rcModel
if caps := buildClaudeCodeCapabilities(rc); caps != "" {
env[rm.EnvVar+"_SUPPORTED_CAPABILITIES"] = caps
}
}
}
func setA2OModelEnv(env map[string]string, connectorID, model string, conn connector.Connector) {
env["ANTHROPIC_BASE_URL"] = fmt.Sprintf("http://127.0.0.1:%d/c/%s", defaultA2OPort, connectorID)
env["ANTHROPIC_API_KEY"] = "dummy"
env["ANTHROPIC_MODEL"] = model
env["ANTHROPIC_DEFAULT_OPUS_MODEL"] = model
env["ANTHROPIC_DEFAULT_SONNET_MODEL"] = model
env["ANTHROPIC_DEFAULT_HAIKU_MODEL"] = model
if !isStandardAnthropicModel(model) {
env["ANTHROPIC_CUSTOM_MODEL_OPTION"] = model
env["ANTHROPIC_CUSTOM_MODEL_OPTION_NAME"] = model
env["ANTHROPIC_DEFAULT_OPUS_MODEL_NAME"] = model
env["ANTHROPIC_DEFAULT_SONNET_MODEL_NAME"] = model
env["ANTHROPIC_DEFAULT_HAIKU_MODEL_NAME"] = model
if caps := buildClaudeCodeCapabilities(conn); caps != "" {
env["ANTHROPIC_CUSTOM_MODEL_OPTION_SUPPORTED_CAPABILITIES"] = caps
env["ANTHROPIC_DEFAULT_OPUS_MODEL_SUPPORTED_CAPABILITIES"] = caps
env["ANTHROPIC_DEFAULT_SONNET_MODEL_SUPPORTED_CAPABILITIES"] = caps
env["ANTHROPIC_DEFAULT_HAIKU_MODEL_SUPPORTED_CAPABILITIES"] = caps
}
}
}
func applyA2ORoleOverrides(
env map[string]string,
roles map[string]connector.Connector,
) {
for role, rm := range claudeRoleEnvMap {
if role == "default" {
continue
}
rc, ok := roles[role]
if !ok || rc == nil {
continue
}
var rcModel string
if lc, ok := rc.(goullm.LLMConnector); ok {
rcModel = lc.GetModel()
}
if rcModel == "" {
rcModel, _ = rc.Setting()["model"].(string)
}
if rcModel == "" {
continue
}
env[rm.EnvVar] = rcModel
if !isStandardAnthropicModel(rcModel) {
env[rm.EnvVar+"_NAME"] = rcModel
if caps := buildClaudeCodeCapabilities(rc); caps != "" {
env[rm.EnvVar+"_SUPPORTED_CAPABILITIES"] = caps
}
}
}
}

View file

@ -8,6 +8,11 @@ import (
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/yaoapp/gou/connector"
goullm "github.com/yaoapp/gou/llm"
gouTypes "github.com/yaoapp/gou/types"
"github.com/yaoapp/xun/dbal/query"
"github.com/yaoapp/xun/dbal/schema"
agentContext "github.com/yaoapp/yao/agent/context"
"github.com/yaoapp/yao/agent/sandbox/v2/types"
infra "github.com/yaoapp/yao/sandbox/v2"
@ -304,7 +309,6 @@ func TestBuildSandboxEnvPrompt(t *testing.T) {
assert.Contains(t, prompt, "darwin")
assert.Contains(t, prompt, "bash")
assert.Contains(t, prompt, "Sandbox Environment")
assert.Contains(t, prompt, ".attachments")
}
func TestBuildSandboxEnvPrompt_WindowsPlatform(t *testing.T) {
@ -434,3 +438,419 @@ func TestBuildArgs_EmptyChatID_Continuation(t *testing.T) {
assert.NotContains(t, args, "--session-id")
assert.NotContains(t, args, "--name")
}
// --- fakeConnector implements connector.Connector for unit tests ---
type fakeConnector struct {
id string
typ int
settings map[string]interface{}
}
func (f *fakeConnector) Register(string, string, []byte) error { return nil }
func (f *fakeConnector) Query() (query.Query, error) { return nil, nil }
func (f *fakeConnector) Schema() (schema.Schema, error) { return nil, nil }
func (f *fakeConnector) Close() error { return nil }
func (f *fakeConnector) ID() string { return f.id }
func (f *fakeConnector) Is(t int) bool { return f.typ == t }
func (f *fakeConnector) Setting() map[string]interface{} { return f.settings }
func (f *fakeConnector) GetMetaInfo() gouTypes.MetaInfo { return gouTypes.MetaInfo{} }
func newOpenAIConnector(id, host, model, key string) *fakeConnector {
return &fakeConnector{
id: id,
typ: connector.OPENAI,
settings: map[string]interface{}{
"host": host,
"model": model,
"key": key,
},
}
}
func newAnthropicConnector(id, host, model, key string) *fakeConnector {
return &fakeConnector{
id: id,
typ: connector.ANTHROPIC,
settings: map[string]interface{}{
"host": host,
"model": model,
"key": key,
},
}
}
func newDualProtoConnector(id, host, model, key string) *fakeConnector {
return &fakeConnector{
id: id,
typ: connector.OPENAI,
settings: map[string]interface{}{
"host": host,
"model": model,
"key": key,
"protocols": []string{"openai", "anthropic"},
},
}
}
// --- helper functions tests ---
func TestConnectorHost(t *testing.T) {
c := newOpenAIConnector("test", "https://api.openai.com", "gpt-4", "k")
assert.Equal(t, "https://api.openai.com", connectorHost(c))
assert.Equal(t, "", connectorHost(nil))
}
func TestConnectorProtocols(t *testing.T) {
oai := newOpenAIConnector("oai", "https://api.openai.com", "gpt-4", "k")
assert.Equal(t, []string{"openai"}, connectorProtocols(oai))
anth := newAnthropicConnector("anth", "https://api.anthropic.com", "claude", "k")
assert.Equal(t, []string{"anthropic"}, connectorProtocols(anth))
dual := newDualProtoConnector("dual", "https://api.yao.run", "model", "k")
assert.Equal(t, []string{"openai", "anthropic"}, connectorProtocols(dual))
assert.Nil(t, connectorProtocols(nil))
}
func TestSupportsProtocol(t *testing.T) {
dual := newDualProtoConnector("dual", "https://api.yao.run", "model", "k")
assert.True(t, supportsProtocol(dual, "anthropic"))
assert.True(t, supportsProtocol(dual, "openai"))
assert.False(t, supportsProtocol(dual, "grpc"))
oai := newOpenAIConnector("oai", "https://api.openai.com", "gpt-4", "k")
assert.False(t, supportsProtocol(oai, "anthropic"))
assert.True(t, supportsProtocol(oai, "openai"))
}
func TestBuildEnv_OpenAI_SingleConnector(t *testing.T) {
oai := newOpenAIConnector("kimi", "https://api.moonshot.cn", "kimi-k2.5", "sk-test")
req := &types.StreamRequest{
Config: &types.SandboxConfig{},
Connector: oai,
}
req.Computer = newFakeComputer("/workspace")
p := testPlatform()
env := buildEnv(req, p)
assert.Contains(t, env["ANTHROPIC_BASE_URL"], "127.0.0.1")
assert.Contains(t, env["ANTHROPIC_BASE_URL"], "kimi")
assert.Equal(t, "kimi-k2.5", env["ANTHROPIC_MODEL"])
assert.Equal(t, "kimi-k2.5", env["ANTHROPIC_DEFAULT_OPUS_MODEL"])
}
func TestBuildEnv_OpenAI_MultiConnector(t *testing.T) {
primary := newOpenAIConnector("kimi", "https://api.moonshot.cn", "kimi-k2.5", "sk-test")
heavyConn := newOpenAIConnector("heavy-conn", "https://api.heavy.com", "heavy-model", "sk-h")
req := &types.StreamRequest{
Config: &types.SandboxConfig{},
Connector: primary,
Roles: map[string]connector.Connector{
"default": primary,
"heavy": heavyConn,
},
}
req.Computer = newFakeComputer("/workspace")
p := testPlatform()
env := buildEnv(req, p)
assert.Equal(t, "heavy-model", env["ANTHROPIC_DEFAULT_OPUS_MODEL"],
"heavy role should use actual model name from connector")
assert.Equal(t, "kimi-k2.5", env["ANTHROPIC_MODEL"],
"primary should use actual model name from connector")
assert.Equal(t, "kimi-k2.5", env["ANTHROPIC_CUSTOM_MODEL_OPTION"],
"non-standard model should set custom model option")
}
func TestBuildEnv_Anthropic_SingleConnector(t *testing.T) {
anth := newAnthropicConnector("claude", "https://api.anthropic.com", "claude-sonnet-4-20250514", "sk-ant-test")
req := &types.StreamRequest{
Config: &types.SandboxConfig{},
Connector: anth,
}
req.Computer = newFakeComputer("/workspace")
p := testPlatform()
env := buildEnv(req, p)
assert.Equal(t, "https://api.anthropic.com", env["ANTHROPIC_BASE_URL"])
assert.Equal(t, "sk-ant-test", env["ANTHROPIC_API_KEY"])
assert.Equal(t, "claude-sonnet-4-20250514", env["ANTHROPIC_MODEL"])
}
func TestBuildEnv_Anthropic_MultiConnector_Compatible(t *testing.T) {
primary := newAnthropicConnector("claude", "https://api.yao.run", "claude-sonnet-4-20250514", "sk-ant")
lightConn := newDualProtoConnector("light-conn", "https://api.yao.run", "claude-haiku-3-5-20241022", "sk-light")
req := &types.StreamRequest{
Config: &types.SandboxConfig{},
Connector: primary,
Roles: map[string]connector.Connector{
"default": primary,
"light": lightConn,
},
}
req.Computer = newFakeComputer("/workspace")
p := testPlatform()
env := buildEnv(req, p)
assert.Equal(t, "claude-haiku-3-5-20241022", env["ANTHROPIC_DEFAULT_HAIKU_MODEL"],
"compatible connector: use real model name")
assert.Equal(t, "https://api.yao.run", env["ANTHROPIC_BASE_URL"],
"primary base URL unchanged")
}
// --- buildSingleA2OConfig + injectA2OConfigWithRoutes ---
func TestBuildSingleA2OConfig_Basic(t *testing.T) {
conn := newOpenAIConnector("test", "https://api.openai.com", "gpt-4", "sk-test")
cfg := buildSingleA2OConfig(conn)
require.NotNil(t, cfg)
assert.Contains(t, cfg.Backend, "api.openai.com")
assert.Contains(t, cfg.Backend, "chat/completions")
assert.Equal(t, "gpt-4", cfg.Model)
assert.Equal(t, "sk-test", cfg.APIKey)
}
func TestBuildSingleA2OConfig_Nil(t *testing.T) {
conn := &fakeConnector{id: "empty", typ: connector.OPENAI, settings: map[string]interface{}{}}
cfg := buildSingleA2OConfig(conn)
assert.Nil(t, cfg, "no host => nil config")
}
func TestInjectA2OConfigWithRoutes_BuildsCorrectJSON(t *testing.T) {
primary := newOpenAIConnector("kimi", "https://api.moonshot.cn", "kimi-k2.5", "sk-kimi")
heavyConn := newOpenAIConnector("heavy", "https://api.heavy.com", "heavy-model", "sk-h")
roleConnectors := map[string]connector.Connector{
"heavy-model": heavyConn,
}
primaryCfg := buildSingleA2OConfig(primary)
require.NotNil(t, primaryCfg)
routes := make(map[string]*a2oConnectorConfig, len(roleConnectors))
for modelName, rc := range roleConnectors {
routeCfg := buildSingleA2OConfig(rc)
if routeCfg != nil {
routes[modelName] = routeCfg
}
}
primaryCfg.Routes = routes
data, err := json.Marshal(primaryCfg)
require.NoError(t, err)
var parsed map[string]interface{}
require.NoError(t, json.Unmarshal(data, &parsed))
routesMap, ok := parsed["routes"].(map[string]interface{})
require.True(t, ok, "routes should be present in JSON")
assert.Len(t, routesMap, 1)
heavyRoute, ok := routesMap["heavy-model"].(map[string]interface{})
require.True(t, ok)
assert.Equal(t, "heavy-model", heavyRoute["model"])
assert.Contains(t, heavyRoute["backend"], "api.heavy.com")
}
func TestResolveAllRoleConnectors_Empty(t *testing.T) {
req := &types.StreamRequest{
Config: &types.SandboxConfig{},
Connector: newOpenAIConnector("test", "h", "m", "k"),
}
result := resolveAllRoleConnectors(req)
assert.Nil(t, result)
}
func TestResolveAllRoleConnectors_WithRoles(t *testing.T) {
primaryConn := newOpenAIConnector("primary", "https://primary.com", "primary-m", "k")
heavyConn := newOpenAIConnector("hvy", "https://heavy.com", "heavy-m", "sk")
req := &types.StreamRequest{
Config: &types.SandboxConfig{},
Connector: primaryConn,
Roles: map[string]connector.Connector{
"default": primaryConn,
"heavy": heavyConn,
},
}
result := resolveAllRoleConnectors(req)
assert.Len(t, result, 2)
assert.Equal(t, primaryConn, result["primary-m"])
assert.Equal(t, heavyConn, result["heavy-m"])
}
func TestBuildEnv_Anthropic_MultiConnector_Incompatible(t *testing.T) {
primary := newAnthropicConnector("claude", "https://api.anthropic.com", "claude-sonnet-4-20250514", "sk-ant")
heavyConn := newOpenAIConnector("heavy-oai", "https://api.openai.com", "gpt-4o", "sk-oai")
req := &types.StreamRequest{
Config: &types.SandboxConfig{},
Connector: primary,
Roles: map[string]connector.Connector{
"default": primary,
"heavy": heavyConn,
},
}
req.Computer = newFakeComputer("/workspace")
p := testPlatform()
env := buildEnv(req, p)
assert.Equal(t, "claude-sonnet-4-20250514", env["ANTHROPIC_DEFAULT_OPUS_MODEL"],
"incompatible connector: heavy should keep primary model (different host, no anthropic protocol)")
}
// --- fakeLLMConnector implements both connector.Connector and goullm.LLMConnector ---
type fakeLLMConnector struct {
fakeConnector
model string
caps *goullm.Capabilities
}
func (f *fakeLLMConnector) GetAuthMode() goullm.AuthMode { return goullm.AuthBearer }
func (f *fakeLLMConnector) GetURL() string { return "" }
func (f *fakeLLMConnector) GetKey() string { return "" }
func (f *fakeLLMConnector) GetModel() string { return f.model }
func (f *fakeLLMConnector) GetSupportedParams() map[string]*goullm.ParamSpec {
return nil
}
func (f *fakeLLMConnector) GetCapabilities() *goullm.Capabilities { return f.caps }
// --- buildModelCapabilityPrompt tests ---
func TestBuildModelCapabilityPrompt_NilConnector(t *testing.T) {
req := &types.StreamRequest{Config: &types.SandboxConfig{}}
result := buildModelCapabilityPrompt(req)
assert.Empty(t, result)
}
func TestBuildModelCapabilityPrompt_NoRoles_NoSpecialCaps(t *testing.T) {
primary := &fakeLLMConnector{
fakeConnector: fakeConnector{id: "test", typ: connector.OPENAI, settings: map[string]interface{}{"model": "deepseek-v4-flash"}},
model: "deepseek-v4-flash",
caps: &goullm.Capabilities{ToolCalls: true, Streaming: true},
}
req := &types.StreamRequest{
Config: &types.SandboxConfig{},
Connector: primary,
}
result := buildModelCapabilityPrompt(req)
assert.Empty(t, result, "no roles and no special caps → empty")
}
func TestBuildModelCapabilityPrompt_WithHeavyAndLight(t *testing.T) {
primary := &fakeLLMConnector{
fakeConnector: fakeConnector{id: "ds-flash", typ: connector.OPENAI, settings: map[string]interface{}{"model": "deepseek-v4-flash"}},
model: "deepseek-v4-flash",
caps: &goullm.Capabilities{ToolCalls: true},
}
heavy := &fakeLLMConnector{
fakeConnector: fakeConnector{id: "ds-pro", typ: connector.OPENAI, settings: map[string]interface{}{"model": "deepseek-v4-pro"}},
model: "deepseek-v4-pro",
caps: &goullm.Capabilities{Reasoning: true, ToolCalls: true},
}
light := &fakeLLMConnector{
fakeConnector: fakeConnector{id: "ds-lite", typ: connector.OPENAI, settings: map[string]interface{}{"model": "deepseek-v4-flash"}},
model: "deepseek-v4-flash",
caps: &goullm.Capabilities{ToolCalls: true},
}
req := &types.StreamRequest{
Config: &types.SandboxConfig{},
Connector: primary,
Roles: map[string]connector.Connector{
"default": primary,
"heavy": heavy,
"light": light,
},
}
result := buildModelCapabilityPrompt(req)
assert.Contains(t, result, "deepseek-v4-flash")
assert.Contains(t, result, "deepseek-v4-pro")
assert.Contains(t, result, "opus")
assert.Contains(t, result, "haiku")
assert.Contains(t, result, "thinking")
assert.Contains(t, result, "tool_calls")
assert.Contains(t, result, "sub-agent")
}
func TestBuildModelCapabilityPrompt_VisionGuidance(t *testing.T) {
primary := &fakeLLMConnector{
fakeConnector: fakeConnector{id: "ds-flash", typ: connector.OPENAI, settings: map[string]interface{}{"model": "deepseek-v4-flash"}},
model: "deepseek-v4-flash",
caps: &goullm.Capabilities{ToolCalls: true},
}
visionConn := &fakeLLMConnector{
fakeConnector: fakeConnector{id: "vision", typ: connector.OPENAI, settings: map[string]interface{}{"model": "gpt-4o"}},
model: "gpt-4o",
caps: &goullm.Capabilities{Vision: true, ToolCalls: true},
}
req := &types.StreamRequest{
Config: &types.SandboxConfig{},
Connector: primary,
Roles: map[string]connector.Connector{
"default": primary,
"vision": visionConn,
},
}
result := buildModelCapabilityPrompt(req)
assert.Contains(t, result, "image_read")
assert.Contains(t, result, "Image/Vision")
}
func TestBuildModelCapabilityPrompt_PrimaryHasVision_NoGuidance(t *testing.T) {
primary := &fakeLLMConnector{
fakeConnector: fakeConnector{id: "gpt4o", typ: connector.OPENAI, settings: map[string]interface{}{"model": "gpt-4o"}},
model: "gpt-4o",
caps: &goullm.Capabilities{Vision: true, ToolCalls: true},
}
req := &types.StreamRequest{
Config: &types.SandboxConfig{},
Connector: primary,
Roles: map[string]connector.Connector{
"default": primary,
"vision": primary,
},
}
result := buildModelCapabilityPrompt(req)
assert.NotContains(t, result, "image_read", "should not suggest image_read when primary has vision")
}
func TestBuildModelCapabilityPrompt_ThinkingFromSettings(t *testing.T) {
primary := &fakeLLMConnector{
fakeConnector: fakeConnector{
id: "ds-pro",
typ: connector.OPENAI,
settings: map[string]interface{}{
"model": "deepseek-v4-pro",
"thinking": map[string]interface{}{
"type": "enabled",
},
},
},
model: "deepseek-v4-pro",
caps: &goullm.Capabilities{ToolCalls: true},
}
heavy := &fakeLLMConnector{
fakeConnector: fakeConnector{id: "ds-pro2", typ: connector.OPENAI, settings: map[string]interface{}{"model": "deepseek-v4-pro-max"}},
model: "deepseek-v4-pro-max",
caps: &goullm.Capabilities{ToolCalls: true},
}
req := &types.StreamRequest{
Config: &types.SandboxConfig{},
Connector: primary,
Roles: map[string]connector.Connector{
"default": primary,
"heavy": heavy,
},
}
result := buildModelCapabilityPrompt(req)
assert.Contains(t, result, "thinking", "should detect thinking from Setting()[\"thinking\"]")
}

View file

@ -11,6 +11,7 @@ import (
"github.com/yaoapp/kun/log"
"github.com/yaoapp/yao/agent/output/message"
"github.com/yaoapp/yao/agent/sandbox/v2/shared"
)
// streamParser is an explicit state machine for Claude CLI stream-json output.
@ -19,16 +20,20 @@ import (
//
// content_block_start -> message_start(id=exec-N-xxx) + ChunkExecute{tool, status:running}
// input_json_delta -> ChunkExecute{input_delta:...} (same message group)
// content_block_stop -> message_end(exec-N-xxx)
// content_block_stop -> message_end(exec-N-xxx) (streaming phase ends, tool kept in buffer)
// ...later...
// user/tool_result -> message_start(id=exec-N-xxx, reuse!) + ChunkExecute{status:completed, output:...} + message_end
//
// For parallel tool calls, multiple tools may be in-flight simultaneously.
// The tools buffer keeps each tool's state until its tool_result arrives.
type streamParser struct {
handler message.StreamFunc
completed bool
textActive bool
toolIndex int
curTool *toolState
textActive bool
toolIndex int
activeToolID string // tool currently receiving content_block_delta
tools map[string]*toolState // tool_id -> buffered tool state
toolNames map[string]string // tool_id -> tool_name
toolMsgIDs map[string]string // tool_id -> message_id (for result reuse)
@ -47,6 +52,7 @@ type toolState struct {
func newStreamParser(handler message.StreamFunc) *streamParser {
return &streamParser{
handler: handler,
tools: make(map[string]*toolState),
toolNames: make(map[string]string),
toolMsgIDs: make(map[string]string),
toolInputs: make(map[string]string),
@ -54,6 +60,13 @@ func newStreamParser(handler message.StreamFunc) *streamParser {
}
}
func (p *streamParser) activeTool() *toolState {
if p.activeToolID == "" {
return nil
}
return p.tools[p.activeToolID]
}
func (p *streamParser) parse(ctx context.Context, stdout io.ReadCloser) error {
doneParsing := make(chan struct{})
defer close(doneParsing)
@ -65,8 +78,7 @@ func (p *streamParser) parse(ctx context.Context, stdout io.ReadCloser) error {
}
}()
scanner := bufio.NewScanner(stdout)
scanner.Buffer(make([]byte, 0, 64*1024), 10*1024*1024)
reader := bufio.NewReaderSize(stdout, 64*1024)
startTime := time.Now()
lineCount := 0
@ -75,17 +87,30 @@ func (p *streamParser) parse(ctx context.Context, stdout io.ReadCloser) error {
log.Trace("[claude-parse] stream started")
for scanner.Scan() {
line := scanner.Text()
if line == "" {
for {
line, skipped, err := shared.ReadJSONLine(reader)
if err == io.EOF {
break
}
if err != nil {
if ctx.Err() != nil {
return ctx.Err()
}
return err
}
if skipped {
log.Warn("[claude-parse] skipped oversized JSONL line (>%dMB)", shared.MaxLineSize/1024/1024)
continue
}
if len(line) == 0 {
continue
}
lineCount++
if time.Since(lastHeartbeat) > 30*time.Second {
builderLen := 0
if p.curTool != nil {
builderLen = p.curTool.inputJSON.Len()
if t := p.activeTool(); t != nil {
builderLen = t.inputJSON.Len()
}
log.Trace("[claude-parse] heartbeat: lines=%d elapsed=%v lastEvent=%s toolBuilderLen=%d",
lineCount, time.Since(startTime).Round(time.Second), lastEventType, builderLen)
@ -93,11 +118,11 @@ func (p *streamParser) parse(ctx context.Context, stdout io.ReadCloser) error {
}
var msg map[string]any
if err := json.Unmarshal([]byte(line), &msg); err != nil {
if err := json.Unmarshal(line, &msg); err != nil {
if len(line) > 200 {
log.Trace("[claude-parse] JSON unmarshal error: %v (line len=%d, prefix=%q)", err, len(line), line[:200])
log.Trace("[claude-parse] JSON unmarshal error: %v (line len=%d, prefix=%q)", err, len(line), string(line[:200]))
} else {
log.Trace("[claude-parse] JSON unmarshal error: %v (line=%q)", err, line)
log.Trace("[claude-parse] JSON unmarshal error: %v (line=%q)", err, string(line))
}
continue
}
@ -129,16 +154,9 @@ func (p *streamParser) parse(ctx context.Context, stdout io.ReadCloser) error {
}
}
log.Trace("[claude-parse] stream ended: lines=%d elapsed=%v completed=%v scanErr=%v",
lineCount, time.Since(startTime).Round(time.Second), p.completed, scanner.Err())
log.Trace("[claude-parse] stream ended: lines=%d elapsed=%v completed=%v",
lineCount, time.Since(startTime).Round(time.Second), p.completed)
if err := scanner.Err(); err != nil {
log.Trace("[claude-parse] scanner error: %v (ctx.Err=%v)", err, ctx.Err())
if ctx.Err() != nil {
return ctx.Err()
}
return err
}
return nil
}
@ -172,28 +190,52 @@ func (p *streamParser) closeTextMessage() {
}
}
// closeCurrentTool closes the in-flight streaming tool message (if any),
// flushing its accumulated input and emitting message_end. This must be
// called before opening a new message group so that the downstream handler
// never sees interleaved message_start/message_end pairs.
func (p *streamParser) closeCurrentTool() {
if p.curTool == nil {
// closeStreamingTool closes the currently streaming tool's message group,
// flushing accumulated input and emitting message_end. The tool remains
// in p.tools so handleUser can later reuse its msgID for the completed phase.
func (p *streamParser) closeStreamingTool() {
t := p.activeTool()
if t == nil {
return
}
toolID := p.curTool.id
inputStr := p.curTool.inputJSON.String()
inputStr := t.inputJSON.String()
if inputStr != "" {
p.toolInputs[toolID] = inputStr
summary := extractSummary(p.curTool.name, inputStr)
p.toolInputs[t.id] = inputStr
summary := extractSummary(t.name, inputStr)
if summary != "" {
p.toolSummaries[toolID] = summary
p.toolSummaries[t.id] = summary
p.emitExecute(map[string]any{
"summary": summary,
})
}
}
p.endMessage()
p.curTool = nil
p.activeToolID = ""
}
// suspendStreamingTool temporarily closes the active tool's message group
// (emits message_end) so another message group can be opened. The tool
// remains in p.tools and p.activeToolID is cleared. Call resumeStreamingTool
// to reopen it.
func (p *streamParser) suspendStreamingTool() {
t := p.activeTool()
if t == nil {
return
}
p.endMessage()
p.activeToolID = ""
}
// resumeStreamingTool reopens a previously suspended tool's message group
// by emitting a new message_start with the same msgID, and restores it as
// the active streaming tool.
func (p *streamParser) resumeStreamingTool(toolID string) {
t, ok := p.tools[toolID]
if !ok {
return
}
p.beginMessageWithID(t.msgID, "execute")
p.activeToolID = toolID
}
func (p *streamParser) ensureTextMessage() (stopped bool) {
@ -255,7 +297,6 @@ func extractSummary(toolName string, inputJSON string) string {
}
}
// Fallback: try common field names
for _, key := range []string{"path", "file_path", "command", "url", "query"} {
if v, ok := obj[key].(string); ok {
return truncate(v, 80)
@ -324,7 +365,9 @@ func (p *streamParser) onContentBlockStart(event map[string]any) (stopped bool)
return true
}
p.curTool = &toolState{id: toolID, name: toolName, msgID: msgID, index: p.toolIndex}
ts := &toolState{id: toolID, name: toolName, msgID: msgID, index: p.toolIndex}
p.tools[toolID] = ts
p.activeToolID = toolID
p.toolIndex++
p.toolNames[toolID] = toolName
p.toolMsgIDs[toolID] = msgID
@ -342,7 +385,7 @@ func (p *streamParser) onContentBlockStart(event map[string]any) (stopped bool)
}
func (p *streamParser) onContentBlockStop() (stopped bool) {
p.closeCurrentTool()
p.closeStreamingTool()
return false
}
@ -360,9 +403,6 @@ func (p *streamParser) onContentBlockDelta(event map[string]any) (stopped bool)
if text == "" {
return false
}
// If there is no active text message and this delta is only
// whitespace, buffer it instead of opening a brand-new message
// group just for spaces/indentation between tool calls.
if !p.textActive && strings.TrimSpace(text) == "" {
return false
}
@ -372,21 +412,22 @@ func (p *streamParser) onContentBlockDelta(event map[string]any) (stopped bool)
return p.emitText(text)
case "input_json_delta":
if p.curTool == nil {
t := p.activeTool()
if t == nil {
return false
}
partial, _ := delta["partial_json"].(string)
if partial == "" {
return false
}
p.curTool.inputJSON.WriteString(partial)
builderLen := p.curTool.inputJSON.Len()
t.inputJSON.WriteString(partial)
builderLen := t.inputJSON.Len()
if builderLen > 0 && builderLen%100000 < len(partial) {
log.Trace("[claude-parse] WARN: tool %s inputJSON growing: %d bytes", p.curTool.name, builderLen)
log.Trace("[claude-parse] WARN: tool %s inputJSON growing: %d bytes", t.name, builderLen)
}
if p.handler != nil {
return p.emitExecute(map[string]any{
"input_delta": p.curTool.inputJSON.String(),
"input_delta": t.inputJSON.String(),
})
}
}
@ -426,7 +467,7 @@ func (p *streamParser) handleAssistant(msg map[string]any) (stopped bool) {
}
p.closeTextMessage()
p.closeCurrentTool()
p.closeStreamingTool()
toolName, _ := ci["name"].(string)
if toolID == "" {
@ -498,18 +539,25 @@ func (p *streamParser) handleUser(msg map[string]any) (stopped bool) {
continue
}
// Close any open text message before opening an execute message.
toolUseID, _ := ci["tool_use_id"].(string)
// If the result belongs to the actively streaming tool, close its
// streaming phase (emits message_end for the running group).
if p.activeToolID == toolUseID {
p.closeStreamingTool()
}
// If a DIFFERENT tool is currently streaming, we must suspend its
// message group before opening the completed-result group, because
// the downstream handler only tracks one currentGroupID at a time.
suspendedToolID := ""
if p.activeToolID != "" && p.activeToolID != toolUseID {
suspendedToolID = p.activeToolID
p.suspendStreamingTool()
}
p.closeTextMessage()
// When Claude CLI executes tools in parallel, tool_result messages
// can arrive while a new tool_use is still streaming. The downstream
// handler (stream.go) tracks only a single currentGroupID, so we
// must close the in-flight streaming tool message before opening
// the result message — otherwise the message_start/message_end
// pairs become interleaved and chunks lose their message_id.
p.closeCurrentTool()
toolUseID, _ := ci["tool_use_id"].(string)
content := ci["content"]
isError, _ := ci["is_error"].(bool)
@ -549,6 +597,14 @@ func (p *streamParser) handleUser(msg map[string]any) (stopped bool) {
return true
}
p.endMessage()
delete(p.tools, toolUseID)
// Resume the suspended tool's message group so subsequent
// content_block_delta events land in the correct group.
if suspendedToolID != "" {
p.resumeStreamingTool(suspendedToolID)
}
}
return false
}

View file

@ -6,625 +6,411 @@ import (
"io"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/yaoapp/yao/agent/output/message"
)
type chunkRecord struct {
Type message.StreamChunkType
Data json.RawMessage
// recordedEvent captures a single StreamFunc callback invocation.
type recordedEvent struct {
chunkType message.StreamChunkType
data []byte
}
func recordingHandler(out *[]chunkRecord) message.StreamFunc {
func mockStreamFunc(events *[]recordedEvent) message.StreamFunc {
return func(chunkType message.StreamChunkType, data []byte) int {
cp := make([]byte, len(data))
copy(cp, data)
*out = append(*out, chunkRecord{Type: chunkType, Data: cp})
*events = append(*events, recordedEvent{chunkType: chunkType, data: cp})
return 0
}
}
func stoppingHandler(stopAfter int) (message.StreamFunc, *[]chunkRecord) {
var out []chunkRecord
count := 0
fn := func(chunkType message.StreamChunkType, data []byte) int {
cp := make([]byte, len(data))
copy(cp, data)
out = append(out, chunkRecord{Type: chunkType, Data: cp})
count++
if count >= stopAfter {
return 1
}
return 0
}
return fn, &out
}
func jsonLine(v interface{}) string {
b, _ := json.Marshal(v)
return string(b)
}
func pipeWithLines(lines ...string) io.ReadCloser {
return io.NopCloser(strings.NewReader(strings.Join(lines, "\n") + "\n"))
}
// --- helper: extract message_id from ChunkMessageStart data ---
func extractMessageID(data json.RawMessage) string {
var d map[string]any
json.Unmarshal(data, &d)
if id, ok := d["message_id"].(string); ok {
return id
}
return ""
}
func TestParser_TextOnly(t *testing.T) {
lines := []string{
jsonLine(map[string]any{"type": "system", "session_id": "abc"}),
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{
"type": "content_block_delta",
"delta": map[string]any{"type": "text_delta", "text": "Hello "},
},
}),
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{
"type": "content_block_delta",
"delta": map[string]any{"type": "text_delta", "text": "world"},
},
}),
jsonLine(map[string]any{
"type": "assistant",
"message": map[string]any{
"stop_reason": "end_turn",
"content": []any{map[string]any{"type": "text", "text": "Hello world"}},
},
}),
jsonLine(map[string]any{
"type": "result",
"total_cost_usd": 0.001,
"duration_ms": 1234,
"num_turns": 1,
"usage": map[string]any{"input_tokens": 10, "output_tokens": 20},
}),
}
var chunks []chunkRecord
p := newStreamParser(recordingHandler(&chunks))
err := p.parse(context.Background(), pipeWithLines(lines...))
require.NoError(t, err)
assert.True(t, p.completed)
hasMessageStart := false
hasText := false
hasMessageEnd := false
hasResultMeta := false
for _, c := range chunks {
switch c.Type {
case message.ChunkMessageStart:
hasMessageStart = true
case message.ChunkText:
hasText = true
case message.ChunkMessageEnd:
hasMessageEnd = true
case message.ChunkMetadata:
var meta map[string]any
json.Unmarshal(c.Data, &meta)
if _, ok := meta["result_summary"]; ok {
hasResultMeta = true
}
}
}
assert.True(t, hasMessageStart, "should emit message_start")
assert.True(t, hasText, "should emit text chunks")
assert.True(t, hasMessageEnd, "should emit message_end")
assert.True(t, hasResultMeta, "should emit result_summary metadata")
}
func TestParser_ToolUseAndResult(t *testing.T) {
lines := []string{
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{
"type": "content_block_start",
"content_block": map[string]any{
"type": "tool_use",
"name": "Bash",
"id": "tool_123",
},
},
}),
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{
"type": "content_block_delta",
"delta": map[string]any{"type": "input_json_delta", "partial_json": `{"command":"ls`},
},
}),
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{
"type": "content_block_delta",
"delta": map[string]any{"type": "input_json_delta", "partial_json": `"}`},
},
}),
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{"type": "content_block_stop"},
}),
jsonLine(map[string]any{
"type": "user",
"message": map[string]any{
"content": []any{
map[string]any{
"type": "tool_result",
"tool_use_id": "tool_123",
"content": "file1.txt\nfile2.txt",
"is_error": false,
},
},
},
}),
jsonLine(map[string]any{
"type": "result",
"num_turns": 1,
}),
}
var chunks []chunkRecord
p := newStreamParser(recordingHandler(&chunks))
err := p.parse(context.Background(), pipeWithLines(lines...))
require.NoError(t, err)
assert.True(t, p.completed)
var execChunks []map[string]any
for _, c := range chunks {
if c.Type == message.ChunkExecute {
var data map[string]any
json.Unmarshal(c.Data, &data)
execChunks = append(execChunks, data)
}
}
require.GreaterOrEqual(t, len(execChunks), 2, "should have at least 2 execute chunks (start + result)")
assert.Equal(t, "Bash", execChunks[0]["tool"])
assert.Equal(t, "tool_123", execChunks[0]["tool_id"])
assert.Equal(t, "running", execChunks[0]["status"])
lastExec := execChunks[len(execChunks)-1]
assert.Equal(t, "tool_123", lastExec["tool_id"])
assert.Equal(t, "completed", lastExec["status"])
assert.Equal(t, "Bash", lastExec["tool"], "tool_result should carry tool name")
}
// TestParser_ToolIndependentMessages verifies that each tool call gets its own
// message_start/message_end pair, and tool_result reuses the same message_id.
func TestParser_ToolIndependentMessages(t *testing.T) {
lines := []string{
// Tool 1: Write
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{
"type": "content_block_start",
"content_block": map[string]any{"type": "tool_use", "name": "Write", "id": "t_write"},
},
}),
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{
"type": "content_block_delta",
"delta": map[string]any{"type": "input_json_delta", "partial_json": `{"file_path":"server.js"}`},
},
}),
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{"type": "content_block_stop"},
}),
// Tool 2: Bash
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{
"type": "content_block_start",
"content_block": map[string]any{"type": "tool_use", "name": "Bash", "id": "t_bash"},
},
}),
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{"type": "content_block_stop"},
}),
// Results
jsonLine(map[string]any{
"type": "user",
"message": map[string]any{
"content": []any{
map[string]any{"type": "tool_result", "tool_use_id": "t_write", "content": "ok"},
map[string]any{"type": "tool_result", "tool_use_id": "t_bash", "content": "done"},
},
},
}),
jsonLine(map[string]any{"type": "result", "num_turns": 1}),
}
var chunks []chunkRecord
p := newStreamParser(recordingHandler(&chunks))
err := p.parse(context.Background(), pipeWithLines(lines...))
require.NoError(t, err)
// Collect all message_start IDs and their order
var msgStarts []string
for _, c := range chunks {
if c.Type == message.ChunkMessageStart {
msgStarts = append(msgStarts, extractMessageID(c.Data))
}
}
// Should have 4 message_starts: Write(running), Bash(running), Write(result), Bash(result)
require.Equal(t, 4, len(msgStarts), "should have 4 message_start events")
writeMsgID := msgStarts[0]
bashMsgID := msgStarts[1]
assert.NotEqual(t, writeMsgID, bashMsgID, "Write and Bash should have different message_ids")
// tool_result should reuse the original message_id
assert.Equal(t, writeMsgID, msgStarts[2], "Write tool_result should reuse Write message_id")
assert.Equal(t, bashMsgID, msgStarts[3], "Bash tool_result should reuse Bash message_id")
// Count message_end events (should match message_start)
endCount := 0
for _, c := range chunks {
if c.Type == message.ChunkMessageEnd {
endCount++
}
}
assert.Equal(t, 4, endCount, "each message_start should have matching message_end")
}
// TestParser_ToolSummaryExtraction verifies that summary is extracted from tool input.
func TestParser_ToolSummaryExtraction(t *testing.T) {
lines := []string{
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{
"type": "content_block_start",
"content_block": map[string]any{"type": "tool_use", "name": "Bash", "id": "t1"},
},
}),
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{
"type": "content_block_delta",
"delta": map[string]any{"type": "input_json_delta", "partial_json": `{"command":"ls -la /workspace"}`},
},
}),
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{"type": "content_block_stop"},
}),
jsonLine(map[string]any{"type": "result", "num_turns": 1}),
}
var chunks []chunkRecord
p := newStreamParser(recordingHandler(&chunks))
err := p.parse(context.Background(), pipeWithLines(lines...))
require.NoError(t, err)
// Look for summary in the last execute chunk before message_end
var summaryFound bool
for _, c := range chunks {
if c.Type == message.ChunkExecute {
var data map[string]any
json.Unmarshal(c.Data, &data)
if s, ok := data["summary"].(string); ok && s != "" {
summaryFound = true
assert.Equal(t, "ls -la /workspace", s)
}
}
}
assert.True(t, summaryFound, "should emit summary from tool input")
}
func TestParser_UsageMetadata(t *testing.T) {
lines := []string{
jsonLine(map[string]any{
"type": "assistant",
"message": map[string]any{
"usage": map[string]any{
"input_tokens": 100,
"output_tokens": 50,
},
"stop_reason": "end_turn",
"content": []any{},
},
}),
jsonLine(map[string]any{"type": "result", "num_turns": 1}),
}
var chunks []chunkRecord
p := newStreamParser(recordingHandler(&chunks))
err := p.parse(context.Background(), pipeWithLines(lines...))
require.NoError(t, err)
var usageMeta map[string]any
for _, c := range chunks {
if c.Type == message.ChunkMetadata {
var meta map[string]any
json.Unmarshal(c.Data, &meta)
if u, ok := meta["usage"]; ok {
usageMeta, _ = u.(map[string]any)
}
}
}
require.NotNil(t, usageMeta, "should emit usage metadata")
assert.Equal(t, float64(100), usageMeta["input_tokens"])
assert.Equal(t, float64(50), usageMeta["output_tokens"])
}
func TestParser_ResultSummary(t *testing.T) {
lines := []string{
jsonLine(map[string]any{
"type": "result",
"total_cost_usd": 0.05,
"duration_ms": 5000,
"num_turns": 3,
"usage": map[string]any{"input_tokens": 500, "output_tokens": 200},
}),
}
var chunks []chunkRecord
p := newStreamParser(recordingHandler(&chunks))
err := p.parse(context.Background(), pipeWithLines(lines...))
require.NoError(t, err)
assert.True(t, p.completed)
var summary map[string]any
for _, c := range chunks {
if c.Type == message.ChunkMetadata {
var meta map[string]any
json.Unmarshal(c.Data, &meta)
if s, ok := meta["result_summary"]; ok {
summary, _ = s.(map[string]any)
}
}
}
require.NotNil(t, summary, "should emit result_summary")
assert.Equal(t, float64(0.05), summary["total_cost_usd"])
assert.Equal(t, float64(5000), summary["duration_ms"])
assert.Equal(t, float64(3), summary["num_turns"])
}
func TestParser_ErrorMessage(t *testing.T) {
lines := []string{
jsonLine(map[string]any{
"type": "error",
"error": map[string]any{"message": "rate limit exceeded"},
}),
}
var chunks []chunkRecord
p := newStreamParser(recordingHandler(&chunks))
err := p.parse(context.Background(), pipeWithLines(lines...))
require.Error(t, err)
assert.Contains(t, err.Error(), "rate limit exceeded")
assert.False(t, p.completed)
hasError := false
for _, c := range chunks {
if c.Type == message.ChunkError {
hasError = true
assert.Contains(t, string(c.Data), "rate limit exceeded")
}
}
assert.True(t, hasError, "should emit error chunk")
}
func TestParser_ResultIsError(t *testing.T) {
lines := []string{
jsonLine(map[string]any{
"type": "result",
"is_error": true,
"result": "authentication failed",
}),
}
var chunks []chunkRecord
p := newStreamParser(recordingHandler(&chunks))
err := p.parse(context.Background(), pipeWithLines(lines...))
require.Error(t, err)
assert.Contains(t, err.Error(), "authentication failed")
}
func TestParser_ContextCancel(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
// runParser feeds JSONL lines to a streamParser and returns all recorded events.
func runParser(t *testing.T, jsonl string) []recordedEvent {
t.Helper()
var events []recordedEvent
p := newStreamParser(mockStreamFunc(&events))
r, w := io.Pipe()
go func() {
time.Sleep(50 * time.Millisecond)
cancel()
defer w.Close()
w.Write([]byte(jsonl))
}()
p := newStreamParser(nil)
err := p.parse(ctx, r)
_ = w.Close()
require.Error(t, err)
assert.ErrorIs(t, err, context.Canceled)
if err := p.parse(context.Background(), r); err != nil {
t.Fatalf("parse error: %v", err)
}
return events
}
func TestParser_HandlerStopsStream(t *testing.T) {
lines := []string{
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{
"type": "content_block_delta",
"delta": map[string]any{"type": "text_delta", "text": "first"},
},
}),
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{
"type": "content_block_delta",
"delta": map[string]any{"type": "text_delta", "text": "second"},
},
}),
}
// --- helpers to inspect recorded events ---
handler, chunks := stoppingHandler(2)
p := newStreamParser(handler)
err := p.parse(context.Background(), pipeWithLines(lines...))
require.NoError(t, err)
assert.LessOrEqual(t, len(*chunks), 3, "should stop early")
type messageGroup struct {
messageID string
events []recordedEvent
}
func TestParser_EmptyAndInvalidLines(t *testing.T) {
lines := []string{
"",
"not json at all",
" ",
`{"type": "result", "num_turns": 1}`,
// extractMessageGroups splits the flat event list into message_start..message_end
// groups, each carrying the messageID from the start event.
func extractMessageGroups(events []recordedEvent) []messageGroup {
var groups []messageGroup
var cur *messageGroup
for _, ev := range events {
if ev.chunkType == message.ChunkMessageStart {
var sd message.EventMessageStartData
json.Unmarshal(ev.data, &sd)
cur = &messageGroup{messageID: sd.MessageID}
}
if cur != nil {
cur.events = append(cur.events, ev)
}
if ev.chunkType == message.ChunkMessageEnd {
if cur != nil {
groups = append(groups, *cur)
cur = nil
}
}
}
var chunks []chunkRecord
p := newStreamParser(recordingHandler(&chunks))
err := p.parse(context.Background(), pipeWithLines(lines...))
require.NoError(t, err)
assert.True(t, p.completed, "should complete despite invalid lines")
return groups
}
func TestParser_MultiTurnConversation(t *testing.T) {
lines := []string{
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{
"type": "content_block_start",
"content_block": map[string]any{"type": "tool_use", "name": "Read", "id": "t1"},
},
}),
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{"type": "content_block_stop"},
}),
jsonLine(map[string]any{
"type": "assistant",
"message": map[string]any{
"stop_reason": "tool_use",
"content": []any{
map[string]any{"type": "tool_use", "name": "Read", "id": "t1", "input": map[string]any{"path": "/tmp"}},
},
},
}),
jsonLine(map[string]any{
"type": "user",
"message": map[string]any{
"content": []any{
map[string]any{"type": "tool_result", "tool_use_id": "t1", "content": "ok"},
},
},
}),
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{
"type": "content_block_delta",
"delta": map[string]any{"type": "text_delta", "text": "Done reading"},
},
}),
jsonLine(map[string]any{
"type": "assistant",
"message": map[string]any{
"stop_reason": "end_turn",
"content": []any{map[string]any{"type": "text", "text": "Done reading"}},
},
}),
jsonLine(map[string]any{"type": "result", "num_turns": 2}),
}
func extractExecuteProps(ev recordedEvent) map[string]any {
var m map[string]any
json.Unmarshal(ev.data, &m)
return m
}
var chunks []chunkRecord
p := newStreamParser(recordingHandler(&chunks))
err := p.parse(context.Background(), pipeWithLines(lines...))
require.NoError(t, err)
assert.True(t, p.completed)
// --- Mock JSONL data based on real terminal logs ---
startCount := 0
endCount := 0
for _, c := range chunks {
switch c.Type {
// parallelToolCallJSONL reproduces the exact interleaved event sequence from
// the production log: two Bash tool calls (pip --version + npm --version) where
// tool0's tool_result arrives while tool1 is still streaming content_block_delta.
const parallelToolCallJSONL = `{"type":"stream_event","event":{"type":"message_start","message":{"id":"msg_mock","type":"message","role":"assistant","content":[],"model":"claude-sonnet-4-20250514","stop_reason":null,"usage":{"input_tokens":100,"output_tokens":1}}}}
{"type":"stream_event","event":{"type":"content_block_start","index":0,"content_block":{"type":"tool_use","id":"toolu_pip","name":"Bash"}}}
{"type":"stream_event","event":{"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"{\"command\""}}}
{"type":"stream_event","event":{"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":":\"pip --version\"}"}}}
{"type":"assistant","message":{"id":"msg_mock","type":"message","role":"assistant","content":[{"type":"tool_use","id":"toolu_pip","name":"Bash","input":{"command":"pip --version"}}],"model":"claude-sonnet-4-20250514","stop_reason":null,"usage":{"input_tokens":100,"output_tokens":30}}}
{"type":"stream_event","event":{"type":"content_block_stop","index":0}}
{"type":"stream_event","event":{"type":"content_block_start","index":1,"content_block":{"type":"tool_use","id":"toolu_npm","name":"Bash"}}}
{"type":"stream_event","event":{"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":"{\"command\""}}}
{"type":"user","message":{"id":"msg_user_pip","type":"message","role":"user","content":[{"type":"tool_result","tool_use_id":"toolu_pip","content":"pip 24.0 (python 3.12)"}]}}
{"type":"stream_event","event":{"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":":\"npm --version\"}"}}}
{"type":"assistant","message":{"id":"msg_mock2","type":"message","role":"assistant","content":[{"type":"tool_use","id":"toolu_npm","name":"Bash","input":{"command":"npm --version"}}],"model":"claude-sonnet-4-20250514","stop_reason":null,"usage":{"input_tokens":200,"output_tokens":60}}}
{"type":"stream_event","event":{"type":"content_block_stop","index":1}}
{"type":"stream_event","event":{"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":60}}}
{"type":"stream_event","event":{"type":"message_stop"}}
{"type":"user","message":{"id":"msg_user_npm","type":"message","role":"user","content":[{"type":"tool_result","tool_use_id":"toolu_npm","content":"10.9.7"}]}}
{"type":"stream_event","event":{"type":"message_start","message":{"id":"msg_final","type":"message","role":"assistant","content":[],"model":"claude-sonnet-4-20250514","stop_reason":null,"usage":{"input_tokens":300,"output_tokens":1}}}}
{"type":"stream_event","event":{"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}}
{"type":"stream_event","event":{"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"pip 和 npm 都可以正常使用。"}}}
{"type":"assistant","message":{"id":"msg_final","type":"message","role":"assistant","content":[{"type":"text","text":"pip 和 npm 都可以正常使用。"}],"model":"claude-sonnet-4-20250514","stop_reason":"end_turn","usage":{"input_tokens":300,"output_tokens":20}}}
{"type":"stream_event","event":{"type":"content_block_stop","index":0}}
{"type":"stream_event","event":{"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":20}}}
{"type":"stream_event","event":{"type":"message_stop"}}
{"type":"result","result":"pip 和 npm 都可以正常使用。","is_error":false}
`
const singleToolCallJSONL = `{"type":"stream_event","event":{"type":"message_start","message":{"id":"msg_s","type":"message","role":"assistant","content":[],"model":"claude-sonnet-4-20250514","stop_reason":null,"usage":{"input_tokens":50,"output_tokens":1}}}}
{"type":"stream_event","event":{"type":"content_block_start","index":0,"content_block":{"type":"tool_use","id":"toolu_single","name":"Bash"}}}
{"type":"stream_event","event":{"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"{\"command\":\"ls\"}"}}}
{"type":"assistant","message":{"id":"msg_s","type":"message","role":"assistant","content":[{"type":"tool_use","id":"toolu_single","name":"Bash","input":{"command":"ls"}}],"model":"claude-sonnet-4-20250514","stop_reason":null,"usage":{"input_tokens":50,"output_tokens":10}}}
{"type":"stream_event","event":{"type":"content_block_stop","index":0}}
{"type":"stream_event","event":{"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":10}}}
{"type":"stream_event","event":{"type":"message_stop"}}
{"type":"user","message":{"id":"msg_u","type":"message","role":"user","content":[{"type":"tool_result","tool_use_id":"toolu_single","content":"file1.txt\nfile2.txt"}]}}
{"type":"stream_event","event":{"type":"message_start","message":{"id":"msg_s2","type":"message","role":"assistant","content":[],"model":"claude-sonnet-4-20250514","stop_reason":null,"usage":{"input_tokens":80,"output_tokens":1}}}}
{"type":"stream_event","event":{"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}}
{"type":"stream_event","event":{"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Done."}}}
{"type":"stream_event","event":{"type":"content_block_stop","index":0}}
{"type":"stream_event","event":{"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":5}}}
{"type":"stream_event","event":{"type":"message_stop"}}
{"type":"result","result":"Done.","is_error":false}
`
// ---------- Test 1: message_start / message_end pairing ----------
func TestParseParallelToolCalls_MessagePairing(t *testing.T) {
events := runParser(t, parallelToolCallJSONL)
var startCount, endCount int
depth := 0
for _, ev := range events {
switch ev.chunkType {
case message.ChunkMessageStart:
startCount++
depth++
if depth > 1 {
t.Fatalf("nested message_start detected (depth %d) — message_start/message_end not strictly paired", depth)
}
case message.ChunkMessageEnd:
endCount++
}
}
// streaming tool_use(start) + streaming tool_use(stop) + assistant tool_use + tool_result + text = 5 starts
assert.GreaterOrEqual(t, startCount, 3, "should have multiple message starts for multi-turn")
assert.Equal(t, startCount, endCount, "each message_start should have matching message_end")
}
func TestParser_ErrorStringFormat(t *testing.T) {
lines := []string{
jsonLine(map[string]any{
"type": "error",
"error": "simple string error",
}),
}
var chunks []chunkRecord
p := newStreamParser(recordingHandler(&chunks))
err := p.parse(context.Background(), pipeWithLines(lines...))
require.Error(t, err)
assert.Contains(t, err.Error(), "simple string error")
}
func TestParser_InputDeltaAccumulation(t *testing.T) {
lines := []string{
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{
"type": "content_block_start",
"content_block": map[string]any{"type": "tool_use", "name": "Bash", "id": "t1"},
},
}),
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{
"type": "content_block_delta",
"delta": map[string]any{"type": "input_json_delta", "partial_json": `{"com`},
},
}),
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{
"type": "content_block_delta",
"delta": map[string]any{"type": "input_json_delta", "partial_json": `mand":"ls"}`},
},
}),
jsonLine(map[string]any{
"type": "stream_event",
"event": map[string]any{"type": "content_block_stop"},
}),
jsonLine(map[string]any{"type": "result", "num_turns": 1}),
}
var chunks []chunkRecord
p := newStreamParser(recordingHandler(&chunks))
err := p.parse(context.Background(), pipeWithLines(lines...))
require.NoError(t, err)
// The last input_delta chunk should contain the full accumulated input
var lastDelta string
for _, c := range chunks {
if c.Type == message.ChunkExecute {
var data map[string]any
json.Unmarshal(c.Data, &data)
if d, ok := data["input_delta"].(string); ok {
lastDelta = d
depth--
if depth < 0 {
t.Fatal("message_end without preceding message_start")
}
}
}
assert.Equal(t, `{"command":"ls"}`, lastDelta, "input_delta should accumulate all fragments")
if startCount != endCount {
t.Fatalf("message_start count (%d) != message_end count (%d)", startCount, endCount)
}
// 4 execute phases (pip running, npm running, pip completed, npm completed) + 1 text = 5
if startCount < 5 {
t.Errorf("expected at least 5 message groups, got %d", startCount)
}
}
// ---------- Test 2: each tool has running + completed lifecycle ----------
func TestParseParallelToolCalls_ToolLifecycle(t *testing.T) {
events := runParser(t, parallelToolCallJSONL)
groups := extractMessageGroups(events)
// Collect execute groups by message_id
type toolPhase struct {
messageID string
status string
tool string
output any
}
var phases []toolPhase
for _, g := range groups {
for _, ev := range g.events {
if ev.chunkType == message.ChunkExecute {
props := extractExecuteProps(ev)
status, _ := props["status"].(string)
if status != "" {
phases = append(phases, toolPhase{
messageID: g.messageID,
status: status,
tool: strDefault(props["tool"]),
output: props["output"],
})
}
}
}
}
// Find pip and npm phases
var pipRunning, pipCompleted, npmRunning, npmCompleted *toolPhase
for i := range phases {
p := &phases[i]
if p.tool == "Bash" && p.status == "running" && pipRunning == nil {
pipRunning = p
} else if p.tool == "Bash" && p.status == "running" && npmRunning == nil {
npmRunning = p
}
out := strDefault(p.output)
if p.status == "completed" && strings.Contains(out, "pip") {
pipCompleted = p
}
if p.status == "completed" && strings.Contains(out, "10.9.7") {
npmCompleted = p
}
}
if pipRunning == nil {
t.Fatal("pip running phase not found")
}
if pipCompleted == nil {
t.Fatal("pip completed phase not found")
}
if npmRunning == nil {
t.Fatal("npm running phase not found")
}
if npmCompleted == nil {
t.Fatal("npm completed phase not found")
}
// running and completed phases must share message_id (reuse)
if pipRunning.messageID != pipCompleted.messageID {
t.Errorf("pip: running message_id %q != completed message_id %q", pipRunning.messageID, pipCompleted.messageID)
}
if npmRunning.messageID != npmCompleted.messageID {
t.Errorf("npm: running message_id %q != completed message_id %q", npmRunning.messageID, npmCompleted.messageID)
}
// two tools should have different message_ids
if pipRunning.messageID == npmRunning.messageID {
t.Error("pip and npm should have different message_ids")
}
}
// ---------- Test 3: interleaved result does not corrupt message_id ----------
func TestParseParallelToolCalls_InterleavedResult(t *testing.T) {
events := runParser(t, parallelToolCallJSONL)
groups := extractMessageGroups(events)
// Find the npm running group (the one with tool_id toolu_npm or second Bash running)
var npmRunningGroup *messageGroup
bashRunningCount := 0
for i := range groups {
for _, ev := range groups[i].events {
if ev.chunkType == message.ChunkExecute {
props := extractExecuteProps(ev)
if props["status"] == "running" && props["tool"] == "Bash" {
bashRunningCount++
if bashRunningCount == 2 {
npmRunningGroup = &groups[i]
}
}
}
}
}
if npmRunningGroup == nil {
t.Fatal("npm running message group not found")
}
// The npm running group must have a non-empty message_id
if npmRunningGroup.messageID == "" {
t.Fatal("npm running group has empty message_id")
}
// All execute chunks in the npm running group must have input_delta data
// (the streaming deltas must be present, proving they weren't lost)
var hasDelta bool
for _, ev := range npmRunningGroup.events {
if ev.chunkType == message.ChunkExecute {
props := extractExecuteProps(ev)
if _, ok := props["input_delta"]; ok {
hasDelta = true
}
}
}
if !hasDelta {
t.Error("npm running group has no input_delta chunks — streaming was interrupted")
}
// pip completed group must not contain npm delta events:
// find pip completed group
var pipCompletedGroup *messageGroup
for i := range groups {
for _, ev := range groups[i].events {
if ev.chunkType == message.ChunkExecute {
props := extractExecuteProps(ev)
out := strDefault(props["output"])
if props["status"] == "completed" && strings.Contains(out, "pip") {
pipCompletedGroup = &groups[i]
}
}
}
}
if pipCompletedGroup == nil {
t.Fatal("pip completed group not found")
}
// pip completed group should only contain its own events, not npm deltas
for _, ev := range pipCompletedGroup.events {
if ev.chunkType == message.ChunkExecute {
props := extractExecuteProps(ev)
if _, ok := props["input_delta"]; ok {
t.Error("pip completed group contains input_delta — npm delta leaked into pip group")
}
}
}
}
// ---------- Test 4: final text after all tools complete ----------
func TestParseParallelToolCalls_FinalText(t *testing.T) {
events := runParser(t, parallelToolCallJSONL)
var textChunks []string
inTextGroup := false
for _, ev := range events {
if ev.chunkType == message.ChunkMessageStart {
var sd message.EventMessageStartData
json.Unmarshal(ev.data, &sd)
inTextGroup = sd.Type == "text"
}
if ev.chunkType == message.ChunkText && inTextGroup {
textChunks = append(textChunks, string(ev.data))
}
if ev.chunkType == message.ChunkMessageEnd {
inTextGroup = false
}
}
fullText := strings.Join(textChunks, "")
if !strings.Contains(fullText, "pip") || !strings.Contains(fullText, "npm") {
t.Errorf("final text should mention pip and npm, got %q", fullText)
}
}
// ---------- Test 5: single tool call regression ----------
func TestParseSingleToolCall(t *testing.T) {
events := runParser(t, singleToolCallJSONL)
groups := extractMessageGroups(events)
// Should have: 1 running group + 1 completed group + 1 text group = 3
if len(groups) < 3 {
t.Fatalf("expected at least 3 message groups for single tool call, got %d", len(groups))
}
// Verify message_start / message_end pairing
depth := 0
for _, ev := range events {
switch ev.chunkType {
case message.ChunkMessageStart:
depth++
if depth > 1 {
t.Fatal("nested message_start in single tool call")
}
case message.ChunkMessageEnd:
depth--
if depth < 0 {
t.Fatal("unmatched message_end in single tool call")
}
}
}
if depth != 0 {
t.Fatalf("unbalanced message_start/message_end (depth=%d)", depth)
}
// Verify tool lifecycle: running then completed with same message_id
var runningID, completedID string
for _, g := range groups {
for _, ev := range g.events {
if ev.chunkType == message.ChunkExecute {
props := extractExecuteProps(ev)
switch props["status"] {
case "running":
runningID = g.messageID
case "completed":
completedID = g.messageID
}
}
}
}
if runningID == "" {
t.Fatal("no running phase found")
}
if completedID == "" {
t.Fatal("no completed phase found")
}
if runningID != completedID {
t.Errorf("running message_id %q != completed message_id %q", runningID, completedID)
}
// Verify text output
var hasText bool
for _, ev := range events {
if ev.chunkType == message.ChunkText && strings.Contains(string(ev.data), "Done") {
hasText = true
}
}
if !hasText {
t.Error("final text 'Done.' not found")
}
}
func strDefault(v any) string {
if v == nil {
return ""
}
s, _ := v.(string)
return s
}

View file

@ -8,11 +8,14 @@ import (
"time"
"github.com/yaoapp/gou/connector"
goullm "github.com/yaoapp/gou/llm"
"github.com/yaoapp/kun/log"
agentContext "github.com/yaoapp/yao/agent/context"
"github.com/yaoapp/yao/agent/output/message"
"github.com/yaoapp/yao/agent/sandbox/v2/shared"
"github.com/yaoapp/yao/agent/sandbox/v2/types"
infra "github.com/yaoapp/yao/sandbox/v2"
"github.com/yaoapp/yao/tools"
)
// Runner implements the sandbox Runner interface for Claude CLI (mode=cli).
@ -48,6 +51,18 @@ func (r *Runner) Prepare(ctx context.Context, req *types.PrepareRequest) error {
steps := append([]types.PrepareStep{}, req.Config.Prepare...)
if ws := req.Computer.Workplace(); ws != nil {
if err := shared.InjectSystemSkills(ws, tools.SkillsFS, ".claude/skills"); err != nil {
r.logger.Warn("inject system skills: %v", err)
}
if err := shared.AppendSystemPrompt(ws, "CLAUDE.md", tools.SystemPrompt); err != nil {
r.logger.Warn("append CLAUDE.md: %v", err)
}
if err := shared.AppendSystemPrompt(ws, "AGENTS.md", tools.SystemPrompt); err != nil {
r.logger.Warn("append AGENTS.md: %v", err)
}
}
if req.SkillsDir != "" {
ws := req.Computer.Workplace()
if ws != nil {
@ -90,7 +105,12 @@ func (r *Runner) Stream(ctx context.Context, req *types.StreamRequest, handler m
// Inject connector config into a2o proxy (best-effort, errors ignored).
if req.Connector != nil && req.Connector.Is(connector.OPENAI) {
injectA2OConfig(ctx, computer, req.Connector)
roleConnectors := resolveAllRoleConnectors(req)
if len(roleConnectors) > 0 {
injectA2OConfigWithRoutes(ctx, computer, req.Connector, roleConnectors)
} else {
injectA2OConfig(ctx, computer, req.Connector)
}
}
if req.ChatID != "" {
@ -179,10 +199,13 @@ func (r *Runner) Cleanup(ctx context.Context, computer infra.Computer) error {
}
type a2oConnectorConfig struct {
Backend string `json:"backend"`
Model string `json:"model"`
APIKey string `json:"api_key"`
Options map[string]interface{} `json:"options,omitempty"`
Backend string `json:"backend"`
Model string `json:"model"`
APIKey string `json:"api_key"`
AuthMode string `json:"auth_mode,omitempty"`
MaxOutputTokens int `json:"max_output_tokens,omitempty"`
Options map[string]interface{} `json:"options,omitempty"`
Routes map[string]*a2oConnectorConfig `json:"routes,omitempty"`
}
func buildSingleA2OConfig(conn connector.Connector) *a2oConnectorConfig {
@ -193,37 +216,119 @@ func buildSingleA2OConfig(conn connector.Connector) *a2oConnectorConfig {
cfg := &a2oConnectorConfig{}
if host, ok := settings["host"].(string); ok && host != "" {
cfg.Backend = connector.BuildAPIURL(host, "/chat/completions")
} else if proxy, ok := settings["proxy"].(string); ok && proxy != "" {
cfg.Backend = connector.BuildAPIURL(proxy, "/chat/completions")
}
if model, ok := settings["model"].(string); ok && model != "" {
cfg.Model = model
}
if key, ok := settings["key"].(string); ok && key != "" {
cfg.APIKey = key
}
extra := make(map[string]interface{})
for k, v := range settings {
switch k {
case "host", "model", "key", "proxy", "type":
continue
default:
extra[k] = v
// Extract standard fields via LLMConnector methods when available
if lc, ok := conn.(goullm.LLMConnector); ok {
if url := lc.GetURL(); url != "" {
cfg.Backend = connector.BuildAPIURL(url, "/chat/completions")
}
cfg.Model = lc.GetModel()
cfg.APIKey = lc.GetKey()
cfg.AuthMode = string(lc.GetAuthMode())
if caps := lc.GetCapabilities(); caps != nil && caps.MaxOutputTokens > 0 {
cfg.MaxOutputTokens = caps.MaxOutputTokens
}
} else {
if host, ok := settings["host"].(string); ok && host != "" {
cfg.Backend = connector.BuildAPIURL(host, "/chat/completions")
} else if proxy, ok := settings["proxy"].(string); ok && proxy != "" {
cfg.Backend = connector.BuildAPIURL(proxy, "/chat/completions")
}
if model, ok := settings["model"].(string); ok && model != "" {
cfg.Model = model
}
if key, ok := settings["key"].(string); ok && key != "" {
cfg.APIKey = key
}
}
// Whitelist-filter remaining settings for the options field
extra := connector.FilterRequestBodyParams(settings, conn)
if len(extra) > 0 {
cfg.Options = extra
}
if cfg.MaxOutputTokens == 0 {
cfg.MaxOutputTokens = defaultA2OMaxOutputTokens
}
if cfg.Backend == "" {
return nil
}
return cfg
}
// resolveAllRoleConnectors maps pre-resolved role connectors from req.Roles
// to actual model names used as A2O proxy route keys.
func resolveAllRoleConnectors(req *types.StreamRequest) map[string]connector.Connector {
if len(req.Roles) == 0 {
return nil
}
result := make(map[string]connector.Connector)
for _, rc := range req.Roles {
var model string
if lc, ok := rc.(goullm.LLMConnector); ok {
model = lc.GetModel()
}
if model == "" {
model, _ = rc.Setting()["model"].(string)
}
if model != "" {
result[model] = rc
}
}
return result
}
// injectA2OConfigWithRoutes pushes the primary connector config along with
// role-based routes to the a2o proxy. Each route maps a virtual model name
// to a different backend connector.
func injectA2OConfigWithRoutes(ctx context.Context, computer infra.Computer, primaryConn connector.Connector, roleConnectors map[string]connector.Connector) {
primaryCfg := buildSingleA2OConfig(primaryConn)
if primaryCfg == nil {
log.Trace("[claude] injectA2OConfigWithRoutes: no valid primary config for connector %s", primaryConn.ID())
return
}
routes := make(map[string]*a2oConnectorConfig, len(roleConnectors))
for modelName, rc := range roleConnectors {
routeCfg := buildSingleA2OConfig(rc)
if routeCfg != nil {
routes[modelName] = routeCfg
}
}
primaryCfg.Routes = routes
data, err := json.Marshal(primaryCfg)
if err != nil {
log.Trace("[claude] injectA2OConfigWithRoutes: marshal error: %v", err)
return
}
connID := primaryConn.ID()
var result *infra.ExecResult
info := computer.ComputerInfo()
if info.Kind == "host" {
result, err = computer.Exec(ctx, []string{"tai", "a2o", "config", "put", connID}, infra.WithStdin(data))
} else {
escaped := strings.ReplaceAll(string(data), "'", "'\\''")
script := fmt.Sprintf("echo '%s' | tai a2o config put %s", escaped, connID)
result, err = computer.Exec(ctx, []string{"sh", "-c", script})
}
if err != nil {
log.Trace("[claude] injectA2OConfigWithRoutes: exec error (ignored): %v", err)
return
}
if result.ExitCode != 0 {
log.Trace("[claude] injectA2OConfigWithRoutes: exit %d stderr=%s (ignored)", result.ExitCode, result.Stderr)
return
}
log.Trace("[claude] injectA2OConfigWithRoutes: connector=%s injected with %d routes", connID, len(routes))
}
// injectA2OConfig pushes the connector config to the a2o proxy.
// For box (Linux container): uses sh pipe since Docker exec stdin may not work.
// For host: uses WithStdin which works reliably on all platforms.

View file

@ -63,6 +63,10 @@ func (s *session) runStream(handler message.StreamFunc) (completed bool, err err
s.logger.Debug("runStream: parse returned completed=%v parseErr=%v", parser.completed, parseErr)
if !parser.completed && parseErr != nil {
s.exec.Cancel()
}
if parser.completed {
s.logger.Info("claude stream completed normally")
return true, nil

View file

@ -2,6 +2,7 @@ package sandboxv2
import (
"github.com/yaoapp/yao/agent/sandbox/v2/claude"
"github.com/yaoapp/yao/agent/sandbox/v2/opencode"
"github.com/yaoapp/yao/agent/sandbox/v2/types"
yaorunner "github.com/yaoapp/yao/agent/sandbox/v2/yao"
)
@ -9,5 +10,7 @@ import (
func init() {
Register("claude", func() types.Runner { return claude.New() })
Register("claude/cli", func() types.Runner { return claude.New() })
Register("opencode", func() types.Runner { return opencode.New() })
Register("opencode/cli", func() types.Runner { return opencode.New() })
Register("yao", func() types.Runner { return yaorunner.New() })
}

Some files were not shown because too many files have changed in this diff Show more