Commit graph

3259 commits

Author SHA1 Message Date
Max
c30334edc9 Update CHAT_STORAGE_DESIGN.md to clarify message types and storage strategies
- Revised the documentation to emphasize that all message types, including built-in and custom types, are stored, with specific notes on the handling of lifecycle events and custom types.
- Enhanced the timeline section to reflect changes in message processing, including updates to the sequence of stored messages and the introduction of concurrent operations storage.
- Improved clarity on the storage approach for concurrent operations, detailing the use of `block_id` and `thread_id` for grouping messages during parallel tasks.
2025-12-09 09:35:35 +08:00
Max
d4189e4d2d Refactor chat storage design to implement a single-write strategy
- Updated the write strategy in CHAT_STORAGE_DESIGN.md to transition from a two-write to a single-write approach, enhancing efficiency by writing all messages (user input and assistant responses) to the database only once upon stream exit.
- Revised related documentation to clarify the new single-write process and its implications for message handling and database interactions.
- Adjusted implementation details in the Stream function to reflect the new strategy, ensuring that steps are only saved on error or interruption, thereby reducing unnecessary database writes.
2025-12-09 09:14:40 +08:00
Max
7ada8f6983 Enhance built-in types and chat storage design for retrieval functionality
- Introduced a new `retrieval` message type in `BUILTIN_TYPES.md` to handle knowledge base and web search results, including structured properties for query, sources, and metadata.
- Updated `CHAT_STORAGE_DESIGN.md` to document the storage approach for retrieval results, emphasizing user feedback, quality analytics, and source attribution.
- Enhanced examples and use cases for retrieval messages to clarify implementation and integration within the chat system.
- Revised related documentation to ensure consistency and understanding of the new retrieval capabilities.
2025-12-09 09:09:31 +08:00
Max
1d432891d4 Revise chat storage design to enhance resume functionality and message types
- Updated the chat storage design to introduce a new `agent_resume` table for storing execution states during interruptions or failures, improving recovery capabilities.
- Replaced the `agent_step` table with a more focused `Resume` model, capturing essential data for resuming operations.
- Expanded the message types to include detailed structures for various content types, including `tool_call`, `error`, and multimedia messages.
- Enhanced documentation to clarify the new storage strategies and message handling processes, ensuring better understanding for future development.
- Updated related functions and tests to support the new resume logic and message structures, reinforcing the integrity of the chat storage system.
2025-12-08 19:10:20 +08:00
Max
42d13ec1cb Refactor chat storage design and enhance data models
- Updated the chat storage design to reflect a shift from "Conversation" to "Chat" terminology, improving clarity in the data structure.
- Revised the data models for Chat and Message tables, including new fields for enhanced metadata management and permissions.
- Introduced a `space_snapshot` field in the Step model to facilitate recovery during execution, improving resilience in chat interactions.
- Enhanced middleware documentation to reflect the modular architecture and provide clearer usage examples for different API routes.
- Updated tests to ensure the integrity of new data structures and functionalities, reinforcing the robustness of the chat storage system.
2025-12-08 18:47:45 +08:00
Max
467d4e2398 Enhance Assistant initialization and configuration management
- Improved the Assistant's initialization process by integrating dynamic configuration loading from `agent/kb.yml`, allowing for more flexible chat session settings.
- Refactored the `initKBConfig` function to streamline the reading and parsing of Knowledge Base settings.
- Optimized conversation initialization to prepare KB collections asynchronously, enhancing performance during chat interactions.
- Updated tests to ensure the correct application of KB settings and robust integration with the Assistant's functionality.
2025-12-08 18:25:15 +08:00
Max
31b627aba0 Implement Knowledge Base configuration loading and enhance Assistant initialization
- Added functionality to load Knowledge Base (KB) configuration from `agent/kb.yml`, allowing dynamic settings for chat sessions.
- Introduced `initKBConfig` function to read and parse KB settings, integrating them into the Assistant's initialization process.
- Enhanced the Assistant's conversation initialization to prepare KB collections asynchronously, improving performance during chat interactions.
- Updated tests to verify the correct loading and application of KB settings, ensuring robust integration with the Assistant's functionality.
- Refactored metadata handling for KB collections to include additional fields for improved context management during chat sessions.
2025-12-08 17:08:32 +08:00
Max
5f803ef4d4 Implement authorized script execution and enhance context handling
- Added ExecuteWithAuthorized method to the Script struct, allowing scripts to be executed with user authorization information.
- Updated existing Execute method to utilize ExecuteWithAuthorized for improved authorization handling.
- Enhanced script execution tests to verify behavior with and without authorized information, ensuring accurate context management.
- Implemented authorized information conversion to a map for easier integration with runtime environments.
- Refactored context handling in script execution to ensure authorized data is correctly passed and utilized.
2025-12-08 11:42:01 +08:00
Max
704c0331b1 Enhance Assistant stream functionality and remove history handling
- Added permission validation in the Assistant's Stream method to ensure user authorization before processing input messages.
- Introduced conversation initialization within the Stream method to prepare the context for chat interactions.
- Removed the history.go file, which previously contained a placeholder method for handling chat history, streamlining the Assistant's codebase.
- Updated the Knowledge Base API integration in collection management, ensuring all collection operations utilize the new API structure for improved consistency and error handling.
2025-12-08 10:28:26 +08:00
Max
a683452d22 Implement GetDel method for space management in JS API
- Added a new GetDel method to the space object, allowing retrieval and immediate deletion of a key's value, enhancing one-time use data handling.
- Updated the JS API to include tests for GetDel functionality, ensuring correct behavior for existing and non-existent keys, as well as complex data structures.
- Removed debug print statements from the Assistant's initializeCapabilities method for cleaner code.
2025-12-06 18:36:06 +08:00
Max
666c1e8d74 Enhance content processing with forceUses configuration
- Updated the Vision function and associated handlers to accept a forceUses parameter, allowing explicit control over the use of external tools regardless of model capabilities.
- Modified the Assistant's BuildContent method to set the AssistantID in the context for better file tracking.
- Enhanced the applyCreateResponseOptions method to merge and prioritize Uses and ForceUses configurations from the createResponse.
- Added comprehensive tests to validate the adjustment of Uses and ForceUses configurations through hooks, ensuring correct application in content processing.
2025-12-06 17:38:21 +08:00
Max
d9b8496a7d Refactor authorized object handling in JS API
- Updated the NewObject method to pass the complete authorized structure instead of individual fields, simplifying the code and improving clarity.
- Enhanced handling for nil authorized scenarios by setting an empty object, ensuring consistent behavior in context management.
- Released Go-side persistent handles for authorized and empty objects to maintain proper resource management.
2025-12-05 20:39:19 +08:00
Max
53a22fd9e4 Enhance context handling in MCP tests and JS API
- Added tests to verify the presence and correctness of authorized and metadata fields in MCP tool context passing.
- Updated MCP tool context tests to include user and tenant ID assertions, ensuring accurate context validation.
- Enhanced JS API to handle authorized and metadata fields, including tests for nil scenarios, improving robustness and clarity in context management.
2025-12-05 20:20:21 +08:00
Max
a66c2c7666 Implement context passing for MCP tool calls
- Added tests to verify that agent context is correctly passed to MCP tools during single and parallel calls.
- Updated the MCP client methods to accept agent context as an additional argument, enhancing the context management during tool execution.
- Improved test coverage for context handling, ensuring that context data is accurately received and validated in tool responses.
2025-12-05 20:09:38 +08:00
Max
f0e445862f Refactor Assistant cache tests and enhance cache functionality
- Updated cache tests to use the testify assertion library for improved readability and maintainability.
- Added new tests for cache operations including basic functionality, LRU eviction, removal, clearing, and concurrent access.
- Enhanced the cache implementation to unregister scripts upon removal and clearing, ensuring proper resource management.
- Introduced an `All` method to retrieve all assistants in the cache, improving accessibility of cached items.
- Streamlined the script registration and unregistration process within the Assistant struct, enhancing script management.
2025-12-05 19:49:44 +08:00
Max
d0981e53af Enhance engine loading process with progress reporting and timing
- Introduced a `loadStep` function to wrap loading operations, providing timing and progress callback functionality.
- Updated the `Load` function to utilize `loadStep` for various components, improving visibility into loading durations.
- Enhanced the `start` command to display loading progress and duration in development mode, improving user experience during application startup.
- Refactored model loading functions to return models without migration, streamlining the loading process and improving performance.
2025-12-05 18:53:21 +08:00
Max
57724b20db Refactor Assistant script loading and management
- Updated the script loading process to load both hook scripts and other scripts from the src directory, improving organization and clarity.
- Introduced a new `Scripts` field in the Assistant struct to store additional scripts, enhancing the flexibility of script management.
- Removed deprecated script loading functions and streamlined the loading logic to ensure better maintainability and performance.
- Adjusted the handling of timestamps for script updates, ensuring accurate tracking of script modifications.
2025-12-05 18:10:07 +08:00
Max
0bc0821646 Refactor Assistant script handling to use HookScript
- Replaced all instances of `Script` with `HookScript` in the Assistant and related files to improve clarity and consistency in naming.
- Updated method calls in the Stream, Create, and Next hooks to utilize the new `HookScript` field.
- Adjusted tests and benchmarks to reflect the changes in script handling, ensuring all functionalities remain intact and operational.
- Enhanced the load and initialization processes to accommodate the new HookScript structure, streamlining the assistant's script management.
2025-12-05 14:45:09 +08:00
Max
a50f43a8bb Refactor MCP client handling for improved categorization and information display
- Updated the `printMCPs` function to enhance the separation of agent and standard MCPs by utilizing the Type field for better clarity.
- Improved error handling and client information retrieval, ensuring accurate display of transport types and labels for each MCP.
- Modified the MCP loading process to specifically load agent clients, streamlining the management of different client types.
- Enhanced output formatting to include tools count only for process transport, improving the readability of MCP information.
2025-12-05 14:17:56 +08:00
Max
cd5b4df503 Add MCP client loading and printing functionality
- Introduced a new `printMCPs` function to display loaded MCP clients, categorizing them into standard and agent clients for better clarity.
- Enhanced the `Load` function to include loading MCP clients from the assistants directory, improving the overall MCP management.
- Added tests for loading assistant MCPs, ensuring that the functionality works as expected and that agent clients are correctly identified and logged.
- Updated model loading to support agent models from assistants, ensuring proper integration and functionality across the system.
2025-12-05 12:28:41 +08:00
Max
311350bfbc Enhance Assistant methods with options parameter for improved context management
- Updated the Stream, BuildContent, and LLM execution methods to accept an Options parameter, allowing for more flexible context handling.
- Removed debug print statements to clean up the code and improve readability.
- Enhanced locale handling in the loadMap function to automatically inject assistant name and description into all locales, ensuring better localization support.
- Introduced output skipping functionality in context options to manage internal A2A calls more effectively.
- Improved output writer resolution logic to prioritize context settings, enhancing output management during agent calls.
2025-12-04 12:03:00 +08:00
Max
a96946d8bb Refactor Assistant context handling to improve options management
- Updated the Assistant methods to accept an Options parameter, enhancing flexibility in context management.
- Removed the Connector field from the context and related structures, transitioning to a more streamlined options-based approach.
- Adjusted various tests to accommodate the new options handling, ensuring comprehensive coverage of the updated functionality.
- Enhanced the Create and Next hooks to return options alongside responses, improving the overall usability of the API.
- Cleaned up deprecated fields and improved context initialization for better maintainability.
2025-12-04 10:43:32 +08:00
Max
beb63b77de Enhance Assistant functionality with improved content handling and debugging
- Added support for converting extended types (file, data) to standard LLM types (text, image_url, input_audio) in the Stream method.
- Introduced a new agentCallerWrapper to facilitate agent calls from the content package.
- Implemented ThreadID management for nested agent calls to improve concurrent stream identification.
- Enhanced message handling to include metadata for message_start and message_end events, allowing for better tracking of message states.
- Removed deprecated vision capability checks from the Assistant initialization process, streamlining the codebase.
- Added new content types (file, data) to the context types for improved message content handling.
2025-12-03 21:35:54 +08:00
Max
1060c0f0e0 Enhance attachment manager with dual text content handling
- Updated `GetText` and `SaveText` methods to support both full content and a preview (first 2000 characters) for improved performance and usability.
- Added comprehensive tests in `manager_test.go` to validate the new functionality, including UTF-8 character handling and content retrieval scenarios.
- Modified the attachment model to include a `content_preview` field alongside the existing `content` field, ensuring efficient data management.
- Enhanced the README.md to document the new dual storage feature, providing clear examples for users on how to utilize the updated methods.
2025-12-03 18:29:24 +08:00
Max
4170fbd13b Add text content storage functionality to attachment manager
- Implemented `GetText` and `SaveText` methods in the attachment manager to handle storing and retrieving parsed text content from files.
- Enhanced the `manager_test.go` with comprehensive tests for various scenarios, including saving, updating, and retrieving text, as well as handling non-existent file IDs.
- Updated the README.md to document the new text content storage features, providing examples for saving and retrieving parsed text.
- Modified the attachment model to include a new `content` field for storing text content, improving the overall functionality of the attachment management system.
2025-12-03 18:18:46 +08:00
Max
60fef5744a Implement custom connector extraction in GetCompletionRequest
- Enhanced the GetCompletionRequest function to support custom connectors by validating the model field against existing connectors.
- If the model is a valid connector ID, it sets the ctx.Connector; otherwise, it defaults to the assistant ID behavior.
- This change improves flexibility in handling different model types while maintaining compatibility with existing functionality.
2025-12-03 12:01:44 +08:00
Max
ede240302a Update ConnectorOptions handling in tests and types
- Changed the `Optional` field in `ConnectorOptions` from a boolean to a pointer to allow for nil values, enhancing flexibility in option handling.
- Updated tests in `load_test.go`, `convert_test.go`, and `assistant_test.go` to reflect the new pointer type for `Optional`, ensuring proper assertions and error handling.
- Added new test cases to validate behavior when `Optional` is nil or false, improving test coverage and robustness of the ConnectorOptions functionality.
2025-12-03 10:57:47 +08:00
Max
bbba6b9fdb Add modes and database support to Assistant model
- Enhanced the `loadMap` function to include handling for `modes` and `default_mode` fields, allowing for flexible operational modes and a specified primary mode.
- Introduced a new `DB` field in the Assistant model to support database configuration, improving data management capabilities.
- Implemented the `ToModes` conversion function to facilitate various input types for modes, ensuring robust handling and validation.
- Added comprehensive tests for the `ToModes` function to validate its functionality across different input scenarios, enhancing overall reliability.
- Updated relevant methods to ensure consistent integration of the new fields and functionalities within the Assistant model.
2025-12-03 10:50:24 +08:00
Max
8d762964b2 Refactor assistant capabilities retrieval and enhance API endpoints
- Replaced the deprecated getConnectorCapabilities method with a unified capability getter in the Assistant model, improving capability retrieval logic.
- Added a new API endpoint to retrieve essential assistant information, including fields like id, name, avatar, and connector options, enhancing the assistant's data accessibility.
- Updated the LLM management to support filtering by capabilities, allowing for more flexible provider listings based on user-defined models.
- Improved overall structure and clarity in the assistant's capabilities and API responses, ensuring better maintainability and usability.
2025-12-03 10:05:23 +08:00
Max
ed62033761 Add modes and default mode fields to Assistant model
- Introduced `modes` field to support multiple operational modes for the assistant, allowing for flexible functionality (e.g., chat, task).
- Added `default_mode` field to specify the primary mode of operation for the assistant.
- Updated the `Map`, `Clone`, and `Update` methods to handle the new fields appropriately.
- Enhanced the `ToAssistantModel` conversion function to include the new fields.
- Added tests to validate the integration and functionality of modes and default mode within the assistant model.
2025-12-02 19:02:31 +08:00
Max
da48896bdf Add database support to Assistant model and related functionalities
- Introduced a new `Database` type to encapsulate database models and options.
- Updated the `Assistant` model to include a `DB` field for database configuration.
- Enhanced the `Map`, `Clone`, and `Update` methods to handle the new `DB` field appropriately.
- Implemented the `ToDatabase` conversion function to support various input types for database initialization.
- Added comprehensive tests to validate the integration of the database functionality within the assistant model.
- Updated relevant fields and methods to ensure consistent handling of the new database configuration.
2025-12-02 18:45:03 +08:00
Max
4b747a5bfa Add GetStorage function and enhance loadSource logic
- Introduced GetStorage function for testing purposes, allowing retrieval of the current storage instance.
- Enhanced loadSource function to load scripts from the source field if present, improving assistant initialization.
- Updated comments for clarity on TypeScript handling in loadSource, ensuring better understanding of script loading mechanics.
2025-12-02 16:54:57 +08:00
Max
ee44432e69 Enhance assistant prompt handling with new preset and global prompt controls
- Introduced prompt preset selection and global prompt disabling features in the assistant's message building process.
- Updated the `buildSystemPrompts` method to prioritize prompt presets and manage global prompt settings based on context and hook responses.
- Added new fields to the `HookCreateResponse` struct for prompt configuration, allowing for dynamic adjustments during message processing.
- Refactored tests to validate the integration of prompt presets and global prompt controls, ensuring robust functionality and context-aware behavior.
2025-12-02 16:24:22 +08:00
Max
42e920eed6 Implement global prompts functionality in the assistant module
- Added methods to set and retrieve global prompts, enhancing the assistant's capabilities.
- Updated the assistant's message building process to include global prompts, ensuring context-aware parsing.
- Introduced tests to validate the integration and functionality of global prompts within the assistant.
- Improved context variable handling for prompt parsing, supporting dynamic content generation.
2025-12-02 16:03:31 +08:00
Max
45029269af Enhance assistant model with global prompts functionality
- Introduced the `disable_global_prompts` field in the Assistant model to control the usage of global prompts.
- Updated the Load and Get methods to initialize and retrieve global prompts from the configuration.
- Refactored tests to validate the new global prompts functionality and ensure proper loading and context handling.
- Improved the overall structure and clarity of the assistant's capabilities and configurations.
2025-12-02 15:37:41 +08:00
Max
51092e3324 Refactor assistant model and enhance data handling
- Removed the GetByConnector method from the Assistant model to streamline the retrieval process.
- Introduced new fields for connector options and prompt presets, allowing for more flexible configurations.
- Updated the Map method to include additional fields such as connector options and prompt presets.
- Enhanced the Clone method to support deep copying of new fields.
- Improved the Update method to handle updates for the new source, connector options, and prompt presets fields.
- Refactored tests to ensure comprehensive coverage of the new functionalities and maintain clarity in the assistant structure.
2025-12-02 14:54:35 +08:00
Max
339a486eb4 Refactor assistant capabilities and update data handling
- Enhanced the getConnectorCapabilities method to prioritize model capabilities and connector settings, improving capability retrieval logic.
- Deprecated the tools field in the Assistant model, transitioning to MCP for tool management, and updated related methods accordingly.
- Introduced new fields for connector options and prompt presets in the Assistant model, allowing for more flexible configurations.
- Updated the GetAssistant method to support field selection, improving data retrieval efficiency and flexibility.
- Refactored tests and documentation to reflect changes in the assistant structure and capabilities, ensuring clarity and maintainability.
2025-12-02 12:36:47 +08:00
Max
3dd63e5530 Refactor model capabilities to use OpenAI struct
- Updated the model capabilities throughout the agent to utilize the new gouOpenAI.Capabilities struct instead of the previous ModelCapabilities.
- Adjusted related methods and types to ensure compatibility with the new capabilities structure, enhancing clarity and maintainability.
- Improved context handling and message processing by directly integrating OpenAI capabilities, streamlining the overall architecture.
2025-12-02 11:33:09 +08:00
Max
7e8b2d8d9f Add LLM failure tracing in executeLLMStream and executeLLMForToolRetry methods
- Implemented traceLLMFail method to mark LLM requests as failed in the trace context.
- Updated executeLLMStream and executeLLMForToolRetry methods to call traceLLMFail on errors, enhancing error tracking and debugging capabilities.
- Improved overall error handling in LLM execution flows, ensuring failures are properly logged in the trace.
2025-12-02 09:32:21 +08:00
Max
5ca78e0133 Enhance tool call handling in stream processing
- Updated handleToolCall method to parse tool call delta data from OpenAI, improving the handling of single and multiple tool calls.
- Implemented dynamic delta actions ("append" for arguments, "merge" for static fields) to optimize message sending.
- Enhanced message structure to include appropriate DeltaAction and DeltaPath for better frontend integration.
- Improved error handling during JSON parsing to ensure robustness in tool call processing.
2025-12-01 16:25:35 +08:00
Max
b45165b0ae Implement thread-safe message and block metadata storage
- Introduced a new messageMetadataStore type for managing message and block metadata with thread-safe operations.
- Added methods for setting, getting, and updating message and block metadata, enhancing concurrency handling.
- Removed redundant metadata store implementation from types.go, centralizing functionality in message.go for improved organization.
2025-12-01 08:35:38 +08:00
Max
a868b4d741 Refactor context initialization and enhance tool call handling
- Updated context creation methods to remove unnecessary payload parameter, simplifying initialization.
- Enhanced handleToolCall method to manage message sequencing and delta actions more effectively, ensuring proper message tracking.
- Improved error handling in message sending to ensure robustness in tool call processing.
2025-11-30 16:40:45 +08:00
Max
4e8508649e Add automatic ChunkID generation for messages in Send method
- Implemented logic to auto-generate ChunkID for messages when not provided, enhancing message tracking.
- Utilized IDGenerator if available; otherwise, fallback to message.GenerateNanoID for unique identification.
- Improved message handling consistency by ensuring ChunkID is always set for non-event messages.
2025-11-30 15:28:09 +08:00
Max
6c295db8d5 Implement EndBlock method and enhance message lifecycle management
- Added EndBlock method to explicitly mark the end of a message block, sending a block_end event.
- Updated message handling to include lifecycle events for message start and end, improving tracking and management of message durations.
- Enhanced message metadata structure to support chunk counting and message types, facilitating better message operations.
- Improved JSAPI documentation to include new lifecycle management features, clarifying usage for developers.
2025-11-30 15:09:27 +08:00
Max
1af1afaaee Refactor delegated assistant context handling in Stream method
- Simplified context management by directly passing the existing context to the delegated assistant's Stream method, ensuring proper stack tracing.
- Updated comments to clarify the delegation process and the relationship between parent and delegated assistants, enhancing code readability.
2025-11-30 11:57:02 +08:00
Max
0fe0843b43 Refactor agent and assistant structure for improved clarity and performance
- Removed deprecated API and vision components, streamlining the agent's architecture.
- Updated the Load function to utilize a new agentDSL variable, enhancing the management of assistant capabilities.
- Enhanced context handling by introducing a message metadata store for thread-safe operations, improving message tracking and management.
- Refactored context methods to eliminate deprecated fields, ensuring cleaner and more maintainable code.
- Improved documentation and comments throughout the codebase to clarify changes and enhance developer understanding.
2025-11-30 11:47:25 +08:00
Max
ab240443f8 Add message metadata handling and new context methods for message operations
- Introduced recordMessageMetadata and getMessageMetadata methods to manage metadata for sent messages, enabling BlockID and ThreadID inheritance in delta operations.
- Implemented new methods (Append, Merge, Set) in the context to enhance message management capabilities, allowing for more flexible message updates.
- Updated the Send method to automatically manage BlockID and ThreadID for delta operations, improving message handling consistency.
- Enhanced JSAPI documentation to reflect new methods and usage patterns, improving developer experience and clarity.
2025-11-30 10:39:51 +08:00
Max
c11bc0a1ae Enhance tracing and message handling in Assistant context
- Added error handling and logging for completion node creation in the traceAgentCompletion method, improving traceability of failures.
- Introduced a new replaceMethod in the context to allow message content replacement, enhancing message management capabilities.
- Updated documentation for sendMethod and replaceMethod to clarify usage and return values, improving developer experience.
2025-11-30 09:40:17 +08:00
Max
78494f8622 Refactor agent tracing and completion handling
- Updated the traceAgentOutput method to be deprecated in favor of the new traceAgentCompletion method, which creates a dedicated completion node for reporting final outputs.
- Enhanced the i18n messages to include labels and descriptions for the new agent completion functionality, improving localization support.
- Removed debug logging related to MCP tools from the buildCompletionOptions method to streamline the code and improve clarity.
2025-11-30 08:31:31 +08:00
Max
3bfd40b701 Refactor Assistant's Stream method to support Next hook responses
- Updated the Stream method to return a flexible final response, accommodating both standard and Next hook responses.
- Introduced NextProcessContext to encapsulate context for processing Next hook responses, enhancing clarity and maintainability.
- Removed deprecated Done and Failback hooks, streamlining the hook management process.
- Enhanced error handling and logging for improved traceability during streaming operations.
2025-11-29 22:52:06 +08:00