Commit graph

2024 commits

Author SHA1 Message Date
Max
0d676b3348 Update Go module dependencies and enhance assistant message handling in Neo API
- Added the github.com/spf13/cast dependency to improve type conversion capabilities.
- Enhanced the requestMessages function to include prompts, allowing for more dynamic message handling.
- Updated the LoadBuiltIn function to ensure the "Built-in" tag is added if it doesn't already exist, improving assistant organization.
- Refactored sorting logic to utilize the cast package for better type safety.

These changes support improved functionality and maintainability in the assistant management process.
2025-01-03 17:17:04 +08:00
Max
55c0580fc3 Refactor Neo API assistant management and remove obsolete functions
- Removed unused assistant-related functions and types from the Neo API, including local chat and file handling, to streamline the codebase.
- Refactored the assistant initialization process, consolidating the loading and configuration of assistants, including Vision and Connector settings.
- Updated the DSL struct to enhance clarity and maintainability, ensuring better configuration management for assistants.
- Improved error handling and modularity in the assistant loading process, paving the way for future enhancements in AI functionalities.
2025-01-03 16:37:43 +08:00
Max
966b0f601f Update Go module dependencies and enhance Neo API with Vision integration
- Removed obsolete AWS SDK v2 dependencies from go.mod and go.sum to streamline the codebase.
- Added Vision-related imports and initialized Vision settings in the Neo API, enhancing AI capabilities.
- Introduced initVision method to configure and initialize the Vision instance, improving modularity and functionality.
- Updated the DSL struct to include Vision settings, allowing for better configuration management.

This commit supports future enhancements in AI functionalities and improves the overall structure of the Neo API.
2025-01-03 15:56:14 +08:00
Max
1ea1852677 Remove prompt documentation and enhance image upload/download functionality in vision module
- Deleted the obsolete prompt.md file, streamlining the codebase.
- Added tests for image upload and download functionality using local and S3 storage drivers, ensuring robust handling of image files.
- Implemented image compression during upload for both local and S3 storage, maintaining aspect ratio and limiting dimensions to a maximum size of 1920x1080.
- Enhanced the local and S3 storage drivers to support image compression, improving storage efficiency.
- Updated tests to verify image dimensions and content type after upload and download operations, ensuring data integrity.
2025-01-03 13:52:45 +08:00
Max
93a6d7a6a0 Update Go module dependencies to include AWS SDK v2
- Added several AWS SDK v2 modules to go.mod and go.sum, enhancing cloud service integration capabilities:
  - github.com/aws/aws-sdk-go-v2 and its related services (S3, STS, SSO, etc.) for improved AWS functionality.
- Updated indirect dependencies to ensure compatibility and performance improvements.
- This update supports future development involving AWS services, streamlining the integration process.
2025-01-03 13:33:58 +08:00
Max
af30bec4f0 Enhance RAG configuration and assistant loading in Neo API
- Updated the SetRAG function to include RAG settings, allowing for more detailed configuration during assistant initialization.
- Refactored the LoadBuiltIn function to streamline the saving process of assistants, improving error handling and code clarity.
- Modified loadPrompts and loadScript functions to return timestamps, enabling tracking of creation and update times for assistants.
- Enhanced the Assistant struct with created_at and updated_at fields, improving data management and traceability.
- Overall improvements to the assistant loading process, ensuring better organization and maintainability of the codebase.
2025-01-02 18:56:35 +08:00
Max
973fb5dd3d Remove assistant management files to streamline codebase
- Deleted the assistant.go and assistant_test.go files, which contained the implementation and tests for assistant management.
- This cleanup reduces complexity and focuses on core functionalities, paving the way for future enhancements in assistant handling.
2025-01-02 17:00:50 +08:00
Max
2903602c6b Refactor Neo API initialization and enhance assistant management
- Introduced new methods for initializing RAG and store, improving modularity and clarity in the Load function.
- Refactored the assistant initialization process to streamline the loading of built-in assistants and set RAG configurations.
- Moved the defaultAssistant method from neo.go to load.go for better organization and accessibility.
- Enhanced error handling in store initialization to support multiple connector types, including Redis and Mongo.
- Updated the assistant struct to include RAG settings, improving the overall assistant management capabilities.
2025-01-02 16:55:21 +08:00
Max
6288e25a8a Refactor conversation table handling and update settings structure in Neo API
- Changed the 'Table' field to 'Prefix' in the Setting struct to better represent its purpose as a table name prefix.
- Updated all relevant instances in the codebase to use the new 'Prefix' field, ensuring consistency across the application.
- Adjusted logging and table retrieval methods to reflect the new naming convention, enhancing clarity and maintainability.
2025-01-02 16:22:10 +08:00
Max
fdf71f4cfd Update Go version and dependencies; enhance Neo API functionality
- Bump Go version from 1.22.2 to 1.23 for improved performance.
- Update dependencies in go.mod and go.sum, including:
  - github.com/PuerkitoBio/goquery to v1.10.1
  - github.com/dchest/captcha to v1.1.0
  - github.com/evanw/esbuild to v0.24.2
  - github.com/fatih/color to v1.18.0
  - github.com/fsnotify/fsnotify to v1.8.0
  - github.com/spf13/cobra to v1.8.1
  - github.com/stretchr/testify to v1.10.0
  - github.com/xuri/excelize/v2 to v2.9.0
- Introduce RAG (Retrieval-Augmented Generation) initialization in the Neo API, enhancing assistant capabilities.
- Refactor workflows to use Go 1.23 for testing and CI processes, ensuring compatibility with the latest features.
2025-01-02 13:06:36 +08:00
Max
0d05842c28 Update Go module dependencies and version in go.mod and go.sum
- Bump Go version from 1.20 to 1.22.2 for improved performance and features.
- Update indirect dependencies to their latest versions, including:
  - github.com/cespare/xxhash/v2 from v2.2.0 to v2.3.0
  - github.com/golang/protobuf from v1.5.3 to v1.5.4
  - github.com/klauspost/compress from v1.17.3 to v1.17.4
  - golang.org/x/oauth2 from v0.14.0 to v0.23.0
  - google.golang.org/grpc from v1.60.1 to v1.69.2
  - google.golang.org/protobuf from v1.34.2 to v1.36.1
- Add new dependencies for OpenTelemetry packages, enhancing observability capabilities.
2025-01-02 11:07:04 +08:00
Max
fd5d701a23 Refactor assistant loading and initialization in Neo API
- Simplified the assistant loading process by removing the asynchronous query for the assistant list and replacing it with a direct call to retrieve the default assistant.
- Introduced a new method, defaultAssistant, to streamline the retrieval of the default assistant based on the current configuration.
- Enhanced the LoadStore function to support loading assistants from a specified path, improving flexibility in assistant management.
- Updated the overall structure for better readability and maintainability, ensuring a more efficient assistant initialization process.
2025-01-01 17:34:17 +08:00
Max
8d320f0e41 Enhance assistant struct with tags and options handling in Neo API
- Added support for initializing assistant tags with a default value of "Built-in" if none are provided, improving data consistency.
- Implemented loading of tags, options, and description from the input data map, enhancing the flexibility of assistant configuration.
- Updated the LoadBuiltIn and loadMap functions to accommodate new fields, streamlining the assistant loading process.
2025-01-01 16:55:11 +08:00
Max
f8ce8fc193 Enhance assistant management and API functionality in Neo
- Added a new endpoint to retrieve all assistant tags, improving data accessibility for clients.
- Updated the assistant list handling to support filtering by built-in status and assistant ID, enhancing the filtering capabilities.
- Introduced a method to load built-in assistants, streamlining the assistant initialization process.
- Enhanced the Assistant struct with new fields for path, built-in status, and sorting, improving data organization.
- Implemented validation and cloning methods for the Assistant struct, ensuring data integrity and ease of use.
- Updated tests to cover new functionalities, including validation, cloning, and tag retrieval, ensuring robust functionality across the assistant management operations.
2025-01-01 16:48:06 +08:00
Max
d0d110b0ec Refactor assistant management and enhance storage retrieval in Neo API
- Removed Weaviate store implementation, streamlining the codebase and focusing on Mongo and Redis backends.
- Introduced GetAssistant method in both Mongo and Redis stores to retrieve a single assistant by ID, improving data access capabilities.
- Updated LoadStore function to utilize the new storage retrieval logic, enhancing the assistant loading process.
- Enhanced the Assistant struct to include a Script field for better management of assistant scripts.
- Improved tests to cover the new GetAssistant functionality, ensuring robust error handling and data retrieval across different scenarios.
2025-01-01 12:31:47 +08:00
Max
e530ccc1cd Refactor conversation management and migrate to store-based architecture in Neo API
- Replaced the conversation handling logic with a new store-based approach, enhancing data management and retrieval capabilities.
- Updated all relevant methods to utilize the new store interface, including GetChats, GetChat, GetHistory, and SaveAssistant, ensuring consistent functionality across the API.
- Removed deprecated conversation-related files and structures, streamlining the codebase and improving maintainability.
- Enhanced the AssistantFilter and AssistantResponse types to support the new store architecture, improving filtering and pagination capabilities.
- Updated tests to cover the new store-based methods and ensure robust functionality across different storage backends.
2025-01-01 10:43:05 +08:00
Max
238347c834 Refactor message handling and enhance file reading capabilities in Neo API
- Updated the message handling logic to improve error reporting and content management, ensuring more robust communication with clients.
- Introduced a new ReadBase64 method in both Local and OpenAI assistant implementations to read files and return their base64 encoded content, enhancing file handling capabilities.
- Removed the deprecated JSON message handling code, streamlining the message processing structure.
- Refactored message struct to include new fields and methods for better data management and response handling.
- Improved the chat functionality to handle attachments and user messages more effectively, ensuring a smoother user experience.
2024-12-31 16:01:48 +08:00
Max
882277d974 Enhance assistant filtering and management in Neo API
- Added support for filtering assistants by keywords, connector, and mentionable status in the handleAssistantList function.
- Introduced a new parseBoolValue utility function to handle various boolean formats for filtering.
- Updated the AssistantFilter structure to include an AssistantID field and a Select field for specifying returned fields.
- Refactored process functions to replace 'add' with 'create' for consistency and clarity in assistant management.
- Implemented new processAssistantFind function to retrieve assistants by ID, improving data retrieval capabilities.
- Enhanced tests to cover new filtering options and ensure robust functionality across assistant management operations.
2024-12-29 17:30:33 +08:00
Max
94f3ddb158 Enhance assistant management and JSON field handling in Neo API
- Updated SaveAssistant and processAssistantAdd methods to return assistant IDs, improving response consistency.
- Implemented logic to handle JSON fields as both strings and native types, ensuring proper storage and retrieval of assistant attributes.
- Refactored tests to cover various JSON field formats, including string, native types, and nil values, enhancing test coverage and reliability.
- Improved the assistant management structure to support mixed JSON formats, ensuring robust functionality across different storage backends.
2024-12-29 17:14:01 +08:00
Max
af65a2d5fa Refactor assistant management and enhance API response structure
- Updated the assistant management functions to utilize a unified response structure, replacing the previous 'Items' field with 'Data' for consistency across the API.
- Refactored the handling of assistant mentions and details to streamline data processing, improving code readability and maintainability.
- Enhanced the AssistantResponse type to include pagination fields directly, facilitating better management of assistant data retrieval.
- Implemented new methods for adding, saving, deleting, and searching assistants, ensuring robust functionality across different storage backends.
- Updated tests to reflect changes in the assistant management logic and response structure, ensuring comprehensive coverage and reliability.
2024-12-29 12:57:17 +08:00
Max
de49788d52 Add utility endpoint for listing connectors in Neo API
- Introduced a new GET endpoint at `/utility/connectors` to list available connectors, enhancing the API's utility features.
- Implemented `handleConnectors` function to filter and format connector data, providing a structured response with labels and values for each connector.
- Updated documentation comments to include usage examples for the new endpoint, improving developer experience and usability.
2024-12-29 11:27:40 +08:00
Max
697facb23d Enhance Neo API with detailed endpoint examples and improved documentation
- Added comprehensive curl examples for all API endpoints, including chat management, assistant management, file handling, and generation functionalities.
- Improved comments throughout the code to clarify the purpose and usage of each endpoint, enhancing developer experience and usability.
- Organized endpoint descriptions into categories for better readability and understanding of the API structure.
2024-12-29 11:13:57 +08:00
Max
8fafdc5855 Enhance assistant management in Neo API and conversation module
- Added new endpoints for managing assistants: list, detail, save, and delete.
- Implemented filtering capabilities for assistants based on keywords, tags, mentionable status, and automation.
- Refactored the assistant structure to include new fields for better management and filtering.
- Removed obsolete test data file, streamlining the codebase.
- Updated tests to cover new assistant functionalities and filtering options, ensuring robust functionality.
2024-12-29 11:06:41 +08:00
Max
b17778afea Add assistant management functionality in conversation module
- Introduced SaveAssistant, DeleteAssistant, and GetAssistants methods across Mongo, Redis, and Weaviate implementations to manage assistant records.
- Enhanced the AssistantFilter and AssistantResponse types for improved pagination and filtering capabilities.
- Updated the conversation interface to include new assistant-related methods, ensuring consistent functionality across different storage backends.
- Added tests for assistant CRUD operations and pagination, ensuring robust functionality and reliability in managing assistants.
2024-12-29 10:03:38 +08:00
Max
8166503058 Refactor assistant table structure in Xun implementation
- Enhanced the assistant table by adding new fields: 'options', 'permissions', 'readonly', and 'automated', improving the management of assistant attributes.
- Updated the fields array to reflect the new structure, ensuring all fields are recognized as required attributes.
- Improved code readability by adding comments to clarify the purpose of each field.
2024-12-28 17:19:01 +08:00
Max
f022b79518 Add 'functions' field to assistant table in Xun implementation
- Introduced a new 'functions' field in the assistant table, enhancing the structure for better management of assistant capabilities.
- Updated the fields array to include 'functions', ensuring it is recognized as a required attribute in the assistant structure.
2024-12-20 16:24:00 +08:00
Max
6f6c0b3d6f Enhance assistant table structure by adding 'files' field
- Introduced a new 'files' field in the assistant table within the Xun implementation, allowing for better management of associated files.
- Updated the fields array to include the new 'files' field, ensuring it is recognized as a required attribute in the assistant structure.
2024-12-20 16:21:53 +08:00
Max
76e7156dac Refactor assistant structure and replace base assistant with local implementation
- Replaced the base assistant package with a local assistant implementation, enhancing modularity and maintainability.
- Updated the newAssistantByConnector function to utilize the local assistant, ensuring consistent error handling and improved functionality.
- Removed obsolete base assistant files, streamlining the codebase and reducing complexity.
- Added a new 'mentionable' field to the assistant table in the Xun implementation, allowing for better management of assistant visibility in mentions.
2024-12-20 15:48:28 +08:00
Max
af52dd35a1 Refactor conversation history handling and enhance Assistant structure
- Updated SaveHistory method across Mongo, Redis, and Weaviate to include a context parameter, improving flexibility in message storage.
- Modified the Assistant struct in types.go to add new fields: Type, Avatar, and Flows, enhancing the representation of assistant attributes.
- Adjusted the Xun implementation to support the new context handling in SaveHistory, ensuring messages can now include contextual information.
- Updated tests to reflect changes in message structure and ensure proper functionality with the new context parameter.
2024-12-20 12:30:31 +08:00
Max
33b28b50f4 Enhance SSE error handling in handleGenerateTitle method
- Added support for Server-Sent Events (SSE) in the handleGenerateTitle function to send error messages in the appropriate SSE format when invalid request bodies are encountered.
- Updated response headers for SSE requests to ensure correct content type and connection settings, improving the API's handling of real-time updates and error reporting.
2024-12-19 17:15:44 +08:00
Max
c1e95374f2 Update GenerateChatTitle method to include silent mode in chat handling
- Modified the handleChatUpdate function to pass an additional parameter for silent mode when generating chat titles, enhancing the flexibility of chat title generation.
- This change allows for improved handling of chat updates, ensuring that the API can better manage user expectations and interactions.
2024-12-19 16:52:43 +08:00
Max
a90d1d5a0a Enhance Neo API with new generation endpoints and improved SSE handling
- Added new GET and POST endpoints for generating custom content, chat titles, and prompts, expanding the API's capabilities.
- Implemented support for Server-Sent Events (SSE) in response handling, allowing for real-time updates and improved user experience.
- Refactored validation and error handling to differentiate between regular and SSE requests, ensuring appropriate responses based on the request type.
- Updated the GenerateWithAI method to support silent mode for regular HTTP requests, enhancing flexibility in content generation.
- Improved overall structure and maintainability of the API by consolidating request handling logic.
2024-12-19 16:27:51 +08:00
Max
fc1cbb2457 Add OPTIONS routes for new generation endpoints in Neo API
- Registered new OPTIONS routes for /generate, /generate/title, and /generate/prompts in the Neo API, enhancing CORS support for the recently introduced generation functionalities.
- This update ensures that the API can handle preflight requests for these endpoints, improving overall API usability and compliance with CORS standards.
2024-12-19 15:16:08 +08:00
Max
375ad96e9d Add generate endpoints and improve response handling in Neo API
- Introduced new API endpoints for generating custom content, chat titles, and prompts, enhancing the functionality of the Neo API.
- Implemented handleGenerateCustom, handleGenerateTitle, and handleGeneratePrompts methods to process generation requests with appropriate validation and error handling.
- Refactored the GenerateWithAI method to support different content types and system prompts, improving the flexibility of AI-generated responses.
- Updated the API router to include the new generation routes, ensuring seamless integration with existing middleware.
2024-12-19 15:13:38 +08:00
Max
ef6c39b87a
Merge pull request #811 from trheyi/main
Refactor Chat Assistant(50%)
2024-12-19 09:59:47 +08:00
Max
67c76fa933 Update go.mod and go.sum to upgrade golang.org/x/net to v0.33.0
- Updated the golang.org/x/net dependency from v0.27.0 to v0.33.0 in go.mod.
- Updated the corresponding checksums in go.sum to reflect the new version.
2024-12-19 09:56:43 +08:00
Max
35168845ae Enhance mentions handling in Neo API
- Convert keywords to lowercase for case-insensitive matching in handleMentions method.
- Introduce test data for mentions to improve testing and demonstration of functionality.
- Implement filtering of mentions based on keywords, allowing users to retrieve relevant mentions more effectively.
- Append test mentions to actual mentions before returning the response, enhancing the API's usability.
2024-12-17 18:45:16 +08:00
Max
13e17ae386 Enhance chat handling and API response in Neo
- Added time-based generation of chat IDs when not provided, improving chat session management.
- Updated context handling in handleChat to ensure valid chat IDs are used.
- Expanded allowed HTTP methods in CORS headers to include DELETE, enhancing API flexibility.
- Removed redundant chat history saving logic from GenerateChatTitle method, streamlining chat processing.
2024-12-17 17:04:22 +08:00
Max
9598d000cd Enhance chat retrieval in Xun by adding filtering and validation
- Updated GetChats method to filter out empty chat IDs, ensuring only valid chats are processed.
- Improved date handling by refining the logic for grouping chats into "This Week" and "Last Week" categories.
- Enhanced error handling for created_at field parsing, supporting multiple date formats for better robustness.
- Streamlined chat grouping logic for clearer organization of chat data in responses.
2024-12-17 11:18:00 +08:00
Max
fe8b3df163 Add chat deletion endpoints and handlers in Neo API
- Introduced new API endpoints for deleting individual chats and clearing all chats for a user, enhancing chat management capabilities.
- Implemented handleChatDelete and handleChatsDeleteAll methods to process deletion requests, including error handling for missing session and chat IDs.
- Updated API router to register new endpoints under the 'dangerous' path, ensuring clear separation of critical operations.
- Enhanced overall API structure and maintainability by adding robust feedback mechanisms for deletion operations.
2024-12-17 10:36:03 +08:00
Max
7f9307dbdd Add chat deletion functionality across conversation implementations
- Implemented DeleteChat and DeleteAllChats methods in Mongo, Redis, Weaviate, and Xun conversation handlers to allow users to delete specific chats and all chats associated with their account.
- Updated the Conversation interface in types.go to include the new deletion methods, ensuring consistent API across different storage backends.
- Added unit tests for DeleteChat and DeleteAllChats methods in xun_test.go to verify functionality and ensure proper deletion of chat histories.
- Enhanced error handling in deletion methods to provide robust feedback during chat removal operations.
2024-12-17 10:27:44 +08:00
Max
9db67a2456 Refactor chat retrieval in Neo API to support filtering and pagination
- Updated GetChats method across conversation implementations to accept a ChatFilter struct, enabling keyword filtering, pagination, and ordering.
- Enhanced handleChatList method in the Neo API to construct a filter from query parameters, improving the flexibility of chat retrieval.
- Introduced new types (ChatFilter, ChatGroup, ChatGroupResponse) to facilitate structured responses and better organization of chat data.
- Improved error handling and response structure for chat retrieval, ensuring robust feedback and clarity in API responses.
2024-12-17 10:18:37 +08:00
Max
f666070bb3 Add user field support in conversation settings and enhance user ID retrieval in Xun
- Introduced a new UserField in the Setting struct to allow customization of the user ID field name.
- Implemented getUserID method in Xun to retrieve user IDs based on the configured UserField, improving flexibility in user identification.
- Updated multiple methods (UpdateChatTitle, GetChats, GetHistory, SaveHistory, GetRequest, SaveRequest, GetChat) to utilize the new user ID retrieval logic, ensuring consistent handling of user sessions.
- Enhanced error handling for user ID retrieval, ensuring robust feedback in case of issues.
2024-12-17 09:54:57 +08:00
Max
b3bc843235 Enhance chat update functionality and introduce chat title generation in Neo API
- Updated handleChatUpdate method to include content field in the request body, allowing for dynamic chat title generation based on user input.
- Implemented GenerateChatTitle method to create a concise title for chats, improving user experience and interaction.
- Modified JSON response structure in handleChatDetail and handleChatUpdate methods for better clarity and consistency.
- Enhanced error handling for chat updates, ensuring robust validation and feedback for users.
2024-12-16 19:59:23 +08:00
Max
869ed3717a Add chat update functionality and enhance API endpoints in Neo API
- Introduced a new endpoint for updating chat details, allowing users to modify chat titles via the handleChatUpdate method.
- Implemented error handling for missing session IDs and chat IDs, ensuring robust validation of input parameters.
- Enhanced existing API structure by adding status, chat list, chat detail, history, file upload, download, and mentions endpoints, improving overall API usability.
- Improved code organization and maintainability by clearly defining new methods for handling chat updates and related functionalities.
2024-12-16 17:46:17 +08:00
Max
eadc550980 Add chat detail and mentions handling in Neo API
- Introduced new endpoints for retrieving chat details and mentions, enhancing the API's functionality.
- Implemented handleChatDetail method to fetch details of a specific chat by ID, including error handling for missing parameters.
- Added handleMentions method to retrieve mentions based on keywords, improving user interaction with chat content.
- Updated existing GetChats method to support keyword filtering, allowing for more refined chat list retrieval.
- Enhanced the DSL structure to include new methods for managing mentions and chat details, improving overall code organization and maintainability.
2024-12-16 17:36:02 +08:00
Max
2da7c30ac5 Refactor query handling in Xun conversation management
- Introduced a NewQuery method to streamline query creation, enhancing code readability and maintainability.
- Updated multiple methods (UpdateChatTitle, GetChats, GetHistory, SaveHistory, GetRequest, SaveRequest, clean) to utilize the new query method, reducing redundancy in query table references.
- Improved overall structure of the conversation management code by centralizing query logic.
2024-12-16 12:37:11 +08:00
Max
b34f97ac2a Add file download functionality to Neo API
- Introduced a new endpoint for downloading files, enhancing the API's capabilities.
- Implemented the handleDownload method to manage download requests, including validation for required parameters (sid and file_id).
- Updated the Download method in the assistant interface to retrieve files based on file_id, ensuring proper error handling and response structure.
- Enhanced file upload logic to include file extensions in generated IDs for better file management.
- Added necessary response headers for file downloads, improving user experience during file retrieval.
2024-12-16 11:07:22 +08:00
Max
3fbcd4a5e1 Add support for JSON file type in AllowedFileTypes across assistant modules 2024-12-15 09:54:18 +08:00
Max
773e050925 Implement file upload functionality in Neo API, adding new endpoint for file uploads and enhancing assistant management. Introduce handleUpload method to process file uploads, including size and type validation. Update context to include upload information and refactor assistant selection logic for improved clarity. Enhance error handling and response structure for file upload operations, ensuring robust communication of success and failure states. 2024-12-14 19:39:14 +08:00