Commit graph

11 commits

Author SHA1 Message Date
Max
455ed57231 Refactor job handling and enhance document processing features
- Renamed the Start method to Push for better semantic clarity in job execution.
- Introduced OnceAndSave, CronAndSave, and DaemonAndSave methods for immediate job creation and saving.
- Added new test cases for OnceAndSave, CronAndSave, and DaemonAndSave methods to ensure proper functionality.
- Updated AddFileProcess, AddTextProcess, and AddURLProcess to utilize new job handling methods, improving consistency across document operations.
- Enhanced error handling and logging in document processing functions for better traceability.
2025-09-02 10:05:51 +08:00
Max
96eef9e27d Implement document and collection count updates in API
- Added DocumentCount and UpdateDocumentCount methods to manage document counts in collections, enhancing metadata accuracy.
- Introduced RemoveDocumentsByCollectionID method for bulk document removal, improving collection management.
- Updated AddFileProcess, AddTextProcess, and AddURLProcess functions to include document and segment count updates after file operations.
- Enhanced RemoveCollection function to report the number of documents removed during collection deletion.
- Implemented segment count updates in RemoveSegments and RemoveSegmentsByDocID functions, ensuring accurate tracking of document segments.
2025-08-29 17:34:09 +08:00
Max
5f217bcd33 Implement segment retrieval enhancements in API
- Added error handling for segment not found and access denied scenarios in GetSegmentParents function.
- Implemented response formatting based on query parameters, including metadata options.
- Introduced formatSegmentTreeResponse function to structure segment data for API responses.
- Improved validation to ensure segments belong to the specified document, enhancing security and clarity in responses.
2025-08-24 10:26:06 +08:00
Max
58626f4fa9 Remove debug print statements from UpdateSegments function to clean up code and improve readability. 2025-08-21 16:43:17 +08:00
Max
5ea5c7835d Refactor document ID handling in API responses
- Updated API response structures across multiple endpoints to replace `document_id` with `doc_id` for consistency.
- Enhanced error handling in segment retrieval to ensure proper validation of document ownership.
- Improved response clarity by ensuring all relevant endpoints reflect the updated field naming convention.
2025-08-21 11:10:13 +08:00
Max
c3abc84057 Refactor API routes for collections and segments management
- Rearranged and updated API endpoints for collections, including adding GetCollections and adjusting the order of operations for better clarity.
- Enhanced segment management by introducing new endpoints for segment scores, weights, and votes, along with async operations for adding and updating segments.
- Improved error handling for missing document IDs in segment-related functions, ensuring robust validation.
- Streamlined response structures to include document and segment IDs in relevant endpoints, enhancing the API's usability.
2025-08-21 08:48:17 +08:00
Max
91d67d3b54 Enhance collection management and segment handling in the API
- Added GetCollection endpoint to retrieve collections by ID, including error handling for missing IDs and uninitialized knowledge base.
- Updated CreateCollectionConfig to use more descriptive field names for embedding provider and option IDs.
- Refactored UpdateSegments function to improve validation and error handling, ensuring document IDs and segment texts are properly checked.
- Introduced new request structures for updating votes, scores, and weights for segments, enhancing the API's capabilities.
- Removed the outdated vote handling file to streamline the codebase.
2025-08-20 18:25:28 +08:00
Max
c4c598db35 Refactor segment handling to improve pagination and endpoint clarity
- Replaced ListSegments function with ScrollSegments to implement iterator-style pagination.
- Updated related comments and error messages to reflect the new scrolling functionality.
- Streamlined API routes by removing the redundant ListSegments endpoint, enhancing overall clarity.
2025-08-15 16:43:22 +08:00
Max
451569e8cb Implement pagination and filtering in ListSegments function
- Enhanced ListSegments function to support pagination with customizable limit and offset parameters.
- Added error handling for missing docID and uninitialized knowledge base instance.
- Introduced filtering options for score, weight, and vote, allowing for refined segment retrieval.
- Improved response structure to return success or error messages based on the operation outcome.
2025-08-15 15:47:21 +08:00
Max
1c4fd3d20c Implement file and segment management API endpoints with error handling
- Added AddFile, AddText, AddURL, and AddSegments functions to handle file and segment uploads, including JSON request parsing and validation.
- Integrated error handling for invalid requests and uninitialized Knowledge Base instances, ensuring robust API responses.
- Enhanced response structures for success and error cases, improving consistency across the API.
- Updated AutoDetectConverter function to use a single content type parameter instead of multiple content types.
2025-07-25 16:35:19 +08:00
Max
aae1bccf88 Add Knowledge Base API endpoints and integrate with OpenAPI router
- Introduced the Knowledge Base (KB) API by creating a new kb package with various endpoints for collection and document management, segment management, and search functionalities.
- Updated the OpenAPI router to attach the KB API, ensuring OAuth protection for all endpoints.
- Implemented logging for configuration validation to enhance error handling and user feedback.
2025-07-23 15:19:40 +08:00