Commit graph

6 commits

Author SHA1 Message Date
Max
9326f4b747 feat(logging): enhance logger output control and add trace diagnostics
- Updated logger methods to respect the new `config.Silent` flag, allowing suppression of console output in development mode.
- Refactored `buildTurnResponse` to utilize a shared `buildTrace` function for improved tool call association.
- Introduced JSON output capabilities in `OutputWriter` for detailed trace and duration reporting.
- Enhanced reporting mechanisms to include trace details in JSON and Markdown formats for better diagnostics.
- Added support for a new `Trace` structure to capture detailed execution information, including tool calls and completion data.
2026-04-28 10:37:04 +08:00
Max
8de302d4e8 Implement Dynamic Testing Features in Agent Test Framework
- Introduced dynamic testing capabilities, allowing for multi-turn conversations with checkpoints through the new `DynamicRunner`.
- Enhanced the `runSingleTest` method to support dynamic mode, including the execution of before/after scripts and detailed output for dynamic test results.
- Added new output methods in `output.go` for dynamic test start, turns, checkpoints, and results, improving console feedback during testing.
- Updated `DESIGN_V2.md` and `TODO_V2.md` to reflect the new dynamic mode features, including simulator configurations and checkpoint definitions.
- Revised the `Case` struct in `types.go` to include fields for dynamic testing, such as `Simulator`, `Checkpoints`, and `MaxTurns`.
2025-12-26 11:17:54 +08:00
Max
6ea982f027 Implement Agent-Driven Input and Dry-Run Mode in Test Framework
- Added support for generating test cases using agents and scripts through the new `LoadFromAgent` and `LoadFromScript` methods in the loader.
- Enhanced the `RunTests` method to handle different input sources, including agent-driven and script-based test case generation.
- Introduced a `--dry-run` flag to allow users to preview generated test cases without executing them.
- Updated relevant documentation in DESIGN_V2.md and TODO_V2.md to reflect the new features and implementation status.
2025-12-26 10:53:11 +08:00
Max
5351128f89 Refactor Before/After Script Integration in Agent Test Framework
- Updated DESIGN_V2.md to reflect changes in the handling of before/after scripts, removing the `scripts:` prefix and clarifying their usage in JSONL test cases.
- Enhanced runner.go to integrate global before/after hooks, ensuring they execute correctly before and after test cases.
- Revised types.go to include new fields for before/after scripts in test case and options structures.
- Improved TODO_V2.md to track the implementation progress of before/after script functionality and related tasks.
- Added utility function LoadAgentTestScripts to facilitate loading of test scripts from the agent's src directory.
2025-12-26 09:44:56 +08:00
Max
5e64c78435 Implement Script Testing Framework and Enhance Test Context Management
- Introduced a new script testing mode to allow testing of agent handler scripts (hooks, tools, etc.) using a Go-like interface, enabling better unit testing of TypeScript/JavaScript code.
- Enhanced the `LoadScripts` function to skip test files during script loading, ensuring only relevant scripts are processed.
- Refactored the test context creation to support custom context configurations via a JSON file, allowing for flexible authorization and metadata management during tests.
- Updated the test runner to handle script tests, including the ability to filter tests using regex patterns and manage custom context data.
- Improved documentation to include details on script testing usage, input formats, and available assertions, enhancing developer experience and clarity.
2025-12-21 10:44:59 +08:00
Max
d21f9c3769 Enhance Logging Functionality in RequestLogger
- Added a `noop` check in multiple logging methods (`LLMComplete`, `ToolStart`, `ToolComplete`, `HookStart`, `HookComplete`, and `HistoryLoad`) to prevent logging when the logger is in no-operation mode.
- Improved command handling in `root.go` by removing minimum argument requirements for commands and providing help output when no arguments are given.
- Introduced an `agent` command for better organization of agent-related functionalities in the CLI.
- Implemented automatic detection of the application root directory in `run.go` to streamline the application startup process.
- Cleaned up debug print statements in `config.go` to reduce clutter in the output.
2025-12-17 12:33:34 +08:00