opencode/picoclaw
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

restore old bytes read_file tool

Browse source
This commit is contained in:
afjcjsbx 2026-03-27 21:25:07 +01:00
parent a731151d5e
commit 6090ab1d20
4 changed files with 516 additions and 669 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

25
docs/configuration.md
View file

@ -293,8 +293,10 @@ Even with `restrict_to_workspace: false`, the `exec` tool blocks these dangerous
| Config Key | Type | Default | Description | | Config Key | Type | Default | Description |
|------------|------|---------|-------------| |------------|------|---------|-------------|
| `tools.read_file.enabled` | bool | `true` | Enables the `read_file` tool |
| `tools.read_file.mode` | string | `bytes` | Selects the `read_file` implementation: `bytes` or `lines` | | `tools.read_file.mode` | string | `bytes` | Selects the `read_file` implementation: `bytes` or `lines` |
| `tools.read_file.max_read_file_size` | int | `65536` | Maximum bytes returned in `bytes` mode and the output budget source for `lines` mode truncation | | `tools.read_file.max_read_file_size` | int | `65536` | Maximum bytes returned by `read_file` and byte budget used by `read_file_lines` |
| `tools.read_file_lines.enabled` | bool | `false` | Enables the separate line-oriented `read_file_lines` tool |
#### Mode: `bytes` #### Mode: `bytes`
@ -314,22 +316,21 @@ Use `bytes` when:
#### Mode: `lines` #### Mode: `lines`
Text-oriented behavior, optimized for source files, markdown, logs, and configs. Text-oriented behavior, optimized for source files, markdown, logs, and configs. The tool reads sequentially by line and stops when the configured byte budget is reached.
Parameters: Parameters:
* `path` (required): File path * `path` (required): File path
* `offset` (optional): Starting line number, 1-indexed, default `1` * `offset` (optional): Starting line number, 1-indexed and inclusive, default `0`
* `limit` (optional): Maximum number of lines to read, default = all remaining lines * `limit` (optional): Maximum number of lines to read, default = all remaining lines until EOF or byte budget
When content exceeds the estimated token budget, PicoClaw truncates it intelligently: Behavior notes:
* Estimates the token count heuristically * Binary-looking files are rejected with guidance to use `read_file`
* Keeps the head and tail of the selected text * Extremely long single lines are truncated rather than skipped
* Cuts on newline boundaries when possible * If truncation happens mid-line, the tool explicitly suggests falling back to `read_file` for byte-wise inspection
* Inserts an explicit truncation notice in the middle
Use `lines` when: Use `read_file_lines` when:
* The agent mostly reads text files * The agent mostly reads text files
* You want line-based pagination in prompts and tool calls * You want line-based pagination in prompts and tool calls
@ -342,8 +343,10 @@ Use `lines` when:
"tools": { "tools": {
"read_file": { "read_file": {
"enabled": true, "enabled": true,
"mode": "lines",
"max_read_file_size": 65536 "max_read_file_size": 65536
},
"read_file_lines": {
"enabled": true
} }
} }
} }

281
docs/it/configuration.md
View file

@ -1,281 +0,0 @@
# ⚙️ Guida alla Configurazione
> Torna al [README](../../README.md)
## ⚙️ Configurazione
File di configurazione: `~/.picoclaw/config.json`
### Variabili d'Ambiente
Puoi sovrascrivere i percorsi predefiniti usando variabili d'ambiente. Questo è utile per installazioni portatili, distribuzioni containerizzate, o per eseguire picoclaw come servizio di sistema. Queste variabili sono indipendenti e controllano percorsi diversi.
| Variabile | Descrizione | Percorso Predefinito |
|-------------------|-----------------------------------------------------------------------------------------------------------------------------------------|---------------------------|
| `PICOCLAW_CONFIG` | Sovrascrive il percorso al file di configurazione. Indica direttamente a picoclaw quale `config.json` caricare, ignorando tutte le altre posizioni. | `~/.picoclaw/config.json` |
| `PICOCLAW_HOME` | Sovrascrive la directory radice per i dati di picoclaw. Modifica la posizione predefinita del `workspace` e delle altre directory dati. | `~/.picoclaw` |
**Esempi:**
```bash
# Esegui picoclaw usando un file di configurazione specifico
# Il percorso del workspace verrà letto da quel file di configurazione
PICOCLAW_CONFIG=/etc/picoclaw/production.json picoclaw gateway
# Esegui picoclaw con tutti i dati salvati in /opt/picoclaw
# La configurazione verrà caricata dal percorso predefinito ~/.picoclaw/config.json
# Il workspace verrà creato in /opt/picoclaw/workspace
PICOCLAW_HOME=/opt/picoclaw picoclaw agent
# Usa entrambi per un setup completamente personalizzato
PICOCLAW_HOME=/srv/picoclaw PICOCLAW_CONFIG=/srv/picoclaw/main.json picoclaw gateway
```
### Struttura del Workspace
PicoClaw salva i dati nel workspace configurato (predefinito: `~/.picoclaw/workspace`):
```
~/.picoclaw/workspace/
├── sessions/ # Sessioni di conversazione e cronologia
├── memory/ # Memoria a lungo termine (MEMORY.md)
├── state/ # Stato persistente (ultimo canale, ecc.)
├── cron/ # Database dei job pianificati
├── skills/ # Skill personalizzate
├── AGENTS.md # Guida al comportamento dell'agent
├── HEARTBEAT.md # Prompt per task periodici (controllato ogni 30 min)
├── IDENTITY.md # Identità dell'agent
├── SOUL.md # Anima dell'agent
└── USER.md # Preferenze dell'utente
```
> **Nota:** Le modifiche a `AGENTS.md`, `SOUL.md`, `USER.md`, `IDENTITY.md` e `memory/MEMORY.md` vengono rilevate automaticamente a runtime tramite il tracciamento della data di modifica (mtime). **Non è necessario riavviare il gateway** dopo aver modificato questi file — l'agent caricherà il nuovo contenuto alla prossima richiesta.
### Sorgenti delle Skill
Per impostazione predefinita, le skill vengono caricate da:
1. `~/.picoclaw/workspace/skills` (workspace)
2. `~/.picoclaw/skills` (globale)
3. `<current-working-directory>/skills` (builtin)
Per configurazioni avanzate/di test, puoi sovrascrivere la directory radice delle skill builtin con:
```bash
export PICOCLAW_BUILTIN_SKILLS=/path/to/skills
```
### Politica Unificata di Esecuzione dei Comandi
- I comandi slash generici vengono eseguiti tramite un unico percorso in `pkg/agent/loop.go` via `commands.Executor`.
- Gli adattatori dei canali non consumano più localmente i comandi generici; inoltrano il testo in entrata al percorso bus/agent. Telegram registra ancora automaticamente i comandi supportati all'avvio.
- Un comando slash sconosciuto (ad esempio `/foo`) viene passato all'elaborazione LLM come se fosse un messaggio dell'utente.
- Un comando registrato ma non supportato sul canale corrente (ad esempio `/show` su WhatsApp) restituisce un errore esplicito all'utente e interrompe l'elaborazione.
### 🔒 Sandbox di Sicurezza
PicoClaw esegue in un ambiente sandboxed per impostazione predefinita. L'agent può accedere solo ai file ed eseguire comandi all'interno del workspace configurato.
#### Configurazione Predefinita
```json
{
"agents": {
"defaults": {
"workspace": "~/.picoclaw/workspace",
"restrict_to_workspace": true
}
}
}
```
| Opzione | Predefinito | Descrizione |
| ----------------------- | ----------------------- | ---------------------------------------------------- |
| `workspace` | `~/.picoclaw/workspace` | Directory di lavoro dell'agent |
| `restrict_to_workspace` | `true` | Limita l'accesso a file/comandi al workspace |
#### Strumenti Protetti
Quando `restrict_to_workspace: true`, i seguenti strumenti sono in sandbox:
| Strumento | Funzione | Restrizione |
| ------------- | ------------------------- | ---------------------------------------------------- |
| `read_file` | Legge file | Solo file all'interno del workspace |
| `write_file` | Scrive file | Solo file all'interno del workspace |
| `list_dir` | Elenca directory | Solo directory all'interno del workspace |
| `edit_file` | Modifica file | Solo file all'interno del workspace |
| `append_file` | Aggiunge ai file | Solo file all'interno del workspace |
| `exec` | Esegue comandi | I percorsi dei comandi devono essere nel workspace |
#### Protezione Exec Aggiuntiva
Anche con `restrict_to_workspace: false`, lo strumento `exec` blocca questi comandi pericolosi:
* `rm -rf`, `del /f`, `rmdir /s` — Cancellazione di massa
* `format`, `mkfs`, `diskpart` — Formattazione del disco
* `dd if=` — Imaging del disco
* Scrittura su `/dev/sd[a-z]` — Scritture dirette su disco
* `shutdown`, `reboot`, `poweroff` — Spegnimento del sistema
* Fork bomb `:(){ :|:& };:`
### Controllo Accesso ai File
| Chiave di configurazione | Tipo | Predefinito | Descrizione |
|--------------------------|------|-------------|-------------|
| `tools.allow_read_paths` | string[] | `[]` | Percorsi aggiuntivi consentiti per la lettura al di fuori del workspace |
| `tools.allow_write_paths` | string[] | `[]` | Percorsi aggiuntivi consentiti per la scrittura al di fuori del workspace |
### Modalità Read File
`read_file` ha due implementazioni mutuamente esclusive, selezionate via configurazione. PicoClaw ne registra esattamente una all'avvio:
| Chiave di configurazione | Tipo | Predefinito | Descrizione |
|--------------------------|------|-------------|-------------|
| `tools.read_file.mode` | string | `bytes` | Seleziona l'implementazione di `read_file`: `bytes` oppure `lines` |
| `tools.read_file.max_read_file_size` | int | `65536` | Numero massimo di byte restituiti in modalità `bytes` e base del budget di output per la truncation in modalità `lines` |
#### Modalità: `bytes`
Comportamento legacy, ottimizzato per file arbitrari e paginazione byte-safe.
Parametri:
* `path` (obbligatorio): percorso del file
* `offset` (opzionale): offset iniziale in byte, default `0`
* `length` (opzionale): numero massimo di byte da leggere, default `max_read_file_size`
Usa `bytes` quando:
* Ti serve retrocompatibilità con prompt o agent esistenti
* Potresti leggere file binari
* Vuoi una paginazione deterministica a byte
#### Modalità: `lines`
Comportamento orientato al testo, ottimizzato per sorgenti, markdown, log e file di configurazione.
Parametri:
* `path` (obbligatorio): percorso del file
* `offset` (opzionale): numero di riga iniziale, 1-indexed, default `1`
* `limit` (opzionale): numero massimo di righe da leggere, default = tutte le righe rimanenti
Quando il contenuto supera il budget token stimato, PicoClaw applica una truncation intelligente:
* Stima il numero di token in modo euristico
* Mantiene testa e coda del testo selezionato
* Tronca, quando possibile, sui boundary di newline
* Inserisce al centro un avviso esplicito di contenuto troncato
Usa `lines` quando:
* L'agent legge soprattutto file testuali
* Vuoi paginazione a righe nei prompt e nei tool call
* Vuoi chunk più leggibili per code review, log e documentazione
#### Esempio
```json
{
"tools": {
"read_file": {
"enabled": true,
"mode": "lines",
"max_read_file_size": 65536
}
}
}
```
### Sicurezza Exec
| Chiave di configurazione | Tipo | Predefinito | Descrizione |
|--------------------------|------|-------------|-------------|
| `tools.exec.allow_remote` | bool | `false` | Consente lo strumento exec da canali remoti (Telegram/Discord ecc.) |
| `tools.exec.enable_deny_patterns` | bool | `true` | Abilita l'intercettazione dei comandi pericolosi |
| `tools.exec.custom_deny_patterns` | string[] | `[]` | Pattern regex personalizzati da bloccare |
| `tools.exec.custom_allow_patterns` | string[] | `[]` | Pattern regex personalizzati da consentire |
> **Nota di sicurezza:** La protezione dei symlink è abilitata per impostazione predefinita — tutti i percorsi file vengono risolti tramite `filepath.EvalSymlinks` prima del confronto con la whitelist, prevenendo attacchi di escape tramite symlink.
#### Limitazione Nota: Processi Figlio degli Strumenti di Build
Il controllo di sicurezza exec ispeziona solo la riga di comando avviata direttamente da PicoClaw. Non ispeziona ricorsivamente i processi figlio generati da strumenti di sviluppo consentiti come `make`, `go run`, `cargo`, `npm run` o script di build personalizzati.
Ciò significa che un comando di primo livello può comunque compilare o avviare altri binari dopo aver superato il controllo iniziale. In pratica, tratta gli script di build, i Makefile, gli script di pacchetti e i binari generati come codice eseguibile che richiede lo stesso livello di revisione di un comando shell diretto.
Per ambienti ad alto rischio:
* Esamina gli script di build prima dell'esecuzione.
* Preferisci l'approvazione/revisione manuale per i workflow di compilazione ed esecuzione.
* Esegui PicoClaw in un container o VM se hai bisogno di un isolamento più forte di quello fornito dal controllo integrato.
#### Esempi di Errore
```
[ERROR] tool: Tool execution failed
{tool=exec, error=Command blocked by safety guard (path outside working dir)}
```
```
[ERROR] tool: Tool execution failed
{tool=exec, error=Command blocked by safety guard (dangerous pattern detected)}
```
#### Disabilitare le Restrizioni (Rischio di Sicurezza)
Se hai bisogno che l'agent acceda a percorsi al di fuori del workspace:
**Metodo 1: File di configurazione**
```json
{
"agents": {
"defaults": {
"restrict_to_workspace": false
}
}
}
```
**Metodo 2: Variabile d'ambiente**
```bash
export PICOCLAW_AGENTS_DEFAULTS_RESTRICT_TO_WORKSPACE=false
```
> ⚠️ **Attenzione**: Disabilitare questa restrizione consente all'agent di accedere a qualsiasi percorso sul tuo sistema. Usare con cautela solo in ambienti controllati.
#### Coerenza dei Confini di Sicurezza
L'impostazione `restrict_to_workspace` si applica in modo coerente a tutti i percorsi di esecuzione:
| Percorso di esecuzione | Confine di sicurezza |
| ---------------------- | --------------------------------- |
| Main Agent | `restrict_to_workspace` ✅ |
| Subagent / Spawn | Eredita la stessa restrizione ✅ |
| Heartbeat tasks | Eredita la stessa restrizione ✅ |
Tutti i percorsi condividono la stessa restrizione del workspace — non è possibile aggirare il confine di sicurezza tramite subagent o task pianificati.
### Heartbeat (Task Periodici)
PicoClaw può eseguire task periodici automaticamente. Crea un file `HEARTBEAT.md` nel tuo workspace:
```markdown
# Periodic Tasks
- Check my email for important messages
- Review my calendar for upcoming events
- Check the weather forecast
```
L'agent leggerà questo file ogni 30 minuti (configurabile) ed eseguirà tutti i task usando gli strumenti disponibili.
#### Task Asincroni con Spawn
Per task di lunga durata (ricerca web, chiamate API), usa lo strumento `spawn` per creare un **subagent**:
```markdown
# Periodic Tasks
```

604
pkg/tools/filesystem.go
View file

@ -4,7 +4,9 @@ import (
"bufio" "bufio"
"bytes" "bytes"
"context" "context"
"errors"
"fmt" "fmt"
"io"
"io/fs" "io/fs"
"math" "math"
"net/http" "net/http"
@ -18,13 +20,10 @@ import (
"github.com/sipeed/picoclaw/pkg/fileutil" "github.com/sipeed/picoclaw/pkg/fileutil"
"github.com/sipeed/picoclaw/pkg/logger" "github.com/sipeed/picoclaw/pkg/logger"
"github.com/sipeed/picoclaw/pkg/routing"
) )
const MaxReadFileSize = 64 * 1024 // 64KB limit to avoid context overflow const MaxReadFileSize = 64 * 1024 // 64KB limit to avoid context overflow
const readFileTruncationSafetyMargin = 0.95
func validatePathWithAllowPaths(path, workspace string, restrict bool, patterns []*regexp.Regexp) (string, error) { func validatePathWithAllowPaths(path, workspace string, restrict bool, patterns []*regexp.Regexp) (string, error) {
if workspace == "" { if workspace == "" {
return path, fmt.Errorf("workspace is not defined") return path, fmt.Errorf("workspace is not defined")
@ -254,14 +253,11 @@ func isWithinWorkspace(candidate, workspace string) bool {
} }
type ReadFileTool struct { type ReadFileTool struct {
readFileBase fs fileSystem
maxSize int64
} }
type ReadFileLinesTool struct { type ReadFileLinesTool struct {
readFileBase
}
type readFileBase struct {
fs fileSystem fs fileSystem
maxSize int64 maxSize int64
} }
@ -271,15 +267,6 @@ func NewReadFileTool(
restrict bool, restrict bool,
maxReadFileSize int, maxReadFileSize int,
allowPaths ...[]*regexp.Regexp, allowPaths ...[]*regexp.Regexp,
) *ReadFileTool {
return NewReadFileBytesTool(workspace, restrict, maxReadFileSize, allowPaths...)
}
func NewReadFileBytesTool(
workspace string,
restrict bool,
maxReadFileSize int,
allowPaths ...[]*regexp.Regexp,
) *ReadFileTool { ) *ReadFileTool {
var patterns []*regexp.Regexp var patterns []*regexp.Regexp
if len(allowPaths) > 0 { if len(allowPaths) > 0 {
@ -292,13 +279,20 @@ func NewReadFileBytesTool(
} }
return &ReadFileTool{ return &ReadFileTool{
readFileBase: readFileBase{ fs: buildFs(workspace, restrict, patterns),
fs: buildFs(workspace, restrict, patterns), maxSize: maxSize,
maxSize: maxSize,
},
} }
} }
func NewReadFileBytesTool(
workspace string,
restrict bool,
maxReadFileSize int,
allowPaths ...[]*regexp.Regexp,
) *ReadFileTool {
return NewReadFileTool(workspace, restrict, maxReadFileSize, allowPaths...)
}
func NewReadFileLinesTool( func NewReadFileLinesTool(
workspace string, workspace string,
restrict bool, restrict bool,
@ -316,10 +310,8 @@ func NewReadFileLinesTool(
} }
return &ReadFileLinesTool{ return &ReadFileLinesTool{
readFileBase: readFileBase{ fs: buildFs(workspace, restrict, patterns),
fs: buildFs(workspace, restrict, patterns), maxSize: maxSize,
maxSize: maxSize,
},
} }
} }
@ -328,15 +320,15 @@ func (t *ReadFileTool) Name() string {
} }
func (t *ReadFileLinesTool) Name() string { func (t *ReadFileLinesTool) Name() string {
return "read_file" return "read_file_lines"
} }
func (t *ReadFileTool) Description() string { func (t *ReadFileTool) Description() string {
return "Read the contents of a file using byte-based pagination via `offset` and `length`." return "Read the contents of a file. Supports pagination via `offset` and `length`."
} }
func (t *ReadFileLinesTool) Description() string { func (t *ReadFileLinesTool) Description() string {
return "Read file contents from the filesystem. Output always includes line numbers in the format `LINE_NUMBER|LINE_CONTENT` (1-indexed). Supports partial reads via `offset` and `limit` for large text files." return "Read a UTF-8 text file from a specific line range. Uses 1-indexed line offsets and stops when the configured byte budget is reached."
} }
func (t *ReadFileTool) Parameters() map[string]any { func (t *ReadFileTool) Parameters() map[string]any {
@ -370,14 +362,14 @@ func (t *ReadFileLinesTool) Parameters() map[string]any {
"type": "string", "type": "string",
"description": "Path to the file to read.", "description": "Path to the file to read.",
}, },
"offset": map[string]any{ "start_line": map[string]any{
"type": "integer", "type": "integer",
"description": "Starting line number (1-indexed). Use for large files to read from a specific line.", "description": "Line number to start reading from (1-indexed, inclusive).",
"default": 1, "default": 1,
}, },
"limit": map[string]any{ "max_lines": map[string]any{
"type": "integer", "type": "integer",
"description": "Number of lines to read. Use with offset for large files to read in chunks.", "description": "Maximum number of lines to read.",
}, },
}, },
"required": []string{"path"}, "required": []string{"path"},
@ -390,114 +382,7 @@ func (t *ReadFileTool) Execute(ctx context.Context, args map[string]any) *ToolRe
return ErrorResult("path is required") return ErrorResult("path is required")
} }
data, err := t.fs.ReadFile(path) // offset (optional, default 0)
if err != nil {
return ErrorResult(err.Error())
}
return t.executeByteRead(path, data, args)
}
func (t *ReadFileLinesTool) Execute(ctx context.Context, args map[string]any) *ToolResult {
path, ok := args["path"].(string)
if !ok {
return ErrorResult("path is required")
}
data, err := t.fs.ReadFile(path)
if err != nil {
return ErrorResult(err.Error())
}
if isBinaryReadFileData(data) {
return ErrorResult(`file appears to be binary`)
}
return t.executeLineRead(path, data, args)
}
func (t *readFileBase) executeLineRead(path string, data []byte, args map[string]any) *ToolResult {
offset, err := getInt64Arg(args, "offset", 1)
if err != nil {
return ErrorResult(err.Error())
}
if offset < 1 {
return ErrorResult("offset must be >= 1")
}
limit := int64(-1)
if _, exists := args["limit"]; exists {
limit, err = getInt64Arg(args, "limit", -1)
if err != nil {
return ErrorResult(err.Error())
}
if limit <= 0 {
return ErrorResult("limit must be > 0")
}
}
lines := splitLinesPreserveNewlines(string(data))
totalLines := int64(len(lines))
if totalLines == 0 {
return NewToolResult("[END OF FILE - no content at this offset]")
}
startIdx := offset - 1
if startIdx >= totalLines {
return NewToolResult("[END OF FILE - no content at this offset]")
}
endExclusive := totalLines
if limit > 0 {
endExclusive = startIdx + limit
if endExclusive > totalLines {
endExclusive = totalLines
}
}
selectedLines := lines[startIdx:endExclusive]
if len(selectedLines) == 0 {
return NewToolResult("[END OF FILE - no content at this offset]")
}
formatted := make([]string, 0, len(lines))
for _, line := range lines {
// Remove only final carriage returns to normalize
lineContent := strings.TrimRight(line, "\r\n")
formatted = append(formatted, lineContent)
}
content := strings.Join(formatted, "\n")
maxTokens := t.maxTokenBudget()
content = truncateReadFileContent(content, maxTokens)
readEndLine := startIdx + int64(len(selectedLines))
readRange := fmt.Sprintf("lines %d-%d", offset, readEndLine)
displayPath := filepath.Base(path)
header := fmt.Sprintf(
"[file: %s | total: %d lines | read: %s]",
displayPath, totalLines, readRange,
)
hasMore := endExclusive < totalLines
if hasMore {
header += fmt.Sprintf(
"\n[TRUNCATED - file has more content. Call read_file again with offset=%d to continue.]",
readEndLine+1,
)
}
logger.DebugCF("tool", "ReadFileTool execution completed successfully",
map[string]any{
"path": path,
"lines_read": len(selectedLines),
"has_more": hasMore,
})
return NewToolResult(header + "\n\n" + content)
}
func (t *readFileBase) executeByteRead(path string, data []byte, args map[string]any) *ToolResult {
offset, err := getInt64Arg(args, "offset", 0) offset, err := getInt64Arg(args, "offset", 0)
if err != nil { if err != nil {
return ErrorResult(err.Error()) return ErrorResult(err.Error())
@ -506,6 +391,7 @@ func (t *readFileBase) executeByteRead(path string, data []byte, args map[string
return ErrorResult("offset must be >= 0") return ErrorResult("offset must be >= 0")
} }
// length (optional, capped at MaxReadFileSize)
length, err := getInt64Arg(args, "length", t.maxSize) length, err := getInt64Arg(args, "length", t.maxSize)
if err != nil { if err != nil {
return ErrorResult(err.Error()) return ErrorResult(err.Error())
@ -517,28 +403,105 @@ func (t *readFileBase) executeByteRead(path string, data []byte, args map[string
length = t.maxSize length = t.maxSize
} }
totalSize := int64(len(data)) file, err := t.fs.Open(path)
if offset >= totalSize { if err != nil {
return ErrorResult(err.Error())
}
defer file.Close()
// measure total size
totalSize := int64(-1) // -1 means unknown
if info, statErr := file.Stat(); statErr == nil {
totalSize = info.Size()
}
// sniff the first 512 bytes to detect binary content before loading
// it into the LLM context. Seeking back to 0 afterwards restores state.
sniff := make([]byte, 512)
sniffN, _ := file.Read(sniff)
// Reset read position to beginning before applying the caller's offset.
if seeker, ok := file.(io.Seeker); ok {
_, err = seeker.Seek(0, io.SeekStart)
if err != nil {
return ErrorResult(fmt.Sprintf("failed to reset file position after sniff: %v", err))
}
} else {
// Non-seekable: we consumed sniffN bytes above; account for them when
// discarding to reach the requested offset below.
// If offset < sniffN the data we already read covers it, which we
// cannot replay on a non-seekable stream — return a clear error.
if offset < int64(sniffN) && offset > 0 {
return ErrorResult(
"non-seekable file: cannot seek to an offset within the first 512 bytes after binary detection",
)
}
}
// Seek to the requested offset.
if seeker, ok := file.(io.Seeker); ok {
_, err = seeker.Seek(offset, io.SeekStart)
if err != nil {
return ErrorResult(fmt.Sprintf("failed to seek to offset %d: %v", offset, err))
}
} else if offset > 0 {
// Fallback for non-seekable streams: discard leading bytes.
// sniffN bytes were already consumed above, so subtract them.
remaining := offset - int64(sniffN)
if remaining > 0 {
_, err = io.CopyN(io.Discard, file, remaining)
if err != nil {
return ErrorResult(fmt.Sprintf("failed to advance to offset %d: %v", offset, err))
}
}
}
// read length+1 bytes to reliably detect whether more content exists
// without relying on totalSize (which may be -1 for non-seekable streams).
// This avoids the false-positive TRUNCATED message on the last page.
probe := make([]byte, length+1)
n, err := io.ReadFull(file, probe)
// FIX: io.ReadFull returns io.ErrUnexpectedEOF for partial reads (0 < n < len),
// and io.EOF only when n == 0. Both are normal terminal conditions — only
// other errors are genuine failures.
if err != nil && err != io.EOF && !errors.Is(err, io.ErrUnexpectedEOF) {
return ErrorResult(fmt.Sprintf("failed to read file content: %v", err))
}
// hasMore is true only when we actually got the extra probe byte.
hasMore := int64(n) > length
data := probe[:min(int64(n), length)]
if len(data) == 0 {
return NewToolResult("[END OF FILE - no content at this offset]") return NewToolResult("[END OF FILE - no content at this offset]")
} }
end := offset + length // Build metadata header.
if end > totalSize { // use filepath.Base(path) instead of the raw path to avoid leaking
end = totalSize // internal filesystem structure into the LLM context.
} readEnd := offset + int64(len(data))
chunk := data[offset:end] // use ASCII hyphen-minus instead of en-dash (U+2013) to keep the
readRange := fmt.Sprintf("bytes %d-%d", offset, end-1) // header parseable by downstream tools and log processors.
displayPath := filepath.Base(path) readRange := fmt.Sprintf("bytes %d-%d", offset, readEnd-1)
header := fmt.Sprintf(
"[file: %s | total: %d bytes | read: %s]", displayPath := filepath.Base(path)
displayPath, totalSize, readRange, var header string
) if totalSize >= 0 {
header = fmt.Sprintf(
"[file: %s | total: %d bytes | read: %s]",
displayPath, totalSize, readRange,
)
} else {
header = fmt.Sprintf(
"[file: %s | read: %s | total size unknown]",
displayPath, readRange,
)
}
hasMore := end < totalSize
if hasMore { if hasMore {
header += fmt.Sprintf( header += fmt.Sprintf(
"\n[TRUNCATED - file has more content. Call read_file again with offset=%d to continue.]", "\n[TRUNCATED - file has more content. Call read_file again with offset=%d to continue.]",
end, readEnd,
) )
} else { } else {
header += "\n[END OF FILE - no further content.]" header += "\n[END OF FILE - no further content.]"
@ -547,36 +510,162 @@ func (t *readFileBase) executeByteRead(path string, data []byte, args map[string
logger.DebugCF("tool", "ReadFileTool execution completed successfully", logger.DebugCF("tool", "ReadFileTool execution completed successfully",
map[string]any{ map[string]any{
"path": path, "path": path,
"bytes_read": len(chunk), "bytes_read": len(data),
"has_more": hasMore, "has_more": hasMore,
"mode": "bytes",
}) })
return NewToolResult(header + "\n\n" + string(chunk)) return NewToolResult(header + "\n\n" + string(data))
} }
func (t *readFileBase) maxTokenBudget() int { func (t *ReadFileLinesTool) Execute(ctx context.Context, args map[string]any) *ToolResult {
if t.maxSize <= 0 { path, ok := args["path"].(string)
return MaxReadFileSize / 4 if !ok {
return ErrorResult("path is required")
} }
budget := int(t.maxSize / 4) startLine, err := getInt64Arg(args, "start_line", 0)
if budget <= 0 { if err != nil {
return 1 return ErrorResult(err.Error())
} }
return budget if startLine < 1 {
} return ErrorResult("offset must be >= 1")
func splitLinesPreserveNewlines(text string) []string {
if text == "" {
return nil
} }
lines := strings.SplitAfter(text, "\n") limit := int64(-1)
if len(lines) > 0 && lines[len(lines)-1] == "" { if raw, exists := args["limit"]; exists && raw != nil {
lines = lines[:len(lines)-1] limit, err = getInt64Arg(args, "limit", -1)
if err != nil {
return ErrorResult(err.Error())
}
if limit <= 0 {
return ErrorResult("limit, if provided, must be > 0")
}
} }
return lines
file, err := t.fs.Open(path)
if err != nil {
return ErrorResult(err.Error())
}
defer file.Close()
if info, statErr := file.Stat(); statErr == nil && info.IsDir() {
return ErrorResult(fmt.Sprintf("failed to open file: path is a directory: %s", path))
}
sample := make([]byte, 512)
sampleN, readErr := file.Read(sample)
if readErr != nil && readErr != io.EOF {
return ErrorResult(fmt.Sprintf("failed to read file: %v", readErr))
}
sample = sample[:sampleN]
if isBinaryReadFileData(sample) {
return ErrorResult("file appears to be binary; use read_file for byte-based inspection")
}
reader := bufio.NewReaderSize(io.MultiReader(bytes.NewReader(sample), file), 32*1024)
var content strings.Builder
var lineIndex int64
var linesRead int64
var bytesRead int64
var reachedEOF bool
var byteBudgetTruncated bool
var lineTruncated bool
for lineIndex < startLine {
hasLine, consumeErr := consumeNextLine(reader)
if consumeErr != nil {
return ErrorResult(fmt.Sprintf("failed to read file content: %v", consumeErr))
}
if !hasLine {
reachedEOF = true
break
}
lineIndex++
}
for !reachedEOF && (limit < 0 || linesRead < limit) {
remaining := t.maxSize - bytesRead
if remaining <= 0 {
byteBudgetTruncated = true
break
}
line, complete, hasLine, readLineErr := readNextLinePrefix(reader, remaining)
if readLineErr != nil {
return ErrorResult(fmt.Sprintf("failed to read file content: %v", readLineErr))
}
if !hasLine {
reachedEOF = true
break
}
content.Write(line)
bytesRead += int64(len(line))
linesRead++
lineIndex++
if !complete {
byteBudgetTruncated = true
lineTruncated = true
break
}
}
if !reachedEOF && !lineTruncated {
hasMoreContent, peekErr := readerHasMoreContent(reader)
if peekErr != nil {
return ErrorResult(fmt.Sprintf("failed to inspect remaining file content: %v", peekErr))
}
if !hasMoreContent {
reachedEOF = true
byteBudgetTruncated = false
}
}
if linesRead == 0 && content.Len() == 0 {
return NewToolResult("[END OF FILE - no content at this offset]")
}
start := startLine
endLine := startLine + linesRead - 1
displayPath := filepath.Base(path)
header := fmt.Sprintf(
"[file: %s | read: lines %d-%d (0-indexed) | bytes: %d]",
displayPath, start, endLine, bytesRead,
)
switch {
case lineTruncated:
header += fmt.Sprintf(
"\n[TRUNCATED - line %d exceeded the %d byte read budget and was cut mid-line. Use read_file for byte-wise inspection of the remaining content.]",
endLine,
t.maxSize,
)
case byteBudgetTruncated:
header += fmt.Sprintf(
"\n[TRUNCATED - byte budget reached. Call read_file_lines again with offset=%d to continue at the next line.]",
startLine+linesRead,
)
case !reachedEOF && limit > 0 && linesRead >= limit:
header += fmt.Sprintf(
"\n[PARTIAL - more content remains. Call read_file_lines again with offset=%d to continue.]",
startLine+linesRead,
)
default:
header += "\n[END OF FILE - no further content.]"
}
logger.DebugCF("tool", "ReadFileTool execution completed successfully",
map[string]any{
"path": path,
"lines_read": linesRead,
"bytes_read": bytesRead,
"truncated": byteBudgetTruncated,
"tool": t.Name(),
})
return NewToolResult(header + "\n\n" + content.String())
} }
func isBinaryReadFileData(data []byte) bool { func isBinaryReadFileData(data []byte) bool {
@ -619,119 +708,82 @@ func isBinaryReadFileData(data []byte) bool {
return float64(controlChars)/float64(len(sample)) > 0.1 return float64(controlChars)/float64(len(sample)) > 0.1
} }
func truncateReadFileContent(text string, maxTokens int) string { func consumeNextLine(reader *bufio.Reader) (bool, error) {
if strings.TrimSpace(text) == "" || maxTokens <= 0 { sawData := false
return text
}
tokenCount := routing.EstimateTokens(text) for {
if tokenCount <= maxTokens { fragment, err := reader.ReadSlice('\n')
return text if len(fragment) > 0 {
} sawData = true
totalChars := utf8.RuneCountInString(text)
if totalChars == 0 {
return text
}
tokenPerChar := float64(tokenCount) / float64(totalChars)
if tokenPerChar <= 0 {
return text
}
allowedChars := int(float64(maxTokens)/tokenPerChar*readFileTruncationSafetyMargin + 0.5)
if allowedChars < 2 {
allowedChars = 2
}
if allowedChars >= totalChars {
return text
}
headChars := allowedChars / 2
tailChars := allowedChars - headChars
headRaw := firstNRunes(text, headChars)
tailRaw := lastNRunes(text, tailChars)
head := trimHeadToLineBoundary(headRaw)
tail := trimTailToLineBoundary(tailRaw)
if head == "" {
head = strings.TrimRight(headRaw, "\n")
}
if tail == "" {
tail = strings.TrimLeft(tailRaw, "\n")
}
notice := fmt.Sprintf(
"\n\n... [Content truncated: %d tokens -> ~%d tokens limit] ...\n\n",
tokenCount,
maxTokens,
)
switch {
case head == "" && tail == "":
return notice
case head == "":
return notice + tail
case tail == "":
return head + notice
default:
return head + notice + tail
}
}
func firstNRunes(text string, n int) string {
if n <= 0 {
return ""
}
if utf8.RuneCountInString(text) <= n {
return text
}
var builder strings.Builder
builder.Grow(n)
count := 0
reader := bufio.NewReader(strings.NewReader(text))
for count < n {
r, _, err := reader.ReadRune()
if err != nil {
break
} }
builder.WriteRune(r)
count++
}
return builder.String() switch {
case err == nil:
return true, nil
case errors.Is(err, bufio.ErrBufferFull):
continue
case errors.Is(err, io.EOF):
return sawData, nil
default:
return false, err
}
}
} }
func lastNRunes(text string, n int) string { func readNextLinePrefix(reader *bufio.Reader, maxBytes int64) ([]byte, bool, bool, error) {
if n <= 0 { if maxBytes <= 0 {
return "" return nil, false, false, nil
} }
runes := []rune(text) var out bytes.Buffer
if len(runes) <= n { sawData := false
return text complete := true
}
return string(runes[len(runes)-n:]) for {
fragment, err := reader.ReadSlice('\n')
if len(fragment) > 0 {
sawData = true
if remaining := maxBytes - int64(out.Len()); remaining > 0 {
take := len(fragment)
if int64(take) > remaining {
take = int(remaining)
complete = false
}
out.Write(fragment[:take])
} else {
complete = false
}
}
switch {
case err == nil:
return out.Bytes(), complete, sawData, nil
case errors.Is(err, bufio.ErrBufferFull):
if !complete {
return out.Bytes(), false, true, nil
}
continue
case errors.Is(err, io.EOF):
if !sawData {
return nil, true, false, nil
}
return out.Bytes(), complete, true, nil
default:
return nil, false, false, err
}
}
} }
func trimHeadToLineBoundary(text string) string { func readerHasMoreContent(reader *bufio.Reader) (bool, error) {
idx := strings.LastIndex(text, "\n") _, err := reader.Peek(1)
if idx < 0 { switch {
return strings.TrimRight(text, "\n") case err == nil:
return true, nil
case errors.Is(err, io.EOF):
return false, nil
default:
return false, err
} }
return strings.TrimRight(text[:idx], "\n")
}
func trimTailToLineBoundary(text string) string {
idx := strings.Index(text, "\n")
if idx < 0 {
return strings.TrimLeft(text, "\n")
}
return strings.TrimLeft(text[idx+1:], "\n")
} }
// getInt64Arg extracts an integer argument from the args map, returning the // getInt64Arg extracts an integer argument from the args map, returning the

273
pkg/tools/filesystem_test.go
View file

@ -6,7 +6,6 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"regexp" "regexp"
"strconv"
"strings" "strings"
"testing" "testing"
@ -60,7 +59,7 @@ func TestFilesystemTool_ReadFile_NotFound(t *testing.T) {
} }
// Should contain error message // Should contain error message
if !strings.Contains(result.ForLLM, "failed to read file") && !strings.Contains(result.ForUser, "failed to read") { if !strings.Contains(result.ForLLM, "failed to open file") && !strings.Contains(result.ForUser, "failed to open") {
t.Errorf("Expected error message, got ForLLM: %s, ForUser: %s", result.ForLLM, result.ForUser) t.Errorf("Expected error message, got ForLLM: %s, ForUser: %s", result.ForLLM, result.ForUser)
} }
} }
@ -722,11 +721,119 @@ func TestWhitelistFs_AllowsResolvedAllowedRootAlias(t *testing.T) {
} }
// TestReadFileTool_ChunkedReading verifies the pagination logic of the tool // TestReadFileTool_ChunkedReading verifies the pagination logic of the tool
// by reading a file in multiple chunks using 1-indexed line offset and limit. // by reading a file in multiple chunks using 'offset' and 'length'.
func TestReadFileTool_ChunkedReading(t *testing.T) { func TestReadFileTool_ChunkedReading(t *testing.T) {
tmpDir := t.TempDir() tmpDir := t.TempDir()
testFile := filepath.Join(tmpDir, "pagination_test.txt") testFile := filepath.Join(tmpDir, "pagination_test.txt")
fullContent := "abcdefghijklmnopqrstuvwxyz"
err := os.WriteFile(testFile, []byte(fullContent), 0o644)
if err != nil {
t.Fatalf("Failed to write test file: %v", err)
}
tool := NewReadFileTool(tmpDir, false, MaxReadFileSize)
ctx := context.Background()
// --- Step 1: Read the first chunk (10 bytes) ---
args1 := map[string]any{
"path": testFile,
"offset": 0,
"length": 10,
}
result1 := tool.Execute(ctx, args1)
if result1.IsError {
t.Fatalf("Chunk 1 failed: %s", result1.ForLLM)
}
if !strings.Contains(result1.ForLLM, "abcdefghij") {
t.Errorf("Chunk 1 should contain 'abcdefghij', got: %s", result1.ForLLM)
}
if !strings.Contains(result1.ForLLM, "[TRUNCATED") {
t.Errorf("Chunk 1 header should indicate truncation, got: %s", result1.ForLLM)
}
if !strings.Contains(result1.ForLLM, "offset=10") {
t.Errorf("Chunk 1 header should suggest next offset=10, got: %s", result1.ForLLM)
}
// Step 2: Read the second chunk (10 bytes) ---
args2 := map[string]any{
"path": testFile,
"offset": 10,
"length": 10,
}
result2 := tool.Execute(ctx, args2)
if result2.IsError {
t.Fatalf("Chunk 2 failed: %s", result2.ForLLM)
}
if !strings.Contains(result2.ForLLM, "klmnopqrst") {
t.Errorf("Chunk 2 should contain 'klmnopqrst', got: %s", result2.ForLLM)
}
if !strings.Contains(result2.ForLLM, "offset=20") {
t.Errorf("Chunk 2 header should suggest next offset=20, got: %s", result2.ForLLM)
}
// Step 3: Read the final chunk (remaining 6 bytes) ---
args3 := map[string]any{
"path": testFile,
"offset": 20,
"length": 10,
}
result3 := tool.Execute(ctx, args3)
if result3.IsError {
t.Fatalf("Chunk 3 failed: %s", result3.ForLLM)
}
if !strings.Contains(result3.ForLLM, "uvwxyz") {
t.Errorf("Chunk 3 should contain 'uvwxyz', got: %s", result3.ForLLM)
}
if !strings.Contains(result3.ForLLM, "[END OF FILE") {
t.Errorf("Chunk 3 header should indicate end of file, got: %s", result3.ForLLM)
}
if strings.Contains(result3.ForLLM, "[TRUNCATED") {
t.Errorf("Chunk 3 header should NOT indicate truncation, got: %s", result3.ForLLM)
}
}
// TestReadFileTool_OffsetBeyondEOF checks the behavior when requesting
// An offset that exceeds the total file size.
func TestReadFileTool_OffsetBeyondEOF(t *testing.T) {
tmpDir := t.TempDir()
testFile := filepath.Join(tmpDir, "short.txt")
err := os.WriteFile(testFile, []byte("12345"), 0o644)
if err != nil {
t.Fatalf("Failed to write test file: %v", err)
}
tool := NewReadFileTool(tmpDir, false, MaxReadFileSize)
ctx := context.Background()
args := map[string]any{
"path": testFile,
"offset": int64(100),
}
result := tool.Execute(ctx, args)
if result.IsError {
t.Errorf("A mistake was not expected, obtained IsError=true: %s", result.ForLLM)
}
expectedMsg := "[END OF FILE - no content at this offset]"
if result.ForLLM != expectedMsg {
t.Errorf("The message %q was expected, obtained: %q", expectedMsg, result.ForLLM)
}
}
func TestReadFileLinesTool_ChunkedReading(t *testing.T) {
tmpDir := t.TempDir()
testFile := filepath.Join(tmpDir, "pagination_lines.txt")
fullContent := strings.Join([]string{ fullContent := strings.Join([]string{
"line 1", "line 1",
"line 2", "line 2",
@ -741,106 +848,57 @@ func TestReadFileTool_ChunkedReading(t *testing.T) {
} }
tool := NewReadFileLinesTool(tmpDir, false, MaxReadFileSize) tool := NewReadFileLinesTool(tmpDir, false, MaxReadFileSize)
ctx := context.Background()
// --- Step 1: Read the first chunk (2 lines) --- result1 := tool.Execute(context.Background(), map[string]any{
args1 := map[string]any{
"path": testFile, "path": testFile,
"offset": 1, "offset": 0,
"limit": 2, "limit": 2,
} })
result1 := tool.Execute(ctx, args1)
if result1.IsError { if result1.IsError {
t.Fatalf("Chunk 1 failed: %s", result1.ForLLM) t.Fatalf("Chunk 1 failed: %s", result1.ForLLM)
} }
if !strings.Contains(result1.ForLLM, "line 1\nline 2\n") {
if !strings.Contains(result1.ForLLM, "line 1\nline 2\nline 3\nline 4\nline 5\nline 6") { t.Fatalf("expected first two lines, got: %s", result1.ForLLM)
t.Errorf("Chunk 1 should contain file content, got: %s", result1.ForLLM)
} }
if !strings.Contains(result1.ForLLM, "[TRUNCATED") { if !strings.Contains(result1.ForLLM, "lines 0-1") {
t.Errorf("Chunk 1 header should indicate truncation, got: %s", result1.ForLLM) t.Fatalf("expected line range 0-1, got: %s", result1.ForLLM)
} }
if !strings.Contains(result1.ForLLM, "offset=3") { if !strings.Contains(result1.ForLLM, "offset=2") {
t.Errorf("Chunk 1 header should suggest next offset=3, got: %s", result1.ForLLM) t.Fatalf("expected continuation offset=2, got: %s", result1.ForLLM)
}
if !strings.Contains(result1.ForLLM, "read: lines 1-2") {
t.Errorf("Chunk 1 header should report line range 1-2, got: %s", result1.ForLLM)
} }
// Step 2: Read the second chunk (2 lines) --- result2 := tool.Execute(context.Background(), map[string]any{
args2 := map[string]any{
"path": testFile, "path": testFile,
"offset": 3, "offset": 2,
"limit": 2, "limit": 2,
} })
result2 := tool.Execute(ctx, args2)
if result2.IsError { if result2.IsError {
t.Fatalf("Chunk 2 failed: %s", result2.ForLLM) t.Fatalf("Chunk 2 failed: %s", result2.ForLLM)
} }
if !strings.Contains(result2.ForLLM, "line 3\nline 4\n") {
if !strings.Contains(result2.ForLLM, "line 1\nline 2\nline 3\nline 4\nline 5\nline 6") { t.Fatalf("expected middle chunk, got: %s", result2.ForLLM)
t.Errorf("Chunk 2 should contain file content, got: %s", result2.ForLLM)
} }
if !strings.Contains(result2.ForLLM, "offset=5") { if !strings.Contains(result2.ForLLM, "offset=4") {
t.Errorf("Chunk 2 header should suggest next offset=5, got: %s", result2.ForLLM) t.Fatalf("expected continuation offset=4, got: %s", result2.ForLLM)
} }
// Step 3: Read the final chunk (remaining 2 lines) --- result3 := tool.Execute(context.Background(), map[string]any{
args3 := map[string]any{
"path": testFile, "path": testFile,
"offset": 5, "offset": 4,
"limit": 2, "limit": 2,
} })
result3 := tool.Execute(ctx, args3)
if result3.IsError { if result3.IsError {
t.Fatalf("Chunk 3 failed: %s", result3.ForLLM) t.Fatalf("Chunk 3 failed: %s", result3.ForLLM)
} }
if !strings.Contains(result3.ForLLM, "line 5\nline 6\n") {
if !strings.Contains(result3.ForLLM, "line 1\nline 2\nline 3\nline 4\nline 5\nline 6") { t.Fatalf("expected final chunk, got: %s", result3.ForLLM)
t.Errorf("Chunk 3 should contain file content, got: %s", result3.ForLLM)
} }
if strings.Contains(result3.ForLLM, "[TRUNCATED") { if !strings.Contains(result3.ForLLM, "[END OF FILE") {
t.Errorf("Chunk 3 header should NOT indicate truncation, got: %s", result3.ForLLM) t.Fatalf("expected EOF marker, got: %s", result3.ForLLM)
} }
} }
// TestReadFileTool_OffsetBeyondEOF checks the behavior when requesting func TestReadFileLinesTool_DefaultOffsetAndRemainingLines(t *testing.T) {
// a starting line that exceeds the total number of lines.
func TestReadFileTool_OffsetBeyondEOF(t *testing.T) {
tmpDir := t.TempDir()
testFile := filepath.Join(tmpDir, "short.txt")
err := os.WriteFile(testFile, []byte("line 1\nline 2\n"), 0o644)
if err != nil {
t.Fatalf("Failed to write test file: %v", err)
}
tool := NewReadFileLinesTool(tmpDir, false, MaxReadFileSize)
ctx := context.Background()
args := map[string]any{
"path": testFile,
"offset": int64(100), // Line offset beyond the end of the file
}
result := tool.Execute(ctx, args)
// It should not be classified as a tool execution error
if result.IsError {
t.Errorf("A mistake was not expected, obtained IsError=true: %s", result.ForLLM)
}
// Must return EXACTLY the string provided in the code
expectedMsg := "[END OF FILE - no content at this offset]"
if result.ForLLM != expectedMsg {
t.Errorf("The message %q was expected, obtained: %q", expectedMsg, result.ForLLM)
}
}
func TestReadFileTool_DefaultOffsetAndRemainingLines(t *testing.T) {
tmpDir := t.TempDir() tmpDir := t.TempDir()
testFile := filepath.Join(tmpDir, "default_lines.txt") testFile := filepath.Join(tmpDir, "default_lines.txt")
@ -854,11 +912,11 @@ func TestReadFileTool_DefaultOffsetAndRemainingLines(t *testing.T) {
if result.IsError { if result.IsError {
t.Fatalf("Execute() error = %s", result.ForLLM) t.Fatalf("Execute() error = %s", result.ForLLM)
} }
if !strings.Contains(result.ForLLM, "line 1\nline 2\nline 3") { if !strings.Contains(result.ForLLM, "line 1\nline 2\nline 3\n") {
t.Fatalf("expected remaining lines by default, got: %s", result.ForLLM) t.Fatalf("expected remaining lines by default, got: %s", result.ForLLM)
} }
if !strings.Contains(result.ForLLM, "read: lines 1-3") { if !strings.Contains(result.ForLLM, "lines 0-2") {
t.Fatalf("expected line range 1-3, got: %s", result.ForLLM) t.Fatalf("expected line range 0-2, got: %s", result.ForLLM)
} }
} }
@ -891,6 +949,28 @@ func TestReadFileTool_LegacyLengthUsesByteModeForText(t *testing.T) {
} }
} }
func TestReadFileLinesTool_OffsetBeyondEOF(t *testing.T) {
tmpDir := t.TempDir()
testFile := filepath.Join(tmpDir, "short_lines.txt")
err := os.WriteFile(testFile, []byte("line 1\nline 2\n"), 0o644)
if err != nil {
t.Fatalf("Failed to write test file: %v", err)
}
tool := NewReadFileLinesTool(tmpDir, false, MaxReadFileSize)
result := tool.Execute(context.Background(), map[string]any{
"path": testFile,
"offset": int64(100),
})
if result.IsError {
t.Fatalf("unexpected error: %s", result.ForLLM)
}
if result.ForLLM != "[END OF FILE - no content at this offset]" {
t.Fatalf("unexpected EOF message: %q", result.ForLLM)
}
}
func TestReadFileLinesTool_BinaryFileRejected(t *testing.T) { func TestReadFileLinesTool_BinaryFileRejected(t *testing.T) {
tmpDir := t.TempDir() tmpDir := t.TempDir()
testFile := filepath.Join(tmpDir, "binary.dat") testFile := filepath.Join(tmpDir, "binary.dat")
@ -909,40 +989,33 @@ func TestReadFileLinesTool_BinaryFileRejected(t *testing.T) {
if !strings.Contains(result.ForLLM, "file appears to be binary") { if !strings.Contains(result.ForLLM, "file appears to be binary") {
t.Fatalf("expected binary file rejection message, got: %s", result.ForLLM) t.Fatalf("expected binary file rejection message, got: %s", result.ForLLM)
} }
if !strings.Contains(result.ForLLM, "use read_file") {
t.Fatalf("expected suggestion to use read_file, got: %s", result.ForLLM)
}
} }
func TestReadFileTool_TruncatesLargeContentByEstimatedTokens(t *testing.T) { func TestReadFileLinesTool_TruncatesSingleLongLineAtByteBudget(t *testing.T) {
tmpDir := t.TempDir() tmpDir := t.TempDir()
testFile := filepath.Join(tmpDir, "long.txt") testFile := filepath.Join(tmpDir, "long_line.txt")
var builder strings.Builder content := strings.Repeat("x", 70*1024) + "\nsecond line\n"
for i := 1; i <= 40; i++ { err := os.WriteFile(testFile, []byte(content), 0o644)
builder.WriteString(strings.Repeat("line-content-", 8))
builder.WriteString(strconv.Itoa(i))
builder.WriteString("\n")
}
err := os.WriteFile(testFile, []byte(builder.String()), 0o644)
if err != nil { if err != nil {
t.Fatalf("Failed to write test file: %v", err) t.Fatalf("Failed to write test file: %v", err)
} }
tool := NewReadFileLinesTool(tmpDir, false, 160) // ~40 token output budget tool := NewReadFileLinesTool(tmpDir, false, MaxReadFileSize)
result := tool.Execute(context.Background(), map[string]any{"path": testFile}) result := tool.Execute(context.Background(), map[string]any{"path": testFile})
if result.IsError { if result.IsError {
t.Fatalf("Execute() error = %s", result.ForLLM) t.Fatalf("Execute() error = %s", result.ForLLM)
} }
if !strings.Contains(result.ForLLM, "was cut mid-line") {
if !strings.Contains(result.ForLLM, "... [Content truncated:") { t.Fatalf("expected explicit mid-line truncation warning, got: %s", result.ForLLM)
t.Fatalf("expected truncation notice, got: %s", result.ForLLM)
} }
if !strings.Contains(result.ForLLM, "-> ~40 tokens limit") { if !strings.Contains(result.ForLLM, "Use read_file for byte-wise inspection") {
t.Fatalf("expected token limit notice derived from config, got: %s", result.ForLLM) t.Fatalf("expected byte-tool guidance for long line, got: %s", result.ForLLM)
} }
if !strings.Contains(result.ForLLM, "line-content-line-content-") { if strings.Contains(result.ForLLM, "second line") {
t.Fatalf("expected head of content to remain visible, got: %s", result.ForLLM) t.Fatalf("did not expect second line after truncation, got: %s", result.ForLLM)
}
if !strings.Contains(result.ForLLM, "line-content-line-content-40") {
t.Fatalf("expected tail of content to remain visible, got: %s", result.ForLLM)
} }
} }