opencode/picoclaw
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge pull request #104 from hobbyistlabs-coder/bolt-web-fetch-optimization-9650873766949108786

Browse source 
⚡ Bolt: Optimize web_fetch HTML detection to prevent large string allocations
This commit is contained in:
hobbyistlabs-coder 2026-03-27 14:17:43 -04:00 • committed by GitHub
commit 552d00b527
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 8 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
.jules/bolt.md
View file

@ -11,3 +11,7 @@
## 2025-03-20 - String Operations Fast Paths and Avoiding Double Searches ## 2025-03-20 - String Operations Fast Paths and Avoiding Double Searches
**Learning:** When trying to optimize `strings.ToLower`, ensure you don't introduce regressions with byte-to-rune casting on UTF-8 strings. Also, `strings.Contains(s, sub)` literally calls `strings.Index(s, sub)` under the hood. Using `strings.Contains` followed immediately by `strings.Index` to extract the position is an anti-pattern that searches the string twice, undermining the intended performance optimization. **Learning:** When trying to optimize `strings.ToLower`, ensure you don't introduce regressions with byte-to-rune casting on UTF-8 strings. Also, `strings.Contains(s, sub)` literally calls `strings.Index(s, sub)` under the hood. Using `strings.Contains` followed immediately by `strings.Index` to extract the position is an anti-pattern that searches the string twice, undermining the intended performance optimization.
**Action:** Always prefer a single `strings.Index` call over `Contains`+`Index`. Stick to one single optimization per PR to reduce risk and review burden. **Action:** Always prefer a single `strings.Index` call over `Contains`+`Index`. Stick to one single optimization per PR to reduce risk and review burden.
## 2025-03-25 - Efficient HTTP Response Prefix Checking
**Learning:** Using `strings.ToLower(string(body))` on large HTTP response payloads (which can be megabytes in size) to check for a small case-insensitive prefix (like `<html` or `<!doctype`) causes massive memory allocation, large garbage collection overhead, and $O(N)$ string iterations.
**Action:** Use bounded byte slice checks combined with `bytes.EqualFold` (e.g., `bytes.EqualFold(body[:5], []byte("<html"))`) for large payloads. This makes the check $O(1)$ without any string allocations or full-body case conversions.

6
pkg/tools/web/fetch.go
View file

@ -1,6 +1,7 @@
package web package web
import ( import (
"bytes"
"context" "context"
"encoding/json" "encoding/json"
"errors" "errors"
@ -158,8 +159,9 @@ func (t *WebFetchTool) Execute(ctx context.Context, args map[string]any) *tools.
text = string(body) text = string(body)
extractor = "raw" extractor = "raw"
} }
} else if strings.Contains(contentType, "text/html") || len(body) > 0 && } else if strings.Contains(contentType, "text/html") ||
(strings.HasPrefix(string(body), "<!DOCTYPE") || strings.HasPrefix(strings.ToLower(string(body)), "<html")) { (len(body) >= 9 && bytes.EqualFold(body[:9], []byte("<!doctype"))) ||
(len(body) >= 5 && bytes.EqualFold(body[:5], []byte("<html"))) {
text = t.extractText(string(body)) text = t.extractText(string(body))
extractor = "text" extractor = "text"
} else { } else {