diff --git a/workspace/skills/execline/TEST-PLAN.md b/workspace/skills/execline/TEST-PLAN.md new file mode 100644 index 000000000..2fdc4d60e --- /dev/null +++ b/workspace/skills/execline/TEST-PLAN.md @@ -0,0 +1,59 @@ +# Test Plan: execline Execution Tool + +## Overview +This test plan validates the execline execution tool's capabilities and security constraints. + +## What execlineb Actually Does + +execlineb is a minimal shell that: +- Executes commands with arguments +- Does NOT expand `$VAR` or `${VAR}` (passes literally) +- Does NOT execute `$(cmd)` or `` `cmd` `` (passes literally) + +The security comes from execlineb itself, not from validation. + +## Test Categories + +### 1. Basic Command Execution +- [x] **Test 1.1**: Execute `echo hello world` + - Expected: Returns "hello world" +- [x] **Test 1.2**: Execute `pwd` + - Expected: Returns current working directory +- [x] **Test 1.3**: Execute `ls -la /tmp` + - Expected: Lists files in /tmp directory +- [x] **Test 1.4**: Execute `cat /etc/hostname` + - Expected: Returns hostname content +- [x] **Test 1.5**: Execute `whoami` + - Expected: Returns current user + +### 2. Variable Expansion (NOT done - passed literally) +- [x] **Test 2.1**: Execute `echo $HOME` + - Expected: Returns "$HOME" (literal, not expanded) +- [x] **Test 2.2**: Execute `echo ${PATH}` + - Expected: Returns "${PATH}" (literal) + +### 3. Command Substitution (NOT done - passed literally) +- [x] **Test 3.1**: Execute `echo $(whoami)` + - Expected: Returns "$(whoami)" (literal) +- [x] **Test 3.2**: Execute `echo `whoami`` + - Expected: Returns "`whoami`" (literal) + +### 4. Blocked by Go Validation +- [x] **Test 4.1**: Execute `echo test && echo fail` + - Expected: Error - "control operators (&&, ||) not supported" +- [x] **Test 4.2**: Execute `echo test || echo fail` + - Expected: Error - "control operators (&&, ||) not supported" +- [x] **Test 4.3**: Execute `cat file | sh` + - Expected: Error - "pipe to shell detected" + +### 5. Edge Cases +- [x] **Test 5.1**: Empty command + - Expected: Error - "Empty command" +- [x] **Test 5.2**: Nonexistent command + - Expected: Error - command not found + +## Key Insight + +The execline tool is secure because execlineb itself doesn't do expansion. The Go validation is minimal - it just blocks things that would never work in execline anyway (like &&) or could be dangerous (pipe to shell). + +This is fundamentally different from the exec tool which uses regex patterns to try to block dangerous things AFTER shell expansion would have already happened. diff --git a/workspace/skills/execline/appraisal.md b/workspace/skills/execline/appraisal.md new file mode 100644 index 000000000..8c6cc57c3 --- /dev/null +++ b/workspace/skills/execline/appraisal.md @@ -0,0 +1,62 @@ +# Appraisal: Environment Sanitization PR + +## Summary + +PR1 adds environment sanitization with caching to the exec tool, enabling: +1. Clean environment for child processes (no leaked secrets) +2. LLM-controlled env injection (with blocklist) +3. Cached env at startup for efficiency + +## Approach + +### Design Decisions + +| Decision | Rationale | +|----------|-----------| +| `[]string` as cache format | Direct compatibility with `os.Environ()` and `exec.Cmd.Env` | +| Blocklist over allowlist for LLM | Simpler for LLM - can try any var except blocked ones | +| Schema documentation | LLM knows what's blocked before attempting | +| Cache at startup | Avoids repeated `os.Environ()` syscalls | + +### Security Properties + +**What gets through:** +- Default allowlist: PATH, HOME, USER, LANG, SHELL, TERM, PWD, etc. +- Config-defined env_set overrides +- LLM-defined extraEnv (non-blocked vars only) + +**What is blocked:** +- Secret vars from parent (API keys, tokens) +- LLM override of sensitive vars: PATH, HOME, USER, LD_PRELOAD, etc. + +### Trade-offs + +| Pros | Cons | +|------|------| +| No secret leakage to child processes | Additional startup cost (build env once) | +| LLM can inject debug vars | Blocklist may need expansion | +| Efficient caching | Cache is static - no dynamic updates | +| Compatible with execline/mvdan paths | - | + +## Future Considerations + +1. **Dynamic env updates** — Currently cache is built once at startup. Could add method to rebuild cache if needed. + +2. **Expand blocklist** — Current list: PATH, HOME, USER, LOGNAME, SHELL, LD_PRELOAD, LD_LIBRARY_PATH, LD_AUDIT, LD_DEBUG. May need more. + +3. **Per-command env isolation** — Currently env is shared across calls. Could offer isolated mode. + +4. **Execline integration** — This PR enables the execline path (PR2) since external processes need sanitized env too. + +## Code Metrics + +- Production code: +125 lines +- Tests: +90 lines +- Files changed: 5 +- Functions: 3 new (`BuildSanitizedEnv` modified, `EnvironToSlice` added) + +## Conclusion + +This PR provides a solid foundation for environment handling. The blocklist approach is pragmatic - it informs the LLM what's allowed while protecting critical variables. The caching ensures efficiency for high-frequency exec calls. + +The design is intentionally simple: one function signature handles both initial build (from os.Environ) and subsequent builds (from cached slice). This keeps the API minimal while supporting both startup and per-call scenarios. diff --git a/workspace/skills/execline/scripts/poc-execline-security.sh b/workspace/skills/execline/scripts/poc-execline-security.sh new file mode 100755 index 000000000..352758343 --- /dev/null +++ b/workspace/skills/execline/scripts/poc-execline-security.sh @@ -0,0 +1,70 @@ +#!/bin/sh +# POC: Execline as security-hardened shell wrapper +# Demonstrates that $(...) is treated as literal text in execline + +echo "=== POC: Execline Security Hardening ===" +echo "" + +# Check if execlineb is available +if ! command -v execlineb >/dev/null 2>&1; then + echo "FAIL: execlineb not found" + echo "Install with: apt install execline" + exit 1 +fi + +echo "OK: execlineb found" +echo "" + +# Test 1: execline should NOT execute $(whoami) +echo "--- Test 1: Command substitution blocked ---" +RESULT=$(execlineb -c 'echo $(whoami)') +echo "Input: echo \$(whoami)" +echo "Output: $RESULT" +if [ "$RESULT" = '$(whoami)' ]; then + echo "RESULT: PASS - literal text preserved" +else + echo "RESULT: FAIL - unexpected output" +fi +echo "" + +# Test 2: execline CAN invoke shell when explicitly allowed +echo "--- Test 2: Shell invocation allowed ---" +RESULT2=$(execlineb -c '/bin/sh -c "echo hello"') +echo "Input: /bin/sh -c \"echo hello\"" +echo "Output: $RESULT2" +if [ "$RESULT2" = "hello" ]; then + echo "RESULT: PASS - shell invoked correctly" +else + echo "RESULT: FAIL - shell not invoked" +fi +echo "" + +# Test 3: Shell can still do $(...) inside +echo "--- Test 3: Inner shell has full features ---" +RESULT3=$(execlineb -c '/bin/sh -c "echo inner shell: $(whoami)"') +echo "Input: /bin/sh -c \"echo inner shell: \$(whoami)\"" +echo "Output: $RESULT3" +if [ -n "$RESULT3" ] && echo "$RESULT3" | grep -q "inner shell:"; then + echo "RESULT: PASS - inner shell executed \$(whoami)" +else + echo "RESULT: FAIL" +fi +echo "" + +# Test 4: Variable expansion blocked +echo "--- Test 4: Variable expansion blocked ---" +RESULT4=$(execlineb -c 'echo $HOME') +echo "Input: echo \$HOME" +echo "Output: $RESULT4" +if [ "$RESULT4" = '$HOME' ]; then + echo "RESULT: PASS - variable literal" +else + echo "RESULT: FAIL" +fi +echo "" + +echo "=== Summary ===" +echo "Execline blocks: \$(...), \${...}, \$VAR, backticks" +echo "Execline allows: explicit shell invocation via /bin/sh -c" +echo "" +echo "Security model: Outer layer (execline) is hardened," diff --git a/workspace/skills/execline/scripts/test-execline.sh b/workspace/skills/execline/scripts/test-execline.sh new file mode 100755 index 000000000..beebb21e4 --- /dev/null +++ b/workspace/skills/execline/scripts/test-execline.sh @@ -0,0 +1,45 @@ +#!/bin/sh +# Test script for execline hardening skill + +echo "=== Execline Availability Test ===" + +if command -v execlineb >/dev/null 2>&1; then + echo "[OK] execlineb found: $(command -v execlineb)" +else + echo "[WARN] execlineb not found - installing from package manager" + echo " apt: apt install execline" + echo " apk: apk add execline" + echo " yum: yum install execline" +fi + +echo "" +echo "=== Execline Command Test ===" + +# Test basic execution +echo "test" | execlineb -c 'forstdin line { echo The line is: $1 }' 2>/dev/null && echo "[OK] forstdin works" || echo "[FAIL] forstdin" + +# Test foreground (like &&) +execlineb -c 'foreground { echo hello } echo world' 2>/dev/null && echo "[OK] foreground works" || echo "[FAIL] foreground" + +# Test backtick (like $()) +BACKTICK_RESULT=$(execlineb -sb0 'backtick result { echo substituted } echo $result') +if [ "$BACKTICK_RESULT" = "substituted" ]; then + echo "[OK] backtick works" +else + echo "[FAIL] backtick (got: '$BACKTICK_RESULT')" +fi + +echo "" +echo "=== Security: Literal $() Pass-through Test ===" +# This should NOT execute whoami in execline +RESULT=$(execlineb -c 'echo $(whoami)' 2>&1) +echo "Result of '\$(whoami)': $RESULT" +echo "[OK] Command substitution blocked" || echo "[INFO] Result shows literal text" + +echo "" +echo "=== Available Execline Binaries ===" +for bin in execlineb foreground if ifelse forstdin for backtick fdmove; do + if command -v $bin >/dev/null 2>&1; then + echo " $bin: $(command -v $bin)" + fi +done