rename security.yml to .security.yml
This commit is contained in:
parent
7bf4831059
commit
310f788f5f
8 changed files with 65 additions and 474 deletions
|
|
@ -1,225 +0,0 @@
|
||||||
# Security Configuration Refactoring Summary
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
Successfully refactored `pkg/config/config.go` to support a separate `security.yml` file for storing all sensitive data (API keys, tokens, secrets, passwords).
|
|
||||||
|
|
||||||
## Changes Made
|
|
||||||
|
|
||||||
### New Files Created
|
|
||||||
|
|
||||||
1. **`pkg/config/security.go`** (New file)
|
|
||||||
- Defines `SecurityConfig` structure for all sensitive data
|
|
||||||
- Implements `LoadSecurityConfig()` to load from YAML
|
|
||||||
- Implements `SaveSecurityConfig()` to save with secure permissions (0o600)
|
|
||||||
- Implements `ResolveReference()` to resolve `ref:` prefixed strings
|
|
||||||
- Supports all model, channel, web tool, and skills security entries
|
|
||||||
|
|
||||||
2. **`pkg/config/security_test.go`** (New file)
|
|
||||||
- Comprehensive unit tests for security config loading
|
|
||||||
- Tests for reference resolution (models, channels, web tools, skills)
|
|
||||||
- Tests for file I/O operations
|
|
||||||
|
|
||||||
3. **`pkg/config/security_integration_test.go`** (New file)
|
|
||||||
- Integration tests for full workflow
|
|
||||||
- Tests backward compatibility with direct values
|
|
||||||
- Tests mixed usage of references and direct values
|
|
||||||
- Tests error handling for invalid references
|
|
||||||
|
|
||||||
4. **`security.example.yml`** (New file)
|
|
||||||
- Template for users to copy and fill in
|
|
||||||
- Includes all possible security entries with placeholder values
|
|
||||||
- Located at project root
|
|
||||||
|
|
||||||
5. **`pkg/config/SECURITY_CONFIG.md`** (New file)
|
|
||||||
- Complete documentation for the security config feature
|
|
||||||
- Usage examples and reference format guide
|
|
||||||
- Migration guide from old config
|
|
||||||
- Security best practices
|
|
||||||
|
|
||||||
6. **`pkg/config/example_security_usage.go`** (New file)
|
|
||||||
- Practical examples in Go comment format
|
|
||||||
- Shows complete workflow from creation to usage
|
|
||||||
- Lists all available reference paths
|
|
||||||
|
|
||||||
### Modified Files
|
|
||||||
|
|
||||||
1. **`pkg/config/config.go`**
|
|
||||||
- Added `applySecurityConfig()` function to resolve all `ref:` references
|
|
||||||
- Modified `LoadConfig()` to:
|
|
||||||
- Load security config from `security.yml`
|
|
||||||
- Apply security references to all config fields
|
|
||||||
- Maintain backward compatibility with direct values
|
|
||||||
- Updated warning message to suggest using `security.yml`
|
|
||||||
|
|
||||||
## Key Features
|
|
||||||
|
|
||||||
### Reference Format
|
|
||||||
|
|
||||||
Uses dot notation for referencing values:
|
|
||||||
- Models: `ref:model_list.<model_name>.api_key`
|
|
||||||
- Channels: `ref:channels.<channel_name>.<field>`
|
|
||||||
- Web Tools: `ref:web.<provider>.<field>`
|
|
||||||
- Skills: `ref:skills.<registry>.<field>`
|
|
||||||
|
|
||||||
### Supported Security Entries
|
|
||||||
|
|
||||||
**Models:**
|
|
||||||
- API keys for all model configurations
|
|
||||||
|
|
||||||
**Channels:**
|
|
||||||
- Telegram: token
|
|
||||||
- Feishu: app_secret, encrypt_key, verification_token
|
|
||||||
- Discord: token
|
|
||||||
- QQ: app_secret
|
|
||||||
- DingTalk: client_secret
|
|
||||||
- Slack: bot_token, app_token
|
|
||||||
- Matrix: access_token
|
|
||||||
- LINE: channel_secret, channel_access_token
|
|
||||||
- OneBot: access_token
|
|
||||||
- WeCom: token, encoding_aes_key
|
|
||||||
- WeComApp: corp_secret, token, encoding_aes_key
|
|
||||||
- WeComAIBot: token, encoding_aes_key
|
|
||||||
- Pico: token
|
|
||||||
- IRC: password, nickserv_password, sasl_password
|
|
||||||
|
|
||||||
**Web Tools:**
|
|
||||||
- Brave: api_key
|
|
||||||
- Tavily: api_key
|
|
||||||
- Perplexity: api_key
|
|
||||||
- GLMSearch: api_key
|
|
||||||
|
|
||||||
**Skills:**
|
|
||||||
- GitHub: token
|
|
||||||
- ClawHub: auth_token
|
|
||||||
|
|
||||||
### Backward Compatibility
|
|
||||||
|
|
||||||
- Direct values in `config.json` still work
|
|
||||||
- Mixed usage of references and direct values is supported
|
|
||||||
- Optional security file (if missing, only references fail)
|
|
||||||
- No breaking changes to existing configurations
|
|
||||||
|
|
||||||
## Testing
|
|
||||||
|
|
||||||
All tests pass successfully:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
go test ./pkg/config -v
|
|
||||||
```
|
|
||||||
|
|
||||||
Test coverage includes:
|
|
||||||
- ✅ Unit tests for reference resolution
|
|
||||||
- ✅ Integration tests for full workflow
|
|
||||||
- ✅ Backward compatibility tests
|
|
||||||
- ✅ Error handling tests
|
|
||||||
- ✅ File I/O and permission tests
|
|
||||||
- ✅ All existing config tests still pass
|
|
||||||
|
|
||||||
## Usage Example
|
|
||||||
|
|
||||||
### config.json
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"version": 1,
|
|
||||||
"model_list": [
|
|
||||||
{
|
|
||||||
"model_name": "gpt-5.4",
|
|
||||||
"model": "openai/gpt-5.4",
|
|
||||||
"api_base": "https://api.openai.com/v1",
|
|
||||||
"api_key": "ref:model_list.gpt-5.4.api_key"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"channels": {
|
|
||||||
"telegram": {
|
|
||||||
"enabled": true,
|
|
||||||
"token": "ref:channels.telegram.token"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### security.yml
|
|
||||||
```yaml
|
|
||||||
model_list:
|
|
||||||
gpt-5.4:
|
|
||||||
api_key: "sk-proj-actual-key-here"
|
|
||||||
|
|
||||||
channels:
|
|
||||||
telegram:
|
|
||||||
token: "1234567890:ABCdefGHIjklMNOpqrsTUVwxyz"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Migration Path
|
|
||||||
|
|
||||||
1. Copy `security.example.yml` to `~/.picoclaw/security.yml`
|
|
||||||
2. Fill in actual API keys and tokens
|
|
||||||
3. Update `config.json` to use `ref:` references
|
|
||||||
4. Set proper permissions: `chmod 600 ~/.picoclaw/security.yml`
|
|
||||||
5. Test with `picoclaw --version`
|
|
||||||
|
|
||||||
## Security Benefits
|
|
||||||
|
|
||||||
1. **Separation of concerns**: Configuration and secrets are in separate files
|
|
||||||
2. **Easier sharing**: Config can be shared without exposing secrets
|
|
||||||
3. **Better version control**: `security.yml` can be added to `.gitignore`
|
|
||||||
4. **Flexible deployment**: Different environments can use different security files
|
|
||||||
5. **Secure file permissions**: Saved with `0o600` by default
|
|
||||||
|
|
||||||
## Implementation Details
|
|
||||||
|
|
||||||
### File Loading Flow
|
|
||||||
|
|
||||||
```
|
|
||||||
LoadConfig()
|
|
||||||
├─ Load config.json
|
|
||||||
├─ Detect version
|
|
||||||
├─ Parse config based on version
|
|
||||||
├─ Load security.yml (optional)
|
|
||||||
├─ Apply security references
|
|
||||||
│ └─ Resolve all "ref:" prefixes
|
|
||||||
├─ Parse environment variables
|
|
||||||
├─ Resolve API keys (file://, enc://)
|
|
||||||
├─ Expand multi-key models
|
|
||||||
└─ Validate and return
|
|
||||||
```
|
|
||||||
|
|
||||||
### Reference Resolution
|
|
||||||
|
|
||||||
The `ResolveReference()` function:
|
|
||||||
1. Checks if string starts with `ref:`
|
|
||||||
2. Parses the dot-notation path
|
|
||||||
3. Navigates the security config structure
|
|
||||||
4. Returns the actual value
|
|
||||||
5. Returns error if path doesn't exist
|
|
||||||
|
|
||||||
### Error Handling
|
|
||||||
|
|
||||||
- Clear error messages with full context
|
|
||||||
- Includes the reference path and field name
|
|
||||||
- Fails early on invalid references
|
|
||||||
- Maintains backward compatibility
|
|
||||||
|
|
||||||
## Dependencies
|
|
||||||
|
|
||||||
Added dependency: `gopkg.in/yaml.v3` for YAML parsing
|
|
||||||
|
|
||||||
## Files Modified Summary
|
|
||||||
|
|
||||||
- **Created**: 6 new files (security.go, tests, docs, examples)
|
|
||||||
- **Modified**: 1 file (config.go - added security integration)
|
|
||||||
- **Lines added**: ~1000+ lines (including tests and documentation)
|
|
||||||
- **Backward compatible**: ✅ Yes
|
|
||||||
- **Breaking changes**: ❌ None
|
|
||||||
|
|
||||||
## Next Steps
|
|
||||||
|
|
||||||
1. Update main README to mention security.yml
|
|
||||||
2. Add security.yml to .gitignore
|
|
||||||
3. Update documentation with security config examples
|
|
||||||
4. Consider adding migration tool for existing users
|
|
||||||
5. Add validation for security.yml schema
|
|
||||||
|
|
||||||
## Conclusion
|
|
||||||
|
|
||||||
The refactoring successfully implements a secure, flexible, and backward-compatible way to manage sensitive configuration data. All tests pass and the feature is ready for use.
|
|
||||||
|
|
@ -2,11 +2,11 @@
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
This refactoring introduces a `security.yml` file to store all sensitive data (API keys, tokens, secrets, passwords) separately from the main configuration. This improves security by:
|
This refactoring introduces a `.security.yml` file to store all sensitive data (API keys, tokens, secrets, passwords) separately from the main configuration. This improves security by:
|
||||||
|
|
||||||
1. **Separation of concerns**: Configuration settings and secrets are in separate files
|
1. **Separation of concerns**: Configuration settings and secrets are in separate files
|
||||||
2. **Easier sharing**: The main config can be shared without exposing sensitive data
|
2. **Easier sharing**: The main config can be shared without exposing sensitive data
|
||||||
3. **Better version control**: `security.yml` can be added to `.gitignore`
|
3. **Better version control**: `.security.yml` can be added to `.gitignore`
|
||||||
4. **Flexible deployment**: Different environments can use different security files
|
4. **Flexible deployment**: Different environments can use different security files
|
||||||
|
|
||||||
## File Structure
|
## File Structure
|
||||||
|
|
@ -14,14 +14,14 @@ This refactoring introduces a `security.yml` file to store all sensitive data (A
|
||||||
```
|
```
|
||||||
~/.picoclaw/
|
~/.picoclaw/
|
||||||
├── config.json # Main configuration (safe to share)
|
├── config.json # Main configuration (safe to share)
|
||||||
└── security.yml # Security data (never share)
|
└── .security.yml # Security data (never share)
|
||||||
```
|
```
|
||||||
|
|
||||||
## Usage
|
## Usage
|
||||||
|
|
||||||
### Basic Configuration
|
### Basic Configuration
|
||||||
|
|
||||||
In your `config.json`, use `ref:` references to point to values in `security.yml`:
|
In your `config.json`, use `ref:` references to point to values in `.security.yml`:
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
|
|
@ -45,7 +45,7 @@ In your `config.json`, use `ref:` references to point to values in `security.yml
|
||||||
|
|
||||||
### Security Configuration
|
### Security Configuration
|
||||||
|
|
||||||
In your `security.yml`, store the actual values:
|
In your `.security.yml`, store the actual values:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
model_list:
|
model_list:
|
||||||
|
|
@ -135,9 +135,9 @@ The refactoring maintains full backward compatibility:
|
||||||
|
|
||||||
1. **Direct values**: You can still use direct values in `config.json` (not recommended for production)
|
1. **Direct values**: You can still use direct values in `config.json` (not recommended for production)
|
||||||
2. **Mixed usage**: You can mix `ref:` references and direct values
|
2. **Mixed usage**: You can mix `ref:` references and direct values
|
||||||
3. **Optional security file**: If `security.yml` doesn't exist, all references will fail (but direct values still work)
|
3. **Optional security file**: If `.security.yml` doesn't exist, all references will fail (but direct values still work)
|
||||||
|
|
||||||
### API Key Formats in security.yml
|
### API Key Formats in .security.yml
|
||||||
|
|
||||||
**Models (gpt-5.4, claude-sonnet-4.6, etc.):**
|
**Models (gpt-5.4, claude-sonnet-4.6, etc.):**
|
||||||
- Must use `api_keys` (array) format
|
- Must use `api_keys` (array) format
|
||||||
|
|
@ -190,16 +190,16 @@ In `config.json`:
|
||||||
|
|
||||||
## Migration Guide
|
## Migration Guide
|
||||||
|
|
||||||
### Step 1: Create security.yml
|
### Step 1: Create .security.yml
|
||||||
|
|
||||||
Copy the example template:
|
Copy the example template:
|
||||||
```bash
|
```bash
|
||||||
cp security.example.yml ~/.picoclaw/security.yml
|
cp security.example.yml ~/.picoclaw/.security.yml
|
||||||
```
|
```
|
||||||
|
|
||||||
### Step 2: Fill in your actual values
|
### Step 2: Fill in your actual values
|
||||||
|
|
||||||
Edit `~/.picoclaw/security.yml` and replace placeholder values with your actual API keys and tokens.
|
Edit `~/.picoclaw/.security.yml` and replace placeholder values with your actual API keys and tokens.
|
||||||
|
|
||||||
### Step 3: Update config.json
|
### Step 3: Update config.json
|
||||||
|
|
||||||
|
|
@ -240,11 +240,11 @@ picoclaw --version
|
||||||
|
|
||||||
## Security Best Practices
|
## Security Best Practices
|
||||||
|
|
||||||
1. **Never commit `security.yml`** to version control
|
1. **Never commit `.security.yml`** to version control
|
||||||
2. **Set file permissions**: `chmod 600 ~/.picoclaw/security.yml`
|
2. **Set file permissions**: `chmod 600 ~/.picoclaw/.security.yml`
|
||||||
3. **Use different keys** for different environments (dev, staging, production)
|
3. **Use different keys** for different environments (dev, staging, production)
|
||||||
4. **Rotate keys regularly** and update `security.yml`
|
4. **Rotate keys regularly** and update `.security.yml`
|
||||||
5. **Backup securely**: Encrypt backups containing `security.yml`
|
5. **Backup securely**: Encrypt backups containing `.security.yml`
|
||||||
|
|
||||||
## API
|
## API
|
||||||
|
|
||||||
|
|
@ -254,7 +254,7 @@ picoclaw --version
|
||||||
func LoadSecurityConfig(securityPath string) (*SecurityConfig, error)
|
func LoadSecurityConfig(securityPath string) (*SecurityConfig, error)
|
||||||
```
|
```
|
||||||
|
|
||||||
Loads the security configuration from `security.yml`. Returns an empty `SecurityConfig` if the file doesn't exist.
|
Loads the security configuration from `.security.yml`. Returns an empty `SecurityConfig` if the file doesn't exist.
|
||||||
|
|
||||||
### SaveSecurityConfig
|
### SaveSecurityConfig
|
||||||
|
|
||||||
|
|
@ -262,7 +262,7 @@ Loads the security configuration from `security.yml`. Returns an empty `Security
|
||||||
func SaveSecurityConfig(securityPath string, sec *SecurityConfig) error
|
func SaveSecurityConfig(securityPath string, sec *SecurityConfig) error
|
||||||
```
|
```
|
||||||
|
|
||||||
Saves the security configuration to `security.yml` with `0o600` permissions.
|
Saves the security configuration to `.security.yml` with `0o600` permissions.
|
||||||
|
|
||||||
### ResolveReference
|
### ResolveReference
|
||||||
|
|
||||||
|
|
@ -278,7 +278,7 @@ Resolves a reference string (e.g., `"ref:model_list.test.api_key"`) and returns
|
||||||
func SecurityPath(configPath string) string
|
func SecurityPath(configPath string) string
|
||||||
```
|
```
|
||||||
|
|
||||||
Returns the path to `security.yml` relative to the config file.
|
Returns the path to `.security.yml` relative to the config file.
|
||||||
|
|
||||||
## Example: Complete Configuration
|
## Example: Complete Configuration
|
||||||
|
|
||||||
|
|
@ -323,7 +323,7 @@ Returns the path to `security.yml` relative to the config file.
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
### security.yml
|
### .security.yml
|
||||||
```yaml
|
```yaml
|
||||||
model_list:
|
model_list:
|
||||||
gpt-5.4:
|
gpt-5.4:
|
||||||
|
|
@ -362,13 +362,13 @@ go test ./pkg/config -run TestSecurityConfig
|
||||||
|
|
||||||
### Error: "model security entry not found"
|
### Error: "model security entry not found"
|
||||||
|
|
||||||
- Ensure the model name in your reference matches exactly in `security.yml`
|
- Ensure the model name in your reference matches exactly in `.security.yml`
|
||||||
- Check that the `model_list` section exists in `security.yml`
|
- Check that the `model_list` section exists in `.security.yml`
|
||||||
- For models with indexed names (e.g., "gpt-5.4:0"), ensure the exact name is used or check the base name without index
|
- For models with indexed names (e.g., "gpt-5.4:0"), ensure the exact name is used or check the base name without index
|
||||||
|
|
||||||
### Error: "failed to load security config"
|
### Error: "failed to load security config"
|
||||||
|
|
||||||
- Verify `security.yml` exists in the same directory as `config.json`
|
- Verify `.security.yml` exists in the same directory as `config.json`
|
||||||
- Check the YAML syntax is valid (use a YAML validator)
|
- Check the YAML syntax is valid (use a YAML validator)
|
||||||
- Ensure file permissions allow reading
|
- Ensure file permissions allow reading
|
||||||
|
|
||||||
|
|
@ -376,7 +376,7 @@ go test ./pkg/config -run TestSecurityConfig
|
||||||
|
|
||||||
- Verify the reference format is correct
|
- Verify the reference format is correct
|
||||||
- Check the path structure matches the examples above
|
- Check the path structure matches the examples above
|
||||||
- Ensure all required sections exist in `security.yml`
|
- Ensure all required sections exist in `.security.yml`
|
||||||
|
|
||||||
## Advanced Features
|
## Advanced Features
|
||||||
|
|
||||||
|
|
@ -392,7 +392,7 @@ Both models and web tools support multiple API keys for improved reliability:
|
||||||
|
|
||||||
#### Example: Model with Multiple Keys
|
#### Example: Model with Multiple Keys
|
||||||
|
|
||||||
**security.yml:**
|
**.security.yml:**
|
||||||
```yaml
|
```yaml
|
||||||
model_list:
|
model_list:
|
||||||
gpt-5.4:
|
gpt-5.4:
|
||||||
|
|
@ -417,7 +417,7 @@ model_list:
|
||||||
|
|
||||||
#### Example: Web Tool with Multiple Keys
|
#### Example: Web Tool with Multiple Keys
|
||||||
|
|
||||||
**security.yml:**
|
**.security.yml:**
|
||||||
```yaml
|
```yaml
|
||||||
web:
|
web:
|
||||||
brave:
|
brave:
|
||||||
|
|
@ -504,7 +504,7 @@ All formats work identically in `config.json` - you always use the same referenc
|
||||||
|
|
||||||
When you have multiple models with the same base name but different API keys, you can use indexed names:
|
When you have multiple models with the same base name but different API keys, you can use indexed names:
|
||||||
|
|
||||||
**security.yml:**
|
**.security.yml:**
|
||||||
```yaml
|
```yaml
|
||||||
model_list:
|
model_list:
|
||||||
gpt-5.4:
|
gpt-5.4:
|
||||||
|
|
@ -538,7 +538,7 @@ Environment variables follow this pattern: `PICOCLAW_<SECTION>_<KEY1>_<KEY2>_<FI
|
||||||
|
|
||||||
### Multiple API Keys Not Working
|
### Multiple API Keys Not Working
|
||||||
|
|
||||||
- Ensure you're using `api_keys` (plural) in `security.yml` for models and web tools (except GLMSearch)
|
- Ensure you're using `api_keys` (plural) in `.security.yml` for models and web tools (except GLMSearch)
|
||||||
- Check that the array format is correct in YAML (proper indentation)
|
- Check that the array format is correct in YAML (proper indentation)
|
||||||
- Remember: Models, Brave, Tavily, Perplexity MUST use `api_keys` (array format)
|
- Remember: Models, Brave, Tavily, Perplexity MUST use `api_keys` (array format)
|
||||||
- GLMSearch MUST use `api_key` (single string format)
|
- GLMSearch MUST use `api_key` (single string format)
|
||||||
|
|
|
||||||
|
|
@ -1341,7 +1341,7 @@ func LoadConfig(path string) (*Config, error) {
|
||||||
return nil, fmt.Errorf("failed to load security config: %w", err)
|
return nil, fmt.Errorf("failed to load security config: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Apply security references from security.yml BEFORE resolveAPIKeys
|
// Apply security references from .security.yml BEFORE resolveAPIKeys
|
||||||
// This resolves ref: references to actual values
|
// This resolves ref: references to actual values
|
||||||
if err := applySecurityConfig(cfg, sec); err != nil {
|
if err := applySecurityConfig(cfg, sec); err != nil {
|
||||||
return nil, fmt.Errorf("failed to apply security config: %w", err)
|
return nil, fmt.Errorf("failed to apply security config: %w", err)
|
||||||
|
|
@ -1404,7 +1404,7 @@ func copyArray[T any](dst, src *[]T) {
|
||||||
}
|
}
|
||||||
|
|
||||||
// applySecurityConfig resolves all security references in config
|
// applySecurityConfig resolves all security references in config
|
||||||
// It checks each field for "ref:" prefixed values and resolves them from security.yml
|
// It checks each field for "ref:" prefixed values and resolves them from .security.yml
|
||||||
func applySecurityConfig(cfg *Config, sec *SecurityConfig) error {
|
func applySecurityConfig(cfg *Config, sec *SecurityConfig) error {
|
||||||
if sec == nil {
|
if sec == nil {
|
||||||
return nil
|
return nil
|
||||||
|
|
@ -1444,7 +1444,7 @@ func applySecurityConfig(cfg *Config, sec *SecurityConfig) error {
|
||||||
}
|
}
|
||||||
|
|
||||||
// Try match without index suffix (e.g., "abc" -> "abc")
|
// Try match without index suffix (e.g., "abc" -> "abc")
|
||||||
// This allows security.yml to use simpler keys like "test-model" instead of "test-model:0"
|
// This allows .security.yml to use simpler keys like "test-model" instead of "test-model:0"
|
||||||
baseName := model.ModelName
|
baseName := model.ModelName
|
||||||
if entry, exists := sec.ModelList[baseName]; exists {
|
if entry, exists := sec.ModelList[baseName]; exists {
|
||||||
copyArray(&model.apiKeys, &entry.APIKeys)
|
copyArray(&model.apiKeys, &entry.APIKeys)
|
||||||
|
|
@ -1838,7 +1838,7 @@ func SaveConfig(path string, cfg *Config) error {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if err := saveSecurityConfig(securityPath(path), cfg.security); err != nil {
|
if err := saveSecurityConfig(securityPath(path), cfg.security); err != nil {
|
||||||
logger.ErrorCF("config", "cannot save security.yml", map[string]any{"error": err})
|
logger.ErrorCF("config", "cannot save .security.yml", map[string]any{"error": err})
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -118,7 +118,7 @@ chmod 600 ~/.picoclaw/security.yml
|
||||||
|
|
||||||
```gitignore
|
```gitignore
|
||||||
# Security configuration
|
# Security configuration
|
||||||
security.yml
|
.security.yml
|
||||||
```
|
```
|
||||||
|
|
||||||
## 5. Verify it works
|
## 5. Verify it works
|
||||||
|
|
@ -136,7 +136,7 @@ Examples:
|
||||||
- ref:model_list.gpt-5.4.api_key
|
- ref:model_list.gpt-5.4.api_key
|
||||||
- ref:model_list.claude-sonnet-4.6.api_key
|
- ref:model_list.claude-sonnet-4.6.api_key
|
||||||
|
|
||||||
**Note:** In security.yml, use `api_keys` (array) format for models.
|
**Note:** In .security.yml, use `api_keys` (array) format for models.
|
||||||
Both single and multiple keys should use the array format.
|
Both single and multiple keys should use the array format.
|
||||||
|
|
||||||
## Channel Tokens/Secrets
|
## Channel Tokens/Secrets
|
||||||
|
|
@ -172,8 +172,8 @@ Both single and multiple keys should use the array format.
|
||||||
- ref:web.glm_search.api_key
|
- ref:web.glm_search.api_key
|
||||||
|
|
||||||
**Note:**
|
**Note:**
|
||||||
- Brave, Tavily, Perplexity: Use `api_keys` (array) format in security.yml
|
- Brave, Tavily, Perplexity: Use `api_keys` (array) format in .security.yml
|
||||||
- GLMSearch: Use `api_key` (single string) format in security.yml
|
- GLMSearch: Use `api_key` (single string) format in .security.yml
|
||||||
|
|
||||||
## Skills Registry Tokens
|
## Skills Registry Tokens
|
||||||
- ref:skills.github.token
|
- ref:skills.github.token
|
||||||
|
|
@ -206,7 +206,7 @@ You can also mix references and direct values:
|
||||||
"model_list": [
|
"model_list": [
|
||||||
{
|
{
|
||||||
"model_name": "cloud-model",
|
"model_name": "cloud-model",
|
||||||
"api_key": "ref:model_list.cloud-model.api_key" // From security.yml
|
"api_key": "ref:model_list.cloud-model.api_key" // From .security.yml
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"model_name": "local-model",
|
"model_name": "local-model",
|
||||||
|
|
@ -226,11 +226,11 @@ cp ~/.picoclaw/config.json ~/.picoclaw/config.json.backup
|
||||||
|
|
||||||
## Step 2: Copy the example security file
|
## Step 2: Copy the example security file
|
||||||
```bash
|
```bash
|
||||||
cp security.example.yml ~/.picoclaw/security.yml
|
cp security.example.yml ~/.picoclaw/.security.yml
|
||||||
```
|
```
|
||||||
|
|
||||||
## Step 3: Fill in your API keys
|
## Step 3: Fill in your API keys
|
||||||
Edit ~/.picoclaw/security.yml and replace placeholders with your actual keys.
|
Edit ~/.picoclaw/.security.yml and replace placeholders with your actual keys.
|
||||||
|
|
||||||
## Step 4: Update config.json references
|
## Step 4: Update config.json references
|
||||||
Replace sensitive values in ~/.picoclaw/config.json with ref: references.
|
Replace sensitive values in ~/.picoclaw/config.json with ref: references.
|
||||||
|
|
@ -255,7 +255,7 @@ You can configure multiple API keys for both models and web tools to enable:
|
||||||
|
|
||||||
### Example: Model with Multiple Keys
|
### Example: Model with Multiple Keys
|
||||||
|
|
||||||
**security.yml:**
|
**.security.yml:**
|
||||||
```yaml
|
```yaml
|
||||||
model_list:
|
model_list:
|
||||||
|
|
||||||
|
|
@ -284,7 +284,7 @@ model_list:
|
||||||
|
|
||||||
### Example: Web Tool with Multiple Keys
|
### Example: Web Tool with Multiple Keys
|
||||||
|
|
||||||
**security.yml:**
|
**.security.yml:**
|
||||||
```yaml
|
```yaml
|
||||||
web:
|
web:
|
||||||
|
|
||||||
|
|
@ -342,12 +342,12 @@ model_list:
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
**Important:** All model keys in security.yml must use the `api_keys` (plural) array format.
|
**Important:** All model keys in .security.yml must use the `api_keys` (plural) array format.
|
||||||
The single `api_key` (singular) format is NOT supported for models.
|
The single `api_key` (singular) format is NOT supported for models.
|
||||||
|
|
||||||
### Model Index Matching
|
### Model Index Matching
|
||||||
|
|
||||||
The system supports intelligent model name matching in security.yml:
|
The system supports intelligent model name matching in .security.yml:
|
||||||
|
|
||||||
**Example 1: Exact Match**
|
**Example 1: Exact Match**
|
||||||
```yaml
|
```yaml
|
||||||
|
|
@ -357,7 +357,7 @@ The system supports intelligent model name matching in security.yml:
|
||||||
"model_name": "gpt-5.4:0"
|
"model_name": "gpt-5.4:0"
|
||||||
}
|
}
|
||||||
|
|
||||||
# security.yml (exact match with index)
|
# .security.yml (exact match with index)
|
||||||
model_list:
|
model_list:
|
||||||
|
|
||||||
gpt-5.4:0:
|
gpt-5.4:0:
|
||||||
|
|
@ -373,7 +373,7 @@ model_list:
|
||||||
"model_name": "gpt-5.4:0"
|
"model_name": "gpt-5.4:0"
|
||||||
}
|
}
|
||||||
|
|
||||||
# security.yml (base name without index)
|
# .security.yml (base name without index)
|
||||||
model_list:
|
model_list:
|
||||||
|
|
||||||
gpt-5.4:
|
gpt-5.4:
|
||||||
|
|
@ -381,7 +381,7 @@ model_list:
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Both methods work. The base name match allows you to use simpler keys in security.yml
|
Both methods work. The base name match allows you to use simpler keys in .security.yml
|
||||||
even when your config uses indexed model names for load balancing.
|
even when your config uses indexed model names for load balancing.
|
||||||
|
|
||||||
### Security File Permissions
|
### Security File Permissions
|
||||||
|
|
@ -389,34 +389,34 @@ even when your config uses indexed model names for load balancing.
|
||||||
The security file should have restricted permissions:
|
The security file should have restricted permissions:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
chmod 600 ~/.picoclaw/security.yml
|
chmod 600 ~/.picoclaw/.security.yml
|
||||||
```
|
```
|
||||||
|
|
||||||
This ensures only the owner can read and write the file.
|
This ensures only the owner can read and write the file.
|
||||||
|
|
||||||
# Security Best Practices
|
# Security Best Practices
|
||||||
|
|
||||||
1. Never commit security.yml to version control
|
1. Never commit .security.yml to version control
|
||||||
2. Set file permissions: chmod 600 ~/.picoclaw/security.yml
|
2. Set file permissions: chmod 600 ~/.picoclaw/.security.yml
|
||||||
3. Use different keys for different environments
|
3. Use different keys for different environments
|
||||||
4. Rotate keys regularly and update security.yml
|
4. Rotate keys regularly and update .security.yml
|
||||||
5. Encrypt backups containing security.yml
|
5. Encrypt backups containing .security.yml
|
||||||
|
|
||||||
# Troubleshooting
|
# Troubleshooting
|
||||||
|
|
||||||
## Error: "model security entry not found"
|
## Error: "model security entry not found"
|
||||||
- Check that the model name in config.json matches exactly in security.yml
|
- Check that the model name in config.json matches exactly in .security.yml
|
||||||
- Verify the model_list section exists in security.yml
|
- Verify the model_list section exists in .security.yml
|
||||||
|
|
||||||
## Error: "failed to load security config"
|
## Error: "failed to load security config"
|
||||||
- Ensure security.yml exists in the same directory as config.json
|
- Ensure .security.yml exists in the same directory as config.json
|
||||||
- Check YAML syntax is valid
|
- Check YAML syntax is valid
|
||||||
- Verify file permissions allow reading
|
- Verify file permissions allow reading
|
||||||
|
|
||||||
## Error: "unknown reference path"
|
## Error: "unknown reference path"
|
||||||
- Verify the reference format is correct
|
- Verify the reference format is correct
|
||||||
- Check the path structure matches the examples above
|
- Check the path structure matches the examples above
|
||||||
- Ensure all required sections exist in security.yml
|
- Ensure all required sections exist in .security.yml
|
||||||
*/
|
*/
|
||||||
package config
|
package config
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -19,7 +19,7 @@ import (
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
SecurityConfigFile = "security.yml"
|
SecurityConfigFile = ".security.yml"
|
||||||
)
|
)
|
||||||
|
|
||||||
// SecurityConfig stores all sensitive data (API keys, tokens, secrets, passwords)
|
// SecurityConfig stores all sensitive data (API keys, tokens, secrets, passwords)
|
||||||
|
|
|
||||||
|
|
@ -80,8 +80,8 @@ func TestSecurityConfigIntegration(t *testing.T) {
|
||||||
err := os.WriteFile(configPath, []byte(configContent), 0o644)
|
err := os.WriteFile(configPath, []byte(configContent), 0o644)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
|
|
||||||
// Create security.yml with actual values
|
// Create .security.yml with actual values
|
||||||
securityPath := filepath.Join(tmpDir, "security.yml")
|
securityPath := filepath.Join(tmpDir, SecurityConfigFile)
|
||||||
securityContent := `model_list:
|
securityContent := `model_list:
|
||||||
test-model:
|
test-model:
|
||||||
api_keys:
|
api_keys:
|
||||||
|
|
@ -141,8 +141,8 @@ func TestSecurityConfigWithAPIKeysArray(t *testing.T) {
|
||||||
err := os.WriteFile(configPath, []byte(configContent), 0o644)
|
err := os.WriteFile(configPath, []byte(configContent), 0o644)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
|
|
||||||
// Create security.yml
|
// Create .security.yml
|
||||||
securityPath := filepath.Join(tmpDir, "security.yml")
|
securityPath := filepath.Join(tmpDir, SecurityConfigFile)
|
||||||
securityContent := `model_list:
|
securityContent := `model_list:
|
||||||
multi-key-model:0:
|
multi-key-model:0:
|
||||||
api_key: "sk-key-1"
|
api_key: "sk-key-1"
|
||||||
|
|
@ -197,7 +197,7 @@ func TestAllSecurityKeysAccessible(t *testing.T) {
|
||||||
err = os.WriteFile(clawhubAuthTokenFile, []byte("clawhub-auth-token-from-file"), 0o600)
|
err = os.WriteFile(clawhubAuthTokenFile, []byte("clawhub-auth-token-from-file"), 0o600)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
|
|
||||||
// Create config.json without sensitive values (they'll be in security.yml)
|
// Create config.json without sensitive values (they'll be in .security.yml)
|
||||||
configPath := filepath.Join(tmpDir, "config.json")
|
configPath := filepath.Join(tmpDir, "config.json")
|
||||||
configContent := `{
|
configContent := `{
|
||||||
"version": 1,
|
"version": 1,
|
||||||
|
|
@ -288,8 +288,8 @@ func TestAllSecurityKeysAccessible(t *testing.T) {
|
||||||
err = os.WriteFile(configPath, []byte(configContent), 0o644)
|
err = os.WriteFile(configPath, []byte(configContent), 0o644)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
|
|
||||||
// Create security.yml with file:// references and plaintext values
|
// Create .security.yml with file:// references and plaintext values
|
||||||
securityPath := filepath.Join(tmpDir, "security.yml")
|
securityPath := filepath.Join(tmpDir, SecurityConfigFile)
|
||||||
securityContent := `model_list:
|
securityContent := `model_list:
|
||||||
test-model-1:
|
test-model-1:
|
||||||
api_keys:
|
api_keys:
|
||||||
|
|
|
||||||
|
|
@ -16,7 +16,7 @@ import (
|
||||||
|
|
||||||
func TestSecurityConfig(t *testing.T) {
|
func TestSecurityConfig(t *testing.T) {
|
||||||
t.Run("LoadNonExistent", func(t *testing.T) {
|
t.Run("LoadNonExistent", func(t *testing.T) {
|
||||||
sec, err := loadSecurityConfig("/nonexistent/security.yml")
|
sec, err := loadSecurityConfig("/nonexistent/.security.yml")
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
assert.NotNil(t, sec)
|
assert.NotNil(t, sec)
|
||||||
assert.Empty(t, sec.ModelList)
|
assert.Empty(t, sec.ModelList)
|
||||||
|
|
@ -32,12 +32,12 @@ func TestSecurityPath(t *testing.T) {
|
||||||
{
|
{
|
||||||
name: "standard path",
|
name: "standard path",
|
||||||
configDir: "/home/user/.picoclaw/config.json",
|
configDir: "/home/user/.picoclaw/config.json",
|
||||||
want: "/home/user/.picoclaw/security.yml",
|
want: "/home/user/.picoclaw/.security.yml",
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
name: "nested path",
|
name: "nested path",
|
||||||
configDir: "/path/to/config/myconfig.json",
|
configDir: "/path/to/config/myconfig.json",
|
||||||
want: "/path/to/config/security.yml",
|
want: "/path/to/config/.security.yml",
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -51,7 +51,7 @@ func TestSecurityPath(t *testing.T) {
|
||||||
|
|
||||||
func TestSaveAndLoadSecurityConfig(t *testing.T) {
|
func TestSaveAndLoadSecurityConfig(t *testing.T) {
|
||||||
tmpDir := t.TempDir()
|
tmpDir := t.TempDir()
|
||||||
secPath := filepath.Join(tmpDir, "security.yml")
|
secPath := filepath.Join(tmpDir, SecurityConfigFile)
|
||||||
|
|
||||||
original := &SecurityConfig{
|
original := &SecurityConfig{
|
||||||
ModelList: map[string]ModelSecurityEntry{
|
ModelList: map[string]ModelSecurityEntry{
|
||||||
|
|
|
||||||
|
|
@ -1,184 +0,0 @@
|
||||||
# PicoClaw Security Configuration
|
|
||||||
# This file stores all sensitive data (API keys, tokens, secrets, passwords)
|
|
||||||
# Keep this file secure and never commit it to version control
|
|
||||||
# Copy this file to security.yml and fill in your actual values
|
|
||||||
|
|
||||||
# Model API Keys
|
|
||||||
# Use dot notation references in config.json like: "ref:model_list.gpt-5.4.api_key"
|
|
||||||
# IMPORTANT: Use 'api_keys' (array) format - both single and multiple keys
|
|
||||||
model_list:
|
|
||||||
# Example: OpenAI GPT-5.4 (multiple keys for load balancing/failover)
|
|
||||||
gpt-5.4:
|
|
||||||
api_keys:
|
|
||||||
- "your-openai-api-key-1"
|
|
||||||
- "your-openai-api-key-2" # Optional: failover key
|
|
||||||
|
|
||||||
# Example: Claude Sonnet (single key in array format)
|
|
||||||
claude-sonnet-4.6:
|
|
||||||
api_keys:
|
|
||||||
- "your-anthropic-api-key-here" # Single key MUST be in array format
|
|
||||||
|
|
||||||
# Example: Zhipu GLM
|
|
||||||
glm-4.7:
|
|
||||||
api_key: "your-zhipu-api-key-here"
|
|
||||||
|
|
||||||
# Example: DeepSeek
|
|
||||||
deepseek-chat:
|
|
||||||
api_key: "your-deepseek-api-key-here"
|
|
||||||
|
|
||||||
# Example: Google Gemini
|
|
||||||
gemini-2.0-flash:
|
|
||||||
api_key: "your-gemini-api-key-here"
|
|
||||||
|
|
||||||
# Example: Qwen
|
|
||||||
qwen-plus:
|
|
||||||
api_key: "your-qwen-api-key-here"
|
|
||||||
|
|
||||||
# Example: Moonshot
|
|
||||||
moonshot-v1-8k:
|
|
||||||
api_key: "your-moonshot-api-key-here"
|
|
||||||
|
|
||||||
# Example: Groq
|
|
||||||
llama-3.3-70b:
|
|
||||||
api_key: "your-groq-api-key-here"
|
|
||||||
|
|
||||||
# Example: OpenRouter
|
|
||||||
openrouter-auto:
|
|
||||||
api_key: "your-openrouter-api-key-here"
|
|
||||||
openrouter-gpt-5.4:
|
|
||||||
api_key: "your-openrouter-api-key-here"
|
|
||||||
|
|
||||||
# Example: NVIDIA
|
|
||||||
nemotron-4-340b:
|
|
||||||
api_key: "your-nvidia-api-key-here"
|
|
||||||
|
|
||||||
# Example: Cerebras
|
|
||||||
cerebras-llama-3.3-70b:
|
|
||||||
api_key: "your-cerebras-api-key-here"
|
|
||||||
|
|
||||||
# Example: Vivgrid
|
|
||||||
vivgrid-auto:
|
|
||||||
api_key: "your-vivgrid-api-key-here"
|
|
||||||
|
|
||||||
# Example: Volcengine
|
|
||||||
ark-code-latest:
|
|
||||||
api_key: "your-volcengine-api-key-here"
|
|
||||||
doubao-pro:
|
|
||||||
api_key: "your-volcengine-api-key-here"
|
|
||||||
|
|
||||||
# Example: ShengsuanYun
|
|
||||||
deepseek-v3:
|
|
||||||
api_key: "your-shengsuanyun-api-key-here"
|
|
||||||
|
|
||||||
# Example: Mistral
|
|
||||||
mistral-small:
|
|
||||||
api_key: "your-mistral-api-key-here"
|
|
||||||
|
|
||||||
# Example: Avian
|
|
||||||
deepseek-v3.2:
|
|
||||||
api_key: "your-avian-api-key-here"
|
|
||||||
kimi-k2.5:
|
|
||||||
api_key: "your-avian-api-key-here"
|
|
||||||
|
|
||||||
# Example: Minimax
|
|
||||||
MiniMax-M2.5:
|
|
||||||
api_key: "your-minimax-api-key-here"
|
|
||||||
|
|
||||||
# Example: LongCat
|
|
||||||
LongCat-Flash-Thinking:
|
|
||||||
api_key: "your-longcat-api-key-here"
|
|
||||||
|
|
||||||
# Example: ModelScope
|
|
||||||
modelscope-qwen:
|
|
||||||
api_key: "your-modelscope-api-key-here"
|
|
||||||
|
|
||||||
# Example: VLLM (local, usually no real key needed)
|
|
||||||
local-model:
|
|
||||||
api_key: ""
|
|
||||||
|
|
||||||
# Example: Azure OpenAI
|
|
||||||
azure-gpt5:
|
|
||||||
api_key: "your-azure-api-key-here"
|
|
||||||
|
|
||||||
# Channel Tokens and Secrets
|
|
||||||
channels:
|
|
||||||
telegram:
|
|
||||||
token: "your-telegram-bot-token"
|
|
||||||
|
|
||||||
feishu:
|
|
||||||
app_secret: "your-feishu-app-secret"
|
|
||||||
encrypt_key: "your-feishu-encrypt-key"
|
|
||||||
verification_token: "your-feishu-verification-token"
|
|
||||||
|
|
||||||
discord:
|
|
||||||
token: "your-discord-bot-token"
|
|
||||||
|
|
||||||
qq:
|
|
||||||
app_secret: "your-qq-app-secret"
|
|
||||||
|
|
||||||
dingtalk:
|
|
||||||
client_secret: "your-dingtalk-client-secret"
|
|
||||||
|
|
||||||
slack:
|
|
||||||
bot_token: "your-slack-bot-token"
|
|
||||||
app_token: "your-slack-app-token"
|
|
||||||
|
|
||||||
matrix:
|
|
||||||
access_token: "your-matrix-access-token"
|
|
||||||
|
|
||||||
line:
|
|
||||||
channel_secret: "your-line-channel-secret"
|
|
||||||
channel_access_token: "your-line-channel-access-token"
|
|
||||||
|
|
||||||
onebot:
|
|
||||||
access_token: "your-onebot-access-token"
|
|
||||||
|
|
||||||
wecom:
|
|
||||||
token: "your-wecom-token"
|
|
||||||
encoding_aes_key: "your-wecom-encoding-aes-key"
|
|
||||||
|
|
||||||
wecom_app:
|
|
||||||
corp_secret: "your-wecom-app-corp-secret"
|
|
||||||
token: "your-wecom-app-token"
|
|
||||||
encoding_aes_key: "your-wecom-app-encoding-aes-key"
|
|
||||||
|
|
||||||
wecom_aibot:
|
|
||||||
token: "your-wecom-aibot-token"
|
|
||||||
encoding_aes_key: "your-wecom-aibot-encoding-aes-key"
|
|
||||||
|
|
||||||
pico:
|
|
||||||
token: "your-pico-token"
|
|
||||||
|
|
||||||
irc:
|
|
||||||
password: "your-irc-password"
|
|
||||||
nickserv_password: "your-irc-nickserv-password"
|
|
||||||
sasl_password: "your-irc-sasl-password"
|
|
||||||
|
|
||||||
# Web Tool API Keys
|
|
||||||
# IMPORTANT: Use 'api_keys' (array) for Brave, Tavily, Perplexity
|
|
||||||
# Use 'api_key' (single string) for GLMSearch only
|
|
||||||
web:
|
|
||||||
brave:
|
|
||||||
api_keys:
|
|
||||||
- "your-brave-api-key-1"
|
|
||||||
- "your-brave-api-key-2" # Optional: failover key
|
|
||||||
|
|
||||||
tavily:
|
|
||||||
api_keys:
|
|
||||||
- "your-tavily-api-key" # Single key MUST be in array format
|
|
||||||
|
|
||||||
perplexity:
|
|
||||||
api_keys:
|
|
||||||
- "your-perplexity-api-key-1"
|
|
||||||
- "your-perplexity-api-key-2"
|
|
||||||
|
|
||||||
glm_search:
|
|
||||||
api_key: "your-glm-search-api-key" # GLMSearch uses single string format (NOT array)
|
|
||||||
|
|
||||||
# Skills Registry Tokens
|
|
||||||
skills:
|
|
||||||
github:
|
|
||||||
token: "your-github-token"
|
|
||||||
|
|
||||||
clawhub:
|
|
||||||
auth_token: "your-clawhub-auth-token"
|
|
||||||
Loading…
Add table
Reference in a new issue