fix(openrouter): ship reasoning suppression preset for Nemotron free
Some OpenRouter reasoning models (e.g. nvidia/nemotron-3-super-120b-a12b:free) emit chain-of-thought into message.content, so even strict prompts like "Reply with exactly: PONG" produce visible reasoning preamble. Add a default model_list entry that sets extra_body.reasoning.exclude=true so OpenRouter strips reasoning server-side, plus a config test and docs note covering the symptom, cause, and per-model tradeoff. No global auto-injection: reasoning visibility stays a per-model choice. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
6e1fab80e2
commit
b7e9f1cd87
4 changed files with 113 additions and 0 deletions
|
|
@ -293,6 +293,31 @@ For direct Anthropic API access or custom endpoints that only support Anthropic'
|
||||||
`api_base` defaults to `http://localhost:1234/v1`. API key is optional unless your LM Studio server enables authentication.<br/>
|
`api_base` defaults to `http://localhost:1234/v1`. API key is optional unless your LM Studio server enables authentication.<br/>
|
||||||
With explicit `provider`, PicoClaw sends `openai/gpt-oss-20b` unchanged to the LM Studio server. The legacy compatibility form `"model": "lmstudio/openai/gpt-oss-20b"` still resolves to the same upstream model ID when `provider` is omitted.
|
With explicit `provider`, PicoClaw sends `openai/gpt-oss-20b` unchanged to the LM Studio server. The legacy compatibility form `"model": "lmstudio/openai/gpt-oss-20b"` still resolves to the same upstream model ID when `provider` is omitted.
|
||||||
|
|
||||||
|
**OpenRouter reasoning models (suppressing thinking output)**
|
||||||
|
|
||||||
|
Some OpenRouter reasoning models (for example `nvidia/nemotron-3-super-120b-a12b:free`) emit their chain-of-thought into `message.content` rather than a separate reasoning channel. With a strict prompt like `Reply with exactly: PONG`, the visible reply may then start with reasoning preamble such as `We need to ...` before the requested text.
|
||||||
|
|
||||||
|
Pass `extra_body.reasoning.exclude = true` so OpenRouter strips the reasoning server-side:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"model_name": "openrouter-nemotron-free",
|
||||||
|
"provider": "openrouter",
|
||||||
|
"model": "nvidia/nemotron-3-super-120b-a12b:free",
|
||||||
|
"api_base": "https://openrouter.ai/api/v1",
|
||||||
|
"api_keys": ["sk-or-v1-..."],
|
||||||
|
"extra_body": {
|
||||||
|
"reasoning": {
|
||||||
|
"exclude": true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
This is shipped as a default `model_list` entry (`openrouter-nemotron-free`); add your `api_keys` to enable it.
|
||||||
|
|
||||||
|
Tradeoff: only set this on entries that should hide reasoning. If you rely on OpenRouter to surface reasoning tokens (for example to render thinking separately), leave `extra_body` unset on those models. PicoClaw does not auto-inject `reasoning.exclude` for OpenRouter, since reasoning visibility is a per-model preference.
|
||||||
|
|
||||||
**Custom Proxy/API**
|
**Custom Proxy/API**
|
||||||
|
|
||||||
```json
|
```json
|
||||||
|
|
|
||||||
|
|
@ -48,3 +48,30 @@ Example snippet:
|
||||||
```
|
```
|
||||||
|
|
||||||
Get your key at [OpenRouter Keys](https://openrouter.ai/keys).
|
Get your key at [OpenRouter Keys](https://openrouter.ai/keys).
|
||||||
|
|
||||||
|
## OpenRouter reasoning model leaks thinking into reply
|
||||||
|
|
||||||
|
**Symptom:** With an OpenRouter reasoning model (for example `nvidia/nemotron-3-super-120b-a12b:free`), even a strict prompt like `Reply with exactly: PONG` produces a reply whose visible content begins with reasoning preamble such as `We need to follow the instruction ...` before the requested text.
|
||||||
|
|
||||||
|
**Cause:** Some OpenRouter reasoning models put their chain-of-thought into `message.content` rather than a separate reasoning channel. Without server-side suppression, that reasoning surfaces as the assistant's visible reply.
|
||||||
|
|
||||||
|
**Fix:** Set `extra_body.reasoning.exclude = true` on the model entry so OpenRouter strips reasoning before returning content:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"model_name": "openrouter-nemotron-free",
|
||||||
|
"provider": "openrouter",
|
||||||
|
"model": "nvidia/nemotron-3-super-120b-a12b:free",
|
||||||
|
"api_base": "https://openrouter.ai/api/v1",
|
||||||
|
"api_keys": ["sk-or-v1-YOUR_OPENROUTER_KEY"],
|
||||||
|
"extra_body": {
|
||||||
|
"reasoning": {
|
||||||
|
"exclude": true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
PicoClaw ships this as a default `model_list` entry (`openrouter-nemotron-free`); set `api_keys` to use it.
|
||||||
|
|
||||||
|
**Tradeoff:** apply this per-model, not globally. Some users want OpenRouter to return reasoning tokens (for example to render thinking separately), so PicoClaw does not auto-inject `reasoning.exclude` for all OpenRouter models.
|
||||||
|
|
|
||||||
|
|
@ -2019,6 +2019,53 @@ func TestDefaultConfig_MinimaxExtraBody(t *testing.T) {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestDefaultConfig_OpenRouterNemotronReasoningExcluded verifies the default
|
||||||
|
// OpenRouter Nemotron entry ships with reasoning suppression so reasoning
|
||||||
|
// preamble does not leak into visible assistant content.
|
||||||
|
func TestDefaultConfig_OpenRouterNemotronReasoningExcluded(t *testing.T) {
|
||||||
|
cfg := DefaultConfig()
|
||||||
|
|
||||||
|
var nemo *ModelConfig
|
||||||
|
for i := range cfg.ModelList {
|
||||||
|
if cfg.ModelList[i].Provider == "openrouter" &&
|
||||||
|
cfg.ModelList[i].Model == "nvidia/nemotron-3-super-120b-a12b:free" {
|
||||||
|
nemo = cfg.ModelList[i]
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if nemo == nil {
|
||||||
|
t.Fatal("OpenRouter Nemotron model not found in ModelList")
|
||||||
|
}
|
||||||
|
if nemo.ExtraBody == nil {
|
||||||
|
t.Fatal("OpenRouter Nemotron ExtraBody should not be nil")
|
||||||
|
}
|
||||||
|
reasoning, ok := nemo.ExtraBody["reasoning"].(map[string]any)
|
||||||
|
if !ok {
|
||||||
|
t.Fatalf("ExtraBody[reasoning] = %T, want map[string]any", nemo.ExtraBody["reasoning"])
|
||||||
|
}
|
||||||
|
if got, ok := reasoning["exclude"]; !ok || got != true {
|
||||||
|
t.Fatalf("ExtraBody[reasoning][exclude] = %v, want true", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Round-trip the model entry through JSON to confirm the nested
|
||||||
|
// reasoning map survives marshal/unmarshal the same way config files do.
|
||||||
|
data, err := json.Marshal(nemo)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("json.Marshal: %v", err)
|
||||||
|
}
|
||||||
|
var rt ModelConfig
|
||||||
|
if err := json.Unmarshal(data, &rt); err != nil {
|
||||||
|
t.Fatalf("json.Unmarshal: %v", err)
|
||||||
|
}
|
||||||
|
rtReasoning, ok := rt.ExtraBody["reasoning"].(map[string]any)
|
||||||
|
if !ok {
|
||||||
|
t.Fatalf("round-tripped ExtraBody[reasoning] = %T, want map[string]any", rt.ExtraBody["reasoning"])
|
||||||
|
}
|
||||||
|
if got, ok := rtReasoning["exclude"]; !ok || got != true {
|
||||||
|
t.Fatalf("round-tripped ExtraBody[reasoning][exclude] = %v, want true", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestFilterSensitiveData(t *testing.T) {
|
func TestFilterSensitiveData(t *testing.T) {
|
||||||
// Test with nil security config
|
// Test with nil security config
|
||||||
cfg := &Config{}
|
cfg := &Config{}
|
||||||
|
|
|
||||||
|
|
@ -144,6 +144,20 @@ func DefaultConfig() *Config {
|
||||||
Model: "openai/gpt-5.4",
|
Model: "openai/gpt-5.4",
|
||||||
APIBase: "https://openrouter.ai/api/v1",
|
APIBase: "https://openrouter.ai/api/v1",
|
||||||
},
|
},
|
||||||
|
// OpenRouter reasoning models that emit thinking into message.content
|
||||||
|
// require server-side reasoning suppression to avoid leaking the
|
||||||
|
// chain-of-thought into the visible assistant reply. See
|
||||||
|
// docs/operations/troubleshooting.md ("OpenRouter reasoning model
|
||||||
|
// leaks thinking into reply").
|
||||||
|
{
|
||||||
|
ModelName: "openrouter-nemotron-free",
|
||||||
|
Provider: "openrouter",
|
||||||
|
Model: "nvidia/nemotron-3-super-120b-a12b:free",
|
||||||
|
APIBase: "https://openrouter.ai/api/v1",
|
||||||
|
ExtraBody: map[string]any{
|
||||||
|
"reasoning": map[string]any{"exclude": true},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
|
||||||
// NVIDIA - https://build.nvidia.com/
|
// NVIDIA - https://build.nvidia.com/
|
||||||
{
|
{
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue