feat: add support for reasoning models (K2-Think, DeepSeek R1, QwQ)

Add support for reasoning models that return responses in the
reasoning_content field instead of the standard content field.

Changes:
- Add reasoning_content field to response parsing
- Fallback to reasoning_content if content is empty
- Add reasoning_effort parameter for K2-Think models
- Strip special tokens (<|im_end|>, <|endoftext|>, etc.) from output

This enables compatibility with:
- LLM360/K2-Think-V2 (with vLLM)
- DeepSeek R1
- QwQ and other reasoning models

Tested with K2-Think-V2 on vLLM with --tool-call-parser hermes.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
Shaurya Rohatgi 2026-02-16 15:34:54 -08:00
parent 3407bf3055
commit 0d7c84a1ca

View file

@ -198,6 +198,15 @@ func (p *HTTPProvider) parseResponse(body []byte) (*LLMResponse, error) {
content = choice.Message.ReasoningContent content = choice.Message.ReasoningContent
} }
// Strip special tokens that some models include in their output
content = strings.TrimSpace(content)
specialTokens := []string{"<|im_end|>", "<|endoftext|>", "</think_fast>", "<|end|>"}
for _, token := range specialTokens {
content = strings.TrimSuffix(content, token)
content = strings.TrimPrefix(content, token)
content = strings.TrimSpace(content)
}
return &LLMResponse{ return &LLMResponse{
Content: content, Content: content,
ToolCalls: toolCalls, ToolCalls: toolCalls,