yao/agent/output/message/STREAMING.md

Message Streaming Architecture

This document explains the hierarchical streaming architecture for Agent/LLM/MCP message delivery.

Overview

The streaming system uses a hierarchical structure to handle complex scenarios including:

• Single LLM calls with multiple message types (thinking, tool calls, text)
• Agent logic with multiple sequential operations (LLM → MCP → LLM)
• Concurrent/parallel calls to multiple LLMs or MCPs
• Real-time delta updates for streaming responses

Hierarchical Structure

Agent Stream (entire conversation)
  └─ ThreadID (concurrent stream, optional: T1, T2, T3...)
      └─ BlockID (output block/section: B1, B2, B3...)
          └─ MessageID (logical message: M1, M2, M3...)
              └─ ChunkID (stream fragment: C1, C2, C3...)

Field Definitions

Message Struct Fields

type Message struct {
    // Core fields
    Type  string                 `json:"type"`
    Props map[string]interface{} `json:"props,omitempty"`

    // Streaming control
    ChunkID   string `json:"chunk_id,omitempty"`
    MessageID string `json:"message_id,omitempty"`
    BlockID   string `json:"block_id,omitempty"`
    ThreadID  string `json:"thread_id,omitempty"`

    // Delta control
    Delta       bool   `json:"delta,omitempty"`
    DeltaPath   string `json:"delta_path,omitempty"`
    DeltaAction string `json:"delta_action,omitempty"`

    // ...
}

Field Responsibilities

Field	Generated By	Purpose	Example Values	Required
ChunkID	System (auto)	Deduplication, ordering, debugging	C1, C2, C3	Always
MessageID	LLM Provider/Handler	Delta merge target	M1, M2, M3 or thinking_msg_1	Required for delta scenarios
BlockID	Agent Logic	UI block/section rendering	B1, B2, B3 or llm_response_1	Required when Agent controls blocks
ThreadID	Agent Logic	Concurrent stream distinction	T1, T2, T3 or thread_llm1	Optional (concurrent only)

Detailed Field Explanation

ChunkID (Stream Fragment Identifier)

• Purpose: Uniquely identifies each chunk in the stream
• Generated: Automatically by the system (sequential: M1, M2, M3...)
• Used For:
  • Deduplication (prevent duplicate chunks)
  • Ordering (maintain correct sequence)
  • Debugging (trace message flow)
• Scope: Unique within entire Agent stream
• Always Present: Yes

Example:

{"chunk_id": "C1", "type": "text", "props": {"content": "Hello"}}
{"chunk_id": "C2", "type": "text", "props": {"content": " World"}}
{"chunk_id": "C3", "type": "thinking", "props": {"content": "..."}}

MessageID (Logical Message Identifier)

• Purpose: Groups multiple chunks into one logical message via delta merging
• Generated: By LLM Provider or Stream Handler
• Used For:
  • Delta merge target (frontend merges all chunks with same MessageID)
  • Distinguishing different messages within a group
• Scope: Unique within a Group
• Present When: Delta streaming is used

Example:

// Multiple chunks combine into one "thinking" message
{"chunk_id": "C1", "message_id": "M1", "type": "thinking", "props": {"content": "Let me"}, "delta": true}
{"chunk_id": "C2", "message_id": "M1", "type": "thinking", "props": {"content": " think"}, "delta": true}
{"chunk_id": "C3", "message_id": "M1", "type": "thinking", "props": {"content": "..."}, "delta": true}

// Another independent message
{"chunk_id": "C4", "message_id": "M2", "type": "text", "props": {"content": "Hello"}, "delta": true}

BlockID (Output Block Identifier)

• Purpose: Represents one output block/section (e.g., one LLM call, one MCP call)
• Generated: By Agent logic
• Used For:
  • Frontend UI block/section rendering (visual blocks)
  • Distinguishing different operations (LLM vs MCP vs custom logic)
  • Organizing related messages together
• Scope: Unique within entire Agent stream
• Present When: Agent explicitly controls output blocks

Key Concept: Block represents a semantic unit of work from Agent's perspective, NOT from LLM's perspective. Each block is rendered as a distinct UI section in the frontend.

Example:

// BLOCK 1: LLM Response (contains thinking + tool_call + text)
{"chunk_id": "C1", "block_id": "B1", "message_id": "M1", "type": "thinking", ...}
{"chunk_id": "C2", "block_id": "B1", "message_id": "M2", "type": "tool_call", ...}
{"chunk_id": "C3", "block_id": "B1", "message_id": "M3", "type": "text", ...}

// BLOCK 2: MCP Call
{"chunk_id": "C4", "block_id": "B2", "message_id": "M4", "type": "loading", ...}
{"chunk_id": "C5", "block_id": "B2", "message_id": "M5", "type": "text", ...}

// BLOCK 3: Another LLM Response
{"chunk_id": "C6", "block_id": "B3", "message_id": "M6", "type": "text", ...}

ThreadID (Concurrent Stream Identifier)

• Purpose: Distinguishes concurrent/parallel output streams
• Generated: By Agent logic when spawning concurrent operations
• Used For:
  • Separating outputs from parallel LLM/MCP calls
  • Maintaining independent streaming contexts
• Scope: Unique within entire Agent stream
• Present When: Agent makes concurrent calls (optional)

Example:

// Main thread
{"chunk_id": "C1", "thread_id": "T1", "block_id": "B1", "message_id": "M1", "type": "text", ...}

// Parallel MCP calls
{"chunk_id": "C2", "thread_id": "T2", "block_id": "B2", "message_id": "M2", "type": "text", ...}
{"chunk_id": "C3", "thread_id": "T3", "block_id": "B3", "message_id": "M3", "type": "text", ...}

Usage Scenarios

Scenario 1: Simple Text Message

No streaming, no grouping

{
  "chunk_id": "C1",
  "type": "text",
  "props": { "content": "Hello World" }
}

Fields Used:

• chunk_id: C1 (auto-generated)
• No message_id, block_id, or thread_id needed

---

Scenario 2: LLM Streaming Response (Single Message)

LLM streams one text message

{"chunk_id": "C1", "message_id": "M1", "type": "text", "props": {"content": "Hello"}, "delta": true}
{"chunk_id": "C2", "message_id": "M1", "type": "text", "props": {"content": " World"}, "delta": true}
{"chunk_id": "C3", "message_id": "M1", "type": "text", "props": {"content": "!"}, "delta": true}

Fields Used:

• chunk_id: C1, C2, C3 (unique per chunk)
• message_id: M1 (same for all, merge target)
• delta: true

Frontend Behavior:

• Merge all chunks with message_id: "M1" into one message
• Display: "Hello World!"

---

Scenario 3: Agent-Controlled LLM Call (One Block)

Agent wraps LLM response in an output block

// Agent code starts a block for the LLM response
// System generates block_id: "B1"
// LLM returns thinking + tool_call + text
// Agent ends the block

// LLM chunks within block B1
{"chunk_id": "C1", "message_id": "M1", "block_id": "B1", "type": "thinking", "props": {...}, "delta": true}
{"chunk_id": "C2", "message_id": "M1", "block_id": "B1", "type": "thinking", "props": {...}, "delta": true}

{"chunk_id": "C3", "message_id": "M2", "block_id": "B1", "type": "tool_call", "props": {...}}

{"chunk_id": "C4", "message_id": "M3", "block_id": "B1", "type": "text", "props": {...}, "delta": true}
{"chunk_id": "C5", "message_id": "M3", "block_id": "B1", "type": "text", "props": {...}, "delta": true}

Fields Used:

• chunk_id: C1~C5 (unique per chunk)
• message_id: M1, M2, M3 (per logical message)
• block_id: B1 (all belong to same LLM call)
• delta: true (for streaming messages)

Frontend Behavior:

• Render one block/section for block_id: "B1"
• Within this block, show 3 messages:
  • Thinking message (chunks C1+C2 merged into M1)
  • Tool call message (chunk C3 = M2)
  • Text message (chunks C4+C5 merged into M3)

---

Scenario 4: Agent Sequential Operations (Multiple Blocks)

Agent orchestrates: LLM → MCP → LLM

// Agent code orchestrates three sequential operations:
// 1. Block B1: First LLM call
// 2. Block B2: MCP call
// 3. Block B3: Second LLM call

// BLOCK 1: First LLM call
{"chunk_id": "C1", "message_id": "M1", "block_id": "B1", "type": "text", ...}
{"chunk_id": "C2", "message_id": "M1", "block_id": "B1", "type": "text", ...}

// BLOCK 2: MCP call
{"chunk_id": "C3", "message_id": "M2", "block_id": "B2", "type": "loading", ...}
{"chunk_id": "C4", "message_id": "M3", "block_id": "B2", "type": "text", ...}

// BLOCK 3: Second LLM call
{"chunk_id": "C5", "message_id": "M4", "block_id": "B3", "type": "text", ...}
{"chunk_id": "C6", "message_id": "M4", "block_id": "B3", "type": "text", ...}

Frontend Behavior:

• Render 3 distinct blocks/sections:
  1. Block 1 (B1): LLM response with text
  2. Block 2 (B2): MCP call with loading + result
  3. Block 3 (B3): LLM response with text

---

Scenario 5: Concurrent Operations (Blocks + Threads)

Agent uses concurrent handler to make parallel calls

// Agent orchestrates parallel operations within one block (B1)
// The concurrent handler automatically assigns thread_id to each operation:
// - MCP call for weather (thread_id: "T1")
// - MCP call for news (thread_id: "T2")
// - LLM call for summary (thread_id: "T3")
//
// Messages from different threads may arrive in any order

// Same block, different threads (may arrive in any order)
{"chunk_id": "C1", "message_id": "M1", "block_id": "B1", "thread_id": "T1", "type": "text", "props": {"content": "Weather: Sunny"}}
{"chunk_id": "C2", "message_id": "M2", "block_id": "B1", "thread_id": "T2", "type": "text", "props": {"content": "News: ..."}}
{"chunk_id": "C3", "message_id": "M1", "block_id": "B1", "thread_id": "T1", "type": "text", "props": {"content": ", 25°C"}}
{"chunk_id": "C4", "message_id": "M3", "block_id": "B1", "thread_id": "T3", "type": "text", "props": {"content": "Summary..."}}

Fields Used:

• chunk_id: C1, C2, C3, C4 (unique per chunk, chronological order)
• message_id: M1, M2, M3 (per operation/message)
• block_id: B1 (all belong to same parallel operation block)
• thread_id: T1, T2, T3 (distinguish concurrent operations)

Frontend Behavior:

• Render one block for block_id: "B1"
• Within this block, separate messages by thread_id:
  • Thread T1 (Weather): M1 (chunks C1+C3 merged) → "Weather: Sunny, 25°C"
  • Thread T2 (News): M2 (chunk C2)
  • Thread T3 (Summary): M3 (chunk C4)
• Or interleave by chunk_id order (C1, C2, C3, C4) to show real-time arrival

---

Summary

Field	Level	Purpose	Example
ChunkID	System	Transport/debug	C1, C2, C3
MessageID	LLM/Handler	Delta merging	M1, M2, M3
BlockID	Agent	UI blocks/sections	B1, B2, B3
ThreadID	Agent	Concurrency	T1, T2, T3

Key Insight: Each field serves a distinct purpose at a specific layer of the architecture. This hierarchical design supports simple single-message scenarios while enabling complex Agent orchestration with concurrent operations. Blocks provide natural UI boundaries for organizing related messages.