Skip to content

Provider API: SSE stream is falsely terminated when model output contains the literal "[DONE]" #1158

Open

Description

@hjyescut

Summary

When the model output (content, reasoning_content, or a tool call's function.arguments) contains the literal string "[DONE]", the SSE stream is cut at that exact point yet finish_reason still reports "stop". Clients receive silently truncated text and/or invalid tool-call JSON.

This is distinct from #785 (large-argument silent drop with NO terminator). Here the stream dies because a "[DONE]" appears mid-payload - i.e. the gateway appears to substring-match "[DONE]" against the raw stream bytes instead of matching the full terminator line "data: [DONE]".

Expected Behavior

A streaming response should deliver the complete generated output regardless of what characters it contains. A "[DONE]" sequence inside content or tool-call arguments must be passed through verbatim.

The stream should be terminated only by a full SSE terminator line "data: [DONE]" emitted by the server itself, followed by finish_reason "stop" / "tool_calls".

Actual Behavior

  1. Plain text: a request asking the model to output exactly "data: [DONE]" returns content = "data" - the "[DONE]" and everything after is dropped.

  2. Tool call: asking the model to call write_file with content containing:
    if line == 'data: [DONE]':
    the returned arguments is invalid JSON; the "[DONE]" segment is swallowed (observed as: if line == 'data break).

  3. Boundary behaviour (all inside tool-call arguments):

    • "[DONE]" with surrounding text -> truncated at the "["
    • "[DONE" (no closing bracket) -> complete
    • "DONE]" (no opening bracket) -> complete
    • "[END]" -> complete
    • "AAA[DONE]BBB" -> truncated to "AAA"
  4. In practice this makes the model unusable for any task that involves SSE / streaming code: every attempt to write the marker corrupts the tool arguments, the agent retries, writes it again, and is cut again - producing repeated empty turns (finish_reason=stop, output_tokens=0).

Steps to reproduce the issue

  1. POST to https://api.commandcode.ai/provider/v1/chat/completions with model "deepseek/deepseek-v4.1-flash", stream: true, and messages: [{"role":"user","content":"Output exactly this line and nothing else: data: [DONE]"}]
  2. Observe the SSE stream stops right at the "[DONE]" and content is just "data".
  3. Repeat with stream: false -> the full line is returned correctly.
  4. Repeat against another OpenAI-compatible provider -> the full line is returned correctly.
  5. Repeat with "[END]" instead of "[DONE]" -> the line is returned correctly.

Command Code Version

Provider API via ZCode 3.18.2 (no local cmd CLI - reported against the hosted API)

Operating System

Windows

Terminal/IDE

ZCode 3.18.2 desktop

Shell

bash (Git Bash, win32)

Session file (optional)

No response

Fix prompt (optional)

No response

Additional context

Controls that isolate the fault to the streaming gateway:

  • Same prompt with stream:false -> complete output (OK)
  • Same prompt via another OpenAI-compatible provider -> complete output (OK)
  • Replace the marker with "[END]" -> complete output (OK)
  • Cut lands exactly at the "[" character, which points to a substring match on the raw byte stream rather than a line-based terminator check.

Client used: ZCode 3.18.2, requests are standard OpenAI-compatible chat.completions with stream:true and tool definitions. Reproducible with stdlib Python only (urllib + json).

Suggested fix: treat only a full line equal to "data: [DONE]" as the stream terminator; do not substring-scan the SSE payload for the marker.

Activity

  1. added theissue type on Nov 18, 2026
  2. hjyescut commented on Nov 18, 2026

    @hjyescut
    Author

    Additional data: the bug is localized to the /chat/completions streaming path

    I ran the same tool-call payload (a write_file call whose content contains if line == 'data: [DONE]':) against both endpoints, 10 requests each, model deepseek/deepseek-v4.1-flash, stream: true:

    Endpoint Complete & valid JSON Truncated
    /provider/v1/chat/completions 5 / 10 5 / 10
    /provider/v1/responses 10 / 10 0 / 10

    Plain-text probe (Output exactly this line: data: [DONE]), 6 requests each:

    Endpoint Complete Result
    /chat/completions 1 / 6 4 empty responses, 1 partial ("data:")
    /responses 6 / 6 full "data: [DONE]" every time

    So the Responses path handles the same payload correctly. That strongly suggests the defect is in the Chat Completions SSE writer specifically, not in the model output or in a shared post-processing step.

    FYI on endpoint compatibility (from /provider/v1/models), in case others hit this:

    • deepseek/deepseek-v4.1-flash -> ["/chat/completions", "/responses"]
    • stealth/glyph-cluster:free -> ["/chat/completions", "/responses"]
    • thinkingmachines/inkling-2-small:free -> ["/chat/completions"] only

    And /messages (Anthropic shape) correctly rejects non-Anthropic models with 400 Model "..." is not supported on this endpoint, which is good - that one is behaving as documented.

    Suggested scope of the fix: line-based terminator matching in the chat.completions stream writer.

  3. hjyescut commented on Nov 18, 2026

    @hjyescut
    Author

    Full comparison matrix (same prompts, same model, both endpoints)

    deepseek/deepseek-v4.1-flash, stream: true, prepared prompt/response pairs. PASS = expected marker survives intact.

    Scenario /chat/completions /responses
    S1 plain text: output exactly data: [DONE] 0 / 6 (all empty) 6 / 6 exact
    S2 explain + quote the marker in prose 0 / 6 (all empty) 6 / 6
    S3 tool call, short arg containing marker 4 / 6 6 / 6
    S4 tool call, 23-line SSE parser with 4 marker occurrences 0 / 5 (truncated at 3rd/4th) 5 / 5 intact
    S5 agent round-trip: tool call w/ marker -> feed result -> follow-up turn 0 / 4 4 / 4

    Failure signatures on /chat/completions for S1 and S2 are consistently content = "" with finish_reason = "stop": the entire visible answer is lost because the model's text mentions the marker. In S4 the arguments end mid-string, e.g. {"content": "import json\n\ndef read_sse(resp):\n... with the closing brace missing.

    On /responses every one of the 22 requests completed with response.completed and byte-identical marker text.

    Practical consequence: switching a client from Chat Completions to the Responses endpoint avoids the defect entirely, so this is a viable workaround for affected users in addition to being a strong signal that the fault is in the Chat Completions stream writer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions