OpenAI compatibility layer: streaming returns finish_reason "stop" for tool calls; non-streaming returns "tool_calls" for the identical request

[alex-xuweilong]

Note on venue: this is an API-behavior report about the hosted OpenAI-compatibility layer (generativelanguage.googleapis.com/v1beta/openai/), reproduced with plain curl — no SDK involved. Filing here because this tracker actively triages Gemini API behavior issues; happy to move it wherever the team prefers. A forum thread reporting the same symptom (discuss.ai.google.dev/t/112704) has a Google response requesting a full payload and repro — this issue provides exactly that.

Summary

For the identical request, the OpenAI-compatibility layer disagrees with itself about finish_reason when the model responds with a tool call:

Mode	Response contains	finish_reason
Non-streaming	message.tool_calls	"tool_calls" ✅
Streaming (stream: true)	tool_calls delta chunk	"stop" ❌

Per OpenAI Chat Completions semantics, finish_reason: "tool_calls" is the signal that tells a client to execute tools and continue the loop. Agent frameworks built on the OpenAI SDK branch on this value, so in streaming mode the loop terminates instead of dispatching the tool — tool calling silently breaks, but only when streaming.

Reproduced 2026-08-17 on gemini-3.7-flash and gemini-3.1-flash-lite-preview (identical behavior on both, so this looks layer-wide rather than model-specific).

Minimal repro

payload.json:

{
  "model": "gemini-3.7-flash",
  "messages": [{"role": "user", "content": "What is the weather in Paris? Use the tool."}],
  "tools": [{
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Get current weather for a city",
      "parameters": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"]
      }
    }
  }]
}

Non-streaming:

curl -s https://generativelanguage.googleapis.com/v1beta/openai/chat/completions \
  -H "Authorization: Bearer $GEMINI_API_KEY" \
  -H "Content-Type: application/json" \
  -d @payload.json

Response (HTTP 200, id: n3iCar2iBJTBqtsP2syK2Qo, signatures truncated for readability):

{
  "choices": [{
    "finish_reason": "tool_calls",
    "index": 0,
    "message": {
      "role": "assistant",
      "tool_calls": [{
        "extra_content": {"google": {"thought_signature": "Ep0CCpoC…(truncated)"}},
        "function": {"arguments": "{\"city\":\"Paris\"}", "name": "get_weather"},
        "id": "call_482567",
        "type": "function"
      }]
    }
  }],
  "created": 1786935455,
  "model": "gemini-3.7-flash",
  "object": "chat.completion",
  "usage": {"completion_tokens": 16, "prompt_tokens": 55, "total_tokens": 117}
}

Streaming — same payload plus "stream": true, "stream_options": {"include_usage": true}:

curl -s https://generativelanguage.googleapis.com/v1beta/openai/chat/completions \
  -H "Authorization: Bearer $GEMINI_API_KEY" \
  -H "Content-Type: application/json" \
  -d @payload_stream.json

Full SSE output (HTTP 200, id: oXiCaoOBBsPg8QGa35aRCw, signatures truncated):

data: {"choices":[{"delta":{"role":"assistant","tool_calls":[{"extra_content":{"google":{"thought_signature":"EpwCCpkC…(truncated)"}},"function":{"arguments":"{\"city\":\"Paris\"}","name":"get_weather"},"id":"call_1839541","type":"function"}]},"index":0}],"created":1786935457,"id":"oXiCaoOBBsPg8QGa35aRCw","model":"gemini-3.7-flash","object":"chat.completion.chunk","usage":{...}}

data: {"choices":[{"delta":{"role":"assistant"},"finish_reason":"stop","index":0}],"created":1786935457,"id":"oXiCaoOBBsPg8QGa35aRCw","model":"gemini-3.7-flash","object":"chat.completion.chunk","usage":{...}}

data: [DONE]

The tool call arrives complete (id + name + full arguments) in a single delta chunk; the terminal chunk then reports finish_reason: "stop" even though the only content produced was a tool call. Expected: "tool_calls", matching the non-streaming response for the same request.

Impact / ecosystem context

Because the non-streaming path already returns "tool_calls", this looks like a bug in the streaming path of the compat layer rather than an intentional mapping. Downstream, several projects now carry independent normalization patches for it, e.g. BerriAI/litellm#21041, BerriAI/litellm#12249, and agent-side breakage reports like anomalyco/opencode#14972; OpenRouter also rewrites the value at their edge.

We run an OpenAI-compatible gateway (clawapi.org) that routes to Gemini among other upstreams; we have already shipped the same normalization on our side (stream-side stop → tool_calls when tool-call deltas were emitted), so this report is not blocking us — filing it so the fix can land at the source and the ecosystem can eventually drop these patches.

Related observation (separate, minor)

In the same streaming capture, entries inside the tool_calls delta array carry no index field (OpenAI clients use it to assemble parallel tool calls; some SDK codepaths KeyError on it). Previously reported on the forum in Jan 2025: discuss.ai.google.dev/t/59886. Mentioning for completeness since the capture above shows it; can file separately if useful.

[kkorpal]

Hi @alex-xuweilong, thanks for the clear curl repro! I'm taking this directly to the engineering team for them to investigate. Really appreciate you flagging this.