fix: Azure OpenAI Tool Calls

[yuhongsun96]

Description

Tool Calls in Azure OpenAI are not given back as streamed objects but rather just a final response.completed object. This now correctly extracts them.

How Has This Been Tested?

Works locally where previously it didnt.

Additional Options

• [Optional] Override Linear Check

---

Summary by cubic

Fixes Azure OpenAI tool calls in streaming by extracting function_call items from response.completed and emitting delta.tool_calls with a tool_calls finish reason. Tracks streamed tool calls to avoid duplicates and correctly signal completion, so tools execute reliably with the Azure Responses API.

^{Written for commit f3a82ad. Summary will update on new commits.}

[greptile-apps]

Greptile Overview

Greptile Summary

Fixes Azure OpenAI tool calls in streaming responses by extracting function_call items from the response.completed event and emitting them as ChatCompletionToolCallChunk objects with a tool_calls finish reason.

What changed:

• Previously, the code only detected tool calls and set finish_reason="tool_calls" but returned tool_use=None
• Now, the code extracts actual tool call data (id, name, arguments) from response.completed.output and returns it as tool_use=tool_calls[0]

Why this matters:
Azure's Responses API behaves differently from standard OpenAI - it includes all tool calls in the final response.completed event rather than streaming them incrementally. Without extracting this data, tool calls were being ignored despite being detected.

Confidence Score: 4/5

• This PR is safe to merge with minimal risk, addressing a specific Azure OpenAI integration bug
• The change is well-scoped to fixing a specific issue with Azure OpenAI tool calls. The logic correctly extracts tool call data from the response.completed event. One minor concern is that only the first tool call is returned when multiple exist, but the comment indicates the streaming handler will merge them. The fix has been tested locally and resolves the reported issue.
• Verify that the streaming handler correctly processes multiple tool calls when only the first is returned from response.completed

Important Files Changed

Filename	Overview
backend/onyx/llm/litellm_singleton/monkey_patches.py	Extracts tool calls from Azure OpenAI response.completed events, fixing issue where tool calls were detected but not returned

Sequence Diagram

sequenceDiagram
    participant Azure as Azure OpenAI API
    participant Parser as chunk_parser
    participant Handler as Streaming Handler
    participant Client as Onyx Client
    
    Azure->>Parser: response.created event
    Parser->>Handler: GenericStreamingChunk (empty)
    
    Azure->>Parser: response.output_item.added (function_call)
    Parser->>Handler: GenericStreamingChunk (with tool_use)
    
    Azure->>Parser: response.function_call_arguments.delta
    Parser->>Handler: GenericStreamingChunk (arguments delta)
    
    Azure->>Parser: response.output_item.done
    Parser->>Handler: GenericStreamingChunk (tool complete)
    
    Note over Azure,Parser: Azure sends ALL tool calls in final event
    
    Azure->>Parser: response.completed (with all function_calls)
    Note over Parser: OLD: Just set finish_reason="tool_calls"
    Note over Parser: NEW: Extract tool calls, return first one
    Parser->>Handler: GenericStreamingChunk (tool_use=tool_calls[0], finish_reason="tool_calls")
    
    Handler->>Client: Merged tool call data

Loading

[greptile-apps]

_{1 file reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Only returning the first tool call when multiple exist. Verify that either: (1) Azure sends each tool call in separate response.output_item.added events before response.completed, or (2) the streaming handler properly merges tool calls even when only the first is returned here.

Prompt To Fix With AI

This is a comment left during a code review.
Path: backend/onyx/llm/litellm_singleton/monkey_patches.py
Line: 515:523

Comment:
Only returning the first tool call when multiple exist. Verify that either: (1) Azure sends each tool call in separate `response.output_item.added` events before `response.completed`, or (2) the streaming handler properly merges tool calls even when only the first is returned here.

How can I resolve this? If you propose a fix, please make it concise.

[cubic-dev-ai]

No issues found across 1 file

[justin-tahara]

If you can validate the Greptile comment and ideally if you can add a simple unit test that would be great.

Approving to unblock