Skip to content

chore(llm): Backend Fallback Logic Tests#8363

Merged
justin-tahara merged 2 commits intomainfrom
jtahara/backend-fallback-tests
Feb 12, 2026
Merged

chore(llm): Backend Fallback Logic Tests#8363
justin-tahara merged 2 commits intomainfrom
jtahara/backend-fallback-tests

Conversation

@justin-tahara
Copy link
Copy Markdown
Contributor

@justin-tahara justin-tahara commented Feb 12, 2026

Description

Introducing new tests to make sure we are covering fallback extraction paths and prevent regressions in tool-call reconstruction.

How Has This Been Tested?

Ran the tests locally

Additional Options

  • [Required] I have considered whether this PR needs to be cherry-picked to the latest beta branch.
  • [Optional] Override Linear Check

Summary by cubic

Add unit and integration tests for fallback tool-call extraction when tool choice is REQUIRED, including deep_research mode and cases where tool-call JSON is embedded in assistant text. Update chat test helpers to pass deep_research in send_message and disconnect, and cover extraction from answer/reasoning, no-op when already attempted or tool_calls exist, and not-extractable cases.

Written for commit b1eae31. Summary will update on new commits.

@justin-tahara justin-tahara requested a review from a team as a code owner February 12, 2026 00:07
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Feb 12, 2026

Greptile Overview

Greptile Summary

This PR adds comprehensive test coverage for the backend fallback logic that extracts tool calls from LLM response text when the model doesn't natively support structured tool calling. The changes introduce both unit tests and integration tests to validate the fallback extraction mechanism.

Key Changes:

  • Added 5 new unit tests in test_llm_loop.py covering edge cases for _try_fallback_tool_extraction function (already attempted, answer extraction, reasoning fallback, no extraction possible, existing tool calls)
  • Added 1 integration test validating the end-to-end fallback extraction flow in deep research mode
  • Extended test utilities with deep_research parameter to support the new test scenario

The tests are well-structured, have clear assertions, and provide good coverage of the fallback extraction logic to prevent regressions.

Confidence Score: 5/5

  • This PR is safe to merge with no risk
  • This is a test-only PR that adds comprehensive coverage for existing fallback logic. No production code changes. All tests follow established patterns, have proper assertions, and cover edge cases thoroughly. No custom instruction violations detected.
  • No files require special attention

Important Files Changed

Filename Overview
backend/tests/integration/common_utils/managers/chat.py Added deep_research parameter to test utilities for supporting reasoning fallback tests
backend/tests/integration/tests/llm_workflows/test_mock_llm_tool_calls.py Added integration test validating fallback tool extraction from reasoning text in deep research mode
backend/tests/unit/onyx/chat/test_llm_loop.py Added comprehensive unit tests for _try_fallback_tool_extraction covering all edge cases

@justin-tahara justin-tahara force-pushed the jtahara/backend-fallback-tests branch from ae37a30 to 1826b66 Compare February 12, 2026 00:09
Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 3 files

@github-actions
Copy link
Copy Markdown
Contributor

Preview Deployment

Status Preview Commit Updated
https://onyx-preview-4jwe806hf-danswer.vercel.app 1826b66 2026-02-12 00:12:55 UTC

@justin-tahara justin-tahara added this pull request to the merge queue Feb 12, 2026
Merged via the queue into main with commit 204328d Feb 12, 2026
82 checks passed
@justin-tahara justin-tahara deleted the jtahara/backend-fallback-tests branch February 12, 2026 01:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants