Skip to content

fix(image): Emit error to user#7644

Merged
Danelegend merged 2 commits intomainfrom
img_gen_prop_fail
Jan 21, 2026
Merged

fix(image): Emit error to user#7644
Danelegend merged 2 commits intomainfrom
img_gen_prop_fail

Conversation

@Danelegend
Copy link
Copy Markdown
Contributor

@Danelegend Danelegend commented Jan 21, 2026

Description

When the image gen tool fails, the user has no insight as to why. There are a multitude of reasons as to why this could fail (eg. moderation, rate limiting, bad request, ...).

This PR enables us to propagate the image generation errors up to the user

Example Look:

Screenshot 2026-01-21 at 1 41 46 PM

How Has This Been Tested?

Manual

Additional Options

Future extensions

  • Creating an image-gen error packet for custom display

  • Update other tools to make use of this new exception instead of raising ValueErrors

  • Custom frontend display and nicer handling of these errors

  • [Optional] Override Linear Check


Summary by cubic

Surface image generation errors to the user by emitting explicit error packets and clearer messages. Improves visibility for failures like moderation blocks, invalid input, and fetch/convert errors.

  • Bug Fixes
    • Added ToolExecutionException with emit_error_packet to control user-facing errors.
    • Image tool now maps known failures (moderation, invalid URL/request, fetch/convert) to user-readable messages.
    • Tool runner catches these exceptions, adds tracing, returns a safe ToolResponse, and emits PacketException when flagged.

Written for commit 978109c. Summary will update on new commits.

@Danelegend Danelegend requested a review from a team as a code owner January 21, 2026 21:46
Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 3 files

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="backend/onyx/tools/tool_runner.py">

<violation number="1" location="backend/onyx/tools/tool_runner.py:159">
P2: Avoid logging the raw exception string; it may contain sensitive URLs/tokens. Log only safe metadata like the exception type or status code instead.

(Based on your team's feedback about not logging raw exception strings that may contain URLs with temporary auth tokens.) [FEEDBACK_USED]</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Jan 21, 2026

Greptile Summary

This PR introduces error propagation for image generation failures but contains critical bugs that prevent it from working.

Major Issues:

  • Backend serialization bug: PacketException stores exception: Exception object, but when serialized via model_dump() in chat_backend.py:521, it doesn't match the frontend's expected format ({type: "error", message?: string})
  • Type mismatch: Backend PacketException.exception is Exception type, frontend PacketError.message expects string
  • Frontend not updated: ImageToolRenderer.tsx:40 hardcodes error: false and never extracts error messages from PacketType.ERROR packets
  • No error display: Even if packets arrive, the error message won't be shown to users

What needs to be fixed:

  • Change PacketException.exception: Exception to message: str in streaming_models.py
  • Update tool_runner.py:177-183 to pass str(e) when creating PacketException
  • Update ImageToolRenderer.tsx to detect error packets and extract the message
  • Display the actual error message to users instead of generic "Image generation failed"

Impact: Without these fixes, users will see a stuck "Generating image..." spinner when image generation fails, defeating the purpose of this PR.

Confidence Score: 1/5

  • This PR has critical bugs that will prevent error messages from reaching users
  • Score reflects critical serialization bug where backend sends Exception object but frontend expects string message field. The error flow is broken end-to-end: backend creates error packets incorrectly, and frontend doesn't extract/display error messages at all. Users will see stuck "Generating image..." state instead of helpful error messages.
  • Pay close attention to backend/onyx/tools/tool_runner.py (line 177-183), backend/onyx/server/query_and_chat/streaming_models.py (line 74-78), and web/src/app/chat/message/messageComponents/renderers/ImageToolRenderer.tsx (line 40-41, 153-163)

Important Files Changed

Filename Overview
backend/onyx/tools/tool_runner.py Adds handler for ToolExecutionException that emits PacketException when emit_error_packet=True - critical serialization bug
backend/onyx/server/query_and_chat/streaming_models.py PacketException model stores Exception object instead of string - incompatible with frontend
web/src/app/chat/message/messageComponents/renderers/ImageToolRenderer.tsx Frontend not updated to extract and display error messages from error packets

Sequence Diagram

sequenceDiagram
    participant User
    participant Frontend
    participant API as Chat API
    participant ToolRunner
    participant ImageTool as Image Generation Tool
    participant OpenAI as OpenAI API
    participant Emitter

    User->>Frontend: Request image generation
    Frontend->>API: POST /send-chat-message
    API->>ToolRunner: run_tool_calls()
    ToolRunner->>ImageTool: run(placement, **kwargs)
    ImageTool->>Emitter: emit(ImageGenerationToolStart)
    Emitter->>Frontend: Stream packet (image_generation_start)
    
    ImageTool->>OpenAI: generate_image()
    
    alt Success
        OpenAI-->>ImageTool: Image data
        ImageTool->>Emitter: emit(ImageGenerationFinal)
        Emitter->>Frontend: Stream packet (image_generation_final)
        ImageTool->>Emitter: emit(SectionEnd)
        Emitter->>Frontend: Stream packet (section_end)
    else Error (Content Policy, Rate Limit, etc)
        OpenAI-->>ImageTool: Exception
        ImageTool->>ImageTool: raise ToolExecutionException(message, emit_error_packet=True)
        Note over ImageTool: ❌ BUG: Exception bubbles up
        ImageTool-->>ToolRunner: ToolExecutionException
        ToolRunner->>ToolRunner: catch ToolExecutionException
        ToolRunner->>Emitter: emit(PacketException(exception=e))
        Note over Emitter: ❌ BUG: Exception object, not string
        Emitter->>Frontend: Stream packet with Exception object
        Note over Frontend: ❌ BUG: Expects {type: "error", message: string}
        Frontend->>Frontend: ImageToolRenderer error=false (hardcoded)
        Frontend->>User: Shows "Generating image..." (stuck)
    end
Loading

Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional Comments (3)

  1. backend/onyx/server/query_and_chat/streaming_models.py, line 74-78 (link)

    logic: The exception field should be a string to properly serialize for the frontend. The frontend expects PacketError with message?: string, but this stores an Exception object.

  2. web/src/app/chat/message/messageComponents/renderers/ImageToolRenderer.tsx, line 40-41 (link)

    logic: The error state is hardcoded to false and never extracted from packets. Need to check for PacketType.ERROR in the packets and extract the error message.

  3. web/src/app/chat/message/messageComponents/renderers/ImageToolRenderer.tsx, line 153-163 (link)

    style: The error display should show the actual error message from the backend, not just "Image generation failed".

    Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

3 files reviewed, 4 comments

Edit Code Review Agent Settings | Greptile

Comment on lines +177 to +183
if e.emit_error_packet:
tool.emitter.emit(
Packet(
placement=tool_call.placement,
obj=PacketException(exception=e),
)
)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: Critical bug: PacketException stores the exception object directly, but frontend expects a message string field. When model_dump() serializes this for SSE streaming (chat_backend.py:521), the Exception object won't serialize to the format the frontend expects.

The frontend interface is:

export interface PacketError extends BaseObj {
  type: "error";
  message?: string;
}

But the backend PacketException has:

exception: Exception  # Not a string!
Suggested change
if e.emit_error_packet:
tool.emitter.emit(
Packet(
placement=tool_call.placement,
obj=PacketException(exception=e),
)
)
if e.emit_error_packet:
tool.emitter.emit(
Packet(
placement=tool_call.placement,
obj=PacketException(exception=str(e)),
)
)

Note: Also need to update PacketException model to accept str instead of Exception type, or add a @field_serializer to convert exception to string.

Prompt To Fix With AI
This is a comment left during a code review.
Path: backend/onyx/tools/tool_runner.py
Line: 177:183

Comment:
**logic:** Critical bug: `PacketException` stores the exception object directly, but frontend expects a `message` string field. When `model_dump()` serializes this for SSE streaming (chat_backend.py:521), the Exception object won't serialize to the format the frontend expects.

The frontend interface is:
```typescript
export interface PacketError extends BaseObj {
  type: "error";
  message?: string;
}
```

But the backend `PacketException` has:
```python
exception: Exception  # Not a string!
```

```suggestion
            if e.emit_error_packet:
                tool.emitter.emit(
                    Packet(
                        placement=tool_call.placement,
                        obj=PacketException(exception=str(e)),
                    )
                )
```

Note: Also need to update `PacketException` model to accept `str` instead of `Exception` type, or add a `@field_serializer` to convert exception to string.

How can I resolve this? If you propose a fix, please make it concise.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems incorrect. PacketException accepts an exception which is of type Exception

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment is incorrect

@Danelegend
Copy link
Copy Markdown
Contributor Author

Danelegend commented Jan 21, 2026

Greptile Summary

This PR introduces error propagation for image generation failures but contains critical bugs that prevent it from working.

Major Issues:

  • Backend serialization bug: PacketException stores exception: Exception object, but when serialized via model_dump() in chat_backend.py:521, it doesn't match the frontend's expected format ({type: "error", message?: string})
  • Type mismatch: Backend PacketException.exception is Exception type, frontend PacketError.message expects string
  • Frontend not updated: ImageToolRenderer.tsx:40 hardcodes error: false and never extracts error messages from PacketType.ERROR packets
  • No error display: Even if packets arrive, the error message won't be shown to users

What needs to be fixed:

  • Change PacketException.exception: Exception to message: str in streaming_models.py
  • Update tool_runner.py:177-183 to pass str(e) when creating PacketException
  • Update ImageToolRenderer.tsx to detect error packets and extract the message
  • Display the actual error message to users instead of generic "Image generation failed"

Impact: Without these fixes, users will see a stuck "Generating image..." spinner when image generation fails, defeating the purpose of this PR.

Confidence Score: 1/5

  • This PR has critical bugs that will prevent error messages from reaching users
  • Score reflects critical serialization bug where backend sends Exception object but frontend expects string message field. The error flow is broken end-to-end: backend creates error packets incorrectly, and frontend doesn't extract/display error messages at all. Users will see stuck "Generating image..." state instead of helpful error messages.
  • Pay close attention to backend/onyx/tools/tool_runner.py (line 177-183), backend/onyx/server/query_and_chat/streaming_models.py (line 74-78), and web/src/app/chat/message/messageComponents/renderers/ImageToolRenderer.tsx (line 40-41, 153-163)

Important Files Changed

Filename Overview
backend/onyx/tools/tool_runner.py Adds handler for ToolExecutionException that emits PacketException when emit_error_packet=True - critical serialization bug
backend/onyx/server/query_and_chat/streaming_models.py PacketException model stores Exception object instead of string - incompatible with frontend
web/src/app/chat/message/messageComponents/renderers/ImageToolRenderer.tsx Frontend not updated to extract and display error messages from error packets

Sequence Diagram

sequenceDiagram
    participant User
    participant Frontend
    participant API as Chat API
    participant ToolRunner
    participant ImageTool as Image Generation Tool
    participant OpenAI as OpenAI API
    participant Emitter

    User->>Frontend: Request image generation
    Frontend->>API: POST /send-chat-message
    API->>ToolRunner: run_tool_calls()
    ToolRunner->>ImageTool: run(placement, **kwargs)
    ImageTool->>Emitter: emit(ImageGenerationToolStart)
    Emitter->>Frontend: Stream packet (image_generation_start)
    
    ImageTool->>OpenAI: generate_image()
    
    alt Success
        OpenAI-->>ImageTool: Image data
        ImageTool->>Emitter: emit(ImageGenerationFinal)
        Emitter->>Frontend: Stream packet (image_generation_final)
        ImageTool->>Emitter: emit(SectionEnd)
        Emitter->>Frontend: Stream packet (section_end)
    else Error (Content Policy, Rate Limit, etc)
        OpenAI-->>ImageTool: Exception
        ImageTool->>ImageTool: raise ToolExecutionException(message, emit_error_packet=True)
        Note over ImageTool: ❌ BUG: Exception bubbles up
        ImageTool-->>ToolRunner: ToolExecutionException
        ToolRunner->>ToolRunner: catch ToolExecutionException
        ToolRunner->>Emitter: emit(PacketException(exception=e))
        Note over Emitter: ❌ BUG: Exception object, not string
        Emitter->>Frontend: Stream packet with Exception object
        Note over Frontend: ❌ BUG: Expects {type: "error", message: string}
        Frontend->>Frontend: ImageToolRenderer error=false (hardcoded)
        Frontend->>User: Shows "Generating image..." (stuck)
    end
Loading

This seems incorrect. Packet Exception accepts an Exception in it, not Str. The exception is then raised in run_chat_loop_with_state_containers and then caught in handle_stream_message_objects, where a StreamingError is yielded to the client.

@Danelegend Danelegend added this pull request to the merge queue Jan 21, 2026
Merged via the queue into main with commit e5dcf31 Jan 21, 2026
76 of 77 checks passed
@Danelegend Danelegend deleted the img_gen_prop_fail branch January 21, 2026 22:54
Danelegend added a commit that referenced this pull request Jan 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants