fix(tests): use crawler-friendly search query in Exa integration test

[nmgarza5]

Description

Fixes flaky test_web_search_endpoints_with_exa integration test. The previous search query "latest ai research news" returned URLs from news sites (e.g., chemistryworld.com) that block web crawlers with bot protection, causing consistent test failures.

Changed to "wikipedia python programming" which returns Wikipedia URLs that are reliably crawlable without bot protection.

How Has This Been Tested?

The test was failing consistently on PR #7745 due to chemistryworld.com blocking the crawler. Wikipedia URLs should be reliably fetchable.

Additional Options

• [Optional] Override Linear Check

---

Summary by cubic

Fixes flaky Exa integration test by using a crawler-friendly query that returns Wikipedia pages. Replaced "latest ai research news" with "wikipedia python programming" to avoid bot-protected news sites.

^{Written for commit fa233cb. Summary will update on new commits.}

[greptile-apps]

Greptile Overview

Greptile Summary

Fixes flaky integration test by replacing a search query that returned bot-protected news URLs with one that returns crawler-friendly Wikipedia URLs.

• Changed query from "latest ai research news" to "wikipedia python programming" in test_web_search_endpoints_with_exa
• Addresses test failures caused by chemistryworld.com and similar sites blocking web crawlers
• Wikipedia URLs are reliably crawlable without bot protection, making the test deterministic

Confidence Score: 5/5

• This PR is safe to merge with minimal risk
• The change is a simple one-line modification to a test query string that addresses a specific, well-documented issue. The new query maintains test validity while improving reliability by using crawler-friendly Wikipedia URLs instead of bot-protected news sites.
• No files require special attention

Important Files Changed

Filename	Overview
backend/tests/integration/tests/web_search/test_web_search_api.py	Changed search query from bot-protected news sites to crawler-friendly Wikipedia URLs to fix flaky test

Sequence Diagram

sequenceDiagram
    participant Test as test_web_search_endpoints_with_exa
    participant ExaAPI as Exa Search Provider
    participant SearchLite as /web-search/search-lite
    participant Crawler as Onyx Web Crawler
    participant OpenURLs as /web-search/open-urls
    participant Search as /web-search/search
    participant Wikipedia as Wikipedia URLs

    Test->>ExaAPI: Activate Exa provider
    ExaAPI-->>Test: Provider ID
    
    Test->>SearchLite: POST {"queries": ["wikipedia python programming"], "max_results": 3}
    SearchLite->>ExaAPI: Search query
    ExaAPI->>Wikipedia: Find matching pages
    Wikipedia-->>ExaAPI: Return Wikipedia URLs
    ExaAPI-->>SearchLite: Search results with URLs
    SearchLite-->>Test: Exa search results
    
    Test->>Test: Extract URLs from results
    Test->>OpenURLs: POST {"urls": [url1, url2]}
    OpenURLs->>Crawler: Fetch content from URLs
    Crawler->>Wikipedia: GET url1
    Wikipedia-->>Crawler: Page content (bot-friendly)
    Crawler->>Wikipedia: GET url2
    Wikipedia-->>Crawler: Page content (bot-friendly)
    Crawler-->>OpenURLs: Crawled content
    OpenURLs-->>Test: Results with content
    
    Test->>Search: POST search request (combined)
    Search->>ExaAPI: Search query
    ExaAPI->>Wikipedia: Find matching pages
    Wikipedia-->>ExaAPI: Return Wikipedia URLs
    ExaAPI-->>Search: Search results
    Search->>Crawler: Fetch content from URLs
    Crawler->>Wikipedia: GET URLs
    Wikipedia-->>Crawler: Page content
    Crawler-->>Search: Full content
    Search-->>Test: Combined search + content results
    
    Test->>Test: Assert all responses valid

Loading

[cubic-dev-ai]

No issues found across 1 file