feat: support configurable Jina Reader endpoints#6
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces support for configuring a custom Jina Reader endpoint via the JINA_READER_BASE_URL environment variable, making the JINA_KEY optional when a custom endpoint is used. The changes span documentation, environment templates, the WebFetch tool implementation, and associated edge-case tests. The review feedback highlights three key improvements: handling empty or whitespace-only values of JINA_READER_BASE_URL to gracefully fall back to the default hosted reader, removing the required is None restriction in missing_required_env to ensure consistent bypass behavior for JINA_KEY, and adding a test case to verify the fallback behavior for empty/whitespace endpoint configurations.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| def jina_reader_base_url() -> str: | ||
| raw_value = os.getenv(JINA_READER_BASE_URL_ENV) | ||
| base_url = DEFAULT_JINA_READER_BASE_URL if raw_value is None else raw_value.strip().rstrip("/") | ||
| parsed = urlparse(base_url) | ||
| if parsed.scheme.lower() not in {"http", "https"} or not parsed.netloc: | ||
| raise ValueError("JINA_READER_BASE_URL must be an absolute HTTP(S) URL.") | ||
| return base_url |
There was a problem hiding this comment.
If JINA_READER_BASE_URL is defined as an empty string or whitespace in the .env file (e.g., JINA_READER_BASE_URL= or JINA_READER_BASE_URL=""), os.getenv returns "". Currently, jina_reader_base_url() only checks if raw_value is None to fall back to the default hosted reader. Consequently, an empty or whitespace-only value will be parsed as an invalid URL and raise a ValueError, causing WebFetch to fail.
We should treat empty or whitespace-only values as unset and fall back to DEFAULT_JINA_READER_BASE_URL.
| def jina_reader_base_url() -> str: | |
| raw_value = os.getenv(JINA_READER_BASE_URL_ENV) | |
| base_url = DEFAULT_JINA_READER_BASE_URL if raw_value is None else raw_value.strip().rstrip("/") | |
| parsed = urlparse(base_url) | |
| if parsed.scheme.lower() not in {"http", "https"} or not parsed.netloc: | |
| raise ValueError("JINA_READER_BASE_URL must be an absolute HTTP(S) URL.") | |
| return base_url | |
| def jina_reader_base_url() -> str: | |
| raw_value = os.getenv(JINA_READER_BASE_URL_ENV) | |
| if raw_value is None or not raw_value.strip(): | |
| return DEFAULT_JINA_READER_BASE_URL | |
| base_url = raw_value.strip().rstrip("/") | |
| parsed = urlparse(base_url) | |
| if parsed.scheme.lower() not in {"http", "https"} or not parsed.netloc: | |
| raise ValueError("JINA_READER_BASE_URL must be an absolute HTTP(S) URL.") | |
| return base_url |
| def missing_required_env(required: Optional[tuple[str, ...]] = None) -> list[str]: | ||
| required_env = REQUIRED_ENV_VARS if required is None else required | ||
| missing = [key for key in required_env if not os.getenv(key, "").strip()] | ||
| if required is None and os.getenv("JINA_READER_BASE_URL", "").strip(): | ||
| missing = [key for key in missing if key != "JINA_KEY"] | ||
| return missing |
There was a problem hiding this comment.
The bypass logic for JINA_KEY is currently restricted to when required is None. If a caller explicitly passes REQUIRED_ENV_VARS (or any custom list containing "JINA_KEY"), the bypass won't be applied even if JINA_READER_BASE_URL is configured.
Removing the required is None restriction makes the function more robust and consistent.
| def missing_required_env(required: Optional[tuple[str, ...]] = None) -> list[str]: | |
| required_env = REQUIRED_ENV_VARS if required is None else required | |
| missing = [key for key in required_env if not os.getenv(key, "").strip()] | |
| if required is None and os.getenv("JINA_READER_BASE_URL", "").strip(): | |
| missing = [key for key in missing if key != "JINA_KEY"] | |
| return missing | |
| def missing_required_env(required: Optional[tuple[str, ...]] = None) -> list[str]: | |
| required_env = REQUIRED_ENV_VARS if required is None else required | |
| missing = [key for key in required_env if not os.getenv(key, "").strip()] | |
| if os.getenv("JINA_READER_BASE_URL", "").strip(): | |
| missing = [key for key in missing if key != "JINA_KEY"] | |
| return missing |
| os.environ["JINA_READER_BASE_URL"] = "file:///tmp/reader" | ||
| calls_before_invalid = len(calls) | ||
| invalid = WebFetch().jina_readpage("https://example.com/invalid") | ||
| invalid_made_request = len(calls) != calls_before_invalid |
There was a problem hiding this comment.
To ensure that empty or whitespace-only values of JINA_READER_BASE_URL are correctly treated as unset and fall back to the hosted reader, we should add a test case to this function.
For example, we can set os.environ["JINA_READER_BASE_URL"] = " " and verify that it successfully falls back to https://r.jina.ai and includes the Authorization header.
1 participant
Background
WebFetchpreviously used the hosted Jina Reader endpoint exclusively. This PR adds support for configurable Reader-compatible endpoints, including locally deployed Reader services.Changes
JINA_READER_BASE_URLenvironment variable.https://r.jina.aibehavior when no custom endpoint is configured.JINA_KEYis no longer required.Testing
Relevant checks pass for:
JINA_KEYJINA_KEY