Repository navigation
feat(google): map the grounding image parameter to the google_search tool - #588
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Google image models never received the unified
groundingimage parameter added in #513: no Google images mapper wrote it, sogrounding=TrueraisedUnsupportedParameterWarning("Parameter 'grounding' is not supported by model 'gemini-3.1-flash-image' and will be ignored.") and was dropped.Google documents Search grounding for image generation:
"tools": [{"type": "google_search"}], and "Grounding with Google Search is not supported by thegemini-3.1-flash-lite-imagemodel."gemini-3-pro-image,gemini-3.1-flash-imageandgemini-nano-banana-2.1, and not supported forgemini-3.1-flash-lite-imageandgemini-2.5-flash-image.POST /v1beta/interactions):GoogleSearch, "A tool that can be used by the model to search Google",type: google_search, optionalsearch_types.googleSearchtool. The GenerateContent reference definesTool.googleSearch("If not set, web search is enabled by default").Fix:
src/celeste/providers/google/interactions/parameters.pyandsrc/celeste/providers/google/generate_content/parameters.py: aGoogleSearchMapperper wire. When the value is true it appends the existingWebSearchMapper().map_tool(WebSearch())output totools:{"type": "google_search"}on Interactions,{"google_search": {}}on GenerateContent, the same shape the text modality sends forTool.WEB_SEARCH.NoneandFalsewrite nothing, because Google's default is no search tool.src/celeste/modalities/images/providers/google/parameters.py:InteractionsGroundingMapperandVertexGroundingMapperbindImageParameter.GROUNDING, appended toGOOGLE_INTERACTIONS_PARAMETER_MAPPERSandGOOGLE_VERTEX_PARAMETER_MAPPERS.src/celeste/modalities/images/providers/google/models.py:ImageParameter.GROUNDING: Bool()on the three documented models, the same constraintflux-3-imageuses. The two unsupported models stay unconstrained, so the value passes through for Google to reject, by design.tests/unit_tests/test_parameter_wire_contracts.py: aTrueand aFalsewire row for each backend list.CHANGELOG.md: one line under Features.Not in this PR:
Trueenables web search only. Image Search (search_types: ["web_search", "image_search"]) exists only on 3.1 Flash Image and Nano Banana 2.1, so mapping it would makegroundingmodel-dependent. It stays reachable throughextra_body.ImageOutput.groundingstaysNone. The search steps (google_search_result.search_suggestions) andgroundingMetadata.searchEntryPointare in the response but not surfaced:metadata["raw_response"]dropssteps/candidatesas content fields. Google requires displaying the search suggestions for grounded results (terms). Parsing them for images, reusingmap_grounding_interactions/map_grounding_vertex, is a parsing change that needs your approval first.Validation:
make cipasses (767 unit tests)._build_request(..., grounding=True)forgemini-3.1-flash-imageemitsUnsupportedParameterWarningand the body has notools.POST /v1beta/interactions,gemini-3.1-flash-image,quality="512"), throughclient.generatewith the raw response captured at_make_request. Prompt: "Visualize the current weather forecast for the next 5 days in San Francisco as a clean, modern weather chart with the dates and temperatures."grounding=True: requesttools: [{"type": "google_search"}], no warning. The response hasgoogle_search_call(queries: ["current 5 day weather forecast in San Francisco", ...],search_type: web_search) andgoogle_search_result(withsearch_suggestions) steps, andusage.grounding_tool_count: [{"type": "google_search", "count": 2}]. The image is dated "Monday, November 9, 2026" (today) through Friday, Nov 13, with Veterans Day on Nov 11.tools, stepsmodel_output/thoughtonly, nogrounding_tool_count. The image shows stale dates, "Oct 26 – Oct 30".GoogleVertexImagesClientwith an API key routes to Gemini APIPOST /v1beta/models/gemini-3.1-flash-image:generateContent, the same wire Vertex serves.grounding=True: requesttools: [{"google_search": {}}]. The candidate carriesgroundingMetadatawithwebSearchQueries("current weather forecast San Francisco 5 days", ...),groundingChunks(weather.gov, accuweather.com) andsearchEntryPoint. The image is dated Nov 10 – Nov 14, 2026.groundingMetadata.aiplatform.endpoints.predict(403 onpublishers/google/models/gemini-3.1-flash-image).