The Gemini (Vertex) API now gives limited access to grounding queries and returns the search phrases in the [‘groundingMetadata’] object. That’s exciting news, in theory, but it turns out to be messy in practice. We encountered two major issues:
1. Gemini is greedy about grounding
Even running Gemini 3.5 Flash with a [MEDIUM] or higher “thinking” level, it’s rare to get more than 1-2 grounding queries. In many cases, you’ll see none at all. If you’ve used AI Mode much, you know it almost always runs multiple grounding queries. We were able to nudge the system instructions and push Google to generate 4-6 grounding queries. While these queries are “real” (Google actually ran the searches), it’s fair to say that we cheated a little bit. AI Mode and actual query fan-out might behave very differently.
2. Google is greedy for your money
On top of token costs, grounding queries currently cost (as of August 2026) $14 per 1,000 queries. This is not a per-prompt cost. In our case, each prompt ran about five grounding queries, so grounding data cost something like $7 per 100 prompts. This is orders of magnitude more than the token cost, and at enterprise-scale, it adds up fast. If you’re using 3rd-party tools that run on your own API keys, keep a close eye on costs.
