Does your AI agent need a vector database?

Not before you have checked how search results reach the model. In the one public head-to-head, grep beat vector retrieval in all ten model and runtime pairs when the results came back inline. Route the same results through a file instead and the 11.3-point average gap collapses to 0.1.

The mechanism is the second hop. On this benchmark the answer usually sits in a verbatim span. Grep hands the span straight back. Send it to a file instead and the agent has to search its own search output before it can read any of it. That second search is where the advantage goes.

What did the grep against vector study measure?

Sahil Sen and four co-authors posted Is Grep All You Need? on 14 May 2026, arXiv 2605.15184. They ran a 116-question subset of LongMemEval-S, the 500-question long-term memory benchmark from ICLR 2025, across five models and four agent runtimes, and crossed that with two retrievers and two ways of handing back the results.

The grep tool loads the per-question turns into memory, runs regex over the raw text and ranks by match count. No embedding model and no index. The vector tool embeds every turn at ingestion into a per-question index, retrieves by approximate nearest neighbour and reranks, with the agent choosing k. GPT-4o graded each answer against the reference as correct or not, under the benchmark's own protocol.

The runtimes were Chronos, the authors' own LangChain agent, plus the Claude Code, Codex and Gemini command-line agents driving those same two tools through bash wrappers.

Why does the lexical advantage vanish when results go to a file?

Because the delivery path changes what the model has to do next. Inline, the search output lands in the conversation and competes with the system prompt for the context budget. File-based, it goes to disk, and the agent has to read or search that file before it can use anything in it. One extra step, and the model can do that step badly.

Here are the averages across the ten model and runtime rows of their Table 1. The rows are theirs. The four means are our arithmetic over them, because the paper publishes the cells and never the summary.

Delivery path	Grep	Vector	Gap	Rows grep wins
Inline into the conversation	83.8%	72.5%	11.3 points	10 of 10
Written to a file	72.5%	72.4%	0.1 points	5 of 10

Grep lost 11.3 points moving to the file. Vector lost a tenth of a point. An index built to return a ranked handful of chunks does not much care whether those chunks arrive by conversation or by filesystem. A regex that returns every match does.

How far does the agent runtime move the same model?

Further than the retriever does. Claude Haiku 4.5 searching with grep over file-based results scored 83.6% inside Chronos and 37.1% inside Claude Code. That is 46.5 points on one model, one retriever, one delivery path and the same 116 questions. The widest grep-against-vector gap anywhere in the table is 23.3 points, Gemini 3.1 Flash-Lite in Chronos, inline, 86.2% against 62.9%.

The delivery path is worth nearly as much. Hold GPT-5.4 and grep fixed inside Codex CLI and the score falls from 93.1% inline to 55.2% through a file, a drop of 37.9 points.

Part of that 46.5 points is a mismatch rather than a quality difference. Chronos was built for this benchmark, with prompting that changes by question category and a preprocessing pass that extracts temporal events from the transcripts. The provider CLIs are general coding agents aimed at a memory task nobody wrote them for. Which is the finding rather than an objection to it.

The retriever finds the text. The runtime decides whether the model reads it.

Is the vector index the expensive part?

No. The cash cost is the number everybody checks and it is close to nothing. Voyage lists voyage-4 at $0.06 per million tokens and the rerank-3 reranker at $0.05, with the first 200 million tokens free on each. A 10-million-token corpus embeds for about 60 cents at list price, or for nothing inside the free allowance. Checked by Unity Horizon on 20 October 2026.

What an index costs is upkeep. Someone opens the ticket that says add semantic search, puts two weeks on it, and the two weeks go on plumbing: a job that re-embeds the rows that changed, somewhere for the vectors to live, a way to find out on a Monday morning that the index is four days behind the source. Sixty cents buys the index. Nobody budgets the person who rebuilds it.

Anthropic says the same thing in its own guidance, published 29 September 2025. Semantic search is "usually faster than agentic search, but less accurate, more difficult to maintain, and less transparent", and the recommendation is to start with agentic search and add semantic search only for faster results or more query variations. Cursor's search documentation now states that it "does not store embeddings of your codebase for search". Its lexical index is built and queried on the user's own machine.

Who gains from the grep answer, and who pays for it?

The teams that gain are the ones whose evidence is literal. Code, chat transcripts, logs, portal form fields: if the answer exists as a string somewhere in the source, a regex finds it and shows you the line it found it on. So does any team with nothing built yet, because grep ships in an afternoon and an ingestion pipeline does not.

The bill lands on teams whose evidence is paraphrased, and the authors say so before anyone else can. They tie their conclusion to long-memory conversational question answering, where answers rest on verbatim spans, and they decline to claim that grep beats vector retrieval in general. Support tickets written in one register, documentation written in another: that is where nearest-neighbour search earns its money. It lands again on the team that tears out a working index because a summary of this paper quoted the five Chronos rows and left out the ones where a provider CLI took a model down to 55.2%.

Our own systems sit on the literal side, and not because we measured this. We built OFFSET, which files declarations directly to the ΑΑΔΕ portal, and OFFSET ingests from Airbnb, Booking.com and PMS systems. Those fields arrive typed: a date, an amount, a tax number, a nightly rate. Nothing in that shape wants a nearest-neighbour search. askHermis answers 85% of guest queries instantly. The questions guests repeat are mostly lookups of a single stated fact, and that is a design judgement we made rather than a retrieval number we have published.

What would you change first?

Hold the retriever you already have, move its output inline, then count how many of your eval questions have the answer sitting in a verbatim span in the source. That is a morning of work and it tells you whether the 11.3 points are available to you at all. It sits next to a number we published earlier, tool selection degrading past 30 to 50 tools, which is the same shape of problem: the surround decides the result and the component takes the blame.

One thing in the paper is unresolved and it matters here. Section 3.3.1 says every Chronos episode opens with a block of the top 15 vector results before the tool-calling loop starts, and says separately that tools are switched on or off per configuration. It never says whether that opening block is switched off in the grep-only condition. Until the authors confirm it, read the grep column as grep plus a possible vector seed. This is 116 questions on one benchmark with one grader model, and we have published no retrieval accuracy of our own to set beside it.

How do your agent's search results reach the model today, inline or through a file, and who chose that? If nobody chose it, that is where we would start.