Turn documents into a pgvector table, then search it the way your app will
A Data Flow can now turn your documents into vectors in any Postgres with pgvector, embedded by the model you choose, and then let you search what it built: type the question your app will ask and see which passages come back, how strongly, and where each one came from.
Most retrieval work happens in two places that never meet. Somewhere a pipeline reads PDFs, splits them into chunks and writes vectors. Somewhere else an app asks a question and gets back whatever is nearest. When the answer is wrong, there is nothing in between to look at: was it the chunking, the model, the filter, or the question?
Data Flows already did the first half: read the files, split each one on paragraph and sentence boundaries, keep the file and page with every chunk, and update only what changed on the next run. What was missing was a choice of where the vectors go, a choice of the model that makes them, and any way to read them back.
pgvector, beside Pinecone
A flow's destination can now be a vector table in Postgres. The flow enables the vector extension if it is missing, creates the table, and adds a cosine index. Each chunk is one row: its text, its metadata (file, page, section, and the columns you chose) and its vector. A re-run updates the chunks whose text changed, skips the rest without paying to embed them again, and can delete the chunks whose source is gone.
Any Postgres with pgvector works, including a SnoutData Cloud project, which ships with it. Your app then queries the same table it already has a connection to.
Your choice of embedding model
Each flow picks what turns its text into vectors: SnoutData AI, your own Ollama server, or your own OpenAI or OpenRouter key. The default is mxbai-embed-large. The keys are the ones you already saved for data generation, so nothing new is typed and nothing new is stored.
The flow records which model built its index. A search of that index embeds its question with the same model, because a question embedded by a different one lands somewhere else and still gets scores, they just do not mean anything. Change the model and the next run re-embeds everything and says so.
Search what it built
Every flow that writes vectors gets a search. Type the question and the passages come back best first, each with its similarity as a percentage, how far ahead it is of the next one, its text, the file and page it came from, and its metadata. You can narrow by metadata, and mark the text an answer should contain to see where the first real answer lands.
Hello Joel → #1 at 56.0%, next at 36.1%
The passage about the company's founder came first, twenty points ahead of the next. A gap that size means the ranking is sure. Three passages within a point of each other mean it is guessing, and that is worth knowing before a user sees it.
Measured: five passages from three PDFs, Postgres 17 + pgvector, mxbai-embed-large on a local Ollama server, 2026-09-18.
Then measure it
One good-looking query proves little. Save the questions your users actually ask, each with the text or the file its answer should come from, and run the set. You get the hit rate (how often an answer is in the top results), MRR (how high the first answer ranks), recall, and p50 and p95 latency, question by question.
Run the same set against two flows side by side: 500-character chunks against 1,000, or one model against another. Then the choice is made on numbers from your own documents, not on a leaderboard built from someone else's.
What it is not, yet
This is plain vector search on cosine similarity. It does not combine it with keyword search, and it does not rerank. Both are on the list, and the benchmark is what will tell us whether they earn their cost on real documents.