We read our own website and got one row of boilerplate
We pointed our own extraction at snoutdata.com and got back one row of boilerplate. Not a bug in the reader: the page had not been built yet. Here is why reading a modern web page means rendering it first, and what we changed.
The empty div problem
Fetching a URL gives you the HTML the server sent. For a great many sites in 2026 that HTML is a shell: a script tag, some metadata, and an empty container the browser is expected to fill in. Our own site is one of them. So a reader that fetches and parses gets the metadata and nothing else, and reports it as a successful read of one record, which is worse than failing.
Render before either reader sees it
The fix is to open the page the way a browser would, on your own session, wait for it to build itself, and read what is there. The interesting decision was where to put that.
It does not belong inside a reader. Whether a page is really there is a fact about the fetch, not about how you intend to parse the result, and both readers need it to be true. So it sits next to the other things that are also facts about fetching: honouring robots.txt, and pacing requests to the same host. Rendering waits its turn in that queue because it is a second request to the same server.
The preview crawls, instead of guessing
A preview that shows you the seed page and then states a row count for a crawl it has not done is a preview of nothing. So the preview actually crawls a sample, the same number of pages for whichever reader you chose, and shows you what came back.
Plan the columns once, not per page
The bug that taught us the most: asking each page "what are the records in this text" produced 44 columns and 6 rows. Every answer was reasonable on its own, and no two agreed on what a column was called. Five honest per-page answers are five tables stacked diagonally.
So a crawl plans its columns once, across the sample, and then reads every page into that plan. The lesson generalises past crawling: when you are extracting a set of things, deciding the shape is a different job from filling it in, and doing them together gets you neither.