Browser

Craft: reading a website into a spreadsheet

The person points at a page. You write the job. A real Chrome walks the site and Jev answers, per page, in one call. Every value in the spreadsheet is a piece of text that was really on the page, because Jev only ever picks from what the reader found. It cannot write a value, so it cannot invent one.

The shape of a site

Almost every site worth reading has two page kinds, and the job assumes them:

a LIST page  →  links to each thing, and a link to the next list page
a THING page →  the details of one thing

A search page is a list page, so "search this site for X" is the same job: put the words in "search" and the harness types them into the site's own search box, then reads the results.

start does not have to be the list. Give it the site and Jev walks there: each step is one call asking whether this page is already the one, and if not which link goes towards it, up to five pages. Where a person gave you the exact page, use it and skip the walking.

Point start at the list where you have it. If you point it at one thing's page, that one row is collected and the run ends. If the things have no page of their own (everything is on the list), say so to the person: this harness collects one row per page, so a list-only site gives one row for the whole list.

Letting Jev write the fields

Put the person's sentence in want, leave fields out, and two Jev calls decide it: one asks what the page lists and which links are the things, the other asks of every piece of text on one of them "is that a fact about this one, worth a column?" and "what kind of value is it?". About a second, and the names come off the page's own labels.

Do that first. Read what it proposed, then add or reword. What it cannot propose is a judgement: "is this remote?", "how urgent is this". Those are yours to write, and they are where the tool earns its keep, because no page prints them.

Writing the fields

A field is what to look for, in the words a person would use.

• Say it as it appears. "the pay or salary range" beats "compensation" if the page says Salary.
• One value per field. If the page prints "Acme · Leeds · £45,000" the reader offers that line and each of its parts, so company, place and salary can each take their own part. A line splits on ·, |, a dash or a bullet, a label keeps its value (UPC: a22124811bfa8350), and any number inside a line is offered alone.
• Ask for a number when you want to sort by one. A page that says "In stock (19 available)" offers both the sentence and 19: ask for "how many copies are available" and you get the number; ask for "the availability line" and you get the sentence. If the page never prints the number on its own and Jev returns the sentence, that is right and honest. Derive the number with a script afterwards and tell the person you did.
• A value that is not on the thing's page cannot be collected. Rating stars drawn as pictures, a price loaded after a click, a number inside an image: none of those are text. Check with the reader (see AGENTS.md, "Look before you write the job") before promising a column.
• yesno is judgement, not text. Use it for "is it remote?", not for "what is the salary?". check.mjs warns when a pick field reads like a question.