Measure six ways to cut a scraping agent’s HTML input without losing card fields
Compare six local controls on three permitted HTML listing pages, count serialized input with a pinned tokenizer, and check every extracted card against an independent reference.
The big change
What changed: The comparison starts with raw HTML, then filters the listing, extracts fixed fields, serializes structured JSON, batches pages under a token cap, skips unchanged pages and reuses versioned deterministic results. Every candidate includes its instruction and source metadata.
Why it matters: On the saved October 24 corpus, the comparable cold-run input fell from 27,863 tokens for raw pages to 2,083 for one bounded JSON batch, while all 24 card records matched an independent six-field reference. This is a local cl100k_base input count, not an API bill or model result.
What to watch: The warm run needed zero new extraction input for three unchanged pages. Reading cached records in a later agent conversation still consumes context. Current live pages may contain different rows, and changed markup requires a checked parser revision.







