Case file № 003 · Data pipeline · Unattended operation
Every show in Denver, gathered by breakfast
- System
- 303.show
- Client
- Public resource, self-directed
- Live
- 303.show ↗
Denver’s concert listings were scattered across venue sites, ticket sellers, and community lists, none of them complete and most of them behind a login, an app, or a wall of sponsored posts. 303.show is an engine on a box in my house that collects fourteen of those sources every morning, merges them into one catalog of 4,600-plus shows, publishes the site unattended, and asks me one question a day.
LiveThe evidence
Screens of the running thing
The system
How the parts connectBefore & after
The account, both columnsBefore
- No single source had the whole city. Venue calendars knew their own room, ticket sellers knew what they sold, and community listings knew the rest, each in a different format and none of them complete.
- Every one of them wanted a login, an app install, or a scroll past sponsored posts first. Two of the big ticket sellers block automated readers outright, and the richest source’s easy export drops the ticket link, the single most valuable field, on the way out.
- “What’s on tonight?” took five tabs and a guess.
After
- A Linux box runs a chain of timed jobs each morning: sweep fourteen sources, each read in the form that keeps its ticket links, merge them, and hand the result to a site build. The site deploys the moment the sweep finishes rather than on a clock, because one of the sources makes the sweep’s duration unguessable.
- Sources are ranked. A later source may add a show or confirm one, never rewrite one, so a show’s identity stays stable and the shows people have saved don’t vanish under them. Any source can be switched off with one line.
- The system decides what it can and queues what it can’t. New venues, duplicate shows, and address disagreements land in a daily digest and are resolved with a single command; accepting a conflict once becomes a standing rule. Triage found that half the conflict queue was the matcher’s own artifact, and fixing that took the queue from 81 rows to 21.
- A fail-safe contract guards the publish: if the schema shifts, the count drops more than a fifth, or the priced share collapses, the job refuses and the last known-good file keeps serving. An export from the wrong machine is refused too, because only the box can see prices.
- Because every change is logged, “what’s new” is free: it powers the Just Added page, an RSS feed, and 96 per-venue calendar subscriptions, with no database behind the website at all.
The verdict
Collecting is the easy half. The work is deciding: which of two disagreeing sources is right, when two listings are one show, and when a drop in the count means a broken source rather than a quiet week. The system settles what it can and brings me the rest, once a day. Traffic is measured and modest, a few hundred visits a month, and reported as such.
Bill of materials
What was used, and the job it did| 01 | Node + SQLite | the catalog engine: 63 migrations, 14 sources |
|---|---|---|
| 02 | systemd timers | the morning chain, on a box I run |
| 03 | Headed Chrome | for the two sellers that block ordinary readers |
| 04 | SvelteKit | static build: pages, feeds, sitemaps, posters |
| 05 | Cloudflare Pages + Worker | hosting; saved shows and a submission queue |