Mirror a source you cannot query
A real database is queried where it is. Some sources cannot be queried at all — an HTTP API, a
partner's catalog that only answers ?page=&since=. Mirror<Shape> keeps a local copy of their
rows, which you can then query like any other. A subclass writes one thing, how to pull:
// services/PartnerCatalog.ts
export default class PartnerCatalog extends Mirror<BookCard> {
async *pull(since?: Date) {
for (let page = 0; page !== null; ) {
const body = await fetch(`${api}?page=${page}&since=${since?.toISOString() ?? ''}`);
const { items, next } = await body.json();
yield items.map(toCard);
page = next;
}
}
}
It is a Repository<BookCard> that fills itself: the copy reads like any
rows, this.list(), and a handler extending it serves the copy and refresh. Everything around the pull is the same for every mirror and lives in the base: refresh(since?)
pulls from the mark it is given and writes each page in one upsertAll. The mark — the date of the
last pass — is the caller's to keep, and to move only once a pass has returned: a pass that failed
halfway must not skip what the source changed in the meantime.
Each page is checked like every other write, strictly: otherwise a partner renaming label would
write nulls without a word. A refused page stops the pass; what
earlier pages wrote stays, because an upsert is idempotent and the next run resumes from
the mark.
It does not schedule itself. A boot hook, a cron, an operation behind a facade are all legitimate and none of them is this class's business.
Mirror has no directory of its own in the frond vocabulary — a mirrors/ folder is not
scanned. Place it in services/, where nothing
serves it, or extend it from a handler to serve the copy.