docs: add a runbook for verifying a run, and record why today happened

The first unattended run is tomorrow morning and nobody will be watching it, so this writes down how to tell whether it worked -- entirely from the journal, local logs and our own viewer's API, without spending a single Instagram request to check. Includes the per-profile file counts as of today so tomorrow's numbers have something to diff against, and says plainly that a stories run adding zero files is a normal result rather than a failure: profiles often have no active story, and someone checking for the first time would reasonably read 0 as broken. Repeats the one rule that matters more than any tuning: a CDN 429 ends the session. The warning order last time was 429 -> 400 on the highlights endpoint -> suspension, so the runbook gives the disable command rather than leaving it to judgement in the moment. Also records what changed today and why, since none of it is recoverable from the diffs: the run, the --abort measurement, the public/private split, and the automation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UXfdJu7QhSJLr47K7koTDF
2026-08-20 20:06:15 -04:00
parent 652747e0f1
commit db0b97dcec
1 changed files with 84 additions and 0 deletions
@@ -194,6 +194,86 @@ cron runs `/bin/sh`, so `$RANDOM` does not exist — hence `shuf`. And `%` in a
 crontab line means newline unless escaped, so avoid it entirely. cron has no
 equivalent of `Persistent=true`.

+## Verifying a run
+
+The first unattended run is **2026-08-21, around 09:36** (09:00 plus the
+randomised delay). Nothing below costs an Instagram request — every check is
+against the journal, local logs, or our own viewer's API.
+
+```sh
+# 1. did it run, and did it exit 0?
+ssh mattellite
+export XDG_RUNTIME_DIR=/run/user/$(id -u)
+systemctl --user list-timers 'gdl-sync@*'          # LAST/PASSED columns
+journalctl --user -u 'gdl-sync@stories.service' --since yesterday --no-pager
+
+# 2. what did it actually fetch? (sidecars vastly outnumber media -- count media)
+ls -1t ~/gdl/logs | head -3
+grep -E '^(==>|    FAILED|done;)' ~/gdl/logs/stories-*.log | tail -20
+grep -ic 429 ~/gdl/logs/stories-*.log            # MUST be 0 -- see below
+```
+
+**A `429` from `scontent-*.cdninstagram.com` ends the session, it is not a
+pacing knob to tune.** The warning order last time was CDN 429 → `400` on the
+highlights endpoint → suspension. If a run logs one, disable the timers and
+stop for the day:
+
+```sh
+systemctl --user disable --now gdl-sync@stories.timer gdl-sync@full.timer gdl-sync@sweep.timer
+```
+
+Then confirm the archive actually grew, from the workstation:
+
+```sh
+# 3. did the publish land? compare against yesterday's counts
+for u in 0ct0ber19 kimxxlip withaseul cher_ryppo zindoriyam official_artms; do
+  n=$(curl -s "https://instaarchive.ergosteur.com/api/archives/$u/files" | python3 -c 'import json,sys; d=json.load(sys.stdin); print(len(d if isinstance(d,list) else d["files"]))')
+  printf '%-18s %s\n' "$u" "$n"
+done
+```
+
+Counts after the 2026-08-20 run, to diff against:
+
+| profile | files |
+|---|---:|
+| 0ct0ber19 | 3151 |
+| official_artms | 6714 |
+| cher_ryppo | 3031 |
+| kimxxlip | 3019 |
+| zindoriyam | 2253 |
+| withaseul | 1760 |
+
+A stories-only run adds few files and often **none** — profiles frequently have
+no active story. "0 new" is a normal result, not a failure. `fileCount` in
+`/api/archives` is stale by design; use the per-profile `/files` listing.
+
+## What changed on 2026-08-20
+
+One session, three separate pieces of work. Recorded because the reasons are
+not recoverable from the diffs.
+
+**The sync run.** First incremental fetch in four days: 184 new media, 299
+files published, 0 failures, 0 CDN 429s. 20 story items, which are the part
+that could not have been recovered later. Cost about half what it would have,
+because the archive DB was already seeded and the state file was primed by hand
+so no probe passes ran.
+
+**`--abort 50`.** The skip-archive suppresses *downloads*, which spends the
+CDN; it does nothing about the *listing pass*, which spends `instagram.com` and
+scales with how big a profile is rather than how much is new. Measured from
+sidecar write times mid-run: 3 new posts took ~100s each, the other 2272 were
+written in one second. Enumerating `cher_ryppo` fell from 2151 posts to 7.
+
+**The repo split.** `main` is public and now carries none of the fetching
+tooling, no host details, and no real account names — its entire history was
+rewritten, the GitHub repo deleted and recreated to clear force-push residue,
+and 22 container images pruned from ghcr because the server bundle had been
+shipping source comments naming real accounts. This branch holds everything
+that was removed. See the caution at the top.
+
+**Automation.** mattellite got a key on the NAS, closing the last manual step,
+and three systemd timers now run the sync unattended.
+
 ## Outstanding

 State as of 2026-08-20, after the sync run and the repo split. Nothing here is
@@ -201,6 +281,10 @@ broken; these are decisions not yet made and cleanups not yet done.

 ### Fetching

+- **The first unattended run has not happened yet** — 2026-08-21 ~09:36. Until
+  it has, the timers are unproven in the one condition that matters: firing
+  with nobody watching. Check it with "Verifying a run" above; the 20h floor
+  means a manual run beforehand would make the automatic one a no-op.
 - `mattellite`'s `~/.ssh/id_ed25519.pub` is in the NAS's `authorized_keys` for
  `agentapi` (added 2026-08-20, alongside the workstation's existing key), so
  the fetch host publishes straight to the archive and no `sshpass` step is