The pause section predicted official_artms and 0ct0ber19 stories would expire
uncollected. They did not: both surfaces were fetched by hand the next day at
roughly double the configured caution, with 0 400s and 0 429s. 16 story media
and 26 posts/reels media, 85 files into the archive.
That confirms the 400s were the challenge state rather than a block -- once
the interstitial was dismissed, the same endpoints served normally.
It does not retire the warning, and the section says so. Two hand-paced runs
are not evidence the old cadence was safe. It also flags the gap that matters
for whenever the timers go back on: the scheduled "full" mode still runs at
the default 6-10s pacing, so the automation would be less careful than the
manual runs that followed a warning.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UXfdJu7QhSJLr47K7koTDF
Instagram flagged the account on 2026-08-21 -- not a suspension, an
interstitial at /accounts/scraping_warning/ saying it suspects automated
behaviour. Dismissed; account healthy. All three timers are disabled.
The sequence is worth keeping because each step misled differently. The daily
timer fired and exited 0 having fetched nothing: yesterday's manual run was
19h earlier, under the 20h floor, so a silent no-op on the one surface that
cannot be backfilled reported as success. Chrome was separately OOM-killed, so
no browser was running to notice. The recovery run then drew 400s from the
stories endpoint on all six profiles.
Cookies were exported and checked before assuming a block -- sessionid
well-formed and unexpired -- which cost nothing and ruled out the local
explanation. Only then did the browser reveal the interstitial.
Records the four things to fix before re-enabling: the floor defeats the daily
job, a skipped stories run must be loud rather than exit 0, the daily cadence
is itself the most machine-like pattern we run, and chrome-devtools.service
has Restart=no.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UXfdJu7QhSJLr47K7koTDF
The first unattended run is tomorrow morning and nobody will be watching it,
so this writes down how to tell whether it worked -- entirely from the
journal, local logs and our own viewer's API, without spending a single
Instagram request to check.
Includes the per-profile file counts as of today so tomorrow's numbers have
something to diff against, and says plainly that a stories run adding zero
files is a normal result rather than a failure: profiles often have no active
story, and someone checking for the first time would reasonably read 0 as
broken.
Repeats the one rule that matters more than any tuning: a CDN 429 ends the
session. The warning order last time was 429 -> 400 on the highlights endpoint
-> suspension, so the runbook gives the disable command rather than leaving it
to judgement in the moment.
Also records what changed today and why, since none of it is recoverable from
the diffs: the run, the --abort measurement, the public/private split, and the
automation.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UXfdJu7QhSJLr47K7koTDF
mattellite's key is now in the NAS's authorized_keys for agentapi, so
gdl-sync.py publishes straight to the archive. That was the last manual step:
the 2026-08-20 run published locally and rsynced by hand with sshpass, which
no scheduled job could do. Verified end to end -- key auth, then a dry-run
rsync from the fetch host into the archive root.
Three timers, installed and enabled: stories daily, full monthly, sweep
quarterly. A templated gdl-sync@.service takes the mode as its instance name.
Settings that are load-bearing rather than decoration:
RandomizedDelaySec=45m a job firing at exactly 09:00 daily is obviously a
machine; list-timers now shows 09:36, not 09:00
Persistent=true catch up a run the host slept through -- cron just
skips, and a skipped stories run is content gone
TimeoutStartSec=infinity a sweep runs for hours at this pacing; the 90s
default would kill it mid-fetch
Two things caught while installing, both documented: month names are invalid
in OnCalendar's date field (Jan,Apr,Jul,Oct-07 is rejected -- use numerics,
and check with systemd-analyze calendar), and `systemctl --user` over
non-interactive ssh needs XDG_RUNTIME_DIR or it cannot find the bus.
Smoke-tested by starting the stories service: it spent zero Instagram requests
because the 20h floor skipped all six sources, which is exactly what a job
firing twice should do.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UXfdJu7QhSJLr47K7koTDF
gdl-sync.py always needed a fistful of paths on the command line, and the
2026-08-20 run published to a local directory and then rsynced to the NAS by
hand -- fine interactively, impossible from cron. gdl-cron.sh takes one
argument (stories|full|sweep) and does the whole thing.
Staging is wiped every run deliberately: what we already hold is decided by
the skip-archive, never by what happens to be sitting in staging, so starting
empty is correct and keeps the publish to just the new files.
Documents the three cadences and what each is for -- notably that only `sweep`
notices carousels edited after we archived them, and that only `stories` loses
content if it is skipped, since stories cannot be backfilled.
Cron notes the two things that would silently break it: cron runs /bin/sh so
$RANDOM does not exist (use shuf), and `%` in a crontab line means newline.
systemd is offered as the better option because RandomizedDelaySec and
Persistent=true are built in -- a host asleep at 09:00 silently skips a cron
job, and a skipped stories run is content gone for good.
The publish step is still blocked on mattellite having no key on the NAS; that
is recorded under Outstanding as the one thing between this and unattended.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UXfdJu7QhSJLr47K7koTDF
Nothing here is broken — these are decisions not made and cleanups not done,
written down before the session's context is lost.
The two that can actually cost something: no daily stories run is scheduled,
and stories are the one surface that cannot be backfilled; and the `seeded`
flags in the state file were reconstructed by hand from a log rather than
derived from the archive DB, so losing artms.db without also clearing the
state file would leave those sources permanently unseeded and re-download
everything.
Also corrects "Scanner work (not done yet)", which shipped in 53b1f80 —
the three .json shapes are told apart structurally in gallery-dl-sidecar.ts
and isReel comes from the sidecar's type. That file lives on main: it parses
archives at display time and is viewer code, not tooling.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UXfdJu7QhSJLr47K7koTDF
Putting it at the top of README.md worked, but it diverged a shared file: a
probe showed main editing the adjacent line conflicts on every merge. The
warning survived the conflict, so nothing was ever silently lost, but a file
that only exists on this branch has no such cost at all.
TOOLING.md carries the warning plus what CLAUDE.md would have said if it could
— the commands, the remote policy, and the two guards, which are local and
unversioned and so are absent from every fresh clone.
Shared files are byte-identical to main again: CLAUDE.md, README.md and
package.json. The divergence is now only files main has never had.
Trade-off worth knowing: gitea renders README.md on the branch page and does
not render TOOLING.md, so this warning is one click less visible than it was.
The pre-push hook, not the documentation, remains the guard that actually
stops a mistake.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UXfdJu7QhSJLr47K7koTDF
ergosteurandClaude Opus 5