All three modes now use the pacing the 2026-08-22 runs used by hand after the
scraping warning -- 12-20s between requests, 5-10s between downloads, 500K --
against gdl-sync.py's defaults of 6-10 / 3-6 / 1M. Those values produced 0 400s
and 0 429s across 18 sources. An archive sync has no deadline; being slow is
free and being restricted is not.
Two things that made "scheduled" not mean much:
`stories` now uses --min-interval 8, not 20. At 20h the daily timer silently
did nothing whenever a manual run had happened the previous afternoon, which is
exactly what happened on 2026-08-21. The floor is there to stop an aborted
restart re-enumerating profiles -- minutes to hours -- and a stories fetch is
one request per profile, so 8h permits about twelve requests a day instead of
six. A run that skips every source now exits 75 and says so.
And the wrapper never reported failure at all. It ran the sync inside
`{ ...; exit $status; } | tee`, and the left side of a pipeline is a subshell,
so that exit set the subshell's status while the script returned tee's --
always 0. Every claim about cron mailing on failure was wrong. Fixed with
bash PIPESTATUS, hence the shebang change.
Verified with a stub sync across five cases: skipped-and-clean exits 75 with a
warning, a real failure propagates its own code, a normal run stays quiet, and
`full` skipping everything is not treated as an anomaly. Testing also caught
that the log name was only second-granular, so runs in the same second shared a
file and the skip check saw the previous run's output; the check now reads a
per-run temp file.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UXfdJu7QhSJLr47K7koTDF
ergosteurandClaude Opus 5
9c574e02eb