feat: guard against a dead Instagram session, show progress, fix missing pings

- check-session.py: preflight in gdl-cron.sh. No unexpired sessionid in the
  Chrome profile gdl reads -> abort before any request, ping /76 (red check).
- stories-streak.py: postflight. Every fetched stories source empty for 3
  consecutive days -> exit 77, for a session Instagram stopped honouring while
  the cookie is still present. The 2026-10-07 session death went unnoticed for
  a week because every run still exited 0.
- gdl-sync.py: [n/N] source headers, a 60s heartbeat with new-file counts, and
  per-source timing, so a healthy paced run no longer looks hung.
- gdl-sync@.service now runs run-%i.sh instead of gdl-cron.sh: HC_URL is set in
  the wrappers, so scheduled runs had never pinged Healthchecks.
- Tests for both guards (offline) and the progress helpers.
This commit is contained in:
2026-10-14 21:02:58 -04:00
parent fcfb8e5fd3
commit 4a2bd7aa19
7 changed files with 433 additions and 6 deletions
+28
View File
@@ -66,6 +66,24 @@ mkdir -p "$GDL_HOME/logs"
[ -n "$HC_URL" ] && curl -fsS -m 10 --retry 3 -o /dev/null "$HC_URL/start" || true
# Preflight: refuse to scrape logged out. gdl reads the Chrome profile's cookie DB, and the
# session in it can die silently -- from 2026-10-07 every stories run fetched "No results"
# for all six profiles and still exited 0, so nothing alerted and a week of stories (which
# cannot be backfilled) was lost. An open Chrome tab keeps showing its logged-in UI after
# that, so only the cookie DB tells the truth. Exit 76 turns the Healthchecks check red.
SESSION_CHECK="$GDL_HOME/check-session.py"
if [ -x "$SESSION_CHECK" ]; then
if ! SESSION_MSG=$("$SESSION_CHECK" 2>&1); then
echo "=== $MODE run $(date -Is) ===" | tee -a "$LOG"
echo "ABORT (no requests sent): $SESSION_MSG" | tee -a "$LOG" >&2
echo "=== exit 76 at $(date -Is) ===" | tee -a "$LOG"
[ -n "$HC_URL" ] && curl -fsS -m 10 --retry 3 --data-binary "@$LOG" \
-o /dev/null "$HC_URL/76" || true
exit 76
fi
echo "preflight: $SESSION_MSG" | tee -a "$LOG"
fi
# Staging is wiped every run ON PURPOSE. What we already hold is decided by the
# skip-archive (--download-archive), never by which files happen to be sitting
# in staging, so starting empty is correct -- and it keeps the publish rsync
@@ -111,6 +129,16 @@ if [ "$MODE" = "stories" ] && grep -q "sources : 0 to sync" "$RUNOUT"; then
[ "$status" -eq 0 ] && status=75
fi
# Every stories source empty for several days running = a session Instagram stopped
# honouring while the cookie is still in the jar (the preflight cannot see that). Stories-only:
# other modes legitimately return few items per source. See stories-streak.py.
if [ "$MODE" = "stories" ] && [ "$status" -eq 0 ] && [ -x "$GDL_HOME/stories-streak.py" ]; then
if ! STREAK_MSG=$("$GDL_HOME/stories-streak.py" "$RUNOUT" "$GDL_HOME/stories-empty-streak" 2>&1); then
echo "WARNING: $STREAK_MSG" | tee -a "$LOG" >&2
status=77
fi
fi
echo "=== exit $status at $(date -Is) ===" | tee -a "$LOG"
# The log becomes the ping body -- healthchecks.io keeps the last ping's body