feat: guard against a dead Instagram session, show progress, fix missing pings
- check-session.py: preflight in gdl-cron.sh. No unexpired sessionid in the Chrome profile gdl reads -> abort before any request, ping /76 (red check). - stories-streak.py: postflight. Every fetched stories source empty for 3 consecutive days -> exit 77, for a session Instagram stopped honouring while the cookie is still present. The 2026-10-07 session death went unnoticed for a week because every run still exited 0. - gdl-sync.py: [n/N] source headers, a 60s heartbeat with new-file counts, and per-source timing, so a healthy paced run no longer looks hung. - gdl-sync@.service now runs run-%i.sh instead of gdl-cron.sh: HC_URL is set in the wrappers, so scheduled runs had never pinged Healthchecks. - Tests for both guards (offline) and the progress helpers.
This commit is contained in:
@@ -66,6 +66,24 @@ mkdir -p "$GDL_HOME/logs"
|
||||
|
||||
[ -n "$HC_URL" ] && curl -fsS -m 10 --retry 3 -o /dev/null "$HC_URL/start" || true
|
||||
|
||||
# Preflight: refuse to scrape logged out. gdl reads the Chrome profile's cookie DB, and the
|
||||
# session in it can die silently -- from 2026-10-07 every stories run fetched "No results"
|
||||
# for all six profiles and still exited 0, so nothing alerted and a week of stories (which
|
||||
# cannot be backfilled) was lost. An open Chrome tab keeps showing its logged-in UI after
|
||||
# that, so only the cookie DB tells the truth. Exit 76 turns the Healthchecks check red.
|
||||
SESSION_CHECK="$GDL_HOME/check-session.py"
|
||||
if [ -x "$SESSION_CHECK" ]; then
|
||||
if ! SESSION_MSG=$("$SESSION_CHECK" 2>&1); then
|
||||
echo "=== $MODE run $(date -Is) ===" | tee -a "$LOG"
|
||||
echo "ABORT (no requests sent): $SESSION_MSG" | tee -a "$LOG" >&2
|
||||
echo "=== exit 76 at $(date -Is) ===" | tee -a "$LOG"
|
||||
[ -n "$HC_URL" ] && curl -fsS -m 10 --retry 3 --data-binary "@$LOG" \
|
||||
-o /dev/null "$HC_URL/76" || true
|
||||
exit 76
|
||||
fi
|
||||
echo "preflight: $SESSION_MSG" | tee -a "$LOG"
|
||||
fi
|
||||
|
||||
# Staging is wiped every run ON PURPOSE. What we already hold is decided by the
|
||||
# skip-archive (--download-archive), never by which files happen to be sitting
|
||||
# in staging, so starting empty is correct -- and it keeps the publish rsync
|
||||
@@ -111,6 +129,16 @@ if [ "$MODE" = "stories" ] && grep -q "sources : 0 to sync" "$RUNOUT"; then
|
||||
[ "$status" -eq 0 ] && status=75
|
||||
fi
|
||||
|
||||
# Every stories source empty for several days running = a session Instagram stopped
|
||||
# honouring while the cookie is still in the jar (the preflight cannot see that). Stories-only:
|
||||
# other modes legitimately return few items per source. See stories-streak.py.
|
||||
if [ "$MODE" = "stories" ] && [ "$status" -eq 0 ] && [ -x "$GDL_HOME/stories-streak.py" ]; then
|
||||
if ! STREAK_MSG=$("$GDL_HOME/stories-streak.py" "$RUNOUT" "$GDL_HOME/stories-empty-streak" 2>&1); then
|
||||
echo "WARNING: $STREAK_MSG" | tee -a "$LOG" >&2
|
||||
status=77
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "=== exit $status at $(date -Is) ===" | tee -a "$LOG"
|
||||
|
||||
# The log becomes the ping body -- healthchecks.io keeps the last ping's body
|
||||
|
||||
Reference in New Issue
Block a user