docs: 2026-10-14 session-death incident, guards, resolution findings
Document the new exit codes (76/77), the wrapper/unit fix, recovery steps, the verbose output, and the archive-resolution check (images match a logged-in fetch; 1080p VP9 exists for newer videos; iOS plays VP9+Opus webm but not VP9+AAC mp4). Correct the stale "timers are disabled" line, and flag that the repo's weekly profiles timer was never installed on the host.
This commit is contained in:
+93
-5
@@ -23,9 +23,10 @@
|
|||||||
|
|
||||||
## Quick reference — getting new content onto the site
|
## Quick reference — getting new content onto the site
|
||||||
|
|
||||||
Timers are **disabled**, so this is manual today. Everything runs on
|
The three systemd timers (see "Scheduling" below) run these unattended; run
|
||||||
`mattellite`; it fetches, publishes to the NAS itself, and the site picks it up
|
them by hand only to get content sooner. Everything runs on `mattellite`; it
|
||||||
with no deploy or restart.
|
fetches, publishes to the NAS itself, and the site picks it up with no deploy
|
||||||
|
or restart.
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
ssh mattellite
|
ssh mattellite
|
||||||
@@ -67,6 +68,74 @@ Long runs: `setsid nohup ~/gdl/run-profiles.sh >/dev/null 2>&1 &` (or `run-full-
|
|||||||
Nothing reaches the archive until a run finishes, so killing one midway is
|
Nothing reaches the archive until a run finishes, so killing one midway is
|
||||||
safe.
|
safe.
|
||||||
|
|
||||||
|
## 2026-10-14 — the Instagram session died unnoticed; guards added
|
||||||
|
|
||||||
|
**What happened.** Between 2026-10-06 09:12 and 2026-10-07 04:36 the dedicated
|
||||||
|
Chrome's Instagram session died. Nothing alerted: the 10-07 full sweep logged
|
||||||
|
`HTTP redirect to login page` on 17 of 18 sources (and "failed"), but every
|
||||||
|
*stories* run after it fetched "No results" for all six profiles and still
|
||||||
|
exited 0 — about a week of stories, which cannot be backfilled, was lost.
|
||||||
|
Two compounding causes:
|
||||||
|
|
||||||
|
- **No scheduled run had ever pinged Healthchecks.** `gdl-sync@.service` ran
|
||||||
|
`gdl-cron.sh %i` directly (written 08-20), but `HC_URL` is exported by the
|
||||||
|
`run-*.sh` wrappers (added 09-14), which the unit bypassed. `gdl-cron.sh`
|
||||||
|
skips the ping silently when `HC_URL` is empty. Fixed: the unit now runs
|
||||||
|
`run-%i.sh`. (`run-full-sweep.sh` has no check URL, so that mode still
|
||||||
|
does not ping.)
|
||||||
|
- **An open Chrome tab keeps showing its logged-in UI after the session is
|
||||||
|
dead** — the SPA only notices on its next failed request. Looking at the
|
||||||
|
window proves nothing; **reload the tab, or read the cookie DB.** The cookie
|
||||||
|
jar had `csrftoken`, `ds_user_id`, `mid`… but no `sessionid`.
|
||||||
|
|
||||||
|
**Guards now in `gdl-cron.sh`** (tests: `scripts/test_session_guards.py`):
|
||||||
|
|
||||||
|
| exit | from | meaning |
|
||||||
|
|---|---|---|
|
||||||
|
| 76 | `check-session.py`, *before any request* | no unexpired `sessionid` in the Chrome profile gdl reads. The run aborts, logs why, pings `/76` (red check). Same DB, same profile as gdl, so it sees exactly what gdl would. Prints a `WARNING` (log only, no alert) when under 7 days of session remain |
|
||||||
|
| 77 | `stories-streak.py`, *after a stories run* | every fetched stories source returned "No results" on 3 consecutive days (`GDL_EMPTY_STREAK_FAIL` to change). Catches a session Instagram stopped honouring while the cookie is still present, which the preflight cannot see. One day per calendar date, so manual re-runs do not inflate it; state in `~/gdl/stories-empty-streak` |
|
||||||
|
| 75 | existing | every stories source skipped by `--min-interval` |
|
||||||
|
|
||||||
|
**To recover from exit 76** the user must log in to Instagram in the Chrome on
|
||||||
|
`mattellite` (VNC, display `:1`; neither password nor 2FA is something to
|
||||||
|
script). Then `~/gdl/check-session.py` should say `session ok ... expires
|
||||||
|
<date>`, and a manual `~/gdl/run-stories.sh` should find stories again.
|
||||||
|
|
||||||
|
**Verbose output.** A healthy run looked hung: pacing is 12–20 s, the config
|
||||||
|
sets `output: null` (no per-file lines), and the live-listing probe captured
|
||||||
|
output until it finished. `gdl-sync.py` now prints `[n/N] user / kind (time)`,
|
||||||
|
a `... fetching: 4m00s elapsed, 12 new file(s) so far (latest: …)` heartbeat
|
||||||
|
every 60 s, and `done in 6m12s, 14 new file(s)`. It only counts files already
|
||||||
|
written; no extra requests, pacing unchanged.
|
||||||
|
|
||||||
|
**Archive resolution check** (read-only scan of header dimensions for the six
|
||||||
|
scraped profiles, plus 8 single-post live metadata fetches, paced):
|
||||||
|
|
||||||
|
- *Logged-out gdl does not download smaller files — it fails* with a login
|
||||||
|
redirect, so no logged-out media ever entered the archive.
|
||||||
|
- Archived image sizes **match** what a logged-in fetch offers (1440×1800,
|
||||||
|
3277×4096, 2976×3720, and the genuinely small ones such as 400×400 or
|
||||||
|
742×928 sitting next to 1440 px siblings in one carousel). No image-upgrade
|
||||||
|
pass is needed.
|
||||||
|
- Videos: ~1,200 of ~1,300 archived videos are 720p. A 2026 reel offers a
|
||||||
|
1080×1920 VP9 stream (video-only DASH, plus separate AAC); a 2024 post tops
|
||||||
|
out at 1276×720, same as the archive. A video upgrade would mostly benefit
|
||||||
|
2025–26 posts (~700), at one request each.
|
||||||
|
- **iOS 18.7 Safari plays VP9 + Opus `.webm` but not VP9 + AAC `.mp4`** (error
|
||||||
|
code 4; `canPlayType` wrongly says "probably"). A video upgrade must
|
||||||
|
therefore output `.webm` — video stream copied, audio re-encoded to Opus
|
||||||
|
96k. **Not built yet.** Open points before building: the viewer mints every
|
||||||
|
video blob as `video/mp4` (`src/lib/archive-cache.ts`,
|
||||||
|
`src/hooks/useArchiveScanner.ts`) and must derive the MIME type from the
|
||||||
|
extension; superseded 720p files should be kept for review in a directory
|
||||||
|
*outside* the Resilio-synced, viewer-scanned `archives/` tree; and the
|
||||||
|
request budget (suggested: ≤40 posts/day, behind the preflight).
|
||||||
|
|
||||||
|
**Open: profiles cadence.** `scripts/systemd/gdl-sync@profiles.timer` says
|
||||||
|
weekly with a randomised day (commit 09-14) but the host is still on the
|
||||||
|
monthly (3rd, 04:00) unit installed 08-22. The repo version was never
|
||||||
|
installed; decide which one is wanted.
|
||||||
|
|
||||||
## 2026-09-14 — a tracked account renamed on Instagram: 0ct0ber19 → heejin
|
## 2026-09-14 — a tracked account renamed on Instagram: 0ct0ber19 → heejin
|
||||||
|
|
||||||
HeeJin's handle changed and the old one is now fully dead — `instagram.com/
|
HeeJin's handle changed and the old one is now fully dead — `instagram.com/
|
||||||
@@ -173,6 +242,13 @@ above, so a fully-skipped stories run alerts, not silently passes.
|
|||||||
Unset `HC_URL` (e.g. `HC_URL= ./run-stories.sh`) to run without pinging.
|
Unset `HC_URL` (e.g. `HC_URL= ./run-stories.sh`) to run without pinging.
|
||||||
`run-full-sweep.sh` and the reels scripts have no check configured yet.
|
`run-full-sweep.sh` and the reels scripts have no check configured yet.
|
||||||
|
|
||||||
|
**The systemd unit must run the wrappers, not `gdl-cron.sh` directly** —
|
||||||
|
`HC_URL` lives in `run-<mode>.sh`, and `gdl-cron.sh` skips the ping silently
|
||||||
|
when it is empty. Until 2026-10-14 the unit bypassed the wrappers, so no
|
||||||
|
scheduled run had ever pinged. The preflight (exit 76) and the all-empty
|
||||||
|
streak (exit 77) are described in "2026-10-14" above; both reach the dashboard
|
||||||
|
as failures through the same trailing-status-code convention.
|
||||||
|
|
||||||
**This is also why `gdl-sync.py`'s `publish()` no longer treats the routine
|
**This is also why `gdl-sync.py`'s `publish()` no longer treats the routine
|
||||||
chown-permission warning as a failure (2026-09-14).** The archive share is
|
chown-permission warning as a failure (2026-09-14).** The archive share is
|
||||||
never owned by the ssh user, so rsync's own `chown` step fails on *every*
|
never owned by the ssh user, so rsync's own `chown` step fails on *every*
|
||||||
@@ -230,6 +306,9 @@ every deletion because of it.
|
|||||||
| `scripts/gdl-cron.sh` | unattended wrapper: `stories` \| `profiles` \| `full-sweep` |
|
| `scripts/gdl-cron.sh` | unattended wrapper: `stories` \| `profiles` \| `full-sweep` |
|
||||||
| `scripts/systemd/` | the timers actually installed on the fetch host |
|
| `scripts/systemd/` | the timers actually installed on the fetch host |
|
||||||
| `scripts/test_gdl_sync.py` | its tests |
|
| `scripts/test_gdl_sync.py` | its tests |
|
||||||
|
| `scripts/check-session.py` | preflight: is there a usable Instagram `sessionid` in the Chrome cookie DB? (exit 1 = no) |
|
||||||
|
| `scripts/stories-streak.py` | postflight: all stories sources empty for several days running? |
|
||||||
|
| `scripts/test_session_guards.py` | tests for the two guards above (offline) |
|
||||||
| `scripts/jd2-sync.ts` | JDownloader `.crawljob` generator, still used elsewhere |
|
| `scripts/jd2-sync.ts` | JDownloader `.crawljob` generator, still used elsewhere |
|
||||||
| `docs/gallery-dl.md` | the measurements behind every option in the fetcher — **read before changing pacing** |
|
| `docs/gallery-dl.md` | the measurements behind every option in the fetcher — **read before changing pacing** |
|
||||||
| `docs/jdownloader.md` | the older JD2 flow |
|
| `docs/jdownloader.md` | the older JD2 flow |
|
||||||
@@ -372,7 +451,11 @@ systemctl --user list-timers 'gdl-sync@*'
|
|||||||
|
|
||||||
Unit files are version-controlled in `scripts/systemd/` and installed to
|
Unit files are version-controlled in `scripts/systemd/` and installed to
|
||||||
`~/.config/systemd/user/`. One templated service, `gdl-sync@.service`, takes
|
`~/.config/systemd/user/`. One templated service, `gdl-sync@.service`, takes
|
||||||
the mode as its instance name and runs `gdl-cron.sh %i`.
|
the mode as its instance name and runs `run-%i.sh` (the fixed-mode wrappers
|
||||||
|
around `gdl-cron.sh` — they carry the Healthchecks URL; see "Healthchecks.io").
|
||||||
|
Before 2026-10-14 it ran `gdl-cron.sh %i` directly, which skipped every ping.
|
||||||
|
The profiles timer in the table is as originally installed; the repo's copy is
|
||||||
|
weekly (see "2026-10-14" → "Open: profiles cadence").
|
||||||
|
|
||||||
Three settings are load-bearing:
|
Three settings are load-bearing:
|
||||||
|
|
||||||
@@ -561,7 +644,12 @@ Two results are **not** failures, despite how they look:
|
|||||||
|
|
||||||
### 2. Check the cookies before concluding you are blocked
|
### 2. Check the cookies before concluding you are blocked
|
||||||
|
|
||||||
Free, and it separates a local fault from a server-side one:
|
First, `~/gdl/check-session.py` — it reads the same cookie DB gdl does and says
|
||||||
|
whether a `sessionid` exists and when it expires (never prints values). Do not
|
||||||
|
trust the open browser tab to tell you: it keeps showing a logged-in page after
|
||||||
|
the session has died; reload it.
|
||||||
|
|
||||||
|
Then, free, and it separates a local fault from a server-side one:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
PATH=$HOME/.local/bin:$PATH gallery-dl \
|
PATH=$HOME/.local/bin:$PATH gallery-dl \
|
||||||
|
|||||||
@@ -146,6 +146,19 @@ Anonymous access is **not** a viable fallback: it serves lower-resolution media,
|
|||||||
caps profile pagination at 12 posts, and returns `AuthRequired` for stories and
|
caps profile pagination at 12 posts, and returns `AuthRequired` for stories and
|
||||||
highlights.
|
highlights.
|
||||||
|
|
||||||
|
(Observed 2026-10-07, after the session died: with no `sessionid` gallery-dl did
|
||||||
|
not fall back to anonymous downloads at all — most sources failed with
|
||||||
|
`HTTP redirect to login page` / `HTTP redirect to home page`, and stories came
|
||||||
|
back as "No results". The lower-resolution claim above was **not** re-measured
|
||||||
|
then; what *was* measured is that archived image sizes match a logged-in fetch.)
|
||||||
|
|
||||||
|
**The session can die silently, and nothing about the browser window shows it** —
|
||||||
|
an open Instagram tab keeps rendering its logged-in UI until its next failed
|
||||||
|
request. The only reliable signal is the cookie DB: `scripts/check-session.py`
|
||||||
|
looks for an unexpired `sessionid` in this same profile, and `gdl-cron.sh` runs
|
||||||
|
it before every scheduled run (exit 76 = refuse to scrape). See `TOOLING.md`,
|
||||||
|
"2026-10-14".
|
||||||
|
|
||||||
## Output format
|
## Output format
|
||||||
|
|
||||||
The viewer's parser is the contract, not JD2's exact bytes. `EXPORT_RE` in
|
The viewer's parser is the contract, not JD2's exact bytes. `EXPORT_RE` in
|
||||||
|
|||||||
Reference in New Issue
Block a user