tooling: scheduled profiles/full-sweep list posts+reels by browser (chrome-sync per profile/tab, early stop for weekly); GDL_LISTING=api fallback; reels-tab sweep note in MANUAL-RUN

tooling: instagram-chrome-scrape.py (was reels-scrape.py) gains --tab posts with early stop; chrome-sync.sh + reels/posts-sync.sh wrappers; MANUAL-RUN.md; atomic-deploy note
tooling: auto-upgrade gallery-dl + yt-dlp before every run and weekly; document 2026-10-22 empty posts listing
2026-10-23 00:49:45 -04:00 · 2026-10-22 23:53:35 -04:00 · 2026-10-22 23:24:15 -04:00 · 2026-10-14 21:39:28 -04:00 · 2026-10-14 21:34:14 -04:00 · 2026-10-14 21:25:57 -04:00
29 changed files with 2326 additions and 232 deletions
@@ -0,0 +1,109 @@
 # Manual run — scrape, archive, push to the NAS
 **`tooling` branch only. Never push this to GitHub.** Like `TOOLING.md`, it names the fetch host,
 the NAS hosts and their paths.
 How to do by hand everything the timers normally do, plus the second NAS hop that feeds
 `cosmo_web`. `TOOLING.md` has the details behind each step.
 ## Where things go
 1. **mattellite → ana-nas.** Every scrape script ends by rsyncing its new files to
   `ana-nas:/volume1/rslsync/sync/Instagram-archive/archives/` (10.20.28.200).
   <https://instaarchive.ergosteur.com> reads straight from there. **There is no separate step
   for this hop.**
 2. **ana-nas → DH2300.** The `archiver-instagram` container on `DH2300-D7CA` pulls that archive,
   makes thumbnails, remuxes video, normalizes into `cosmo_web`'s data and translates captions.
   It runs weekly (Wednesday 04:00). Step 6 below runs it by hand.
 ## 1. Pre-flight (mattellite)
 ```sh
 ssh matt@mattellite
 ~/gdl/check-session.py                                   # want: "session ok ... expires <date>"
 curl -s http://localhost:9222/json/version | head -2     # Chrome DevTools port is up
 ~/gdl/gdl-upgrade.sh                                     # gallery-dl + yt-dlp to latest
 ```
 If the session check fails, log in to Instagram in the VNC Chrome (display `:1`), **reload the
 tab** (a dead session can still look logged in) and re-check.
 ## 2. Stories — first; they expire in 24 h
 ```sh
 ~/gdl/run-stories.sh
 ```
 About 2 minutes. `rsync ... (code 23)` / `failed to set permissions` is expected: the script
 says "files landed" when that is the only problem. `No results` for a profile just means it has
 no active story.
 ## 3. Posts and reels
 **While gallery-dl's listing is broken** ([gh#9829](https://github.com/mikf/gallery-dl/issues/9829),
 from 2026-10-22), use the browser scraper, one profile at a time:
 ```sh
 for p in heejin kimxxlip; do ~/gdl/posts-sync.sh $p; done
 for p in withaseul cher_ryppo zindoriyam; do ~/gdl/posts-sync.sh $p; done
 ~/gdl/posts-sync.sh official_artms
 ```
 - `posts-sync.sh` scrolls `/<user>/` in the signed-in Chrome, stops once it reaches posts the
  archive already holds, and fetches only the new ones as single posts. Each profile takes a few
  minutes, mostly the 12–20 s gap between fetches.
 - It prints `no new posts for <user>` when a profile is caught up.
 - Logs: `~/gdl/logs/posts-<user>-*.log`.
 - **Then sweep the Reels tab too:** `~/gdl/run-reels-all.sh` (~15 min). Most reels never
  appear in the main grid. On 2026-10-23 only 3 of heejin's 114 reels did, so `posts-sync.sh`
  alone misses them. `reels-sync.sh <user>` does a single profile.
 **Or the whole thing in one command**, the same run the weekly timer does: gallery-dl for
 stories and highlights, then the browser for every profile's posts grid and Reels tab, each
 stopping once it reaches archived content:
 ```sh
 ~/gdl/run-profiles.sh
 ```
 Once a gallery-dl release fixes the listing, `GDL_LISTING=api ~/gdl/run-profiles.sh` does posts
 and reels through gallery-dl again (~5 min).
 For long runs, detach:
 `setsid nohup ~/gdl/run-profiles.sh >/dev/null 2>&1 &`, then
 `tail -f ~/gdl/logs/$(ls -1t ~/gdl/logs | head -1)`.
 **Stop at once** on a `429` from `scontent-*.cdninstagram.com` or a `400 Bad Request` on
 `/api/v1/feed/reels_media/`. See "When a run fails" in `TOOLING.md`.
 ## 4. Highlights
 These only come through `run-profiles.sh`. Expect them to fail with the same
 `Requested user could not be found` until gh#9829 is fixed in a release, so skip them until then.
 ## 5. Check it landed (any machine)
 ```sh
 for u in heejin kimxxlip withaseul cher_ryppo zindoriyam official_artms; do
  n=$(curl -s "https://instaarchive.ergosteur.com/api/archives/$u/files" | python3 -c 'import json,sys; d=json.load(sys.stdin); print(len(d if isinstance(d,list) else d["files"]))')
  printf '%-16s %s\n' "$u" "$n"
 done
 ```
 Or browse <https://instaarchive.ergosteur.com>. New files appear without a redeploy.
 ## 6. Push to `cosmo_web` (the DH2300 hop)
 This runs the same command the container's cron line runs (from `docker/archiver-instagram-entrypoint.sh`
 in Cosmo-Live-Downloads):
 ```sh
 ssh matt@DH2300-D7CA
 docker exec cosmo-web-archiver-instagram-1 sh -c \
  'cd /work && . /app/instagram-archiver.env && python3 scripts/instagram_archive_run.py'
 docker logs --tail 50 cosmo-web-archiver-instagram-1
 ```
 It pings that container's Healthchecks check, same as a scheduled run. The pipeline itself
 (mirror, thumbnail, remux, normalize, collaborators, translate) is described in the docstring of
 `scripts/instagram_archive_run.py` in Cosmo-Live-Downloads.
@@ -23,21 +23,33 @@
 ## Quick reference — getting new content onto the site
-Timers are **disabled**, so this is manual today. Everything runs on
+For the whole thing by hand, step by step (including the DH2300 / `cosmo_web` hop), see
-`mattellite`; it fetches, publishes to the NAS itself, and the site picks it up
+[MANUAL-RUN.md](MANUAL-RUN.md).
-with no deploy or restart.
+
 The three systemd timers (see "Scheduling" below) run these unattended; run
 them by hand only to get content sooner. Everything runs on `mattellite`; it
 fetches, publishes to the NAS itself, and the site picks it up with no deploy
 or restart.
 ```sh
 ssh mattellite
-~/gdl/gdl-cron.sh stories     # ~1 min.  DO THIS OFTEN - stories die in 24h
+~/gdl/run-stories.sh      # ~1 min.    DO THIS OFTEN - stories die in 24h
-~/gdl/gdl-cron.sh profiles    # ~5 min.  posts, reels, highlights, stories
+~/gdl/run-profiles.sh     # ~5 min.    posts, reels, highlights, stories
 ~/gdl/run-full-sweep.sh   # rarely, ~420 requests. Read its own comment first.
 ```
-`profiles` covers everything, so it is the one to run if you only run one — but
+Fixed-mode entry points around `gdl-cron.sh`, so the mode never has to be
-it is no substitute for `stories`, because a story posted and expired between
+remembered or typed — each just runs its own thing with the right flags
-two `profiles` runs is simply gone. Both are safe to run back-to-back; the
+already set. `run-profiles.sh` covers every surface, so it is the one to run
-`--min-interval` floor makes a repeat a no-op rather than a re-fetch.
+if you only run one, but it is no substitute for `run-stories.sh`: a story
 posted and expired between two `run-profiles.sh` runs is simply gone. Both
 are safe to run back-to-back; the `--min-interval` floor makes a repeat a
 no-op rather than a re-fetch.
 **`run-reels-all.sh` (browser-scrolling) is no longer needed for routine use
 as of 2026-09-14** — see "Reels: the API is blocked" below, now fixed
 upstream. `run-profiles.sh`'s own `reels` surface covers it again.
 Then check <https://instaarchive.ergosteur.com> — new posts appear without a
 redeploy. The archive index re-reads a directory when its mtime changes.
@@ -54,9 +66,135 @@ redeploy. The archive index re-reads a directory when its mtime changes.
 - `400 Bad Request` on `/api/v1/feed/reels_media/` — the account is behind a
  scraping-warning interstitial; check the browser before anything else.
-Long runs: `setsid nohup ~/gdl/gdl-cron.sh profiles >/dev/null 2>&1 &` and
+Long runs: `setsid nohup ~/gdl/run-profiles.sh >/dev/null 2>&1 &` (or `run-full-sweep.sh`
-`tail -f ~/gdl/logs/$(ls -1t ~/gdl/logs | head -1)`. Nothing reaches the
+/ `run-reels-all.sh`) and `tail -f ~/gdl/logs/$(ls -1t ~/gdl/logs | head -1)`.
-archive until a run finishes, so killing one midway is safe.
+Nothing reaches the archive until a run finishes, so killing one midway is
 safe.
 ## 2026-10-14 — the Instagram session died unnoticed; guards added
 **What happened.** Between 2026-10-06 09:12 and 2026-10-07 04:36 the dedicated
 Chrome's Instagram session died. Nothing alerted: the 10-07 full sweep logged
 `HTTP redirect to login page` on 17 of 18 sources (and "failed"), but every
 *stories* run after it fetched "No results" for all six profiles and still
 exited 0 — about a week of stories, which cannot be backfilled, was lost.
 Two compounding causes:
 - **No scheduled run had ever pinged Healthchecks.** `gdl-sync@.service` ran
  `gdl-cron.sh %i` directly (written 08-20), but `HC_URL` is exported by the
  `run-*.sh` wrappers (added 09-14), which the unit bypassed. `gdl-cron.sh`
  skips the ping silently when `HC_URL` is empty. Fixed: the unit now runs
  `run-%i.sh`. (`run-full-sweep.sh` has no check URL, so that mode still
  does not ping.)
 - **An open Chrome tab keeps showing its logged-in UI after the session is
  dead** — the SPA only notices on its next failed request. Looking at the
  window proves nothing; **reload the tab, or read the cookie DB.** The cookie
  jar had `csrftoken`, `ds_user_id`, `mid`… but no `sessionid`.
 **Guards now in `gdl-cron.sh`** (tests: `scripts/test_session_guards.py`):
 | exit | from | meaning |
 |---|---|---|
 | 76 | `check-session.py`, *before any request* | no unexpired `sessionid` in the Chrome profile gdl reads. The run aborts, logs why, pings `/76` (red check). Same DB, same profile as gdl, so it sees exactly what gdl would. Prints a `WARNING` (log only, no alert) when under 7 days of session remain |
 | 77 | `stories-streak.py`, *after a stories run* | every fetched stories source returned "No results" on 3 consecutive days (`GDL_EMPTY_STREAK_FAIL` to change). Catches a session Instagram stopped honouring while the cookie is still present, which the preflight cannot see. One day per calendar date, so manual re-runs do not inflate it; state in `~/gdl/stories-empty-streak` |
 | 75 | existing | every stories source skipped by `--min-interval` |
 **To recover from exit 76** the user must log in to Instagram in the Chrome on
 `mattellite` (VNC, display `:1`; neither password nor 2FA is something to
 script). Then `~/gdl/check-session.py` should say `session ok ... expires
 <date>`, and a manual `~/gdl/run-stories.sh` should find stories again.
 **Verbose output.** A healthy run looked hung: pacing is 12–20 s, the config
 sets `output: null` (no per-file lines), and the live-listing probe captured
 output until it finished. `gdl-sync.py` now prints `[n/N] user / kind (time)`,
 a `... fetching: 4m00s elapsed, 12 new file(s) so far (latest: …)` heartbeat
 every 60 s, and `done in 6m12s, 14 new file(s)`. It only counts files already
 written; no extra requests, pacing unchanged.
 **Archive resolution check** (read-only scan of header dimensions for the six
 scraped profiles, plus 8 single-post live metadata fetches, paced):
 - *Logged-out gdl does not download smaller files — it fails* with a login
  redirect, so no logged-out media ever entered the archive.
 - Archived image sizes **match** what a logged-in fetch offers (1440×1800,
  3277×4096, 2976×3720, and the genuinely small ones such as 400×400 or
  742×928 sitting next to 1440 px siblings in one carousel). No image-upgrade
  pass is needed.
 - Videos: ~1,200 of ~1,300 archived videos are 720p. A 2026 reel offers a
  1080×1920 VP9 stream (video-only DASH, plus separate AAC); a 2024 post tops
  out at 1276×720, same as the archive. A video upgrade would mostly benefit
  2025–26 posts (~700), at one request each.
 - **iOS 18.7 Safari plays VP9 + Opus `.webm` but not VP9 + AAC `.mp4`** (error
  code 4; `canPlayType` wrongly says "probably"). A video upgrade must
  therefore output `.webm` — video stream copied, audio re-encoded to Opus
  96k. **Not built yet.** Open points before building: the viewer mints every
  video blob as `video/mp4` (`src/lib/archive-cache.ts`,
  `src/hooks/useArchiveScanner.ts`) and must derive the MIME type from the
  extension; superseded 720p files should be kept for review in a directory
  *outside* the Resilio-synced, viewer-scanned `archives/` tree; and the
  request budget (suggested: ≤40 posts/day, behind the preflight).
 **Account swapped (same day).** The scraping account is now `erg05teur`
 (also signed in on the owner's phone). It is a fresh account, so the
 "replacement account" rules in `docs/gallery-dl.md` apply — and were not fully
 followed on day one: the resolution check and an unplanned catch-up run (below)
 spent roughly 90 requests within hours of the login. The owner chose to leave the
 scheduled timers as they are rather than pause them; the preflight (exit 76) and
 the all-empty-stories streak (exit 77) are the safety net. Worth watching: the
 weekly profiles timer's first fire (~2026-10-23) is a much larger spend than the
 daily stories runs.
 **Host brought in line with the repo (same day).** Every tracked script and unit
 on `mattellite` is now byte-identical to `scripts/`, including the weekly
 profiles timer (it had been left monthly since 08-22). Installing it fired an
 unplanned catch-up profiles run (see "Scheduling"); that run honoured the
 20 h floor (20 of 24 sources skipped) and fetched the four `reels` sources,
 ~15 min each at the usual pacing.
 ## 2026-09-14 — a tracked account renamed on Instagram: 0ct0ber19 → heejin
 HeeJin's handle changed and the old one is now fully dead — `instagram.com/
 0ct0ber19/` returns "Sorry, this page isn't available", no redirect. Verified
 `@heejin` is her (verified badge, ARTMS affiliation, followed by
 `official_artms`, same profile picture) before touching anything.
 Directory naming is forced from **our own tracked profile name**, not derived
 from Instagram's live username (see `gdl_command()` in `gdl-sync.py`) — so
 updating the URL alone would have started a brand-new `heejin/` directory
 tree alongside the existing `0ct0ber19/` one, splitting one person into two
 profiles. Fixed by migrating the existing archive instead of leaving it
 stale:
 - `0ct0ber19` → `heejin`, `0ct0ber19 - reels` → `heejin - reels`,
  `story - 0ct0ber19` → `story - heejin`, and both her highlight directories
  — 3,353 files total, done directly on the NAS (`ana-nas.lan.1qaz.ca`,
  `/volume1/rslsync/sync/Instagram-archive/archives`), not via `gdl-sync.py`.
 - Only files whose embedded `{username}` token was **exactly** `0ct0ber19`
  were renamed — matched as `_0ct0ber19 - ` (dated stem) or `^0ct0ber19 - `
  (the older undated highlight naming) or the bare `0ct0ber19.<ext>` profile
  picture, never a substring match. 7 files inside her own directories were
  correctly left untouched: real collab posts (`chuuo3o`, `findyourmapp`)
  physically stored there but truly owned by someone else — see "Where a
  shared reel lands" above for why that happens at all.
 - Verified by content after each `rename()`, not exit code — this volume can
  report a rename as failed when it actually succeeded (see
  `instaarchive-deployment` in project memory). Every directory came back
  with zero missing targets and zero leftover old names.
 - `docs/artms-instagram-accounts.txt` and `~/gdl/artms_account_links.txt`
  updated to the new URL. The live site picked up the rename immediately, no
  cache-bust needed — the archive index invalidates on directory mtime, which
  a rename changes.
 - **Not changed:** the skip-archive DB (`artms.db`) keys purely on
  `instagram_<media_id>`, no username involved, so it needed nothing. The
  per-source `state.json` timestamps *are* keyed by URL, though, so `heejin`'s
  entries look "never seeded" to `gdl-sync.py` now — the next run against her
  will do one seeding listing pass to catch up, same as any newly added
  profile. Cheap, and expected.
 If this happens again for another tracked account: confirm the new handle is
 really the same person before touching anything (verified badge + a mutual
 signal like "Followed by official_artms" + matching profile picture is
 usually enough), then this section is the template.
 ## Safety net: a global gallery-dl config
@@ -105,6 +243,39 @@ files, it is host-local runtime config, and it embeds the same real
 Chrome-profile path already documented above. Recreate it by hand (or from
 this section) after a fresh `mattellite` setup.
 ## Healthchecks.io
 `run-stories.sh` and `run-profiles.sh` each ping a healthchecks.io check —
 `instagram-archiver-stories` / `instagram-archiver-posts` — via `HC_URL`,
 plumbed through to `gdl-cron.sh`. A start ping fires before the sync, a
 finish ping fires after with the run's own log attached as the body (visible
 in the healthchecks.io dashboard, so a failure is diagnosable without SSHing
 in). The trailing URL segment is the exit code: `0` means success, anything
 else fails the check — including the `stories: 0 to sync` → exit 75 remap
 above, so a fully-skipped stories run alerts, not silently passes.
 Unset `HC_URL` (e.g. `HC_URL= ./run-stories.sh`) to run without pinging.
 `run-full-sweep.sh` and the reels scripts have no check configured yet.
 **The systemd unit must run the wrappers, not `gdl-cron.sh` directly** —
 `HC_URL` lives in `run-<mode>.sh`, and `gdl-cron.sh` skips the ping silently
 when it is empty. Until 2026-10-14 the unit bypassed the wrappers, so no
 scheduled run had ever pinged. The preflight (exit 76) and the all-empty
 streak (exit 77) are described in "2026-10-14" above; both reach the dashboard
 as failures through the same trailing-status-code convention.
 **This is also why `gdl-sync.py`'s `publish()` no longer treats the routine
 chown-permission warning as a failure (2026-09-14).** The archive share is
 never owned by the ssh user, so rsync's own `chown` step fails on *every*
 run — exit 23, "some files/attrs were not transferred" — even though every
 file lands; "Reading the output" above has documented this as cosmetic since
 this file was created. Before this fix the exit *code* didn't know that, so
 `gdl-sync.py` returned 1 on every clean run too — harmless when a human was
 reading the log anyway, but it would have made every healthcheck ping report
 a normal run as a failure. Only that specific permission-warning shape is
 masked; a real transfer failure sharing the same exit code (23) still fails
 loudly, verified with both a real captured rsync failure and unit tests.
 ## Remotes
 | remote | what goes there |
@@ -115,6 +286,68 @@ this section) after a fresh `mattellite` setup.
 The other 22 release tags stay on gitea. Pushing them all to GitHub triggers
 one container build per tag, because each tag carries its own workflow file.
 ## 2026-10-22 — gallery-dl auto-upgrade; posts listing went empty
 Scheduled and manual runs were failing quietly from early October: after the session was
 restored, `posts` listings for `heejin`/`kimxxlip` returned `No results` in ~3 s (a caught-up
 profile skips; it does not say that), while stories worked and the posts opened fine in the
 browser. The host was on gallery-dl 1.32.14; 1.32.17 (2026-10-17) lists "instagram: update
 `user` & `posts` query parameters". After upgrading, the profile lookup failed instead
 (`KeyError: 'data'`, `Requested user could not be found`). That is upstream
 [gh#9829](https://github.com/mikf/gallery-dl/issues/9829) -- Instagram changed the query again
 on ~10-21; a fix is merged upstream but **not in a release** as of 10-22. Single-post fetches
 (`/p/<code>/`) still work, so the gap is the *listing*, which the browser covers -- see
 "Posts grid by browser" below.
 `scripts/gdl-upgrade.sh` (`pipx upgrade --include-injected gallery-dl`, so yt-dlp too) now runs:
 - at the start of every `gdl-cron.sh` run, after the session preflight and before any request;
  logs `upgrade: up to date` or `upgrade: CHANGED a -> b`, and never fails the run;
 - weekly from `gdl-upgrade.timer` (systemd user timer, installed on `mattellite` with the
  others -- systemd rather than cron, to match the rest of the host).
 It costs no Instagram requests. The trade-off: a bad upstream release is picked up
 automatically; roll back with `pipx install --force gallery-dl==<old>` and re-inject yt-dlp.
 ### Posts grid by browser: `posts-sync.sh`
 The reels workaround from August, generalised. `reels-scrape.py` is now
 **`instagram-chrome-scrape.py`** with `--tab reels|posts`, and `reels-sync.sh`'s body is now
 **`chrome-sync.sh`** (`GDL_TAB=reels|posts`); `reels-sync.sh` and the new `posts-sync.sh` are
 two-line wrappers that set the tab, so every older command still works.
 ```sh
 ssh mattellite
 ~/gdl/posts-sync.sh heejin          # scroll /heejin/, fetch new /p/ links as single posts
 ```
 `--tab posts` scrolls the main grid (`/<user>/`), collects `/p/` and `/reel/` links, and dedupes
 against the whole archive like the reels tab does. Unlike reels it **stops early**: the grid is
 newest-first, so after 2 scroll rounds in a row that turned up nothing unarchived
 (`--max-known-rounds`, round 1 exempt because pinned posts sit on top) there is nothing newer to
 find. First run, `heejin`: 3 scrolls, 36 posts seen, 30 already held, 6 new.
 **Scheduled since 2026-10-23.** `gdl-cron.sh profiles` and `full-sweep` now list posts *and*
 reels this way: gallery-dl only does `--only stories,highlights`, then `chrome-sync.sh` runs per
 profile for the `posts` grid and the `reels` tab (both tabs needed: on 10-23 only 3 of heejin's
 114 reels were in the main grid). `profiles` uses the early stop (2 known rounds) on both tabs,
 and `full-sweep` scrolls every grid to the bottom. A failed browser step fails the run, which
 shows on Healthchecks. This reverses the earlier "by hand only" decision on purpose: the API
 listing is broken and the owner chose the browser. The trade-off is that unattended runs now
 need Chrome up on `:9222` with a live session, and they open and close their own tabs in it.
 `GDL_LISTING=api` restores the all-gallery-dl run once a release fixes gh#9829.
 **Deploy scripts atomically, never in place.** bash reads a script as it runs, so overwriting
 `gdl-cron.sh` under a running sync makes bash resume at the same byte offset in the *new* text.
 On 10-22 an `scp` over it mid-run made the wrapper die after the sync with
 `line 163: status: unbound variable`. The fetch and publish had finished, but the exit line and
 the Healthchecks ping were skipped. Copy to a temp name and `mv` it into place (a new inode; the
 running bash keeps the old one), or check `pgrep -af gdl-` first.
 **A `--min-interval` trap:** a run that returns `No results` still counts as a fetch, so the
 next run skips the source for 20 h. Re-run after a bad result with `--min-interval 0`
 (not `--force`, which also drops the probe cache).
 ## Guards — recreate these after a fresh clone
 Neither guard is versioned, so a new clone has **no protection at all**:
@@ -147,9 +380,15 @@ every deletion because of it.
 | path | what it is |
 |---|---|
 | `scripts/gdl-sync.py` | the gallery-dl fetcher; replaced JD2 for the ARTMS profiles |
 | `scripts/gdl-upgrade.sh` | `pipx upgrade` gallery-dl + yt-dlp; run by `gdl-cron.sh` and `gdl-upgrade.timer` |
 | `scripts/instagram-chrome-scrape.py` | list a profile's reels tab or posts grid by scrolling the signed-in Chrome over CDP |
 | `scripts/chrome-sync.sh` | scrape + fetch new posts for one profile; `reels-sync.sh` / `posts-sync.sh` set the tab |
 | `scripts/gdl-cron.sh` | unattended wrapper: `stories` \| `profiles` \| `full-sweep` |
 | `scripts/systemd/` | the timers actually installed on the fetch host |
 | `scripts/test_gdl_sync.py` | its tests |
 | `scripts/check-session.py` | preflight: is there a usable Instagram `sessionid` in the Chrome cookie DB? (exit 1 = no) |
 | `scripts/stories-streak.py` | postflight: all stories sources empty for several days running? |
 | `scripts/test_session_guards.py` | tests for the two guards above (offline) |
 | `scripts/jd2-sync.ts` | JDownloader `.crawljob` generator, still used elsewhere |
 | `docs/gallery-dl.md` | the measurements behind every option in the fetcher — **read before changing pacing** |
 | `docs/jdownloader.md` | the older JD2 flow |
@@ -252,7 +491,7 @@ the old names with a pointer rather than a bare error.
 | mode | cadence | cost | why |
 |---|---|---|---|
 | `stories` | daily | ~6 requests | stories expire in 24h and **cannot be backfilled**; this is the only run that loses content if skipped |
-| `profiles` | monthly | ~40-60 requests | every surface, `--abort 50` — stops enumerating a profile once it reaches content already held. Catches everything **new** |
+| `profiles` | weekly, randomized day | ~40-60 requests | every surface, `--abort 50` — stops enumerating a profile once it reaches content already held. Catches everything **new** |
 | `full-sweep` | rarely, by hand | **~420 requests** | no abort; walks every profile to the end. The only run that notices posts **edited** after we archived them (test case 15) |
 The skip-archive means an infrequent `profiles` run costs barely more than a frequent
@@ -287,12 +526,22 @@ systemctl --user list-timers 'gdl-sync@*'
 | unit | schedule | next fire (as installed) |
 |---|---|---|
 | `gdl-sync@stories.timer` | daily 09:00 | 09:36:45 — the delay is the randomisation working |
-| `gdl-sync@profiles.timer` | 3rd of each month, 04:00 | 04:37:44 |
+| `gdl-sync@profiles.timer` | weekly (`OnCalendar=weekly`) + `RandomizedDelaySec=6d` — a random day each week, not pinned | 2026-10-24 06:44 (first weekly fire after the 10-14 install) |
 | `gdl-sync@full-sweep.timer` | 7th of Jan/Apr/Jul/Oct, 04:00 | 04:42:39 |
 Unit files are version-controlled in `scripts/systemd/` and installed to
 `~/.config/systemd/user/`. One templated service, `gdl-sync@.service`, takes
-the mode as its instance name and runs `gdl-cron.sh %i`.
+the mode as its instance name and runs `run-%i.sh` (the fixed-mode wrappers
 around `gdl-cron.sh` — they carry the Healthchecks URL; see "Healthchecks.io").
 Before 2026-10-14 it ran `gdl-cron.sh %i` directly, which skipped every ping.
 The profiles timer was monthly (3rd, 04:00) until 2026-10-14, when the repo's
 weekly/random-day unit (committed 09-14 but never installed) was put on the host.
 Timer installs: copy the unit to `~/.config/systemd/user/`, `systemctl --user
 daemon-reload`, `restart` the timer. **`Persistent=true` makes that restart fire
 a catch-up run immediately if a scheduled elapse was missed** (it did, 2026-10-14:
 a real profiles run started the moment the timer was restarted). Touch
 `~/.local/share/systemd/timers/stamp-gdl-sync@<mode>.timer` first if a run has
 just happened by hand and you do not want one.
 Three settings are load-bearing:
@@ -481,7 +730,12 @@ Two results are **not** failures, despite how they look:
 ### 2. Check the cookies before concluding you are blocked
-Free, and it separates a local fault from a server-side one:
+First, `~/gdl/check-session.py` — it reads the same cookie DB gdl does and says
 whether a `sessionid` exists and when it expires (never prints values). Do not
 trust the open browser tab to tell you: it keeps showing a logged-in page after
 the session has died; reload it.
 Then, free, and it separates a local fault from a server-side one:
 ```sh
 PATH=$HOME/.local/bin:$PATH gallery-dl \
@@ -531,16 +785,35 @@ before something worse.
 ## Reels: the API is blocked, scrape by scrolling instead
-As of 2026-08-26/27, `gallery-dl`'s dedicated reels extractor fails on every
+**Fixed upstream 2026-09-14 — read this note before the rest of the section.**
 `gallery-dl` 1.32.12 (released 2026-09-12; we were on 1.32.9) resolved it: its
 changelog cites "posts & reels extractors... content retrieval failures", and
 upgrading (`pipx upgrade gallery-dl`) confirmed it -- a full `run-profiles.sh`
 across all six profiles came back with **zero** `HTTP redirect` errors on
 posts or reels, first time since 2026-08-26. `reels-scrape.py` (now `instagram-chrome-scrape.py`)/`reels-sync.sh`/
 `run-reels-all.sh` are no longer needed for routine use; left in place as a
 fallback (and because the whole-archive dedup fix they picked up along the
 way, see below, is independently useful). If this regresses, check the
 gallery-dl changelog for a newer release before re-reaching for the scroll
 workaround -- it may already be fixed there too.
 The rest of this section is kept as-is: the diagnosis, the workaround, and
 what it taught along the way are all still accurate history, just no longer
 the operative fix.
 As of 2026-08-26/27, `gallery-dl`'s dedicated reels extractor failed on every
 profile with `HTTP redirect to home page` — confirmed hours apart, with a
 freshly-warmed session and a correct `X-IG-WWW-Claim` header (that was the
-first suspect; ruled out by tracing the raw HTTP exchange). It is not a
+first suspect; ruled out by tracing the raw HTTP exchange). It was not a
-scraping-warning interstitial — the reels tab loads completely normally in a
+scraping-warning interstitial — the reels tab loaded completely normally in a
-real, already-signed-in browser — so this is Meta blocking the specific
+real, already-signed-in browser — so this was Meta blocking the specific
 `/api/v1/clips/user/` endpoint gallery-dl calls, not an account-health issue.
-Posts, highlights and stories are unaffected; only reels breaks.
+Posts, highlights and stories were unaffected at the time; only reels broke
 (posts later broke too, some time before 2026-09-13, in the same way — same
 fix resolved both).
-`scripts/reels-scrape.py` works around it by never calling that endpoint:
+`scripts/reels-scrape.py` (renamed `instagram-chrome-scrape.py` on 2026-10-22, when it gained
 `--tab posts`) works around it by never calling that endpoint:
 it drives the same signed-in Chrome via its loopback CDP port (`:9222`,
 already exposed for MCP automation), scrolls the reels tab like a person
 would, and scrapes `/reel/<code>/` links out of the rendered page. It only
@@ -569,11 +842,17 @@ after — left behind for inspection, same as `gdl-cron.sh`'s `staging-*`.
 Verified end to end against `zindoriyam` on 2026-08-27: 26 reels found on the
 page, 10 new ones fetched and published cleanly on the first run.
-**Not wired into `gdl-cron.sh` or the timers.** Both scripts drive your
+**Not wired into `gdl-cron.sh` or the timers** (as of 2026-08; **superseded 2026-10-23**: the
-actual browser session rather than firing a background request, and are
+weekly and quarterly runs now use the browser listing, see "Posts grid by browser" above). Both
-slower by design (real scrolling, not an API call) — both good reasons not
+scripts drive your actual browser session rather than firing a background request, and are
-to run this unattended without deciding that deliberately. Today it is a
+slower by design (real scrolling, not an API call), both good reasons not to run this
-per-profile, by-hand tool only.
+unattended without deciding that deliberately.
 For all six ARTMS profiles in one go, `~/gdl/run-reels-all.sh` just calls
 `reels-sync.sh` once per profile in `artms_account_links.txt`, same
 reasoning as above — real scrolling, so still by hand. Run 2026-09-03:
 `official_artms` alone has 609 real reels and took ~14 minutes of scrolling;
 the whole six-profile run took about 15 minutes total, 2 new reels found.
 ### Where a shared reel lands
@@ -607,7 +886,7 @@ PROFILE=someuser
 # 1. Scrape by scrolling; dedupe against the archive; write new URLs to a file.
 #    Needs gallery-dl's own pipx venv python -- that's where websocket-client
 #    (the one extra dependency this needs) got injected.
-~/.local/share/pipx/venvs/gallery-dl/bin/python3 ~/gdl/reels-scrape.py \
+~/.local/share/pipx/venvs/gallery-dl/bin/python3 ~/gdl/instagram-chrome-scrape.py \
  --profile "$PROFILE" \
  --index https://instaarchive.ergosteur.com \
  --out ~/gdl/"$PROFILE"-reels.txt
@@ -623,7 +902,7 @@ cd ~/gdl && PATH=$HOME/.local/bin:$PATH ./gdl-sync.py \
 Notes:
- `reels-scrape.py` fails fast if Chrome/CDP isn't up
+- `instagram-chrome-scrape.py` fails fast if Chrome/CDP isn't up
  (`curl -s http://localhost:9222/json/version` to check first).
 - If it finds nothing new, `gdl-sync.py --post-urls-file` refuses to run
  ("no usable URLs") rather than doing nothing quietly — expected when a
@@ -646,7 +925,7 @@ them, filed them correctly under `{username}` (the post's *true* owner, per
 its own metadata) — where they already existed from that profile's own
 regular sync — and `rsync --ignore-existing` silently skipped every one, so
 nothing was lost or duplicated. But 9 Instagram requests were spent finding
-that out. Since a shortcode is globally unique, `reels-scrape.py` now checks
+that out. Since a shortcode is globally unique, the scraper now checks
 every archived profile's listing, not just the one being scraped — all local
 requests to the viewer's own API, never to `instagram.com`, so checking all
 16 profiles costs nothing on the budget that actually matters. Confirmed
@@ -685,6 +964,12 @@ and three systemd timers now run the sync unattended.
 State as of 2026-08-20, after the sync run and the repo split. Nothing here is
 broken; these are decisions not yet made and cleanups not yet done.
 > **Update 2026-10-14:** the "Fetching" notes below predate the timers being enabled — all three
 > `gdl-sync@*` timers are installed and enabled on `mattellite` (see "Scheduling" and
 > "2026-10-14"). Newer open work: **the video-upgrade pass**
 > ([`docs/video-upgrade-plan.md`](docs/video-upgrade-plan.md) — measured and decided, not built),
 > and watching the fresh scraping account through its first weekly profiles run (~2026-10-23).
 ### Fetching
 - **The timers are DISABLED** after the 2026-08-21 scraping warning; see that
@@ -1,4 +1,4 @@
-https://www.instagram.com/0ct0ber19/
+https://www.instagram.com/heejin/
 https://www.instagram.com/kimxxlip/
 https://www.instagram.com/withaseul/
 https://www.instagram.com/cher_ryppo/
@@ -0,0 +1,221 @@
 [
  [
    2,
    {
      "category": "instagram",
      "coauthors": [
        {
          "full_name": "\ucd5c\ub9ac \ucd5c\uc608\ub9bc Choerry",
          "id": "57668363710",
          "username": "cher_ryppo"
        },
        {
          "full_name": "\uc9c4\uc194 \ud835\udc09\ud835\udc08\ud835\udc0d\ud835\udc12\ud835\udc0e\ud835\udc14\ud835\udc0b",
          "id": "55465048543",
          "username": "zindoriyam"
        },
        {
          "full_name": "Official ARTMS",
          "id": "58524253183",
          "username": "official_artms"
        },
        {
          "full_name": "HEEJIN",
          "id": "57723176039",
          "username": "0ct0ber19"
        },
        {
          "full_name": "\uae40\ub9bd KIMLIP",
          "id": "57506288936",
          "username": "kimxxlip"
        }
      ],
      "count": 1,
      "date": "2026-08-18 07:18:54",
      "description": "#\uc81c\uc791\uc9c0\uc6d0 ARTMS(\uc544\ub974\ud14c\ubbf8\uc2a4) \u2018Blue Blood Night\u2019 \ub9ac\uc2a4\ub2dd \ud30c\ud2f0 \ud83c\udf19\n\n\uc131\uc218\ub3d9\uc5d0\uc11c \ud3bc\uccd0\uc9c4 ARTMS\uc640 300\uba85 \ud32c\ub4e4\uc758 \uc5ed\ub300\uae09 \ubc24!\n\nARTMS\uac00 \uc57d 300\uba85\uc758 \ud32c\ub4e4\uacfc \ud568\uaed8 \ud074\ub7fd \ud615\ud0dc\uc758\n\ub3c5\ud2b9\ud558\uace0 \ubabd\ud658\uc801\uc778 \uacf5\uac04\uc5d0\uc11c \uc624\ud504\ub77c\uc778 \ud32c \uc774\ubca4\ud2b8\n\u2018Blue Blood Night\u2019 \ub9ac\uc2a4\ub2dd \ud30c\ud2f0\ub97c \uac1c\ucd5c\ud588\uc2b5\ub2c8\ub2e4 \u2728\n\n\ud558\uc774 \ub370\ud328\ub274\ub3c4 \uc678\uce58\uace0 ARTMS \uc0ac\ub791\ud574\uc694\ub3c4 \uc678\ucce4\ub358\n\ub728\uac70\uc6e0\ub358 \ud604\uc7a5\uc5d0 \ub370\ud328\ub274\uac00 \ub2e4\ub140\uc654\uc2b5\ub2c8\ub2e4 \ud83d\ude0e\n\n\ud83d\udccc\uc544\ub974\ud14c\ubbf8\uc2a4 (ARTMS) \n@official_artms \n\n\u25aa\ufe0f2024\ub144\uc5d0 \ub370\ubdd4\ud55c \ubaa8\ub4dc\ud558\uc6b0\uc2a4(MODHAUS) \uc18c\uc18d\uc758 5\uc778\uc870 \uac78\uadf8\ub8f9 \n\u25aa\ufe0f\uba64\ubc84: \ud76c\uc9c4, \ud558\uc2ac, \uae40\ub9bd, \uc9c4\uc194, \ucd5c\ub9ac \n\u25aa\ufe0f\uc774\ub2ec\uc758 \uc18c\ub140 \ucd9c\uc2e0 \uba64\ubc84\ub4e4\uc774 \ub73b\uc744 \ubaa8\uc544 \uacb0\uc131\ud55c \ud504\ub85c\uc81d\ud2b8\uc774\uc790 \uc815\uc2dd \uac78\uadf8\ub8f9 \n\u25aa\ufe0f\uccab \uc815\uaddc \uc568\ubc94\uacfc \ud0c0\uc774\ud2c0\uace1 \u2018Virtual Angel\u2019 \ub4f1\uc744 \uc120\ubcf4\uc784\n\n#ARTMS #\uc544\ub974\ud14c\ubbf8\uc2a4 #BlueBloodNight\n\n\ud83c\udfa5 @dailyfashion_news",
      "fullname": "\u270d\ud83c\udffb\ud83e\udd13\ud328\uc158 \ub274\uc2a4 \uc694\uc57d \u2b50\ufe0f \ub370\ud328\ub274\ud83d\ude0e\ud83d\udc85\ud83c\udffb",
      "liked": false,
      "likes": 7219,
      "owner_id": "38980901318",
      "pinned": [],
      "post_date": "2026-08-18 07:18:54",
      "post_id": "3966279735525149864",
      "post_shortcode": "DcLDme7zoyo",
      "post_url": "https://www.instagram.com/reel/DcLDme7zoyo/",
      "subcategory": "reel",
      "tags": [
        "#ARTMS",
        "#BlueBloodNight",
        "#\uc544\ub974\ud14c\ubbf8\uc2a4",
        "#\uc81c\uc791\uc9c0\uc6d0"
      ],
      "type": "reel",
      "username": "dailyfashion_news"
    }
  ],
  [
    3,
    "ytdl:https://www.instagram.com/reel/DcLDme7zoyo/1.mp4",
    {
      "category": "instagram",
      "coauthors": [
        {
          "full_name": "\ucd5c\ub9ac \ucd5c\uc608\ub9bc Choerry",
          "id": "57668363710",
          "username": "cher_ryppo"
        },
        {
          "full_name": "\uc9c4\uc194 \ud835\udc09\ud835\udc08\ud835\udc0d\ud835\udc12\ud835\udc0e\ud835\udc14\ud835\udc0b",
          "id": "55465048543",
          "username": "zindoriyam"
        },
        {
          "full_name": "Official ARTMS",
          "id": "58524253183",
          "username": "official_artms"
        },
        {
          "full_name": "HEEJIN",
          "id": "57723176039",
          "username": "0ct0ber19"
        },
        {
          "full_name": "\uae40\ub9bd KIMLIP",
          "id": "57506288936",
          "username": "kimxxlip"
        }
      ],
      "count": 1,
      "date": "2026-08-18 07:18:54",
      "description": "#\uc81c\uc791\uc9c0\uc6d0 ARTMS(\uc544\ub974\ud14c\ubbf8\uc2a4) \u2018Blue Blood Night\u2019 \ub9ac\uc2a4\ub2dd \ud30c\ud2f0 \ud83c\udf19\n\n\uc131\uc218\ub3d9\uc5d0\uc11c \ud3bc\uccd0\uc9c4 ARTMS\uc640 300\uba85 \ud32c\ub4e4\uc758 \uc5ed\ub300\uae09 \ubc24!\n\nARTMS\uac00 \uc57d 300\uba85\uc758 \ud32c\ub4e4\uacfc \ud568\uaed8 \ud074\ub7fd \ud615\ud0dc\uc758\n\ub3c5\ud2b9\ud558\uace0 \ubabd\ud658\uc801\uc778 \uacf5\uac04\uc5d0\uc11c \uc624\ud504\ub77c\uc778 \ud32c \uc774\ubca4\ud2b8\n\u2018Blue Blood Night\u2019 \ub9ac\uc2a4\ub2dd \ud30c\ud2f0\ub97c \uac1c\ucd5c\ud588\uc2b5\ub2c8\ub2e4 \u2728\n\n\ud558\uc774 \ub370\ud328\ub274\ub3c4 \uc678\uce58\uace0 ARTMS \uc0ac\ub791\ud574\uc694\ub3c4 \uc678\ucce4\ub358\n\ub728\uac70\uc6e0\ub358 \ud604\uc7a5\uc5d0 \ub370\ud328\ub274\uac00 \ub2e4\ub140\uc654\uc2b5\ub2c8\ub2e4 \ud83d\ude0e\n\n\ud83d\udccc\uc544\ub974\ud14c\ubbf8\uc2a4 (ARTMS) \n@official_artms \n\n\u25aa\ufe0f2024\ub144\uc5d0 \ub370\ubdd4\ud55c \ubaa8\ub4dc\ud558\uc6b0\uc2a4(MODHAUS) \uc18c\uc18d\uc758 5\uc778\uc870 \uac78\uadf8\ub8f9 \n\u25aa\ufe0f\uba64\ubc84: \ud76c\uc9c4, \ud558\uc2ac, \uae40\ub9bd, \uc9c4\uc194, \ucd5c\ub9ac \n\u25aa\ufe0f\uc774\ub2ec\uc758 \uc18c\ub140 \ucd9c\uc2e0 \uba64\ubc84\ub4e4\uc774 \ub73b\uc744 \ubaa8\uc544 \uacb0\uc131\ud55c \ud504\ub85c\uc81d\ud2b8\uc774\uc790 \uc815\uc2dd \uac78\uadf8\ub8f9 \n\u25aa\ufe0f\uccab \uc815\uaddc \uc568\ubc94\uacfc \ud0c0\uc774\ud2c0\uace1 \u2018Virtual Angel\u2019 \ub4f1\uc744 \uc120\ubcf4\uc784\n\n#ARTMS #\uc544\ub974\ud14c\ubbf8\uc2a4 #BlueBloodNight\n\n\ud83c\udfa5 @dailyfashion_news",
      "display_url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.82787-15/777982919_18124593284301319_6754878398807305029_n.jpg?stp=dst-jpg_e15_tt6&_nc_cat=107&ig_cache_key=Mzk2NjI3OTczNTUyNTE0OTg2NDE4MTI0NTkzMjgxMzAxMzE5.3-ccb7-5&ccb=7-5&_nc_sid=58cdad&efg=eyJ2ZW5jb2RlX3RhZyI6IkNMSVBTLnhwaWRzLjEzMjAuc2RyLnZpZGVvX2RlZmF1bHRfY292ZXJfZnJhbWUuQzMifQ%3D%3D&_nc_ohc=VTiIkvHCoBMQ7kNvwFnD08V&_nc_oc=Adp2K1e-pF18avq4GpMbXRgiZv_T6ySFEqTFdXPRtemyWnFeG8p6cgvxaXxZtyDDw-U&_nc_ad=z-m&_nc_cid=0&_nc_zt=23&_nc_ht=instagram.fyto3-1.fna&_nc_gid=-Kd8mdFfght8nFvwkJmRPA&_nc_ss=7a22e&oh=00_AQIKaJEGeU5iyrLIjnfXozkpbabHpemHWlDoSEEx11SMFQ&oe=6A9D5761",
      "extension": "mp4",
      "filename": "AQNbHAasCyJwuQWdYi7vWw3w6iCiMdBAXZQ58cnOKhanFZgw_IAhVC_f380GVo23sNeeCD559rVhmUSr0bmruolOjAgXMVwrmoRu88A",
      "fullname": "\u270d\ud83c\udffb\ud83e\udd13\ud328\uc158 \ub274\uc2a4 \uc694\uc57d \u2b50\ufe0f \ub370\ud328\ub274\ud83d\ude0e\ud83d\udc85\ud83c\udffb",
      "height": 1920,
      "height_original": 1920,
      "liked": false,
      "likes": 7219,
      "media_id": "3966279735525149864",
      "num": 1,
      "owner": {
        "account_badges": [],
        "account_type": 3,
        "can_see_quiet_post_attribution": true,
        "eligible_for_text_app_activation_badge": false,
        "fan_club_info": {
          "autosave_to_exclusive_highlight": null,
          "connected_member_count": null,
          "fan_club_id": null,
          "fan_club_name": null,
          "fan_consideration_page_revamp_eligiblity": null,
          "has_created_ssc": null,
          "has_enough_subscribers_for_ssc": null,
          "is_fan_club_gifting_eligible": null,
          "is_fan_club_referral_eligible": null,
          "is_free_trial_eligible": null,
          "largest_public_bc_id": null,
          "should_show_playlists_in_profile_tab": null,
          "subscriber_count": null
        },
        "fbid_v2": "17841439039552612",
        "feed_post_reshare_disabled": false,
        "friendship_status": {
          "followed_by": false,
          "following": false,
          "is_bestie": false,
          "is_feed_favorite": false,
          "is_muting_reel": false,
          "is_private": false,
          "is_restricted": false
        },
        "full_name": "\u270d\ud83c\udffb\ud83e\udd13\ud328\uc158 \ub274\uc2a4 \uc694\uc57d \u2b50\ufe0f \ub370\ud328\ub274\ud83d\ude0e\ud83d\udc85\ud83c\udffb",
        "has_anonymous_profile_picture": false,
        "hd_profile_pic_url_info": {
          "height": 1080,
          "url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.2885-19/283399158_1201757910664534_1998021056617019217_n.jpg?efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gE8DUM4skVKFnTxJ_6S8QN3RWaHsgKT9sigiWMRE8sqvl9r488ADsJFU6ErFHln2eg&_nc_ohc=LjOAkQaNPtMQ7kNvwHhtSrv&_nc_gid=-Kd8mdFfght8nFvwkJmRPA&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQLdX6t5krhp725dzvNOvQbGOFpVdvwgZWqDkFvv1AeQPA&oe=6A9D6555&_nc_sid=fc8dfb",
          "width": 1080
        },
        "hd_profile_pic_versions": [
          {
            "height": 320,
            "url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.2885-19/283399158_1201757910664534_1998021056617019217_n.jpg?stp=dst-jpg_s320x320_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gE8DUM4skVKFnTxJ_6S8QN3RWaHsgKT9sigiWMRE8sqvl9r488ADsJFU6ErFHln2eg&_nc_ohc=LjOAkQaNPtMQ7kNvwHhtSrv&_nc_gid=-Kd8mdFfght8nFvwkJmRPA&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQKux7g7JpZsP_Q7z6rX3bFlcdVCUCwwDAQfmRRWbB6ePQ&oe=6A9D6555&_nc_sid=fc8dfb",
            "width": 320
          },
          {
            "height": 640,
            "url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.2885-19/283399158_1201757910664534_1998021056617019217_n.jpg?stp=dst-jpg_s640x640_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gE8DUM4skVKFnTxJ_6S8QN3RWaHsgKT9sigiWMRE8sqvl9r488ADsJFU6ErFHln2eg&_nc_ohc=LjOAkQaNPtMQ7kNvwHhtSrv&_nc_gid=-Kd8mdFfght8nFvwkJmRPA&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQJJz0fXTbLsfMqkiG7KHQtZbaHwHAU8nS1eTzB0sQqRSA&oe=6A9D6555&_nc_sid=fc8dfb",
            "width": 640
          }
        ],
        "id": "38980901318",
        "is_active_on_text_post_app": true,
        "is_embeds_disabled": false,
        "is_favorite": false,
        "is_private": false,
        "is_ring_creator": false,
        "is_unpublished": false,
        "is_verified": true,
        "latest_reel_media": 1788306389,
        "pk": "38980901318",
        "pk_id": "38980901318",
        "profile_pic_id": "2844231369895585868_38980901318",
        "profile_pic_url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.2885-19/283399158_1201757910664534_1998021056617019217_n.jpg?stp=dst-jpg_s150x150_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gE8DUM4skVKFnTxJ_6S8QN3RWaHsgKT9sigiWMRE8sqvl9r488ADsJFU6ErFHln2eg&_nc_ohc=LjOAkQaNPtMQ7kNvwHhtSrv&_nc_gid=-Kd8mdFfght8nFvwkJmRPA&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQI28hrlvg9XAWGFkTjCyv96HLuMEZb5q6AlNbUUqw2WVQ&oe=6A9D6555&_nc_sid=fc8dfb",
        "show_account_transparency_details": true,
        "show_ring_award": false,
        "strong_id__": "38980901318",
        "text_post_app_is_private": false,
        "third_party_downloads_enabled": 2,
        "transparency_product_enabled": false,
        "user_activation_info": {},
        "username": "dailyfashion_news"
      },
      "owner_id": "38980901318",
      "pinned": [],
      "post_date": "2026-08-18 07:18:54",
      "post_id": "3966279735525149864",
      "post_shortcode": "DcLDme7zoyo",
      "post_url": "https://www.instagram.com/reel/DcLDme7zoyo/",
      "shortcode": "DcLDme7zoyo",
      "subcategory": "reel",
      "tagged_users": [
        {
          "full_name": "\uc9c4\uc194 \ud835\udc09\ud835\udc08\ud835\udc0d\ud835\udc12\ud835\udc0e\ud835\udc14\ud835\udc0b",
          "id": "55465048543",
          "username": "zindoriyam"
        },
        {
          "full_name": "\uae40\ub9bd KIMLIP",
          "id": "57506288936",
          "username": "kimxxlip"
        },
        {
          "full_name": "\ucd5c\ub9ac \ucd5c\uc608\ub9bc Choerry",
          "id": "57668363710",
          "username": "cher_ryppo"
        },
        {
          "full_name": "HEEJIN",
          "id": "57723176039",
          "username": "0ct0ber19"
        },
        {
          "full_name": "Official ARTMS",
          "id": "58524253183",
          "username": "official_artms"
        }
      ],
      "tags": [
        "#ARTMS",
        "#BlueBloodNight",
        "#\uc544\ub974\ud14c\ubbf8\uc2a4",
        "#\uc81c\uc791\uc9c0\uc6d0"
      ],
      "type": "reel",
      "username": "dailyfashion_news",
      "video_url": "https://instagram.fyto3-1.fna.fbcdn.net/o1/v/t2/f2/m86/AQNbHAasCyJwuQWdYi7vWw3w6iCiMdBAXZQ58cnOKhanFZgw_IAhVC_f380GVo23sNeeCD559rVhmUSr0bmruolOjAgXMVwrmoRu88A.mp4?_nc_cat=105&_nc_oc=AdomTMuitkVGoe5U3qTaa9WVmrQNaUmC-hpxQMgtcKNDrT2Htj3cu6iPiBDfh9M0Hyk&_nc_sid=5e9851&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_ohc=O3edipmWdLQQ7kNvwFP9r4s&efg=eyJ2ZW5jb2RlX3RhZyI6Inhwdl9wcm9ncmVzc2l2ZS5JTlNUQUdSQU0uQ0xJUFMuQzMuNzIwLmRhc2hfYmFzZWxpbmVfMV92MSIsInhwdl9hc3NldF9pZCI6MTgxMTkyNTgxMzEyMzA0OSwiYXNzZXRfYWdlX2RheXMiOjE0LCJ2aV91c2VjYXNlX2lkIjoxMDA5OSwiZHVyYXRpb25fcyI6NTQsInVybGdlbl9zb3VyY2UiOiJ3d3cifQ%3D%3D&ccb=17-1&vs=69de96aad46e550e&_nc_vs=HBksFQIYUmlnX3hwdl9yZWVsc19wZXJtYW5lbnRfc3JfcHJvZC83RTRFMjc3MDhFMjVCMzUxNjA5MURDQUM2QTQ3MTlBQV92aWRlb19kYXNoaW5pdC5tcDQVAALIARIAFQIYUWlnX3hwdl9wbGFjZW1lbnRfcGVybWFuZW50X3YyL0I3NDdFMUM4NTRCODk2NDRENDM2NTIwQzVBODBCOTk5X2F1ZGlvX2Rhc2hpbml0Lm1wNBUCAsgBEgAoABgAGwKIB3VzZV9vaWwBMRJwcm9ncmVzc2l2ZV9yZWNpcGUBMRUAACbS35qn-_u3BhUCKAJDMywXQEtRBiTdLxsYEmRhc2hfYmFzZWxpbmVfMV92MREAdf4HZeadAQA&_nc_gid=-Kd8mdFfght8nFvwkJmRPA&_nc_zt=28&_nc_ss=7a22e&oh=00_AQLoXUFyQixJHvIcp3UE-6sEsjKuOlmhiPWmQphL7U7o5g&oe=6A996A42",
      "width": 1080,
      "width_original": 1080
    }
  ]
 ]
@@ -0,0 +1,181 @@
 [
  [
    2,
    {
      "audio_artist": null,
      "audio_duration": 15.0,
      "audio_timestamps": null,
      "audio_title": "Original audio",
      "audio_user": {
        "full_name": "\uc2e0\ubaa8\ucc0c \ud835\udde0\ud835\uddfc\ud835\uddf0\ud835\uddf5\ud835\uddf6 \ud83c\udf58",
        "id": "61396137942",
        "is_private": false,
        "is_verified": false,
        "pk": "61396137942",
        "pk_id": "61396137942",
        "profile_pic_id": "3506506412954824214_61396137942",
        "profile_pic_url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.2885-19/468037362_1017459960091671_5806837525855946557_n.jpg?stp=dst-jpg_s150x150_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby42MDAuYzIifQ&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=103&_nc_oc=Q6cZ2gEBkafhMKkLe-YbSz6FVQiMLEw4juIKWvM8PNmZ2nTcMpNyjdl0kV_-fWna0tytdLA&_nc_ohc=sPpx_bESe4cQ7kNvwFvI70J&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQIS_52Qhu7n6Fo7XNtQaIKXGEhkwhgz-C9vpWo4yb6dnw&oe=6A9D5107&_nc_sid=fc8dfb",
        "strong_id__": "61396137942",
        "username": "mochi.053"
      },
      "category": "instagram",
      "coauthors": [
        {
          "full_name": "HEEJIN",
          "id": "57723176039",
          "username": "0ct0ber19"
        }
      ],
      "count": 2,
      "date": "2026-08-22 02:48:05",
      "description": "\u3060\u3044\u3058\u3087\u270c\ufe0f\u3067\u3057\u3087",
      "fullname": "\uce04",
      "liked": true,
      "likes": 178797,
      "owner_id": "54301371254",
      "pinned": [],
      "post_date": "2026-08-22 02:48:05",
      "post_id": "3969041070976751617",
      "post_shortcode": "DcU3dM-gVgB",
      "post_url": "https://www.instagram.com/p/DcU3dM-gVgB/",
      "subcategory": "post",
      "type": "post",
      "username": "chuuo3o"
    }
  ],
  [
    3,
    "ytdl:https://www.instagram.com/p/DcU3dM-gVgB/1.mp4",
    {
      "audio_artist": null,
      "audio_duration": 15.0,
      "audio_timestamps": null,
      "audio_title": "Original audio",
      "audio_user": {
        "full_name": "\uc2e0\ubaa8\ucc0c \ud835\udde0\ud835\uddfc\ud835\uddf0\ud835\uddf5\ud835\uddf6 \ud83c\udf58",
        "id": "61396137942",
        "is_private": false,
        "is_verified": false,
        "pk": "61396137942",
        "pk_id": "61396137942",
        "profile_pic_id": "3506506412954824214_61396137942",
        "profile_pic_url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.2885-19/468037362_1017459960091671_5806837525855946557_n.jpg?stp=dst-jpg_s150x150_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby42MDAuYzIifQ&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=103&_nc_oc=Q6cZ2gEBkafhMKkLe-YbSz6FVQiMLEw4juIKWvM8PNmZ2nTcMpNyjdl0kV_-fWna0tytdLA&_nc_ohc=sPpx_bESe4cQ7kNvwFvI70J&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQIS_52Qhu7n6Fo7XNtQaIKXGEhkwhgz-C9vpWo4yb6dnw&oe=6A9D5107&_nc_sid=fc8dfb",
        "strong_id__": "61396137942",
        "username": "mochi.053"
      },
      "category": "instagram",
      "coauthors": [
        {
          "full_name": "HEEJIN",
          "id": "57723176039",
          "username": "0ct0ber19"
        }
      ],
      "count": 2,
      "date": "2026-08-22 02:48:05",
      "description": "\u3060\u3044\u3058\u3087\u270c\ufe0f\u3067\u3057\u3087",
      "display_url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.82787-15/783856831_18043412798811255_1724949997749570795_n.jpg?stp=dst-jpg_e15_tt6&_nc_cat=108&ig_cache_key=Mzk2OTA0MTA3MDk3Njc1MTYxNzE4MDQzNDEyNzkyODExMjU1.3-ccb7-5&ccb=7-5&_nc_sid=58cdad&efg=eyJ2ZW5jb2RlX3RhZyI6IkNMSVBTLnhwaWRzLjcyMC5zZHIudmlkZW9fZGVmYXVsdF9jb3Zlcl9mcmFtZS5DMyJ9&_nc_ohc=JWzEND3gUxAQ7kNvwGDGCLf&_nc_oc=AdpwpFkh8Wj84hyF4zWcsaNB5zUnYJh2bJijNe9AxrOR1WKa08Uy35khb5BpFnG9XhM&_nc_ad=z-m&_nc_cid=0&_nc_zt=23&_nc_ht=instagram.fyto3-1.fna&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&_nc_ss=7a22e&oh=00_AQLQlu6EItD2G6QrlOmQSas2IHDOpucLIhxVWcCPFE1BOw&oe=6A9D4B4F",
      "extension": "mp4",
      "filename": "AQN8nhc_L3hdDSQVVRF2OXm2gKwDpsdh_jw81ZnA9Uh4nqOwukKke8QXrGCe1jrMtOXzPT5Bkuipu8RShT651VV7q6EVbjvxC-fxgUE",
      "fullname": "\uce04",
      "height": 1920,
      "height_original": 1920,
      "liked": true,
      "likes": 178797,
      "media_id": "3969041070976751617",
      "num": 1,
      "owner": {
        "account_badges": [],
        "account_type": 3,
        "can_see_quiet_post_attribution": true,
        "eligible_for_text_app_activation_badge": false,
        "fan_club_info": {
          "autosave_to_exclusive_highlight": null,
          "connected_member_count": null,
          "fan_club_id": null,
          "fan_club_name": null,
          "fan_consideration_page_revamp_eligiblity": null,
          "has_created_ssc": null,
          "has_enough_subscribers_for_ssc": null,
          "is_fan_club_gifting_eligible": null,
          "is_fan_club_referral_eligible": null,
          "is_free_trial_eligible": null,
          "largest_public_bc_id": null,
          "should_show_playlists_in_profile_tab": null,
          "subscriber_count": null
        },
        "fbid_v2": "17841454337627671",
        "feed_post_reshare_disabled": false,
        "friendship_status": {
          "followed_by": false,
          "following": true,
          "is_bestie": false,
          "is_feed_favorite": false,
          "is_muting_reel": false,
          "is_private": false,
          "is_restricted": false
        },
        "full_name": "\uce04",
        "has_anonymous_profile_picture": false,
        "hd_profile_pic_url_info": {
          "height": 1080,
          "url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.82787-19/714930835_18032224961811255_6816644959905468206_n.jpg?efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gEBkafhMKkLe-YbSz6FVQiMLEw4juIKWvM8PNmZ2nTcMpNyjdl0kV_-fWna0tytdLA&_nc_ohc=hpoiFipmAqsQ7kNvwE3y5Ow&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQJiPho8ymPf1bC2B5dAlKCbdXW8d4KK-9qqsVrcOytVGQ&oe=6A9D4CB4&_nc_sid=fc8dfb",
          "width": 1080
        },
        "hd_profile_pic_versions": [
          {
            "height": 320,
            "url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.82787-19/714930835_18032224961811255_6816644959905468206_n.jpg?stp=dst-jpg_s320x320_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gEBkafhMKkLe-YbSz6FVQiMLEw4juIKWvM8PNmZ2nTcMpNyjdl0kV_-fWna0tytdLA&_nc_ohc=hpoiFipmAqsQ7kNvwE3y5Ow&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQIn43F6SprbhopBuJxmswE2wd4C39mDZoL9Fbqs8Ih-cw&oe=6A9D4CB4&_nc_sid=fc8dfb",
            "width": 320
          },
          {
            "height": 640,
            "url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.82787-19/714930835_18032224961811255_6816644959905468206_n.jpg?stp=dst-jpg_s640x640_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gEBkafhMKkLe-YbSz6FVQiMLEw4juIKWvM8PNmZ2nTcMpNyjdl0kV_-fWna0tytdLA&_nc_ohc=hpoiFipmAqsQ7kNvwE3y5Ow&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQJF1vYqhGCDkV-v6WbrTb_PoVC4GA3QCbEGJU0y0F96BA&oe=6A9D4CB4&_nc_sid=fc8dfb",
            "width": 640
          }
        ],
        "id": "54301371254",
        "is_active_on_text_post_app": false,
        "is_embeds_disabled": false,
        "is_favorite": false,
        "is_private": false,
        "is_ring_creator": false,
        "is_unpublished": false,
        "is_verified": true,
        "latest_reel_media": 0,
        "pk": "54301371254",
        "pk_id": "54301371254",
        "profile_pic_id": "3912796961274695044_54301371254",
        "profile_pic_url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.82787-19/714930835_18032224961811255_6816644959905468206_n.jpg?stp=dst-jpg_s150x150_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gEBkafhMKkLe-YbSz6FVQiMLEw4juIKWvM8PNmZ2nTcMpNyjdl0kV_-fWna0tytdLA&_nc_ohc=hpoiFipmAqsQ7kNvwE3y5Ow&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQJu85hrHiCGK3enXnfDQExy_ZbgzCV_AGYJ4yOy7mi_QQ&oe=6A9D4CB4&_nc_sid=fc8dfb",
        "show_account_transparency_details": true,
        "show_ring_award": false,
        "strong_id__": "54301371254",
        "text_post_app_is_private": true,
        "third_party_downloads_enabled": 1,
        "transparency_product_enabled": false,
        "user_activation_info": {},
        "username": "chuuo3o"
      },
      "owner_id": "54301371254",
      "pinned": [],
      "post_date": "2026-08-22 02:48:05",
      "post_id": "3969041070976751617",
      "post_shortcode": "DcU3dM-gVgB",
      "post_url": "https://www.instagram.com/p/DcU3dM-gVgB/",
      "shortcode": "DcU3dM-gVgB",
      "subcategory": "post",
      "tagged_users": [
        {
          "full_name": "HEEJIN",
          "id": "57723176039",
          "username": "0ct0ber19"
        }
      ],
      "type": "post",
      "username": "chuuo3o",
      "video_url": "https://instagram.fyto3-1.fna.fbcdn.net/o1/v/t2/f2/m86/AQN8nhc_L3hdDSQVVRF2OXm2gKwDpsdh_jw81ZnA9Uh4nqOwukKke8QXrGCe1jrMtOXzPT5Bkuipu8RShT651VV7q6EVbjvxC-fxgUE.mp4?_nc_cat=102&_nc_oc=AdqR865Bo8jW90mx5U9WR94Q0Eh1YI3ycT_ZhhW71XQNFHxUR3aBwhQEgcQ9Z2Hnrzs&_nc_sid=5e9851&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_ohc=2F0lG8gquXwQ7kNvwGIIkUT&efg=eyJ2ZW5jb2RlX3RhZyI6Inhwdl9wcm9ncmVzc2l2ZS5JTlNUQUdSQU0uQ0xJUFMuQzMuNzIwLmRhc2hfYmFzZWxpbmVfMV92MSIsInhwdl9hc3NldF9pZCI6MTYxMDAzNjE0Mzk3NzkxMywiYXNzZXRfYWdlX2RheXMiOjExLCJ2aV91c2VjYXNlX2lkIjoxMDA5OSwiZHVyYXRpb25fcyI6MTUsInVybGdlbl9zb3VyY2UiOiJ3d3cifQ%3D%3D&ccb=17-1&vs=93986893873e15f7&_nc_vs=HBksFQIYUmlnX3hwdl9yZWVsc19wZXJtYW5lbnRfc3JfcHJvZC80ODQ5NDdDRjM0NzcyNkI2RkE4RDZDRkFGQjQzQTlCQl92aWRlb19kYXNoaW5pdC5tcDQVAALIARIAFQIYUWlnX3hwdl9wbGFjZW1lbnRfcGVybWFuZW50X3YyLzkwNDc3OEIwRTY2MDYxNEY5RjA4MUMxQzNDMDVBMkFDX2F1ZGlvX2Rhc2hpbml0Lm1wNBUCAsgBEgAoABgAGwKIB3VzZV9vaWwBMRJwcm9ncmVzc2l2ZV9yZWNpcGUBMRUAACby1v-GuJTcBRUCKAJDMywXQC6qfvnbItEYEmRhc2hfYmFzZWxpbmVfMV92MREAdf4HZeadAQA&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&_nc_zt=28&_nc_ss=7a22e&oh=00_AQIrUM_ZU8b3LTxlgefSs0x6vjvCX9BV4piQfWrs74dXIg&oe=6A99835F",
      "width": 1080,
      "width_original": 1080
    }
  ]
 ]
@@ -16,6 +16,23 @@ by hand.
 ## The account was suspended on 2026-08-17 — read this first
 > [!NOTE]
 > **The scraping account was replaced on 2026-10-14.** It is now **`erg05teur`**,
 > which is also signed in on the owner's phone (a separate session; the cookie
 > gdl uses lives only in the dedicated Chrome on `mattellite`). Everything below
 > about the suspension is the **previous** account's history and the reasons for
 > every rule; none of it has stopped applying, and the new account has no track
 > record, so treat it as a replacement account (see "For a replacement account"
 > below). The new session cookie was created 2026-10-14 ~10:07 EDT.
 >
 > What this account has been asked since then, honestly counted, because the
 > warm-up advice below was not followed to the letter: ~13 single-post / format
 > probes (resolution check, 2026-10-14), plus ~80 listing requests from an
 > **unplanned** reels catch-up run (installing the weekly profiles timer
 > triggered it via `Persistent=true`; stopped partway, `heejin / reels`
 > completed, `kimxxlip / reels` was abandoned ~7 min in). Scheduled runs were
 > deliberately left on their normal timers by the owner.
 The account used for all of the below was suspended the same day this tooling
 was built, for "activity that doesn't follow our Community Standards on spam".
 The fetching was not the expensive part. **Verification was.**
@@ -79,6 +96,9 @@ complaining, the rate-limited one has been unhappy for a while.
 ### For a replacement account
 (Applies to `erg05teur`, 2026-10-14 onward — see the note at the top of this
 section for what it has already been asked.)
 - Let it exist and be used normally for a while before pointing any tool at it.
 - Keep the cookie on one machine and one public IP, as before.
 - Start with a single small profile and stop for the day afterwards.
@@ -146,6 +166,19 @@ Anonymous access is **not** a viable fallback: it serves lower-resolution media,
 caps profile pagination at 12 posts, and returns `AuthRequired` for stories and
 highlights.
 (Observed 2026-10-07, after the session died: with no `sessionid` gallery-dl did
 not fall back to anonymous downloads at all — most sources failed with
 `HTTP redirect to login page` / `HTTP redirect to home page`, and stories came
 back as "No results". The lower-resolution claim above was **not** re-measured
 then; what *was* measured is that archived image sizes match a logged-in fetch.)
 **The session can die silently, and nothing about the browser window shows it** —
 an open Instagram tab keeps rendering its logged-in UI until its next failed
 request. The only reliable signal is the cookie DB: `scripts/check-session.py`
 looks for an unexpired `sessionid` in this same profile, and `gdl-cron.sh` runs
 it before every scheduled run (exit 76 = refuse to scrape). See `TOOLING.md`,
 "2026-10-14".
 ## Output format
 The viewer's parser is the contract, not JD2's exact bytes. `EXPORT_RE` in
@@ -276,6 +309,79 @@ viewer retire the lone-video heuristic in `src/lib/post-tabs.ts` — see
 `/p/<shortcode>/` URL leaves it `null`. Sync always uses listing URLs, so this
 only matters when testing by hand.
 ### Collab posts: the JSON and the media disagree about whose post it is
 Verified directly against two saved raw API responses (`docs/example-api-
 response-DcU3dM-gVgB.json`, a 2-way collab, and `docs/example-api-response-
 DcLDme7zoyo.json`, a 5-way collab owned by an external account), 2026-09-01.
 For an Instagram Collab, `username`/`fullname`/`owner_id` in the listing
 response are always the **original poster's**, never the scraped account's —
 even when the scraped account is one of the collaborators, not the owner. The
 metadata `.json` sidecar (built from `include`, above) is filed under that
 same original-poster identity, since its filename template uses
 `{username}`. But the **media file** for that same post is written into
 whichever profile's own crawl directory triggered the download, and its
 filename's `{username}` slot took the *scraped* account's name, not the true
 owner's. Net effect: one physical post produces a JSON named for the real
 owner and a media file (or files, for a carousel) named for whoever we were
 crawling — two different identities for one post, in the same directory.
 Downstream (`cosmo_normalize_instagram.py` in `Cosmo-Live-Downloads`) had to
 stop matching media to its JSON by username and match on shortcode/`{num}`
 only, and separately built a cross-profile `collab_with` pass so every
 participant's page shows the post, not just the one whose directory JD2/
 gallery-dl happened to land the JSON in. See that repo's `NOTES.md`,
 2026-09-01 entries, for the full fix.
 **`coauthors` is a native, richer signal for this, and is now captured.**
 The raw API response carries a `coauthors` array (`{"full_name", "id",
 "username"}` per collaborator) that **excludes the post's own owner** —
 confirmed on both saved examples, including one where the owner
 (`dailyfashion_news`) is an external account with no ARTMS members in her
 own name, present only via `coauthors`. Added to the post-level metadata
 JSON's `include` list on 2026-09-01: a direct field instead of inferring a
 collab from filename/directory identity mismatches, and it never needs the
 "is this the owner" branch `coauthors` already excludes for us.
 **Per-carousel-item fields got their own sidecar, added the same day.**
 `width`, `height`, `width_original`, `height_original` and `tagged_users`
 live on the per-FILE kwdict, not the per-post one the metadata JSON above
 reads — a carousel's items can each have different dimensions and tags,
 which one post-level JSON can't represent. `gdl-sync.py`'s `media_pp` is a
 second `metadata` postprocessor, `event: "file"` (gallery-dl's default when
 omitted), so it runs once per downloaded file and writes `<filename>.json`
 alongside it — e.g. `... - 01.jpg.json` next to `... - 01.jpg`, never
 colliding with the post-level `....json`, which has no per-item number.
 `owner` — a full user object (profile pic URLs, privacy flags) for whoever
 posted that specific item — is deliberately left out, the same reasoning as
 `audio_user` above. Verified against the same two saved examples: correct
 per-item `width`/`height` and `tagged_users` came back on a live re-fetch of
 an already-archived carousel, with zero media re-downloaded (the existing
 skip-archive still applies; only the new sidecars are new files).
 ### The shared archive-db dedups media across profiles too, not just within one
 The skip-archive DB (see "Incremental sync" above) keys purely on
 `instagram_<media_id>`, with no per-profile scoping. When the *same* media_id
 is reachable from more than one profile's listing — a Collab post, or a
 repost — whichever profile's crawl reaches it **first** downloads the file;
 every other profile that later lists the same media_id sees it as
 already-in-the-archive-db and skips the download, even though that file has
 never actually landed in *that profile's own* directory tree. The metadata
 `.json`/`.txt` sidecars are written regardless (they aren't gated by the
 download-archive), so the symptom is a JSON with zero matching media files
 in its own directory — 510 such posts were found across the real archive on
 2026-09-01, entirely from this mechanism, not from anything actually missing
 from Instagram.
 This isn't fixable on the gallery-dl side without per-profile archive DBs
 (which would defeat the point of skip-archive — re-downloading anything a
 sibling profile already fetched). The fix lives downstream instead: `cosmo_
 normalize_instagram.py` builds one archive-wide `shortcode → path` index once
 and falls back to it when a post's own directory has no matching media. Worth
 knowing before assuming a JSON-with-no-media post reflects a real scrape gap.
 ## Cadence, and the budget that enforces it
 **Monthly for everything, daily for stories only.** Stories expire in 24h and
@@ -285,7 +391,7 @@ means an infrequent full sync costs barely more than a frequent one, because it
 only fetches what is new.
 ```
-# monthly, everything
+# weekly, everything
 gdl-sync.py --index <viewer-url> --staging ~/gdl/staging \
    --publish <user>@<nas>:<archives> --archive-db ~/gdl/artms.db \
    --urls-file artms_account_links.txt --execute
@@ -559,6 +665,22 @@ Verify by asking gallery-dl's own interpreter, not the shell:
 - **Highlights already have two naming styles on disk**, and every undated file
  has a dated twin. The scanner dedupes by index so they render once; it is
  wasted disk, not a display bug.
 - **Some `video_versions` entries are VP9, and format selection is
  codec-blind.** The extractor picks `max(video_versions, key=lambda x:
  (x["width"], x["height"], x["type"]))` — resolution only, no codec check
  (`instagram.py`). Instagram appears to have started serving VP9-encoded
  highest-resolution variants for some posts around when the new gdl-based
  workflow started (2026-08); 87 such files were found archive-wide on
  2026-09-01. VP9-in-MP4 plays fine everywhere gallery-dl was tested from
  except **Safari/WebKit**, which wires VP9 decode only into its WebM
  demuxer, never its MP4/ISOBMFF path — confirmed via WebKit bug trackers,
  not guessed. Fetching a lower-resolution non-VP9 variant instead was
  considered and rejected (quality loss); the fix is downstream, a one-time
  `-c:v copy -c:a libopus` remux to `.webm` (`cosmo_remux_instagram_vp9.py`
  in `Cosmo-Live-Downloads`) that keeps VP9 losslessly and only re-encodes
  audio (WebM disallows AAC). Confirmed live via gallery-dl/yt-dlp that
  Instagram never offers a native WebM option to request instead — this has
  to be done locally, there's no source-side fix.
 ## Scanner work — done
@@ -0,0 +1,184 @@
 slug,username,archive_dir,shortcode,type,date,post_url,json_file
 choerry,cher_ryppo,cher_ryppo,C5SBC--Syik,post,2024-04-03 01:50:25,https://www.instagram.com/p/C5SBC--Syik/,2024-04-02_cher_ryppo - C5SBC--Syik.json
 choerry,cher_ryppo,cher_ryppo,C811ib_ys7D,post,2024-06-30 13:16:41,https://www.instagram.com/p/C811ib_ys7D/,2024-06-30_cher_ryppo - C811ib_ys7D.json
 choerry,cher_ryppo,cher_ryppo,C9JTMgnSCHF,post,2024-07-08 02:41:25,https://www.instagram.com/p/C9JTMgnSCHF/,2024-07-07_cher_ryppo - C9JTMgnSCHF.json
 choerry,cher_ryppo,cher_ryppo,C-K6kxZy6qq,post,2024-08-02 14:16:09,https://www.instagram.com/p/C-K6kxZy6qq/,2024-08-02_cher_ryppo - C-K6kxZy6qq.json
 choerry,cher_ryppo,cher_ryppo,DBXI0zVs_Iv,post,2024-10-20 21:45:44,https://www.instagram.com/p/DBXI0zVs_Iv/,2024-10-20_cher_ryppo - DBXI0zVs_Iv.json
 choerry,cher_ryppo,cher_ryppo,DRRDiezjdyV,post,2025-11-20 05:26:24,https://www.instagram.com/p/DRRDiezjdyV/,2025-11-20_cher_ryppo - DRRDiezjdyV.json
 haseul,withaseul,withaseul,Cqps3ZWBlav,post,2023-04-05 10:44:56,https://www.instagram.com/p/Cqps3ZWBlav/,2023-04-05_withaseul - Cqps3ZWBlav.json
 haseul,withaseul,withaseul,CqsjfUzSelq,post,2023-04-06 13:20:43,https://www.instagram.com/p/CqsjfUzSelq/,2023-04-06_withaseul - CqsjfUzSelq.json
 haseul,withaseul,withaseul,CqxiTfjhNJ2,post,2023-04-08 11:46:34,https://www.instagram.com/p/CqxiTfjhNJ2/,2023-04-08_withaseul - CqxiTfjhNJ2.json
 haseul,withaseul,withaseul,Cq4qymBhftL,post,2023-04-11 06:15:24,https://www.instagram.com/p/Cq4qymBhftL/,2023-04-11_withaseul - Cq4qymBhftL.json
 haseul,withaseul,withaseul,CrnhJtrhoZi,post,2023-04-29 10:55:29,https://www.instagram.com/p/CrnhJtrhoZi/,2023-04-29_withaseul - CrnhJtrhoZi.json
 haseul,withaseul,withaseul,Cr7fZZyB4Sp,post,2023-05-07 05:04:58,https://www.instagram.com/p/Cr7fZZyB4Sp/,2023-05-07_withaseul - Cr7fZZyB4Sp.json
 haseul,withaseul,withaseul,Cr8QeaFhFsq,post,2023-05-07 12:13:49,https://www.instagram.com/p/Cr8QeaFhFsq/,2023-05-07_withaseul - Cr8QeaFhFsq.json
 haseul,withaseul,withaseul,CsBIA4HBUOX,post,2023-05-09 09:36:05,https://www.instagram.com/p/CsBIA4HBUOX/,2023-05-09_withaseul - CsBIA4HBUOX.json
 haseul,withaseul,withaseul,CsgbxFDhNmj,post,2023-05-21 13:25:08,https://www.instagram.com/p/CsgbxFDhNmj/,2023-05-21_withaseul - CsgbxFDhNmj.json
 haseul,withaseul,withaseul,Csn9aO4B1r-,post,2023-05-24 11:33:48,https://www.instagram.com/p/Csn9aO4B1r-/,2023-05-24_withaseul - Csn9aO4B1r-.json
 haseul,withaseul,withaseul,CsvY2jrBjMc,post,2023-05-27 08:48:17,https://www.instagram.com/p/CsvY2jrBjMc/,2023-05-27_withaseul - CsvY2jrBjMc.json
 haseul,withaseul,withaseul,CtEHmRDPesc,post,2023-06-04 10:01:34,https://www.instagram.com/p/CtEHmRDPesc/,2023-06-04_withaseul - CtEHmRDPesc.json
 haseul,withaseul,withaseul,CtRmUjsBUoc,post,2023-06-09 15:40:09,https://www.instagram.com/p/CtRmUjsBUoc/,2023-06-09_withaseul - CtRmUjsBUoc.json
 haseul,withaseul,withaseul,Cuzar_3LqAA,post,2023-07-17 15:24:04,https://www.instagram.com/p/Cuzar_3LqAA/,2023-07-17_withaseul - Cuzar_3LqAA.json
 haseul,withaseul,withaseul,CvHe1VMr7C-,post,2023-07-25 10:25:06,https://www.instagram.com/p/CvHe1VMr7C-/,2023-07-25_withaseul - CvHe1VMr7C-.json
 haseul,withaseul,withaseul,CvhRFq7rnHc,post,2023-08-04 10:45:15,https://www.instagram.com/p/CvhRFq7rnHc/,2023-08-04_withaseul - CvhRFq7rnHc.json
 haseul,withaseul,withaseul,CwIr3qEBPgF,post,2023-08-19 18:09:39,https://www.instagram.com/p/CwIr3qEBPgF/,2023-08-19_withaseul - CwIr3qEBPgF.json
 haseul,withaseul,withaseul,CwzPlMThh5m,post,2023-09-05 06:49:48,https://www.instagram.com/p/CwzPlMThh5m/,2023-09-05_withaseul - CwzPlMThh5m.json
 haseul,withaseul,withaseul,Cw9s3LNBhsq,post,2023-09-09 08:18:04,https://www.instagram.com/p/Cw9s3LNBhsq/,2023-09-09_withaseul - Cw9s3LNBhsq.json
 haseul,withaseul,withaseul,CxIma4nBlaP,post,2023-09-13 13:53:26,https://www.instagram.com/p/CxIma4nBlaP/,2023-09-13_withaseul - CxIma4nBlaP.json
 haseul,withaseul,withaseul,CxLcZ2jBME5,post,2023-09-14 16:23:38,https://www.instagram.com/p/CxLcZ2jBME5/,2023-09-14_withaseul - CxLcZ2jBME5.json
 haseul,withaseul,withaseul,CxN3g95hVDH,post,2023-09-15 14:59:00,https://www.instagram.com/p/CxN3g95hVDH/,2023-09-15_withaseul - CxN3g95hVDH.json
 haseul,withaseul,withaseul,CxTGBb_BAYw,post,2023-09-17 15:41:59,https://www.instagram.com/p/CxTGBb_BAYw/,2023-09-17_withaseul - CxTGBb_BAYw.json
 haseul,withaseul,withaseul,CxnTyJIhUc0,post,2023-09-25 12:07:02,https://www.instagram.com/p/CxnTyJIhUc0/,2023-09-25_withaseul - CxnTyJIhUc0.json
 haseul,withaseul,withaseul,CxsPZAur6Xv,post,2023-09-27 10:04:51,https://www.instagram.com/p/CxsPZAur6Xv/,2023-09-27_withaseul - CxsPZAur6Xv.json
 haseul,withaseul,withaseul,Cx7mOFqhU7R,post,2023-10-03 09:12:57,https://www.instagram.com/p/Cx7mOFqhU7R/,2023-10-03_withaseul - Cx7mOFqhU7R.json
 haseul,withaseul,withaseul,CyqV0lfruX-,post,2023-10-21 12:53:58,https://www.instagram.com/p/CyqV0lfruX-/,2023-10-21_withaseul - CyqV0lfruX-.json
 haseul,withaseul,withaseul,Cy7RByYh2BH,post,2023-10-28 02:39:10,https://www.instagram.com/p/Cy7RByYh2BH/,2023-10-27_withaseul - Cy7RByYh2BH.json
 haseul,withaseul,withaseul,Cz7nefxBJT0,post,2023-11-22 02:26:43,https://www.instagram.com/p/Cz7nefxBJT0/,2023-11-21_withaseul - Cz7nefxBJT0.json
 haseul,withaseul,withaseul,C0YxsROhG4W,post,2023-12-03 10:13:57,https://www.instagram.com/p/C0YxsROhG4W/,2023-12-03_withaseul - C0YxsROhG4W.json
 haseul,withaseul,withaseul,C0zOk2JBXTY,post,2023-12-13 16:46:36,https://www.instagram.com/p/C0zOk2JBXTY/,2023-12-13_withaseul - C0zOk2JBXTY.json
 haseul,withaseul,withaseul,C1q_9TzBrdt,post,2024-01-04 08:36:20,https://www.instagram.com/p/C1q_9TzBrdt/,2024-01-04_withaseul - C1q_9TzBrdt.json
 haseul,withaseul,withaseul,C1t-aeHBCef,post,2024-01-05 12:20:34,https://www.instagram.com/p/C1t-aeHBCef/,2024-01-05_withaseul - C1t-aeHBCef.json
 haseul,withaseul,withaseul,C1yrgmMhsmz,post,2024-01-07 08:11:35,https://www.instagram.com/p/C1yrgmMhsmz/,2024-01-07_withaseul - C1yrgmMhsmz.json
 haseul,withaseul,withaseul,C1yrxqIhGBZ,post,2024-01-07 08:13:54,https://www.instagram.com/p/C1yrxqIhGBZ/,2024-01-07_withaseul - C1yrxqIhGBZ.json
 haseul,withaseul,withaseul,C10p70xh-6x,post,2024-01-08 02:36:18,https://www.instagram.com/p/C10p70xh-6x/,2024-01-07_withaseul - C10p70xh-6x.json
 haseul,withaseul,withaseul,C2APujnBSF-,post,2024-01-12 14:38:11,https://www.instagram.com/p/C2APujnBSF-/,2024-01-12_withaseul - C2APujnBSF-.json
 haseul,withaseul,withaseul,C3enxknhzeF,post,2024-02-18 06:16:55,https://www.instagram.com/p/C3enxknhzeF/,2024-02-18_withaseul - C3enxknhzeF.json
 haseul,withaseul,withaseul,C4f8UQIBZAe,post,2024-03-14 15:07:03,https://www.instagram.com/p/C4f8UQIBZAe/,2024-03-14_withaseul - C4f8UQIBZAe.json
 haseul,withaseul,withaseul,C4qIrJTh3po,post,2024-03-18 14:07:26,https://www.instagram.com/p/C4qIrJTh3po/,2024-03-18_withaseul - C4qIrJTh3po.json
 haseul,withaseul,withaseul,C4rjDNhhJu8,post,2024-03-19 03:17:09,https://www.instagram.com/p/C4rjDNhhJu8/,2024-03-18_withaseul - C4rjDNhhJu8.json
 haseul,withaseul,withaseul,C4xSiGGhP40,post,2024-03-21 08:48:16,https://www.instagram.com/p/C4xSiGGhP40/,2024-03-21_withaseul - C4xSiGGhP40.json
 haseul,withaseul,withaseul,C5GBz4FB-kE,post,2024-03-29 10:06:12,https://www.instagram.com/p/C5GBz4FB-kE/,2024-03-29_withaseul - C5GBz4FB-kE.json
 haseul,withaseul,withaseul,C5GJmlABYug,post,2024-03-29 11:14:17,https://www.instagram.com/p/C5GJmlABYug/,2024-03-29_withaseul - C5GJmlABYug.json
 haseul,withaseul,withaseul,C5LgFFtB8ri,post,2024-03-31 13:06:54,https://www.instagram.com/p/C5LgFFtB8ri/,2024-03-31_withaseul - C5LgFFtB8ri.json
 haseul,withaseul,withaseul,C5VZGPDB-Qo,post,2024-04-04 09:18:17,https://www.instagram.com/p/C5VZGPDB-Qo/,2024-04-04_withaseul - C5VZGPDB-Qo.json
 haseul,withaseul,withaseul - reels,C5bWrT3BaGy,post,2024-04-06 16:52:50,https://www.instagram.com/p/C5bWrT3BaGy/,2024-04-06_withaseul - C5bWrT3BaGy.json
 haseul,withaseul,withaseul,C6LAH8vBs_a,post,2024-04-25 04:59:04,https://www.instagram.com/p/C6LAH8vBs_a/,2024-04-25_withaseul - C6LAH8vBs_a.json
 haseul,withaseul,withaseul,C6ypIzVBmiL,post,2024-05-10 14:27:49,https://www.instagram.com/p/C6ypIzVBmiL/,2024-05-10_withaseul - C6ypIzVBmiL.json
 haseul,withaseul,withaseul,C68cTpzhUzQ,post,2024-05-14 09:48:07,https://www.instagram.com/p/C68cTpzhUzQ/,2024-05-14_withaseul - C68cTpzhUzQ.json
 haseul,withaseul,withaseul,C68oiiahwZd,post,2024-05-14 11:35:00,https://www.instagram.com/p/C68oiiahwZd/,2024-05-14_withaseul - C68oiiahwZd.json
 haseul,withaseul,withaseul,C7Oh6mYB7oF,post,2024-05-21 10:23:27,https://www.instagram.com/p/C7Oh6mYB7oF/,2024-05-21_withaseul - C7Oh6mYB7oF.json
 haseul,withaseul,withaseul - reels,C7RnCv6BZT3,post,2024-05-22 15:06:48,https://www.instagram.com/p/C7RnCv6BZT3/,2024-05-22_withaseul - C7RnCv6BZT3.json
 haseul,withaseul,withaseul,C7R7owoBd4n,post,2024-05-22 18:05:56,https://www.instagram.com/p/C7R7owoBd4n/,2024-05-22_withaseul - C7R7owoBd4n.json
 haseul,withaseul,withaseul,C7mPzLRhCRz,post,2024-05-30 15:26:55,https://www.instagram.com/p/C7mPzLRhCRz/,2024-05-30_withaseul - C7mPzLRhCRz.json
 haseul,withaseul,withaseul,C7tNIlPh7NK,post,2024-06-02 08:18:19,https://www.instagram.com/p/C7tNIlPh7NK/,2024-06-02_withaseul - C7tNIlPh7NK.json
 haseul,withaseul,withaseul,C70yRTmBgng,post,2024-06-05 06:57:30,https://www.instagram.com/p/C70yRTmBgng/,2024-06-05_withaseul - C70yRTmBgng.json
 haseul,withaseul,withaseul,C71aPMWBfqs,post,2024-06-05 12:46:44,https://www.instagram.com/p/C71aPMWBfqs/,2024-06-05_withaseul - C71aPMWBfqs.json
 haseul,withaseul,withaseul,C8NBuqRhCJv,post,2024-06-14 16:54:21,https://www.instagram.com/p/C8NBuqRhCJv/,2024-06-14_withaseul - C8NBuqRhCJv.json
 haseul,withaseul,withaseul,C8PPpUKhyLf,post,2024-06-15 13:34:26,https://www.instagram.com/p/C8PPpUKhyLf/,2024-06-15_withaseul - C8PPpUKhyLf.json
 haseul,withaseul,withaseul,C8UjYG8B74s,post,2024-06-17 15:03:03,https://www.instagram.com/p/C8UjYG8B74s/,2024-06-17_withaseul - C8UjYG8B74s.json
 haseul,withaseul,withaseul,C87PlKyBTU6,post,2024-07-02 15:40:27,https://www.instagram.com/p/C87PlKyBTU6/,2024-07-02_withaseul - C87PlKyBTU6.json
 haseul,withaseul,withaseul,C9FqomJhky4,post,2024-07-06 16:49:16,https://www.instagram.com/p/C9FqomJhky4/,2024-07-06_withaseul - C9FqomJhky4.json
 haseul,withaseul,withaseul,C9ICMZ5BUfr,post,2024-07-07 14:53:36,https://www.instagram.com/p/C9ICMZ5BUfr/,2024-07-07_withaseul - C9ICMZ5BUfr.json
 haseul,withaseul,withaseul,C9V8bjJBPdk,post,2024-07-13 00:32:37,https://www.instagram.com/p/C9V8bjJBPdk/,2024-07-12_withaseul - C9V8bjJBPdk.json
 haseul,withaseul,withaseul,C9xK8b1SZAY,post,2024-07-23 14:18:56,https://www.instagram.com/p/C9xK8b1SZAY/,2024-07-23_withaseul - C9xK8b1SZAY.json
 haseul,withaseul,withaseul,C-DLjJ6BE0y,post,2024-07-30 14:10:33,https://www.instagram.com/p/C-DLjJ6BE0y/,2024-07-30_withaseul - C-DLjJ6BE0y.json
 haseul,withaseul,withaseul,C-Ncp1MyhiD,post,2024-08-03 13:52:25,https://www.instagram.com/p/C-Ncp1MyhiD/,2024-08-03_withaseul - C-Ncp1MyhiD.json
 haseul,withaseul,withaseul,C_A4QkApuLO,post,2024-08-23 13:14:54,https://www.instagram.com/p/C_A4QkApuLO/,2024-08-23_withaseul - C_A4QkApuLO.json
 haseul,withaseul,withaseul,C_CcGblt8X9,post,2024-08-24 03:47:20,https://www.instagram.com/p/C_CcGblt8X9/,2024-08-23_withaseul - C_CcGblt8X9.json
 haseul,withaseul,withaseul,C_ZCyXPylSu,post,2024-09-01 22:28:40,https://www.instagram.com/p/C_ZCyXPylSu/,2024-09-01_withaseul - C_ZCyXPylSu.json
 haseul,withaseul,withaseul,C_gRp4gPuY3,post,2024-09-04 17:53:16,https://www.instagram.com/p/C_gRp4gPuY3/,2024-09-04_withaseul - C_gRp4gPuY3.json
 haseul,withaseul,withaseul,C_7XyvJMmm7,post,2024-09-15 06:26:24,https://www.instagram.com/p/C_7XyvJMmm7/,2024-09-15_withaseul - C_7XyvJMmm7.json
 haseul,withaseul,withaseul,DACAVOGyOEc,post,2024-09-17 20:16:04,https://www.instagram.com/p/DACAVOGyOEc/,2024-09-17_withaseul - DACAVOGyOEc.json
 haseul,withaseul,withaseul,DAqwdZIBjY5,post,2024-10-03 16:06:14,https://www.instagram.com/p/DAqwdZIBjY5/,2024-10-03_withaseul - DAqwdZIBjY5.json
 haseul,withaseul,withaseul,DAv8DxvyGbr,post,2024-10-05 16:23:48,https://www.instagram.com/p/DAv8DxvyGbr/,2024-10-05_withaseul - DAv8DxvyGbr.json
 haseul,withaseul,withaseul,DBhdsL1NIvC,post,2024-10-24 22:00:28,https://www.instagram.com/p/DBhdsL1NIvC/,2024-10-24_withaseul - DBhdsL1NIvC.json
 haseul,withaseul,withaseul,DBrP-pYSDEn,post,2024-10-28 17:13:03,https://www.instagram.com/p/DBrP-pYSDEn/,2024-10-28_withaseul - DBrP-pYSDEn.json
 haseul,withaseul,withaseul,DBu5zCHBgrT,post,2024-10-30 03:16:12,https://www.instagram.com/p/DBu5zCHBgrT/,2024-10-29_withaseul - DBu5zCHBgrT.json
 haseul,withaseul,withaseul,DByVUvOBSQy,post,2024-10-31 11:14:27,https://www.instagram.com/p/DByVUvOBSQy/,2024-10-31_withaseul - DByVUvOBSQy.json
 haseul,withaseul,withaseul,DB5re5DhW8a,post,2024-11-03 07:42:45,https://www.instagram.com/p/DB5re5DhW8a/,2024-11-03_withaseul - DB5re5DhW8a.json
 haseul,withaseul,withaseul,DCBsfwGhFYd,post,2024-11-06 10:25:32,https://www.instagram.com/p/DCBsfwGhFYd/,2024-11-06_withaseul - DCBsfwGhFYd.json
 haseul,withaseul,withaseul,DCObB0SR64m,post,2024-11-11 09:03:02,https://www.instagram.com/p/DCObB0SR64m/,2024-11-11_withaseul - DCObB0SR64m.json
 haseul,withaseul,withaseul,DCRhOPphLWU,post,2024-11-12 13:54:52,https://www.instagram.com/p/DCRhOPphLWU/,2024-11-12_withaseul - DCRhOPphLWU.json
 haseul,withaseul,withaseul,DCbgyVghiT-,post,2024-11-16 11:03:28,https://www.instagram.com/p/DCbgyVghiT-/,2024-11-16_withaseul - DCbgyVghiT-.json
 haseul,withaseul,withaseul,DCe4qpXhgBV,post,2024-11-17 18:29:51,https://www.instagram.com/p/DCe4qpXhgBV/,2024-11-17_withaseul - DCe4qpXhgBV.json
 haseul,withaseul,withaseul,DC_4KddBmo_,post,2024-11-30 14:00:24,https://www.instagram.com/p/DC_4KddBmo_/,2024-11-30_withaseul - DC_4KddBmo_.json
 haseul,withaseul,withaseul,DDRNi9Bhicn,post,2024-12-07 07:34:20,https://www.instagram.com/p/DDRNi9Bhicn/,2024-12-07_withaseul - DDRNi9Bhicn.json
 haseul,withaseul,withaseul,DDW2JdFBUMs,post,2024-12-09 12:05:19,https://www.instagram.com/p/DDW2JdFBUMs/,2024-12-09_withaseul - DDW2JdFBUMs.json
 haseul,withaseul,withaseul,DDjWETEBWw2,post,2024-12-14 08:35:07,https://www.instagram.com/p/DDjWETEBWw2/,2024-12-14_withaseul - DDjWETEBWw2.json
 haseul,withaseul,withaseul,DDpkLTyhUdt,post,2024-12-16 18:33:51,https://www.instagram.com/p/DDpkLTyhUdt/,2024-12-16_withaseul - DDpkLTyhUdt.json
 haseul,withaseul,withaseul,DE1NVFihF2g,post,2025-01-15 03:36:30,https://www.instagram.com/p/DE1NVFihF2g/,2025-01-14_withaseul - DE1NVFihF2g.json
 haseul,withaseul,withaseul,DE5GO2-S5PB,post,2025-01-16 15:51:26,https://www.instagram.com/p/DE5GO2-S5PB/,2025-01-16_withaseul - DE5GO2-S5PB.json
 haseul,withaseul,withaseul - reels,DE5H7LcB5aX,post,2025-01-16 16:07:31,https://www.instagram.com/p/DE5H7LcB5aX/,2025-01-16_withaseul - DE5H7LcB5aX.json
 haseul,withaseul,withaseul,DFFtju9hTdv,post,2025-01-21 13:25:58,https://www.instagram.com/p/DFFtju9hTdv/,2025-01-21_withaseul - DFFtju9hTdv.json
 haseul,withaseul,withaseul,DFsLxphhwqE,post,2025-02-05 12:01:09,https://www.instagram.com/p/DFsLxphhwqE/,2025-02-05_withaseul - DFsLxphhwqE.json
 haseul,withaseul,withaseul,DHvlOgAxE_j,post,2025-03-28 12:44:03,https://www.instagram.com/p/DHvlOgAxE_j/,2025-03-28_withaseul - DHvlOgAxE_j.json
 haseul,withaseul,withaseul,DH96B0QtxfT,post,2025-04-03 02:15:11,https://www.instagram.com/p/DH96B0QtxfT/,2025-04-02_withaseul - DH96B0QtxfT.json
 haseul,withaseul,withaseul,DIL6wNXRGDC,post,2025-04-08 12:50:53,https://www.instagram.com/p/DIL6wNXRGDC/,2025-04-08_withaseul - DIL6wNXRGDC.json
 haseul,withaseul,withaseul,DIP1mift6QV,post,2025-04-10 01:22:50,https://www.instagram.com/p/DIP1mift6QV/,2025-04-09_withaseul - DIP1mift6QV.json
 haseul,withaseul,withaseul,DIfvDIKxwuT,post,2025-04-16 05:33:25,https://www.instagram.com/p/DIfvDIKxwuT/,2025-04-16_withaseul - DIfvDIKxwuT.json
 haseul,withaseul,withaseul,DIm-MshvCJy,post,2025-04-19 01:00:03,https://www.instagram.com/p/DIm-MshvCJy/,2025-04-18_withaseul - DIm-MshvCJy.json
 haseul,withaseul,withaseul,DIuQyfphWnR,post,2025-04-21 20:57:37,https://www.instagram.com/p/DIuQyfphWnR/,2025-04-21_withaseul - DIuQyfphWnR.json
 haseul,withaseul,withaseul,DI3hOzrBLs-,post,2025-04-25 11:14:27,https://www.instagram.com/p/DI3hOzrBLs-/,2025-04-25_withaseul - DI3hOzrBLs-.json
 haseul,withaseul,withaseul,DK0lklBhrH6,post,2025-06-13 00:57:27,https://www.instagram.com/p/DK0lklBhrH6/,2025-06-12_withaseul - DK0lklBhrH6.json
 haseul,withaseul,withaseul,DK1EdUZBJtr,post,2025-06-13 05:27:20,https://www.instagram.com/p/DK1EdUZBJtr/,2025-06-13_withaseul - DK1EdUZBJtr.json
 haseul,withaseul,withaseul,DLJiMzrhM83,post,2025-06-21 04:12:02,https://www.instagram.com/p/DLJiMzrhM83/,2025-06-21_withaseul - DLJiMzrhM83.json
 haseul,withaseul,withaseul,DLL1iePBcwm,post,2025-06-22 01:39:30,https://www.instagram.com/p/DLL1iePBcwm/,2025-06-21_withaseul - DLL1iePBcwm.json
 haseul,withaseul,withaseul,DLM-sUIhK5P,post,2025-06-22 12:18:44,https://www.instagram.com/p/DLM-sUIhK5P/,2025-06-22_withaseul - DLM-sUIhK5P.json
 haseul,withaseul,withaseul,DLU66yCvWZh,post,2025-06-25 14:19:41,https://www.instagram.com/p/DLU66yCvWZh/,2025-06-25_withaseul - DLU66yCvWZh.json
 haseul,withaseul,withaseul,DLxAZg5hpnW,post,2025-07-06 12:06:18,https://www.instagram.com/p/DLxAZg5hpnW/,2025-07-06_withaseul - DLxAZg5hpnW.json
 haseul,withaseul,withaseul,DMFfDqABJgM,post,2025-07-14 10:59:00,https://www.instagram.com/p/DMFfDqABJgM/,2025-07-14_withaseul - DMFfDqABJgM.json
 haseul,withaseul,withaseul,DM0NH9yhkzT,post,2025-08-01 14:26:37,https://www.instagram.com/p/DM0NH9yhkzT/,2025-08-01_withaseul - DM0NH9yhkzT.json
 haseul,withaseul,withaseul,DM9_Ol7SzmK,post,2025-08-05 09:37:35,https://www.instagram.com/p/DM9_Ol7SzmK/,2025-08-05_withaseul - DM9_Ol7SzmK.json
 haseul,withaseul,withaseul,DNnih4pBd_M,post,2025-08-21 12:54:55,https://www.instagram.com/p/DNnih4pBd_M/,2025-08-21_withaseul - DNnih4pBd_M.json
 haseul,withaseul,withaseul,DOIxptrgXgv,post,2025-09-03 10:42:00,https://www.instagram.com/p/DOIxptrgXgv/,2025-09-03_withaseul - DOIxptrgXgv.json
 haseul,withaseul,withaseul,DOa8a_ugc2Z,post,2025-09-10 12:02:26,https://www.instagram.com/p/DOa8a_ugc2Z/,2025-09-10_withaseul - DOa8a_ugc2Z.json
 haseul,withaseul,withaseul,DPMLxsmgUJJ,post,2025-09-29 14:59:24,https://www.instagram.com/p/DPMLxsmgUJJ/,2025-09-29_withaseul - DPMLxsmgUJJ.json
 haseul,withaseul,withaseul,DPRez1QgUxE,post,2025-10-01 16:21:55,https://www.instagram.com/p/DPRez1QgUxE/,2025-10-01_withaseul - DPRez1QgUxE.json
 haseul,withaseul,withaseul,DQg9WOOgaKD,post,2025-11-01 13:08:45,https://www.instagram.com/p/DQg9WOOgaKD/,2025-11-01_withaseul - DQg9WOOgaKD.json
 haseul,withaseul,withaseul,DQ_JrMdDciC,post,2025-11-13 06:33:42,https://www.instagram.com/p/DQ_JrMdDciC/,2025-11-13_withaseul - DQ_JrMdDciC.json
 haseul,withaseul,withaseul,DROhLjLjVqY,post,2025-11-19 05:47:42,https://www.instagram.com/p/DROhLjLjVqY/,2025-11-19_withaseul - DROhLjLjVqY.json
 haseul,withaseul,withaseul,DRwiTEDDRHR,post,2025-12-02 10:51:38,https://www.instagram.com/p/DRwiTEDDRHR/,2025-12-02_withaseul - DRwiTEDDRHR.json
 haseul,withaseul,withaseul,DTw-4d9gRUx,post,2026-01-21 08:04:12,https://www.instagram.com/p/DTw-4d9gRUx/,2026-01-21_withaseul - DTw-4d9gRUx.json
 haseul,withaseul,withaseul,DT0rHDCjPpE,post,2026-01-22 18:28:24,https://www.instagram.com/p/DT0rHDCjPpE/,2026-01-22_withaseul - DT0rHDCjPpE.json
 haseul,withaseul,withaseul,DT6HqajDJfF,post,2026-01-24 21:14:05,https://www.instagram.com/p/DT6HqajDJfF/,2026-01-24_withaseul - DT6HqajDJfF.json
 haseul,withaseul,withaseul,DU0oU25gWso,post,2026-02-16 14:19:54,https://www.instagram.com/p/DU0oU25gWso/,2026-02-16_withaseul - DU0oU25gWso.json
 haseul,withaseul,withaseul,DWZPNAKAZcl,post,2026-03-27 16:19:14,https://www.instagram.com/p/DWZPNAKAZcl/,2026-03-27_withaseul - DWZPNAKAZcl.json
 haseul,withaseul,withaseul,DW3yBm2gctK,post,2026-04-08 13:00:00,https://www.instagram.com/p/DW3yBm2gctK/,2026-04-08_withaseul - DW3yBm2gctK.json
 haseul,withaseul,withaseul,DZj6uVWhV8M,post,2026-06-14 09:26:10,https://www.instagram.com/p/DZj6uVWhV8M/,2026-06-14_withaseul - DZj6uVWhV8M.json
 haseul,withaseul,withaseul,DaZm_xLAZeL,post,2026-07-05 05:52:43,https://www.instagram.com/p/DaZm_xLAZeL/,2026-07-05_withaseul - DaZm_xLAZeL.json
 haseul,withaseul,withaseul,Dbnq4k9gb5U,post,2026-08-04 13:27:27,https://www.instagram.com/p/Dbnq4k9gb5U/,2026-08-04_withaseul - Dbnq4k9gb5U.json
 heejin,0ct0ber19,0ct0ber19,CrdsY5CrSsO,post,2023-04-25 15:21:16,https://www.instagram.com/p/CrdsY5CrSsO/,2023-04-25_0ct0ber19 - CrdsY5CrSsO.json
 heejin,0ct0ber19,0ct0ber19,CtohvHxLnWO,post,2023-06-18 13:22:37,https://www.instagram.com/p/CtohvHxLnWO/,2023-06-18_0ct0ber19 - CtohvHxLnWO.json
 heejin,0ct0ber19,0ct0ber19,CuEOMWppp5S,post,2023-06-29 07:30:35,https://www.instagram.com/p/CuEOMWppp5S/,2023-06-29_0ct0ber19 - CuEOMWppp5S.json
 heejin,0ct0ber19,0ct0ber19,CwC0Y-prGoB,post,2023-08-17 11:28:40,https://www.instagram.com/p/CwC0Y-prGoB/,2023-08-17_0ct0ber19 - CwC0Y-prGoB.json
 heejin,0ct0ber19,0ct0ber19,CxsKPWgpzU4,post,2023-09-27 09:19:51,https://www.instagram.com/p/CxsKPWgpzU4/,2023-09-27_0ct0ber19 - CxsKPWgpzU4.json
 heejin,0ct0ber19,0ct0ber19,CzJf78xryub,post,2023-11-02 15:18:48,https://www.instagram.com/p/CzJf78xryub/,2023-11-02_0ct0ber19 - CzJf78xryub.json
 heejin,0ct0ber19,0ct0ber19,CzOVVzFLTnD,post,2023-11-04 12:22:25,https://www.instagram.com/p/CzOVVzFLTnD/,2023-11-04_0ct0ber19 - CzOVVzFLTnD.json
 heejin,0ct0ber19,0ct0ber19,C14VKVmpdtC,post,2024-01-09 12:51:44,https://www.instagram.com/p/C14VKVmpdtC/,2024-01-09_0ct0ber19 - C14VKVmpdtC.json
 heejin,0ct0ber19,0ct0ber19,C2FnIelpue9,post,2024-01-14 16:38:54,https://www.instagram.com/p/C2FnIelpue9/,2024-01-14_0ct0ber19 - C2FnIelpue9.json
 heejin,0ct0ber19,0ct0ber19,C5SDY_UJIn6,post,2024-04-03 02:10:53,https://www.instagram.com/p/C5SDY_UJIn6/,2024-04-02_0ct0ber19 - C5SDY_UJIn6.json
 heejin,0ct0ber19,0ct0ber19,C5WB6zwp_rX,post,2024-04-04 15:15:00,https://www.instagram.com/p/C5WB6zwp_rX/,2024-04-04_0ct0ber19 - C5WB6zwp_rX.json
 heejin,0ct0ber19,0ct0ber19,C5dJ0xXpJpX,post,2024-04-07 09:38:46,https://www.instagram.com/p/C5dJ0xXpJpX/,2024-04-07_0ct0ber19 - C5dJ0xXpJpX.json
 heejin,0ct0ber19,0ct0ber19,C5oJdT-pQBI,post,2024-04-11 16:07:12,https://www.instagram.com/p/C5oJdT-pQBI/,2024-04-11_0ct0ber19 - C5oJdT-pQBI.json
 heejin,0ct0ber19,0ct0ber19,C53YPQzp7Wj,post,2024-04-17 14:04:58,https://www.instagram.com/p/C53YPQzp7Wj/,2024-04-17_0ct0ber19 - C53YPQzp7Wj.json
 heejin,0ct0ber19,0ct0ber19,C5--hdUJKZH,post,2024-04-20 12:54:11,https://www.instagram.com/p/C5--hdUJKZH/,2024-04-20_0ct0ber19 - C5--hdUJKZH.json
 heejin,0ct0ber19,0ct0ber19,C71iAmbpekA,post,2024-06-05 13:54:39,https://www.instagram.com/p/C71iAmbpekA/,2024-06-05_0ct0ber19 - C71iAmbpekA.json
 heejin,0ct0ber19,0ct0ber19,C8M8TxnSOKI,post,2024-06-14 16:07:00,https://www.instagram.com/p/C8M8TxnSOKI/,2024-06-14_0ct0ber19 - C8M8TxnSOKI.json
 heejin,0ct0ber19,0ct0ber19,C8hxpuYyWGv,post,2024-06-22 18:17:55,https://www.instagram.com/p/C8hxpuYyWGv/,2024-06-22_0ct0ber19 - C8hxpuYyWGv.json
 heejin,0ct0ber19,0ct0ber19,C-AbtWSJbHB,post,2024-07-29 12:34:02,https://www.instagram.com/p/C-AbtWSJbHB/,2024-07-29_0ct0ber19 - C-AbtWSJbHB.json
 heejin,0ct0ber19,0ct0ber19,C_WxFtuvqXK,post,2024-09-01 01:15:32,https://www.instagram.com/p/C_WxFtuvqXK/,2024-08-31_0ct0ber19 - C_WxFtuvqXK.json
 heejin,0ct0ber19,0ct0ber19,DcgTbz-iTAH,post,2026-08-26 13:19:15,https://www.instagram.com/p/DcgTbz-iTAH/,2026-08-26_0ct0ber19 - DcgTbz-iTAH.json
 jinsoul,zindoriyam,zindoriyam,CndmboMBHeB,post,2023-01-16 04:23:38,https://www.instagram.com/p/CndmboMBHeB/,2023-01-15_zindoriyam - CndmboMBHeB.json
 jinsoul,zindoriyam,zindoriyam,Ctb5AnZxCwh,post,2023-06-13 15:35:51,https://www.instagram.com/p/Ctb5AnZxCwh/,2023-06-13_zindoriyam - Ctb5AnZxCwh.json
 jinsoul,zindoriyam,zindoriyam,Cu6ZRz8hj_T,post,2023-07-20 08:26:26,https://www.instagram.com/p/Cu6ZRz8hj_T/,2023-07-20_zindoriyam - Cu6ZRz8hj_T.json
 jinsoul,zindoriyam,zindoriyam,Cwrydt6OXmo,post,2023-09-02 09:20:42,https://www.instagram.com/p/Cwrydt6OXmo/,2023-09-02_zindoriyam - Cwrydt6OXmo.json
 jinsoul,zindoriyam,zindoriyam,Cxsj3_IhwNv,post,2023-09-27 13:03:51,https://www.instagram.com/p/Cxsj3_IhwNv/,2023-09-27_zindoriyam - Cxsj3_IhwNv.json
 jinsoul,zindoriyam,zindoriyam,Cza0C4gBkb5,post,2023-11-09 08:41:36,https://www.instagram.com/p/Cza0C4gBkb5/,2023-11-09_zindoriyam - Cza0C4gBkb5.json
 jinsoul,zindoriyam,zindoriyam,C5LqGJUBTIK,post,2024-03-31 14:34:25,https://www.instagram.com/p/C5LqGJUBTIK/,2024-03-31_zindoriyam - C5LqGJUBTIK.json
 jinsoul,zindoriyam,zindoriyam,C5QZg9BBc12,post,2024-04-02 10:45:44,https://www.instagram.com/p/C5QZg9BBc12/,2024-04-02_zindoriyam - C5QZg9BBc12.json
 jinsoul,zindoriyam,zindoriyam,C8NAliShdHm,post,2024-06-14 16:44:22,https://www.instagram.com/p/C8NAliShdHm/,2024-06-14_zindoriyam - C8NAliShdHm.json
 jinsoul,zindoriyam,zindoriyam,C9-Ot__hyGY,post,2024-07-28 16:02:02,https://www.instagram.com/p/C9-Ot__hyGY/,2024-07-28_zindoriyam - C9-Ot__hyGY.json
 jinsoul,zindoriyam,zindoriyam,DANQU24um-N,post,2024-09-22 05:07:29,https://www.instagram.com/p/DANQU24um-N/,2024-09-22_zindoriyam - DANQU24um-N.json
 jinsoul,zindoriyam,zindoriyam,DHz88e2sS_J,post,2025-03-30 05:28:16,https://www.instagram.com/p/DHz88e2sS_J/,2025-03-30_zindoriyam - DHz88e2sS_J.json
 jinsoul,zindoriyam,zindoriyam,DJXAVrANWfp,post,2025-05-07 16:42:44,https://www.instagram.com/p/DJXAVrANWfp/,2025-05-07_zindoriyam - DJXAVrANWfp.json
 jinsoul,zindoriyam,zindoriyam,DLhdKkZJ2_5,post,2025-06-30 11:09:49,https://www.instagram.com/p/DLhdKkZJ2_5/,2025-06-30_zindoriyam - DLhdKkZJ2_5.json
 jinsoul,zindoriyam,zindoriyam,DMiZJbaJZXv,post,2025-07-25 16:25:21,https://www.instagram.com/p/DMiZJbaJZXv/,2025-07-25_zindoriyam - DMiZJbaJZXv.json
 jinsoul,zindoriyam,zindoriyam,DNJCd9jpT-J,post,2025-08-09 16:37:33,https://www.instagram.com/p/DNJCd9jpT-J/,2025-08-09_zindoriyam - DNJCd9jpT-J.json
 jinsoul,zindoriyam,zindoriyam,DRl8jAxFnwE,post,2025-11-28 08:09:22,https://www.instagram.com/p/DRl8jAxFnwE/,2025-11-28_zindoriyam - DRl8jAxFnwE.json
 jinsoul,zindoriyam,zindoriyam,DTQR5cDk6kY,post,2026-01-08 15:15:25,https://www.instagram.com/p/DTQR5cDk6kY/,2026-01-08_zindoriyam - DTQR5cDk6kY.json
 jinsoul,zindoriyam,zindoriyam,DVv1oRSEyIy,post,2026-03-11 14:26:54,https://www.instagram.com/p/DVv1oRSEyIy/,2026-03-11_zindoriyam - DVv1oRSEyIy.json
 jinsoul,zindoriyam,zindoriyam,DW_6oxSkz8u,post,2026-04-11 16:49:54,https://www.instagram.com/p/DW_6oxSkz8u/,2026-04-11_zindoriyam - DW_6oxSkz8u.json
 jinsoul,zindoriyam,zindoriyam,DcRLAVJEwlG,post,2026-08-20 16:16:59,https://www.instagram.com/p/DcRLAVJEwlG/,2026-08-20_zindoriyam - DcRLAVJEwlG.json
 kimlip,kimxxlip,kimxxlip,CndmBanNyrR,post,2023-01-16 04:20:04,https://www.instagram.com/p/CndmBanNyrR/,2023-01-15_kimxxlip - CndmBanNyrR.json
 kimlip,kimxxlip,kimxxlip,CwXG1L3vQFH,post,2023-08-25 08:34:37,https://www.instagram.com/p/CwXG1L3vQFH/,2023-08-25_kimxxlip - CwXG1L3vQFH.json
 kimlip,kimxxlip,kimxxlip,CxsL_FQvHna,post,2023-09-27 09:35:06,https://www.instagram.com/p/CxsL_FQvHna/,2023-09-27_kimxxlip - CxsL_FQvHna.json
@@ -0,0 +1,85 @@
 # Video upgrade pass — plan and findings (2026-10-14)
 **Status: designed and partly measured, NOT built. Nothing in the archive has been changed.**
 Written so this can be picked up cold. Account context: the scraping account is the
 fresh `erg05teur` (see `docs/gallery-dl.md`, top of "The account was suspended…") — every
 request in this plan is spent on a new account, so the budget rules there apply in full.
 ## Why this exists
 The question was whether a logged-out fetch had given us lower-resolution media and whether a
 logged-in run could upgrade it. Measured result:
 - **Images: no upgrade needed.** Header-scan of 38.9k archived images, plus live metadata for 8
  posts across 2023–2026 (including the oddly small ones): archived sizes match exactly what a
  logged-in fetch offers (1440×1800, 3277×4096, 2976×3720; 400×400 / 742×928 items are small at
  the source, sitting next to 1440 px siblings in the same carousel).
 - **Logged-out gdl does not download smaller files — it fails** (login redirect). No logged-out
  media ever entered the archive.
 - **Videos: this is where there is something to gain.** Of ~1,300 archived videos in the six scraped
  profiles, ~1,200 are 720p (min side ≤ 720) and ~100 (2026 reels) are 1080p. A 2026 reel
  (`DTpycSlDKHs`, archived 720×1280) offers **1080×1920 VP9 video-only DASH + separate AAC audio**.
  A 2024 post (`C7RmlCcSWNR`) tops out at 1276×720 — same as the archive — so older videos
  probably gain nothing. Expected beneficiaries: **2025–26 videos, roughly 700 posts**. Not
  verified how far back 1080p exists.
 ## Decisions already made (by the owner)
 1. **Videos in a separate pass** from anything else (images need none).
 2. **Keep the replaced files for review** in a `_superseded/` area — never overwrite/delete.
 3. **Output `.webm`, not `.mp4`** (video stream copied, audio re-encoded AAC → Opus 96 kbps; the
   audio stream is ~43 kbps HE-AAC so the loss is negligible). **Tested on real hardware:**
   iPhone, iOS 18.7 Safari plays the VP9+Opus `.webm` at 1080×1920 and **fails the VP9+AAC `.mp4`**
   (`MediaError` code 4, no video track) — even though `canPlayType` reports "probably" for it, so
   never trust `canPlayType` here. VP9-in-MP4 (the lossless remux) is therefore out.
   Re-encoding VP9 → H.264 was rejected (bigger, softer, no gain over the 720p original).
 ## How the sample was produced (reproducible, one post)
 ```sh
 # portrait video: 1080 is the WIDTH — `height=1080` selects nothing and fails "format not available"
 yt-dlp --cookies-from-browser chrome:/home/matt/.config/google-chrome-devtools \
  -f "bv*[width>=1080]+ba" --merge-output-format mp4 -o 'X.%(ext)s' https://www.instagram.com/p/<shortcode>/
 ffmpeg -i X.mp4 -c:v copy -c:a libopus -b:a 96k X.webm        # ffmpeg 7.1.5 on mattellite
 ```
 Sizes for that post: 6.46 MB `.mp4` / 6.53 MB `.webm`. For a browser test you need HTTP **Range**
 support (Python's `http.server` has none; Safari requires it) — the throwaway server used is not kept.
 `gallery-dl` needs `$HOME/.local/bin` on PATH over plain ssh.
 ## Open decisions / work before building
 | item | notes |
 |---|---|
 | **Viewer MIME type** | `src/lib/archive-cache.ts:127` and `src/hooks/useArchiveScanner.ts:272,344,391` mint every video blob URL as `video/mp4`. Derive the type from the extension (`.webm` → `video/webm`) or Safari/iOS may refuse a WebM labelled mp4. `MEDIA_RE` / `isVideo` already recognise `.webm`. |
 | **Filename / dedup** | The archive names `YYYY-MM-DD_<user> - <shortcode>[ - <n>].mp4`. Swapping to `.webm` must not leave **both** in `archives/` (the viewer would show duplicates) — the old `.mp4` has to be *moved out*, not just joined by a `.webm`. Check `parseArchiveFilename` (`src/lib/archive-patterns.ts`) for the extension handling and the JD2-vs-gdl same-stem duplicate logic. |
 | **Skip-archive** | `artms.db` keys on `instagram<media_id>`, so a normal run will never re-fetch these. The upgrade tool must fetch targeted items deliberately and must not touch/poison `artms.db` semantics. |
 | **`_superseded/` location** | Must be **outside** `/volume1/rslsync/sync/Instagram-archive/archives/` — that tree is scanned by the viewer **and** inside the Resilio-synced share (it would sync every replaced file to devices). Not decided. Also unchecked: whether `agentapi` can write outside that share. Use rsync `--backup --backup-dir=<dir>/<date>/` so the old copy is kept. |
 | **Carousel videos** | yt-dlp on a carousel post URL fails ("No video formats found") — see `DIa5NpWRnIP`. gdl's own metadata gives per-child `ytdl:` URLs for videos in carousels; the tool needs to handle child items, not post URLs. |
 | **Request budget** | One `yt-dlp -F`-style request per post to learn what 1080p exists (plus the download). ~700 candidates ⇒ suggested **≤ 40 posts/day**, resumable, behind `check-session.py`, hand-started or a slow timer — never a burst. Candidates can be pre-ranked from the archive alone (720p, 2025–26) at zero request cost. |
 | **Storage** | 1080p VP9 ≈ 1.6× the 720p file (6 MB vs ~4 MB for that sample, plus the kept original). Estimate before running at scale. |
 ## Planned design
 1. **Candidate list, zero requests:** scan the NAS archive (read-only, ssh `agentapi@<nas>` from
   mattellite; the scans in this doc used header parsing in plain Python over stdin because the
   NAS has no ffprobe) for videos ≤ 720 min-side from 2025-01 onward, newest first.
 2. **Dry-run mode (default):** for each candidate, one request to learn the best offered stream,
   record `{shortcode, archived WxH, offered WxH}` in a state file; print counts + estimated GB.
   Downloads nothing.
 3. **Execute mode:** only where offered width > archived width; download video+audio, `ffmpeg
   -c:v copy -c:a libopus -b:a 96k`, verify with `ffprobe` (dimensions, duration within 0.5 s of
   the original, both streams present), publish the `.webm`, move the old file to `_superseded/`.
   Cap per run (`--max-posts`), pace with the same 12–20 s gaps, stop on any 429 / login redirect.
 4. **First step when resuming:** do ONE reel end to end by hand (`DTpycSlDKHs`, ~3 requests) to prove
   download → remux → verify → publish → supersede → viewer playback, *then* the dry-run, and only after
   the weekly profiles timer has run cleanly on the new account (first fire ≈ 2026-10-23).
 ## Things measured along the way (for reference)
 - Image header scan by year (six scraped profiles): ≥ 1440 px for the vast majority; ~75 images
  < 1000 px, all checked samples genuinely small at the source. `official_artms` 2023–24 has a
  cluster < 1440 px (~1080-class originals) — not probed.
 - Video min-side classes (six profiles): 2023 ≤720: 304, 2024 ≤720: 248, 2025 ≤720: 368 (+1 at
  1080), 2026 ≤720: 302 and ≤1080: 100 (+1 at 1440×2560). A handful ≤540.
 - The scan tools (read-only, stdlib only, run on the NAS user over ssh stdin) were throwaway
  and are not in the repo; recreate from this description if needed.
@@ -0,0 +1,70 @@
 #!/usr/bin/env python3
 """
 Preflight for gdl-cron.sh: is there a usable Instagram login in the cookie jar gdl reads?
 gdl-sync.py runs gallery-dl with `--cookies-from-browser chrome:<profile>`, so what matters
 is not what the interactive Chrome window LOOKS like (an open tab keeps showing its logged-in
 UI long after the session died -- found 2026-10-14, stories had been empty for a week) but
 whether the profile's cookie DB holds a `sessionid`. This reads the same DB, the same way.
 Never prints cookie values -- only the name, expiry, and a verdict.
 Exit 0  logged in (a non-expired sessionid exists)
 Exit 1  logged out / unreadable -- the caller must not scrape
 """
 import datetime as dt
 import os
 import shutil
 import sqlite3
 import sys
 import tempfile
 import time
 from pathlib import Path
 PROFILE = Path(os.environ.get("GDL_COOKIE_PROFILE",
                              str(Path.home() / ".config/google-chrome-devtools")))
 WARN_DAYS = 7
 def main() -> int:
    src = PROFILE / "Default" / "Cookies"
    if not src.exists():
        print(f"no cookie DB at {src} -- Chrome profile missing?")
        return 1
    with tempfile.TemporaryDirectory() as tmp:
        # Copy first: Chrome holds the live DB open, and a copy keeps this read-only.
        for suffix in ("", "-journal", "-wal", "-shm"):
            f = Path(str(src) + suffix)
            if f.exists():
                shutil.copy2(f, Path(tmp) / f.name)
        try:
            con = sqlite3.connect(str(Path(tmp) / "Cookies"))
            rows = con.execute(
                "SELECT expires_utc, has_expires FROM cookies "
                "WHERE host_key LIKE '%instagram.com' AND name = 'sessionid'").fetchall()
        except sqlite3.Error as exc:
            print(f"cannot read cookie DB: {exc}")
            return 1
    if not rows:
        print("NOT LOGGED IN: no Instagram sessionid in the Chrome profile "
              f"{PROFILE}. Log in to Instagram in that Chrome window (the open tab can "
              "look logged in while stale -- reload it), then re-run.")
        return 1
    expires, has_expires = rows[0]
    if has_expires:
        exp = dt.datetime.fromtimestamp(expires / 1_000_000 - 11644473600, dt.timezone.utc)
        left = (exp.timestamp() - time.time()) / 86400
        if left <= 0:
            print(f"NOT LOGGED IN: the Instagram sessionid expired {exp:%Y-%m-%d}.")
            return 1
        note = f"expires {exp:%Y-%m-%d}"
        if left < WARN_DAYS:
            note += f" (WARNING: only {left:.1f} days left)"
        print(f"session ok: sessionid present, {note}")
    else:
        print("session ok: sessionid present (session cookie, no expiry)")
    return 0
 if __name__ == "__main__":
    sys.exit(main())
@@ -0,0 +1,120 @@
 #!/bin/bash
 # The full scrape-then-fetch pipeline for one profile: scroll the real page in
 # the signed-in Chrome (instagram-chrome-scrape.py), then fetch and publish
 # whatever's new as single posts. See TOOLING.md ("Reels: the API is blocked,
 # scrape by scrolling instead", and 2026-10-22 for the posts grid) for why this
 # exists at all -- it drives the actual signed-in browser session instead of
 # the listing API, and is slower by design.
 #
 # Usage: GDL_TAB=reels|posts ./chrome-sync.sh <username-or-profile-url>
 # Normally called through reels-sync.sh / posts-sync.sh, which set GDL_TAB.
 #
 # Also run by gdl-cron.sh (profiles, full-sweep) once per profile and tab since
 # 2026-10-23, with GDL_MAX_KNOWN_ROUNDS set -- see TOOLING.md "Posts grid by browser".
 # Exits non-zero if either step does. Everything is logged.
 set -eu
 RAW="${1:?usage: GDL_TAB=reels|posts chrome-sync.sh <username-or-profile-url>}"
 TAB="${GDL_TAB:?set GDL_TAB=reels or posts (or use reels-sync.sh / posts-sync.sh)}"
 GDL_HOME="${GDL_HOME:-$HOME/gdl}"
 GDL_PYTHON="${GDL_PYTHON:-$HOME/.local/share/pipx/venvs/gallery-dl/bin/python3}"
 INDEX="${GDL_INDEX:-https://instaarchive.ergosteur.com}"
 PUBLISH="${GDL_PUBLISH:-agentapi@10.20.28.200:/volume1/rslsync/sync/Instagram-archive/archives/}"
 # Same hand-paced pacing gdl-cron.sh uses -- see its comment for why. Override
 # per-run with GDL_SLEEP_REQUEST etc. if you ever need to, but raise them
 # rather than lower them.
 SLEEP_REQUEST="${GDL_SLEEP_REQUEST:-12 20}"
 SLEEP="${GDL_SLEEP:-5 10}"
 RATE="${GDL_RATE:-500K}"
 SCROLL_PAUSE="${GDL_SCROLL_PAUSE:-2.0 3.5}"
 MAX_IDLE_ROUNDS="${GDL_MAX_IDLE_ROUNDS:-3}"
 # Empty = instagram-chrome-scrape.py's own default (2 for posts, 0 = scroll to the end for reels).
 MAX_KNOWN_ROUNDS="${GDL_MAX_KNOWN_ROUNDS:-}"
 PATH="$HOME/.local/bin:$PATH"; export PATH
 if [ ! -x "$GDL_PYTHON" ]; then
  echo "instagram-chrome-scrape.py needs gallery-dl's own pipx venv python (websocket-client" >&2
  echo "was injected there, not into the system python): $GDL_PYTHON not found" >&2
  exit 2
 fi
 # Same username-from-URL parsing gdl-sync.py already does for --urls-file,
 # reused rather than re-implemented so the two never drift apart.
 PROFILE=$("$GDL_PYTHON" -c "
 import re, sys, importlib.util
 from pathlib import Path
 spec = importlib.util.spec_from_file_location('gdl_sync', Path('$GDL_HOME/gdl-sync.py'))
 gdl = importlib.util.module_from_spec(spec)
 sys.modules['gdl_sync'] = gdl
 spec.loader.exec_module(gdl)
 raw = '$RAW'
 m = gdl.RE_PROFILE_URL.match(raw)
 user = m.group('user') if m else raw.strip('/')
 if not user or '/' in user or ' ' in user:
    print(f'cannot read a username from {raw!r}', file=sys.stderr)
    sys.exit(1)
 print(user)
 ")
 mkdir -p "$GDL_HOME/logs"
 LOG="$GDL_HOME/logs/$TAB-$PROFILE-$(date +%Y%m%d-%H%M%S)-$$.log"
 URLS_FILE="$GDL_HOME/$PROFILE-$TAB.txt"
 STAGING="$GDL_HOME/staging-$TAB-$PROFILE"
 echo "=== $TAB-sync $PROFILE $(date -Is) ===" | tee -a "$LOG"
 # The exit status has to survive the pipe into tee -- see gdl-cron.sh's
 # comment on PIPESTATUS for why this needs to be bash, not sh.
 set +e
 "$GDL_PYTHON" "$GDL_HOME/instagram-chrome-scrape.py" \
    --profile "$PROFILE" \
    --tab "$TAB" \
    --index "$INDEX" \
    --scroll-pause $SCROLL_PAUSE \
    --max-idle-rounds "$MAX_IDLE_ROUNDS" \
    ${MAX_KNOWN_ROUNDS:+--max-known-rounds "$MAX_KNOWN_ROUNDS"} \
    --out "$URLS_FILE" 2>&1 | tee -a "$LOG"
 scrape_status=${PIPESTATUS[0]}
 set -e
 if [ "$scrape_status" -ne 0 ]; then
  echo "=== exit $scrape_status (scrape failed) at $(date -Is) ===" | tee -a "$LOG"
  exit "$scrape_status"
 fi
 if [ ! -s "$URLS_FILE" ]; then
  echo "no new $TAB for $PROFILE; nothing to fetch" | tee -a "$LOG"
  echo "=== exit 0 at $(date -Is) ===" | tee -a "$LOG"
  exit 0
 fi
 # Staging is wiped every run on purpose -- same reasoning as gdl-cron.sh: what
 # we already hold is decided by the archive dedupe in instagram-chrome-scrape.py, not by
 # what happens to be sitting in staging.
 rm -rf "$STAGING"
 set +e
 # shellcheck disable=SC2086
 "$GDL_HOME/gdl-sync.py" \
    --publish "$PUBLISH" \
    --staging "$STAGING" \
    --post-urls-file "$URLS_FILE" \
    --sleep-request $SLEEP_REQUEST \
    --sleep $SLEEP \
    --rate "$RATE" \
    --execute 2>&1 | tee -a "$LOG"
 status=${PIPESTATUS[0]}
 set -e
 echo "=== exit $status at $(date -Is) ===" | tee -a "$LOG"
 # Keep the log directory from growing without bound -- scoped to this
 # script's own logs so it never touches gdl-cron.sh's rotation.
 ls -1t "$GDL_HOME/logs" | grep "^$TAB-[^0-9]" | tail -n +30 | while read -r old; do
  rm -f "$GDL_HOME/logs/$old"
 done
 exit "$status"
@@ -2,7 +2,7 @@
 # Unattended wrapper around gdl-sync.py. One argument: the run mode.
 #
 #   stories     daily      ~6 requests; the only surface that cannot be backfilled
-#   profiles    monthly    every surface, --abort 50: stops enumerating a profile
+#   profiles    weekly     every surface, --abort 50: stops enumerating a profile
 #                          once it reaches content already held, so it costs
 #                          ~40-60 requests and catches everything NEW
 #   full-sweep  rarely     every surface, no abort: walks each profile to the end
@@ -10,11 +10,23 @@
 #                          EDITED after we archived them, and by far the most
 #                          expensive thing here -- see TOOLING.md before running.
 #
 # Posts and reels are LISTED by scrolling the signed-in Chrome (chrome-sync.sh,
 # per profile and tab) rather than by gallery-dl's listing API, which broke on
 # 2026-10-22 (gh#9829) while single-post fetches kept working. gallery-dl still
 # does stories and highlights. GDL_LISTING=api restores the old all-API run.
 #
 # Exits non-zero if the sync does, so cron mails you. Everything is logged.
 set -eu
 MODE="${1:?usage: gdl-cron.sh stories|profiles|full-sweep}"
 # healthchecks.io (or compatible) ping URL, e.g. https://hc-ping.com/<uuid>.
 # Set by the run-*.sh wrapper for whichever mode has a check configured;
 # unset here means no ping, silently. A ping failure (network blip, check
 # deleted) must never fail the sync itself -- hence `|| true` throughout, and
 # a short timeout (-m) so a hung ping can't hang the run.
 HC_URL="${HC_URL:-}"
 GDL_HOME="${GDL_HOME:-$HOME/gdl}"
 INDEX="${GDL_INDEX:-https://instaarchive.ergosteur.com}"
 PUBLISH="${GDL_PUBLISH:-agentapi@10.20.28.200:/volume1/rslsync/sync/Instagram-archive/archives/}"
@@ -34,6 +46,15 @@ SLEEP_REQUEST="${GDL_SLEEP_REQUEST:-12 20}"
 SLEEP="${GDL_SLEEP:-5 10}"
 RATE="${GDL_RATE:-500K}"
 LISTING="${GDL_LISTING:-browser}"
 case "$LISTING" in
  browser|api) ;;
  *) echo "GDL_LISTING must be browser or api, not '$LISTING'" >&2; exit 2 ;;
 esac
 GDL_PYTHON="${GDL_PYTHON:-$HOME/.local/share/pipx/venvs/gallery-dl/bin/python3}"
 # Set per mode below when posts/reels go through the browser: empty = no browser step.
 BROWSER_KNOWN_ROUNDS=""
 case "$MODE" in
  # --min-interval 8, not the 20h default. The floor exists to stop an ABORTED
  # RESTART re-enumerating profiles -- a minutes-to-hours concern. At 20h the
@@ -43,10 +64,23 @@ case "$MODE" in
  # request per profile, so the worst case 8h permits is roughly twelve
  # requests in a day instead of six.
  stories)    ARGS="--only stories --min-interval 8" ;;
-  profiles)   ARGS="--only posts,reels,stories,highlights --abort 50" ;;
+  # Browser listing: stop scrolling a grid after 2 rounds of nothing unarchived,
  # the browser's equivalent of --abort 50.
  profiles)
    if [ "$LISTING" = browser ]; then
      ARGS="--only stories,highlights --abort 50"; BROWSER_KNOWN_ROUNDS=2
    else
      ARGS="--only posts,reels,stories,highlights --abort 50"
    fi ;;
  # No --abort: the whole point of full-sweep is enumerating to the end, so it
-  # is the only run that notices carousels edited after we archived them.
+  # is the only run that notices carousels edited after we archived them. The
-  full-sweep) ARGS="--only posts,reels,stories,highlights" ;;
+  # browser equivalent is scrolling every grid to the bottom (0 = no early stop).
  full-sweep)
    if [ "$LISTING" = browser ]; then
      ARGS="--only stories,highlights"; BROWSER_KNOWN_ROUNDS=0
    else
      ARGS="--only posts,reels,stories,highlights"
    fi ;;
  # Renamed 2026-08-22: "full" was misleading (it is the abort-LIMITED run) and
  # "sweep" did not say it was the exhaustive one. Catch the old names rather
  # than failing with a bare error, in case something still passes them.
@@ -57,6 +91,34 @@ esac
 mkdir -p "$GDL_HOME/logs"
 [ -n "$HC_URL" ] && curl -fsS -m 10 --retry 3 -o /dev/null "$HC_URL/start" || true
 # Preflight: refuse to scrape logged out. gdl reads the Chrome profile's cookie DB, and the
 # session in it can die silently -- from 2026-10-07 every stories run fetched "No results"
 # for all six profiles and still exited 0, so nothing alerted and a week of stories (which
 # cannot be backfilled) was lost. An open Chrome tab keeps showing its logged-in UI after
 # that, so only the cookie DB tells the truth. Exit 76 turns the Healthchecks check red.
 SESSION_CHECK="$GDL_HOME/check-session.py"
 if [ -x "$SESSION_CHECK" ]; then
  if ! SESSION_MSG=$("$SESSION_CHECK" 2>&1); then
    echo "=== $MODE run $(date -Is) ===" | tee -a "$LOG"
    echo "ABORT (no requests sent): $SESSION_MSG" | tee -a "$LOG" >&2
    echo "=== exit 76 at $(date -Is) ===" | tee -a "$LOG"
    [ -n "$HC_URL" ] && curl -fsS -m 10 --retry 3 --data-binary "@$LOG" \
      -o /dev/null "$HC_URL/76" || true
    exit 76
  fi
  echo "preflight: $SESSION_MSG" | tee -a "$LOG"
 fi
 # Upgrade gallery-dl/yt-dlp first (PyPI only, no Instagram requests; never fatal). Instagram
 # endpoints change and gallery-dl follows within days -- on 1.32.14 the posts listing silently
 # returned "No results" after 1.32.17 had already fixed the query. See gdl-upgrade.sh.
 UPGRADE="$GDL_HOME/gdl-upgrade.sh"
 if [ -x "$UPGRADE" ]; then
  "$UPGRADE" 2>&1 | tee -a "$LOG" || true
 fi
 # Staging is wiped every run ON PURPOSE. What we already hold is decided by the
 # skip-archive (--download-archive), never by which files happen to be sitting
 # in staging, so starting empty is correct -- and it keeps the publish rsync
@@ -90,6 +152,39 @@ set +e
 status=${PIPESTATUS[0]}
 set -e
 # Posts and reels by browser: scrape each profile's grid and Reels tab, fetch only
 # the new shortcodes as single posts. chrome-sync.sh logs and publishes on its own;
 # its output is copied into this run's log so the Healthchecks body shows it too.
 if [ -n "$BROWSER_KNOWN_ROUNDS" ]; then
  mapfile -t PROFILES < <("$GDL_PYTHON" -c "
 import sys, importlib.util
 from pathlib import Path
 spec = importlib.util.spec_from_file_location('gdl_sync', Path('$GDL_HOME/gdl-sync.py'))
 gdl = importlib.util.module_from_spec(spec)
 sys.modules['gdl_sync'] = gdl
 spec.loader.exec_module(gdl)
 for p in gdl.read_urls_file(Path('$GDL_HOME/artms_account_links.txt')):
    print(p)
 ")
  browser_failed=0
  for p in "${PROFILES[@]}"; do
    for tab in posts reels; do
      echo "==> browser: $p / $tab" | tee -a "$LOG"
      set +e
      GDL_TAB=$tab GDL_MAX_KNOWN_ROUNDS=$BROWSER_KNOWN_ROUNDS \
        "$GDL_HOME/chrome-sync.sh" "$p" 2>&1 | tee -a "$LOG" >/dev/null
      s=${PIPESTATUS[0]}
      set -e
      if [ "$s" -ne 0 ]; then
        echo "    browser $p / $tab FAILED (exit $s)" | tee -a "$LOG" >&2
        browser_failed=$((browser_failed + 1))
      fi
    done
  done
  echo "browser listing: ${#PROFILES[@]} profile(s), $browser_failed step(s) failed" | tee -a "$LOG"
  [ "$browser_failed" -gt 0 ] && [ "$status" -eq 0 ] && status=1
 fi
 # A stories run that skipped every source is NOT a success. It means the
 # min-interval floor blocked the one surface that cannot be backfilled, and
 # without this it looks identical to a clean run: exit 0, "0 step(s) failed".
@@ -102,8 +197,26 @@ if [ "$MODE" = "stories" ] && grep -q "sources  : 0 to sync" "$RUNOUT"; then
  [ "$status" -eq 0 ] && status=75
 fi
 # Every stories source empty for several days running = a session Instagram stopped
 # honouring while the cookie is still in the jar (the preflight cannot see that). Stories-only:
 # other modes legitimately return few items per source. See stories-streak.py.
 if [ "$MODE" = "stories" ] && [ "$status" -eq 0 ] && [ -x "$GDL_HOME/stories-streak.py" ]; then
  if ! STREAK_MSG=$("$GDL_HOME/stories-streak.py" "$RUNOUT" "$GDL_HOME/stories-empty-streak" 2>&1); then
    echo "WARNING: $STREAK_MSG" | tee -a "$LOG" >&2
    status=77
  fi
 fi
 echo "=== exit $status at $(date -Is) ===" | tee -a "$LOG"
 # The log becomes the ping body -- healthchecks.io keeps the last ping's body
 # and shows it in the dashboard, so a failure is diagnosable without SSHing
 # in. The trailing URL segment is the exit code by convention: 0 means
 # success, anything else means fail (this is how the "0 skipped" -> 75
 # remap above also reaches the dashboard as a failure, not a quiet success).
 [ -n "$HC_URL" ] && curl -fsS -m 10 --retry 3 --data-binary "@$LOG" \
  -o /dev/null "$HC_URL/$status" || true
 # Keep the log directory from growing without bound.
 ls -1t "$GDL_HOME/logs" | tail -n +30 | while read -r old; do
  rm -f "$GDL_HOME/logs/$old"
@@ -94,6 +94,7 @@ import re
 import shutil
 import subprocess
 import sys
 import threading
 import time
 from dataclasses import dataclass, field
 from pathlib import Path
@@ -224,7 +225,7 @@ def read_urls_file(path: Path) -> list[str]:
 def read_post_urls_file(path: Path) -> list[str]:
    """
    Read individual post/reel URLs from a file, one per line -- the output
-    format `reels-scrape.py` writes. Blank lines and `#` comments are skipped;
+    format `instagram-chrome-scrape.py` writes. Blank lines and `#` comments are skipped;
    order is kept and duplicates dropped, same conventions as read_urls_file.
    """
    urls: list[str] = []
@@ -325,6 +326,32 @@ def build_config(rate: str, sleep_request: list[float],
            "post_shortcode", "post_id", "type", "date", "post_date",
            "username", "fullname", "owner_id", "description", "count",
            "likes", "post_url", "sidecar_shortcode",
            # A Collab's `username`/`fullname`/`owner_id` above are always the
            # original poster's, never the scraped account's -- see
            # docs/gallery-dl.md. `coauthors` is the API's own list of every
            # OTHER collaborator (it excludes the post's owner), a direct
            # signal instead of inferring a collab from identity mismatches.
            "coauthors",
        ],
    }
    # Per-carousel-item fields (`width`/`height`/`tagged_users`/`owner`) live
    # on the per-FILE kwdict, not the per-post one `meta_pp` reads -- a
    # carousel's items can each have different dimensions and tags, which one
    # post-level JSON can't represent. `owner` is deliberately left out: it is
    # a full user object (profile pic URLs, privacy flags) for whoever posted
    # that specific item, the same reason `audio_user` is excluded above.
    media_pp = {
        "name": "metadata",
        "mode": "json",
        # event defaults to "file" when omitted -- runs once per downloaded
        # file, appending ".json" to that file's own full name, e.g.
        # "... - 01.jpg" gets "... - 01.jpg.json" alongside it. Never
        # collides with the post-level "....json" above, which has no
        # per-item number.
        "include": [
            "media_id", "shortcode", "num",
            "width", "height", "width_original", "height_original",
            "tagged_users",
        ],
    }
@@ -363,6 +390,7 @@ def build_config(rate: str, sleep_request: list[float],
            "postprocessors": [
                {**caption_pp, "filename": stem + ".txt"},
                {**meta_pp, "filename": stem + ".json"},
                media_pp,
            ],
        }
@@ -433,6 +461,71 @@ def build_config(rate: str, sleep_request: list[float],
 # Planning and execution
 # --------------------------------------------------------------------------
 def _fmt_elapsed(sec: float) -> str:
    sec = int(sec)
    h, rest = divmod(sec, 3600)
    m, s = divmod(rest, 60)
    return f"{h}h{m:02d}m" if h else (f"{m}m{s:02d}s" if m else f"{s}s")
 def _count_new(root: Path, since: float) -> tuple[int, str | None]:
    """Media files under `root` written since `since`, and the newest one's name."""
    n, newest, newest_t = 0, None, 0.0
    for d, _dirs, files in os.walk(root):
        for f in files:
            if f.endswith((".txt", ".json", ".part")):
                continue
            try:
                mt = os.stat(os.path.join(d, f)).st_mtime
            except OSError:
                continue
            if mt >= since:
                n += 1
                if mt > newest_t:
                    newest, newest_t = f, mt
    return n, newest
 class Heartbeat:
    """
    Prints a progress line every `every` seconds while a quiet step runs.
    A healthy run looks hung otherwise: pacing is 12-20s between requests, the
    generated config sets `output: null` (no per-file lines), and the live
    listing probe captures gallery-dl output until it finishes. With `watch`
    set, the line also reports how many files have landed there so far.
    """
    def __init__(self, label: str, watch: Path | None = None, every: float = 60.0):
        self.label, self.watch, self.every = label, watch, every
        self._stop = threading.Event()
        self._thread = threading.Thread(target=self._run, daemon=True)
        self.t0 = time.time()
    def _run(self) -> None:
        while not self._stop.wait(self.every):
            msg = f"    ... {self.label}: {_fmt_elapsed(time.time() - self.t0)} elapsed"
            if self.watch is not None:
                n, newest = _count_new(self.watch, self.t0)
                msg += f", {n} new file(s) so far"
                if newest:
                    msg += f" (latest: {newest})"
            print(msg, flush=True)
    def __enter__(self) -> "Heartbeat":
        self.t0 = time.time()
        self._thread.start()
        return self
    def __exit__(self, *exc) -> None:
        self._stop.set()
        self._thread.join()
    @property
    def elapsed(self) -> float:
        return time.time() - self.t0
 def gdl_command(src: Source, staging: Path, config: Path, cookies: str,
                archive_db: Path | None) -> list[str]:
    cmd = [
@@ -721,7 +814,27 @@ def publish(staging: Path, dest: str, dry_run: bool) -> int:
        return 0
    cmd = rsync_command(staging, dest, dry_run)
    print("    " + " ".join(cmd))
-    return subprocess.run(cmd).returncode
+    result = subprocess.run(cmd, capture_output=True, text=True)
    sys.stdout.write(result.stdout)
    sys.stderr.write(result.stderr)
    if result.returncode == 0:
        return 0
    # The archive share is never owned by the ssh user, so rsync's `chown`
    # step fails on every single run -- exit 23 ("some files/attrs were not
    # transferred"), even though every file landed. This is long documented
    # (TOOLING.md "Reading the output") as something to read past, not a
    # real failure -- but until now the exit CODE didn't know that, so it
    # was 1 on every clean run too, silently. Only mask it when every rsync
    # error line is that specific permission warning; anything else (a real
    # transfer failure sharing the same exit code) still fails loudly.
    err_lines = [l for l in result.stderr.splitlines() if l.startswith("rsync:")]
    if (result.returncode == 23 and err_lines
            and all("failed to set permissions" in l for l in err_lines)):
        print("    (rsync exit 23 was only the expected chown-permission "
              "warning; files landed)")
        return 0
    return result.returncode
 def run_post_urls(args) -> int:
@@ -816,7 +929,7 @@ def main() -> int:
                   help="file of post/reel URLs, one per line; # comments and "
                        "blank lines allowed. Same handling as --post-url, "
                        "paced with --sleep-request between items. This is "
-                        "the format reels-scrape.py writes")
+                        "the format instagram-chrome-scrape.py writes")
    ap.add_argument("--cookies", default="chrome:/home/matt/.config/google-chrome-devtools",
                    help="gallery-dl --cookies-from-browser value")
    ap.add_argument("--archive-db", type=Path, default=None,
@@ -956,8 +1069,9 @@ def main() -> int:
                        0.0 if args.force else args.probe_ttl)
    failures = 0
-    for prof, src, seed in plan:
+    for n_src, (prof, src, seed) in enumerate(plan, 1):
-        print(f"==> {prof.user} / {src.kind}")
+        print(f"==> [{n_src}/{len(plan)}] {prof.user} / {src.kind}  "
              f"({dt.datetime.now().strftime('%H:%M:%S')})")
        stage_dir = args.staging / (src.directory or ".")
        stage_dir.mkdir(parents=True, exist_ok=True)
@@ -969,7 +1083,10 @@ def main() -> int:
            try:
                live = probes.get(src.url, now_ts)
                if live is None:
-                    live = probe_live(src, config_path, args.cookies)
+                    print("    listing live items (metadata only; quiet until done)...")
                    with Heartbeat("listing") as hb:
                        live = probe_live(src, config_path, args.cookies)
                    print(f"    listed {len(live)} item(s) in {_fmt_elapsed(hb.elapsed)}")
                    probes.put(src.url, live, stamp)
                    probes.save()
                else:
@@ -987,12 +1104,16 @@ def main() -> int:
        cmd = gdl_command(src, args.staging, config_path, args.cookies,
                          args.archive_db)
-        result = subprocess.run(cmd)
+        with Heartbeat("fetching", watch=stage_dir) as hb:
            result = subprocess.run(cmd)
        n_new, _ = _count_new(stage_dir, hb.t0)
        if result.returncode != 0:
            failures += 1
            # Keep going: one private or renamed profile must not abort the run.
-            print(f"    FAILED (exit {result.returncode})", file=sys.stderr)
+            print(f"    FAILED (exit {result.returncode}) after "
                  f"{_fmt_elapsed(hb.elapsed)}", file=sys.stderr)
        else:
            print(f"    done in {_fmt_elapsed(hb.elapsed)}, {n_new} new file(s)")
            # Recorded even for an empty fetch: the request was still spent.
            state.mark_fetched(src.url, stamp)
            state.save()
@@ -0,0 +1,34 @@
 #!/bin/bash
 # Upgrade gallery-dl and its injected yt-dlp (both pipx). Instagram changes its
 # endpoints often and gallery-dl follows within days -- 1.32.17 (2026-10-17)
 # "update user & posts query parameters" landed while we were still on 1.32.14
 # and the posts listing silently returned "No results".
 #
 # Called by gdl-cron.sh before every scrape and weekly by gdl-upgrade.timer.
 # Never fatal: a failed upgrade (offline, PyPI down) must not stop a stories
 # run, which cannot be backfilled -- it just scrapes with what is installed.
 # Costs no Instagram requests (PyPI only).
 set -u
 PATH="$HOME/.local/bin:$PATH"; export PATH
 VENV_PY="${GDL_VENV_PY:-$HOME/.local/share/pipx/venvs/gallery-dl/bin/python}"
 versions() {
  printf 'gallery-dl %s, yt-dlp %s' \
    "$(gallery-dl --version 2>/dev/null || echo '?')" \
    "$("$VENV_PY" -c 'import yt_dlp; print(yt_dlp.version.__version__)' 2>/dev/null || echo '?')"
 }
 before=$(versions)
 if ! timeout 180 pipx upgrade --quiet --include-injected gallery-dl >/dev/null 2>&1; then
  echo "upgrade: WARNING pipx upgrade failed; continuing with $before" >&2
  exit 0
 fi
 after=$(versions)
 if [ "$before" = "$after" ]; then
  echo "upgrade: up to date ($after)"
 else
  echo "upgrade: CHANGED $before -> $after"
 fi
@@ -1,6 +1,6 @@
 #!/usr/bin/env python3
 """
-Find a profile's reels by scrolling the real page, not calling the API.
+Find a profile's reels (or posts) by scrolling the real page, not calling the API.
 gallery-dl's dedicated reels extractor POSTs to /api/v1/clips/user/, which
 Instagram now 302-redirects to the home page for this account -- confirmed on
@@ -11,11 +11,19 @@ on loopback CDP for MCP automation -- see TOOLING.md) via the DevTools
 protocol, scrolls it like a person would, and scrapes `/reel/<code>/` links
 out of the rendered page instead.
 `--tab posts` does the same for the main profile grid (`/<user>/`, both `/p/` and
 `/reel/` links), for when gallery-dl's posts listing breaks -- on 2026-10-22,
 1.32.14 returned "No results" and 1.32.17 "Requested user could not be found"
 (upstream gh#9829), while single-post fetches kept working. Since the grid
 runs newest-first, it stops scrolling once rounds turn up nothing but
 already-archived posts (`--max-known-rounds`) instead of walking a whole
 profile's history.
 This only finds shortcodes; it never downloads anything itself. What comes
 out the other end is deduped against the archive (via the same --index used
 elsewhere) and printed as plain post URLs -- feed them to gdl-sync.py:
-    ./scripts/reels-scrape.py --profile someuser \\
+    ./scripts/instagram-chrome-scrape.py --profile someuser \\
        --index https://instaarchive.ergosteur.com > /tmp/someuser-reels.txt
    ./scripts/gdl-sync.py --publish user@host:/path --staging /var/tmp/gdl \\
@@ -38,7 +46,13 @@ import time
 from pathlib import Path
 from urllib.request import urlopen
-RE_REEL_HREF = re.compile(r"/reel/([^/?#]+)")
+RE_POST_HREF = re.compile(r"/(p|reel)/([^/?#]+)")
 # tab -> (page path under the profile, link kinds to collect)
 TABS = {
    "reels": ("reels/", ("reel",)),
    "posts": ("", ("p", "reel")),
 }
 # The reels tab has finished loading once a scroll round finds no NEW
 # shortcodes this many times in a row -- lazy-loaded feeds sometimes stall
@@ -46,21 +60,29 @@ RE_REEL_HREF = re.compile(r"/reel/([^/?#]+)")
 DEFAULT_MAX_IDLE_ROUNDS = 3
 DEFAULT_MAX_SCROLLS = 200
 DEFAULT_SCROLL_PAUSE = (2.0, 3.5)
-
+# Posts tab only: stop after this many consecutive scroll rounds whose new
-SCRAPE_JS = (
+# shortcodes were all already archived. The first round is exempt -- it holds
-    "(() => {"
+# the pinned posts, which are old by nature and sit above the new ones.
-    "window.scrollTo(0, document.body.scrollHeight);"
+DEFAULT_MAX_KNOWN_ROUNDS = 2
    "return Array.from(document.querySelectorAll('a[href*=\"/reel/\"]'))"
    ".map(a => a.getAttribute('href'));"
    "})()"
 )
-def extract_shortcodes(hrefs: list[str]) -> list[str]:
+def scrape_js(kinds: tuple[str, ...]) -> str:
    selector = ", ".join(f'a[href*="/{k}/"]' for k in kinds)
    return (
        "(() => {"
        "window.scrollTo(0, document.body.scrollHeight);"
        f"return Array.from(document.querySelectorAll({json.dumps(selector)}))"
        ".map(a => a.getAttribute('href'));"
        "})()"
    )
 def extract_shortcodes(hrefs: list[str],
                       kinds: tuple[str, ...] = ("reel",)) -> list[str]:
    codes = []
    for href in hrefs:
-        if m := RE_REEL_HREF.search(href):
+        if (m := RE_POST_HREF.search(href)) and m.group(1) in kinds:
-            codes.append(m.group(1))
+            codes.append(m.group(2))
    return codes
@@ -100,23 +122,45 @@ class CDP:
        self.ws.close()
-def scrape_reels(user: str, cdp_port: int = 9222,
+def should_stop_known(round_no: int, new: list[str], have: set[str],
-                 scroll_pause: tuple[float, float] = DEFAULT_SCROLL_PAUSE,
+                      streak: int, max_known_rounds: int) -> tuple[bool, int]:
                 max_idle_rounds: int = DEFAULT_MAX_IDLE_ROUNDS,
                 max_scrolls: int = DEFAULT_MAX_SCROLLS,
                 log=lambda msg: None) -> list[str]:
    """
-    Open the profile's reels tab in a NEW tab of the already-signed-in Chrome,
+    Early-stop bookkeeping for a newest-first grid: returns (stop, streak).
-    scroll it to the bottom repeatedly, and collect every unique `/reel/`
+    A round counts toward the streak when it found new shortcodes and every
-    shortcode that appears. Closes the tab when done either way.
+    one is already archived; a round with anything unarchived resets it. An
    empty round leaves it alone (that is what max_idle_rounds is for), and
    round 1 never counts -- pinned posts come first.
    """
    if max_known_rounds <= 0 or round_no == 1 or not new:
        return False, streak
    if all(c in have for c in new):
        streak += 1
    else:
        streak = 0
    return streak >= max_known_rounds, streak
 def scrape_grid(user: str, tab: str = "reels", cdp_port: int = 9222,
                scroll_pause: tuple[float, float] = DEFAULT_SCROLL_PAUSE,
                max_idle_rounds: int = DEFAULT_MAX_IDLE_ROUNDS,
                max_scrolls: int = DEFAULT_MAX_SCROLLS,
                have: set[str] | None = None,
                max_known_rounds: int = 0,
                log=lambda msg: None) -> list[str]:
    """
    Open the profile's reels tab (or main grid) in a NEW tab of the
    already-signed-in Chrome, scroll it to the bottom repeatedly, and collect
    every unique shortcode that appears. Closes the tab when done either way.
    """
    path, kinds = TABS[tab]
    js = scrape_js(kinds)
    version = json.loads(urlopen(f"http://localhost:{cdp_port}/json/version",
                                 timeout=10).read())
    browser = CDP(version["webSocketDebuggerUrl"])
    target_id = None
    try:
        target = browser.send("Target.createTarget", {
-            "url": f"https://www.instagram.com/{user}/reels/"})
+            "url": f"https://www.instagram.com/{user}/{path}"})
        target_id = target["targetId"]
        attach = browser.send("Target.attachToTarget", {
            "targetId": target_id, "flatten": True})
@@ -128,16 +172,24 @@ def scrape_reels(user: str, cdp_port: int = 9222,
        seen: set[str] = set()
        idle = 0
        known_streak = 0
        for i in range(max_scrolls):
            result = browser.send(
                "Runtime.evaluate",
-                {"expression": SCRAPE_JS, "returnByValue": True},
+                {"expression": js, "returnByValue": True},
                session_id=session_id)
            hrefs = result.get("result", {}).get("value") or []
-            codes = extract_shortcodes(hrefs)
+            codes = extract_shortcodes(hrefs, kinds)
-            new = [c for c in codes if c not in seen]
+            new = list(dict.fromkeys(c for c in codes if c not in seen))
            seen.update(new)
-            log(f"  scroll {i + 1}: {len(seen)} unique reels so far (+{len(new)})")
+            log(f"  scroll {i + 1}: {len(seen)} unique {tab} so far (+{len(new)})")
            stop, known_streak = should_stop_known(
                i + 1, new, have or set(), known_streak, max_known_rounds)
            if stop:
                log(f"  {known_streak} round(s) of already-archived {tab} in a row; "
                    f"stopping")
                break
            if new:
                idle = 0
@@ -162,7 +214,9 @@ def main() -> int:
    ap = argparse.ArgumentParser(description=__doc__,
                                 formatter_class=argparse.RawDescriptionHelpFormatter)
    ap.add_argument("--profile", required=True,
-                    help="Instagram username to scrape reels for")
+                    help="Instagram username to scrape")
    ap.add_argument("--tab", choices=sorted(TABS), default="reels",
                    help="reels tab, or the main posts grid (default: reels)")
    ap.add_argument("--index", required=True,
                    help="existing archive listing, same as gdl-sync.py's "
                         "--index: a local root, or the viewer's base URL. "
@@ -179,6 +233,11 @@ def main() -> int:
    ap.add_argument("--max-scrolls", type=int, default=DEFAULT_MAX_SCROLLS,
                    help="hard ceiling on scroll rounds, in case a page never "
                         "goes idle")
    ap.add_argument("--max-known-rounds", type=int, default=None,
                    help="stop after this many consecutive scroll rounds of "
                         "only already-archived shortcodes; 0 disables "
                         f"(default: {DEFAULT_MAX_KNOWN_ROUNDS} for posts, "
                         "0 for reels)")
    ap.add_argument("--out", type=Path,
                    help="write new reel URLs here, one per line (default: "
                         "stdout)")
@@ -208,20 +267,30 @@ def main() -> int:
    print(f"archive already holds {len(have)} shortcode(s) across "
          f"{len(profiles)} profile(s)", file=sys.stderr)
-    print(f"scraping https://www.instagram.com/{args.profile}/reels/ ...",
+    max_known = args.max_known_rounds
    if max_known is None:
        max_known = DEFAULT_MAX_KNOWN_ROUNDS if args.tab == "posts" else 0
    path, _ = TABS[args.tab]
    print(f"scraping https://www.instagram.com/{args.profile}/{path} ...",
          file=sys.stderr)
-    codes = scrape_reels(args.profile, cdp_port=args.cdp_port,
+    codes = scrape_grid(args.profile, tab=args.tab, cdp_port=args.cdp_port,
                        scroll_pause=tuple(args.scroll_pause),
                        max_idle_rounds=args.max_idle_rounds,
                        max_scrolls=args.max_scrolls,
                        have=have, max_known_rounds=max_known,
                        log=lambda msg: print(msg, file=sys.stderr))
    new_codes = [c for c in codes if c not in have]
-    print(f"found {len(codes)} reel(s) on the page, {len(codes) - len(new_codes)} "
+    print(f"found {len(codes)} {args.tab} on the page, {len(codes) - len(new_codes)} "
          f"already archived, {len(new_codes)} new", file=sys.stderr)
-    urls = [f"https://www.instagram.com/{args.profile}/reel/{c}/"
+    # /p/<code>/ resolves reels too, so the posts grid needs no per-kind URL.
-            for c in new_codes]
+    if args.tab == "posts":
        urls = [f"https://www.instagram.com/p/{c}/" for c in new_codes]
    else:
        urls = [f"https://www.instagram.com/{args.profile}/reel/{c}/"
                for c in new_codes]
    text = "\n".join(urls)
    if args.out:
        args.out.write_text(text + ("\n" if text else ""))
@@ -0,0 +1,11 @@
 #!/bin/bash
 # chrome-sync.sh for the main profile grid: scroll /<user>/ in the signed-in
 # Chrome, collect /p/ and /reel/ links, fetch only the new ones as single
 # posts. A by-hand fallback for when gallery-dl's posts listing is broken
 # (2026-10-22: upstream gh#9829) while single-post fetches still work -- see
 # TOOLING.md. Stops scrolling once it reaches already-archived posts.
 #
 # Usage: ./posts-sync.sh <username-or-profile-url>
 set -eu
 export GDL_TAB=posts
 exec "$(dirname "$0")/chrome-sync.sh" "$@"
@@ -1,114 +1,8 @@
 #!/bin/bash
-# The full reels pipeline for one profile: scrape by scrolling the real page,
+# chrome-sync.sh for a profile's reels tab. See TOOLING.md ("Reels: the API is
-# then fetch and publish whatever's new. See TOOLING.md ("Reels: the API is
+# blocked, scrape by scrolling instead").
 # blocked, scrape by scrolling instead") for why this exists at all -- the
 # dedicated reels API is blocked, this drives the actual signed-in browser
 # session instead, and is slower by design.
 #
 # Usage: ./reels-sync.sh <username-or-profile-url>
 #   ./reels-sync.sh zindoriyam
 #   ./reels-sync.sh https://www.instagram.com/zindoriyam/
 #
 # Deliberately NOT wired into gdl-cron.sh or the timers -- see TOOLING.md.
 # Exits non-zero if either step does. Everything is logged.
 set -eu
-
+export GDL_TAB=reels
-RAW="${1:?usage: reels-sync.sh <username-or-profile-url>}"
+exec "$(dirname "$0")/chrome-sync.sh" "$@"
 GDL_HOME="${GDL_HOME:-$HOME/gdl}"
 GDL_PYTHON="${GDL_PYTHON:-$HOME/.local/share/pipx/venvs/gallery-dl/bin/python3}"
 INDEX="${GDL_INDEX:-https://instaarchive.ergosteur.com}"
 PUBLISH="${GDL_PUBLISH:-agentapi@10.20.28.200:/volume1/rslsync/sync/Instagram-archive/archives/}"
 # Same hand-paced pacing gdl-cron.sh uses -- see its comment for why. Override
 # per-run with GDL_SLEEP_REQUEST etc. if you ever need to, but raise them
 # rather than lower them.
 SLEEP_REQUEST="${GDL_SLEEP_REQUEST:-12 20}"
 SLEEP="${GDL_SLEEP:-5 10}"
 RATE="${GDL_RATE:-500K}"
 SCROLL_PAUSE="${GDL_SCROLL_PAUSE:-2.0 3.5}"
 MAX_IDLE_ROUNDS="${GDL_MAX_IDLE_ROUNDS:-3}"
 PATH="$HOME/.local/bin:$PATH"; export PATH
 if [ ! -x "$GDL_PYTHON" ]; then
  echo "reels-scrape.py needs gallery-dl's own pipx venv python (websocket-client" >&2
  echo "was injected there, not into the system python): $GDL_PYTHON not found" >&2
  exit 2
 fi
 # Same username-from-URL parsing gdl-sync.py already does for --urls-file,
 # reused rather than re-implemented so the two never drift apart.
 PROFILE=$("$GDL_PYTHON" -c "
 import re, sys, importlib.util
 from pathlib import Path
 spec = importlib.util.spec_from_file_location('gdl_sync', Path('$GDL_HOME/gdl-sync.py'))
 gdl = importlib.util.module_from_spec(spec)
 sys.modules['gdl_sync'] = gdl
 spec.loader.exec_module(gdl)
 raw = '$RAW'
 m = gdl.RE_PROFILE_URL.match(raw)
 user = m.group('user') if m else raw.strip('/')
 if not user or '/' in user or ' ' in user:
    print(f'cannot read a username from {raw!r}', file=sys.stderr)
    sys.exit(1)
 print(user)
 ")
 mkdir -p "$GDL_HOME/logs"
 LOG="$GDL_HOME/logs/reels-$PROFILE-$(date +%Y%m%d-%H%M%S)-$$.log"
 URLS_FILE="$GDL_HOME/$PROFILE-reels.txt"
 STAGING="$GDL_HOME/staging-reels-$PROFILE"
 echo "=== reels-sync $PROFILE $(date -Is) ===" | tee -a "$LOG"
 # The exit status has to survive the pipe into tee -- see gdl-cron.sh's
 # comment on PIPESTATUS for why this needs to be bash, not sh.
 set +e
 "$GDL_PYTHON" "$GDL_HOME/reels-scrape.py" \
    --profile "$PROFILE" \
    --index "$INDEX" \
    --scroll-pause $SCROLL_PAUSE \
    --max-idle-rounds "$MAX_IDLE_ROUNDS" \
    --out "$URLS_FILE" 2>&1 | tee -a "$LOG"
 scrape_status=${PIPESTATUS[0]}
 set -e
 if [ "$scrape_status" -ne 0 ]; then
  echo "=== exit $scrape_status (scrape failed) at $(date -Is) ===" | tee -a "$LOG"
  exit "$scrape_status"
 fi
 if [ ! -s "$URLS_FILE" ]; then
  echo "no new reels for $PROFILE; nothing to fetch" | tee -a "$LOG"
  echo "=== exit 0 at $(date -Is) ===" | tee -a "$LOG"
  exit 0
 fi
 # Staging is wiped every run on purpose -- same reasoning as gdl-cron.sh: what
 # we already hold is decided by the archive dedupe in reels-scrape.py, not by
 # what happens to be sitting in staging.
 rm -rf "$STAGING"
 set +e
 # shellcheck disable=SC2086
 "$GDL_HOME/gdl-sync.py" \
    --publish "$PUBLISH" \
    --staging "$STAGING" \
    --post-urls-file "$URLS_FILE" \
    --sleep-request $SLEEP_REQUEST \
    --sleep $SLEEP \
    --rate "$RATE" \
    --execute 2>&1 | tee -a "$LOG"
 status=${PIPESTATUS[0]}
 set -e
 echo "=== exit $status at $(date -Is) ===" | tee -a "$LOG"
 # Keep the log directory from growing without bound -- scoped to this
 # script's own logs so it never touches gdl-cron.sh's rotation.
 ls -1t "$GDL_HOME/logs" | grep '^reels-' | tail -n +30 | while read -r old; do
  rm -f "$GDL_HOME/logs/$old"
 done
 exit "$status"
@@ -0,0 +1,8 @@
 #!/bin/bash
 # Rarely: every surface, no --abort -- walks each profile to the end, ~420
 # requests. The only run that notices posts EDITED after they were archived
 # (e.g. a caption added later -- see TOOLING.md "Reels: the API is blocked"
 # for how that showed up), and by far the most expensive thing here. Read
 # TOOLING.md before running this; prefer running it by hand, not on a timer.
 set -eu
 exec "$(dirname "$0")/gdl-cron.sh" full-sweep "$@"
@@ -0,0 +1,9 @@
 #!/bin/bash
 # Routine run: every surface, --abort 50 (only gdl-cron.sh's "profiles" mode
 # does this), ~5 min. Covers everything but is no substitute for stories.sh --
 # a story posted and expired between two runs of this is simply gone.
 set -eu
 # healthchecks.io check "instagram-archiver-posts". Override or unset (e.g.
 # HC_URL= ./run-profiles.sh) to run without pinging it.
 export HC_URL="${HC_URL:-https://hc-ping.com/3f40abd8-e9bd-43af-a105-c583efaad88c}"
 exec "$(dirname "$0")/gdl-cron.sh" profiles "$@"
@@ -0,0 +1,56 @@
 #!/bin/bash
 # Run reels-sync.sh for every tracked ARTMS profile, one after another.
 # TOOLING.md ("Reels: the API is blocked, scrape by scrolling instead")
 # covers what this actually does and why it exists; this is just the loop
 # version of running reels-sync.sh six times by hand, which is what the
 # 2026-09-03 run was before this script existed.
 #
 # A by-hand full sweep of every Reels tab (no early stop). The weekly
 # gdl-cron.sh profiles run lists reels through chrome-sync.sh itself, with an
 # early stop, since 2026-10-23 -- see TOOLING.md "Posts grid by browser".
 #
 # Usage: ./reels-all.sh
 #   GDL_ARTMS_LINKS=/path/to/list ./reels-all.sh   # override the profile list
 set -eu
 GDL_HOME="${GDL_HOME:-$HOME/gdl}"
 LINKS="${GDL_ARTMS_LINKS:-$GDL_HOME/artms_account_links.txt}"
 GDL_PYTHON="${GDL_PYTHON:-$HOME/.local/share/pipx/venvs/gallery-dl/bin/python3}"
 if [ ! -f "$LINKS" ]; then
  echo "profile list not found: $LINKS" >&2
  exit 2
 fi
 # Same parser gdl-sync.py uses for --urls-file (URLs or bare usernames, #
 # comments, blank lines, duplicates dropped), reused so this never drifts
 # from what a routine profiles.sh run actually tracks.
 mapfile -t PROFILES < <("$GDL_PYTHON" -c "
 import sys, importlib.util
 from pathlib import Path
 spec = importlib.util.spec_from_file_location('gdl_sync', Path('$GDL_HOME/gdl-sync.py'))
 gdl = importlib.util.module_from_spec(spec)
 sys.modules['gdl_sync'] = gdl
 spec.loader.exec_module(gdl)
 for p in gdl.read_urls_file(Path('$LINKS')):
    print(p)
 ")
 if [ "${#PROFILES[@]}" -eq 0 ]; then
  echo "no usable profiles in $LINKS" >&2
  exit 2
 fi
 echo "=== reels-all: ${#PROFILES[@]} profile(s) $(date -Is) ==="
 status=0
 for p in "${PROFILES[@]}"; do
  echo "=== starting $p ==="
  if ! "$(dirname "$0")/reels-sync.sh" "$p"; then
    status=1
    echo "=== $p FAILED ===" >&2
  fi
  echo "=== finished $p ==="
 done
 echo "=== reels-all: exit $status at $(date -Is) ==="
 exit "$status"
@@ -0,0 +1,12 @@
 #!/bin/bash
 # Daily run: stories only, ~1 min. The one surface that cannot be backfilled
 # -- do this often. See TOOLING.md "Quick reference" for the full picture.
 #
 # Just a fixed-mode entry point so you don't have to remember gdl-cron.sh's
 # mode argument; override pacing/paths the same way gdl-cron.sh does, with
 # GDL_* env vars (see gdl-cron.sh's own comments for what's available).
 set -eu
 # healthchecks.io check "instagram-archiver-stories". Override or unset (e.g.
 # HC_URL= ./run-stories.sh) to run without pinging it.
 export HC_URL="${HC_URL:-https://hc-ping.com/9fe7347f-95be-47aa-a46b-affd27691554}"
 exec "$(dirname "$0")/gdl-cron.sh" stories "$@"
@@ -0,0 +1,57 @@
 #!/usr/bin/env python3
 """
 Detect a silently dead Instagram session from a stories run's output.
 A dead session (killed server-side while the sessionid is still in the cookie jar, so
 check-session.py passes) shows up as every fetched stories source returning
 "No results" -- the daily run still exits 0. From 2026-10-07 all six profiles came back
 empty every day and nothing alerted. One all-empty day is normal (nobody posted a story);
 several in a row across ALL sources is not.
 Usage: stories-streak.py <this run's output file> [state file]
 Counts at most one day per calendar date (manual re-runs don't inflate it). Prints a message
 and exits 1 once the streak reaches GDL_EMPTY_STREAK_FAIL days (default 3); exits 0 otherwise.
 Runs that fetched nothing (all skipped / no sources) leave the streak untouched.
 """
 import datetime as dt
 import os
 import re
 import sys
 from pathlib import Path
 THRESHOLD = int(os.environ.get("GDL_EMPTY_STREAK_FAIL", "3"))
 def main() -> int:
    out = Path(sys.argv[1]).read_text(errors="replace")
    state = Path(sys.argv[2] if len(sys.argv) > 2
                 else Path.home() / "gdl" / "stories-empty-streak")
    m = re.search(r"sources\s*:\s*(\d+) to sync", out)
    to_sync = int(m.group(1)) if m else 0
    if to_sync == 0:
        return 0                                    # nothing fetched: no evidence either way
    empty = len(re.findall(r"No results for \S*/stories/", out))
    today = dt.date.today().isoformat()
    last_date, streak = "", 0
    try:
        last_date, s = state.read_text().split()
        streak = int(s)
    except (OSError, ValueError):
        pass
    if empty >= to_sync:                            # every fetched source came back empty
        if last_date != today:
            streak += 1
    else:
        streak = 0
    state.write_text(f"{today} {streak}\n")
    if streak >= THRESHOLD:
        print(f"SESSION LIKELY DEAD: stories came back empty for ALL {to_sync} profile(s) on "
              f"{streak} consecutive day(s). The cookie is present but Instagram is probably "
              "no longer honouring it -- log in again in the Chrome window (reload the tab to "
              "check) and re-run.")
        return 1
    return 0
 if __name__ == "__main__":
    sys.exit(main())
@@ -8,7 +8,7 @@ Wants=network-online.target
 [Service]
 Type=oneshot
-ExecStart=%h/gdl/gdl-cron.sh %i
+ExecStart=%h/gdl/run-%i.sh
 # A sync has no deadline and the pacing is deliberately slow; a full sweep can
 # run for hours. Never let systemd kill one midway -- a half-published run is
 # the one state the publish step is designed to avoid.
@@ -1,9 +1,17 @@
 [Unit]
-Description=Monthly Instagram profile sync (all surfaces, --abort 50)
+Description=Weekly Instagram profile sync (all surfaces, --abort 50)
 [Timer]
-OnCalendar=*-*-03 04:00:00
+# Weekly, but deliberately not pinned to a day: OnCalendar anchors to Monday
-RandomizedDelaySec=45m
+# 00:00:00 and RandomizedDelaySec spreads the actual fire up to 6 days later,
 # so which day it lands on varies week to week -- same "don't look like a
 # machine" reasoning as the stories timer's 45m jitter, just applied to the
 # day instead of the minute. See TOOLING.md's healthchecks.io section: the
 # matching instagram-archiver-posts check uses Simple Schedule (period +
 # grace) rather than Cron Schedule for exactly this reason -- a fixed cron
 # day/time would fight this jitter instead of tolerating it.
 OnCalendar=weekly
 RandomizedDelaySec=6d
 Persistent=true
 [Install]
@@ -0,0 +1,9 @@
 [Unit]
 Description=Upgrade gallery-dl and yt-dlp (pipx)
 After=network-online.target
 Wants=network-online.target
 [Service]
 Type=oneshot
 ExecStart=%h/gdl/gdl-upgrade.sh
 Nice=10
@@ -0,0 +1,12 @@
 [Unit]
 Description=Weekly gallery-dl and yt-dlp upgrade
 # gdl-cron.sh also upgrades before every scrape; this keeps the tools current
 # on weeks when no run happens to call it.
 [Timer]
 OnCalendar=weekly
 RandomizedDelaySec=1d
 Persistent=true
 [Install]
 WantedBy=timers.target
@@ -12,10 +12,12 @@ whose absence got the archive's Instagram account suspended.
 import datetime as dt
 import importlib.util
 import json
 import subprocess
 import sys
 import tempfile
 import unittest
 from pathlib import Path
 from unittest import mock
 _spec = importlib.util.spec_from_file_location(
    "gdl_sync", Path(__file__).with_name("gdl-sync.py"))
@@ -208,6 +210,48 @@ class Publishing(unittest.TestCase):
            self.assertIn(pattern, cmd)
        self.assertIn("--dry-run", cmd)
    def _publish_with(self, returncode, stderr=""):
        with tempfile.TemporaryDirectory() as d:
            staging = Path(d)
            (staging / "file.txt").write_text("x")  # publish() skips empty dirs
            completed = subprocess.CompletedProcess(
                args=["rsync"], returncode=returncode, stdout="", stderr=stderr)
            with mock.patch("subprocess.run", return_value=completed):
                return gdl.publish(staging, "host:/dest", dry_run=False)
    def test_a_clean_rsync_is_success(self):
        self.assertEqual(self._publish_with(0), 0)
    def test_the_expected_chown_warning_is_not_a_failure(self):
        # The exact shape seen on every real run: the archive share is never
        # owned by the ssh user, so rsync's own chown step fails -- exit 23,
        # "some files/attrs were not transferred" -- even though every file
        # landed. TOOLING.md documents this as something to read past; the
        # exit CODE needs to agree, or every healthcheck ping reports a
        # clean run as a failure.
        stderr = (
            'rsync: failed to set permissions on "/dest/.": Operation not permitted (1)\n'
            'rsync: failed to set permissions on "/dest/user": Operation not permitted (1)\n'
            'rsync error: some files/attrs were not transferred (see previous errors) '
            '(code 23) at main.c(1347) [sender=3.4.1]\n'
        )
        self.assertEqual(self._publish_with(23, stderr), 0)
    def test_a_real_transfer_failure_still_fails(self):
        # Same exit code (23) can also mean a real problem -- e.g. a file
        # vanishing mid-transfer -- so only the SPECIFIC permission-warning
        # shape above is masked, not the exit code in general.
        stderr = (
            'rsync: [sender] readlink_stat("/stage/broken") failed: '
            'No such file or directory (2)\n'
            'rsync error: some files/attrs were not transferred (see previous errors) '
            '(code 23) at main.c(1347) [sender=3.4.1]\n'
        )
        self.assertEqual(self._publish_with(23, stderr), 23)
    def test_an_unrelated_exit_code_still_fails(self):
        self.assertEqual(self._publish_with(1, "rsync: some other error\n"), 1)
 class PostUrl(unittest.TestCase):
    """--post-url: a one-off fetch outside the tracked profile list, whose
@@ -232,6 +276,37 @@ class PostUrl(unittest.TestCase):
            self.assertNotIn(str(staging / subcategory), cmd)
 class MetadataFields(unittest.TestCase):
    """Extra fields captured from the raw API response, verified against two
    saved real examples on 2026-09-01 (see docs/gallery-dl.md)."""
    def test_post_level_json_captures_coauthors(self):
        config = gdl.build_config("1M", [6.0, 10.0], [3.0, 6.0])
        pps = config["extractor"]["instagram"]["posts"]["postprocessors"]
        post_json = next(pp for pp in pps if pp.get("filename", "").endswith(".json"))
        self.assertIn("coauthors", post_json["include"])
    def test_per_item_dimensions_and_tags_get_their_own_sidecar(self):
        config = gdl.build_config("1M", [6.0, 10.0], [3.0, 6.0])
        pps = config["extractor"]["instagram"]["posts"]["postprocessors"]
        # No "filename" of its own -- unlike the other two, which are keyed
        # by POST_STEM -- because it must vary per carousel item, not per post.
        media_pp = next(pp for pp in pps if "filename" not in pp)
        for field in ("width", "height", "width_original", "height_original",
                      "tagged_users", "shortcode", "num"):
            self.assertIn(field, media_pp["include"])
        # `owner` is a full user object (profile pic URLs, privacy flags) for
        # whoever posted that item -- deliberately excluded, same reasoning
        # as `audio_user` on the post-level json.
        self.assertNotIn("owner", media_pp["include"])
    def test_reels_get_the_same_capture_as_posts(self):
        config = gdl.build_config("1M", [6.0, 10.0], [3.0, 6.0])
        posts_pps = config["extractor"]["instagram"]["posts"]["postprocessors"]
        reels_pps = config["extractor"]["instagram"]["reels"]["postprocessors"]
        self.assertEqual(posts_pps, reels_pps)
 class UrlsFile(unittest.TestCase):
    def test_reads_every_form_a_person_might_paste(self):
        with tempfile.TemporaryDirectory() as d:
@@ -251,7 +326,7 @@ class UrlsFile(unittest.TestCase):
 class PostUrlsFile(unittest.TestCase):
-    """The format reels-scrape.py writes: whole URLs, not usernames."""
+    """The format instagram-chrome-scrape.py writes: whole URLs, not usernames."""
    def test_skips_comments_blanks_and_duplicates(self):
        with tempfile.TemporaryDirectory() as d:
@@ -268,5 +343,43 @@ class PostUrlsFile(unittest.TestCase):
            ])
 class ProgressOutput(unittest.TestCase):
    """The heartbeat that stops a healthy paced run (12-20s between requests, no per-file
    output) from looking hung."""
    def test_elapsed_formatting(self):
        self.assertEqual(gdl._fmt_elapsed(5), "5s")
        self.assertEqual(gdl._fmt_elapsed(125), "2m05s")
        self.assertEqual(gdl._fmt_elapsed(7300), "2h01m")
    def test_count_new_counts_media_written_since_and_ignores_sidecars(self):
        with tempfile.TemporaryDirectory() as d:
            root = Path(d)
            (root / "sub").mkdir()
            old = root / "old.jpg"
            old.write_text("x")
            import os
            os.utime(old, (1, 1))                              # long before `since`
            (root / "sub" / "new.jpg").write_text("x")
            (root / "sub" / "new.mp4").write_text("x")
            for sidecar in ("new.txt", "new.json", "half.jpg.part"):
                (root / "sub" / sidecar).write_text("x")
            n, newest = gdl._count_new(root, since=1000)
            self.assertEqual(n, 2)
            self.assertIn(newest, ("new.jpg", "new.mp4"))
    def test_heartbeat_prints_while_running_and_stops_cleanly(self):
        with tempfile.TemporaryDirectory() as d:
            with mock.patch("builtins.print") as pr:
                with gdl.Heartbeat("fetching", watch=Path(d), every=0.05) as hb:
                    (Path(d) / "a.jpg").write_text("x")
                    import time
                    time.sleep(0.2)
            lines = [str(c.args[0]) for c in pr.call_args_list]
            self.assertTrue(any("fetching" in s and "new file(s) so far" in s for s in lines))
            self.assertFalse(hb._thread.is_alive())
            self.assertGreater(hb.elapsed, 0.1)
 if __name__ == "__main__":
    unittest.main(verbosity=2)
@@ -0,0 +1,71 @@
 #!/usr/bin/env python3
 """
 Tests for the pure parts of instagram-chrome-scrape.py -- everything except the actual
 CDP session, which needs a live signed-in Chrome and is exercised by hand.
    python3 -m unittest discover -s scripts -p 'test_*.py'
 """
 import importlib.util
 import sys
 import unittest
 from pathlib import Path
 _spec = importlib.util.spec_from_file_location(
    "instagram_chrome_scrape", Path(__file__).with_name("instagram-chrome-scrape.py"))
 scrape = importlib.util.module_from_spec(_spec)
 sys.modules["instagram_chrome_scrape"] = scrape
 _spec.loader.exec_module(scrape)
 class ExtractShortcodes(unittest.TestCase):
    def test_reads_relative_and_absolute_hrefs(self):
        hrefs = [
            "/someuser/reel/ABC123/",
            "https://www.instagram.com/someuser/reel/DEF456/",
            "/reel/GHI789/?img_index=1",
        ]
        self.assertEqual(scrape.extract_shortcodes(hrefs),
                         ["ABC123", "DEF456", "GHI789"])
    def test_ignores_non_reel_links(self):
        hrefs = ["/someuser/", "/someuser/p/ABC123/", "/explore/tags/foo/"]
        self.assertEqual(scrape.extract_shortcodes(hrefs), [])
    def test_empty_input(self):
        self.assertEqual(scrape.extract_shortcodes([]), [])
    def test_posts_kinds_read_both_post_and_reel_links(self):
        hrefs = ["/someuser/p/ABC123/", "/p/DEF456/?img_index=2",
                 "/someuser/reel/GHI789/", "/explore/tags/foo/"]
        self.assertEqual(scrape.extract_shortcodes(hrefs, ("p", "reel")),
                         ["ABC123", "DEF456", "GHI789"])
 class ShouldStopKnown(unittest.TestCase):
    have = {"OLD1", "OLD2", "OLD3", "OLD4"}
    def step(self, round_no, new, streak=0, max_known=2):
        return scrape.should_stop_known(round_no, new, self.have, streak, max_known)
    def test_first_round_never_counts(self):
        # pinned posts: old, but on top of the grid
        self.assertEqual(self.step(1, ["OLD1", "OLD2"]), (False, 0))
    def test_stops_after_consecutive_known_rounds(self):
        stop, streak = self.step(2, ["OLD1"])
        self.assertEqual((stop, streak), (False, 1))
        self.assertEqual(self.step(3, ["OLD2"], streak), (True, 2))
    def test_unarchived_resets_streak(self):
        self.assertEqual(self.step(3, ["OLD3", "NEW1"], streak=1), (False, 0))
    def test_empty_round_leaves_streak(self):
        self.assertEqual(self.step(3, [], streak=1), (False, 1))
    def test_disabled(self):
        self.assertEqual(self.step(5, ["OLD1"], streak=9, max_known=0), (False, 9))
 if __name__ == "__main__":
    unittest.main(verbosity=2)
@@ -1,40 +0,0 @@
 #!/usr/bin/env python3
 """
 Tests for the pure parts of reels-scrape.py -- everything except the actual
 CDP session, which needs a live signed-in Chrome and is exercised by hand.
    python3 -m unittest discover -s scripts -p 'test_*.py'
 """
 import importlib.util
 import sys
 import unittest
 from pathlib import Path
 _spec = importlib.util.spec_from_file_location(
    "reels_scrape", Path(__file__).with_name("reels-scrape.py"))
 scrape = importlib.util.module_from_spec(_spec)
 sys.modules["reels_scrape"] = scrape
 _spec.loader.exec_module(scrape)
 class ExtractShortcodes(unittest.TestCase):
    def test_reads_relative_and_absolute_hrefs(self):
        hrefs = [
            "/someuser/reel/ABC123/",
            "https://www.instagram.com/someuser/reel/DEF456/",
            "/reel/GHI789/?img_index=1",
        ]
        self.assertEqual(scrape.extract_shortcodes(hrefs),
                         ["ABC123", "DEF456", "GHI789"])
    def test_ignores_non_reel_links(self):
        hrefs = ["/someuser/", "/someuser/p/ABC123/", "/explore/tags/foo/"]
        self.assertEqual(scrape.extract_shortcodes(hrefs), [])
    def test_empty_input(self):
        self.assertEqual(scrape.extract_shortcodes([]), [])
 if __name__ == "__main__":
    unittest.main(verbosity=2)
@@ -0,0 +1,160 @@
 """
 Tests for the two guards that stop a dead Instagram session going unnoticed:
  check-session.py    preflight: is there a usable `sessionid` in the Chrome cookie DB?
  stories-streak.py   postflight: did every stories source come back empty for days running?
 Both are run as subprocesses (the way gdl-cron.sh runs them); neither touches the network.
    python3 -I -m unittest scripts/test_session_guards.py
 """
 import datetime as dt
 import os
 import sqlite3
 import subprocess
 import sys
 import tempfile
 import time
 import unittest
 from pathlib import Path
 HERE = Path(__file__).resolve().parent
 CHROME_EPOCH = 11644473600  # seconds between 1601-01-01 and 1970-01-01
 def run(script, *args, env=None):
    e = {**os.environ, **(env or {})}
    return subprocess.run([sys.executable, str(HERE / script), *args],
                          capture_output=True, text=True, env=e)
 class CheckSession(unittest.TestCase):
    def profile(self, rows):
        """A fake Chrome profile whose cookie DB holds `rows` = [(host, name, expires_unix|None)]."""
        tmp = tempfile.TemporaryDirectory()
        self.addCleanup(tmp.cleanup)
        d = Path(tmp.name) / "Default"
        d.mkdir()
        con = sqlite3.connect(d / "Cookies")
        con.execute("CREATE TABLE cookies (host_key TEXT, name TEXT, expires_utc INTEGER, has_expires INTEGER)")
        for host, name, exp in rows:
            con.execute("INSERT INTO cookies VALUES (?,?,?,?)",
                        (host, name, 0 if exp is None else (exp + CHROME_EPOCH) * 1_000_000,
                         0 if exp is None else 1))
        con.commit()
        con.close()
        return tmp.name
    def check(self, rows):
        return run("check-session.py", env={"GDL_COOKIE_PROFILE": self.profile(rows)})
    def test_valid_sessionid_passes(self):
        r = self.check([(".instagram.com", "sessionid", time.time() + 200 * 86400)])
        self.assertEqual(r.returncode, 0, r.stdout)
        self.assertIn("session ok", r.stdout)
        self.assertNotIn("WARNING", r.stdout)
    def test_missing_sessionid_fails_even_with_other_instagram_cookies(self):
        # What the real failure looked like: csrftoken/ds_user_id/mid linger after logout.
        r = self.check([(".instagram.com", "csrftoken", time.time() + 86400),
                        (".instagram.com", "ds_user_id", time.time() + 86400),
                        (".instagram.com", "mid", time.time() + 86400)])
        self.assertEqual(r.returncode, 1)
        self.assertIn("NOT LOGGED IN", r.stdout)
    def test_expired_sessionid_fails(self):
        r = self.check([(".instagram.com", "sessionid", time.time() - 86400)])
        self.assertEqual(r.returncode, 1)
        self.assertIn("expired", r.stdout)
    def test_near_expiry_warns_but_passes(self):
        r = self.check([(".instagram.com", "sessionid", time.time() + 3 * 86400)])
        self.assertEqual(r.returncode, 0)
        self.assertIn("WARNING", r.stdout)
    def test_session_cookie_without_expiry_passes(self):
        r = self.check([(".instagram.com", "sessionid", None)])
        self.assertEqual(r.returncode, 0)
    def test_other_sites_sessionid_does_not_count(self):
        r = self.check([(".example.com", "sessionid", time.time() + 86400)])
        self.assertEqual(r.returncode, 1)
    def test_missing_profile_fails_loudly(self):
        r = run("check-session.py", env={"GDL_COOKIE_PROFILE": "/nonexistent/profile"})
        self.assertEqual(r.returncode, 1)
        self.assertIn("no cookie DB", r.stdout)
    def test_never_prints_cookie_values(self):
        # The fake DB has no value column at all; the script must not need one.
        r = self.check([(".instagram.com", "sessionid", time.time() + 90 * 86400)])
        self.assertNotIn("sessionid=", r.stdout + r.stderr)
 class StoriesStreak(unittest.TestCase):
    def setUp(self):
        tmp = tempfile.TemporaryDirectory()
        self.addCleanup(tmp.cleanup)
        self.dir = Path(tmp.name)
        self.state = self.dir / "streak"
        self.today = dt.date.today().isoformat()
        self.yesterday = (dt.date.today() - dt.timedelta(days=1)).isoformat()
    def out(self, to_sync, empty, skipped=0):
        lines = [f"sources  : {to_sync} to sync, {skipped} skipped"]
        lines += [f"[instagram][info] No results for https://www.instagram.com/stories/user{i}/"
                  for i in range(empty)]
        p = self.dir / "run.out"
        p.write_text("\n".join(lines) + "\n")
        return p
    def streak(self, to_sync, empty, skipped=0):
        return run("stories-streak.py", str(self.out(to_sync, empty, skipped)), str(self.state))
    def stored(self):
        return self.state.read_text().split()
    def test_all_empty_for_three_days_fails(self):
        self.state.write_text(f"{self.yesterday} 2\n")
        r = self.streak(6, 6)
        self.assertEqual(r.returncode, 1)
        self.assertIn("SESSION LIKELY DEAD", r.stdout)
        self.assertEqual(self.stored(), [self.today, "3"])
    def test_first_empty_day_is_quiet(self):
        r = self.streak(6, 6)
        self.assertEqual(r.returncode, 0)
        self.assertEqual(self.stored(), [self.today, "1"])
    def test_same_day_rerun_does_not_inflate_the_streak(self):
        self.state.write_text(f"{self.today} 2\n")
        r = self.streak(6, 6)
        self.assertEqual(r.returncode, 0)
        self.assertEqual(self.stored(), [self.today, "2"])
    def test_any_story_resets_the_streak(self):
        self.state.write_text(f"{self.yesterday} 2\n")
        r = self.streak(6, 5)              # one profile had a story
        self.assertEqual(r.returncode, 0)
        self.assertEqual(self.stored(), [self.today, "0"])
    def test_run_that_fetched_nothing_leaves_state_alone(self):
        self.state.write_text(f"{self.yesterday} 2\n")
        r = self.streak(0, 0, skipped=6)
        self.assertEqual(r.returncode, 0)
        self.assertEqual(self.stored(), [self.yesterday, "2"])
    def test_threshold_is_configurable(self):
        r = run("stories-streak.py", str(self.out(6, 6)), str(self.state),
                env={"GDL_EMPTY_STREAK_FAIL": "1"})
        self.assertEqual(r.returncode, 1)
    def test_corrupt_state_file_is_treated_as_no_history(self):
        self.state.write_text("garbage\n")
        r = self.streak(6, 6)
        self.assertEqual(r.returncode, 0)
        self.assertEqual(self.stored(), [self.today, "1"])
 if __name__ == "__main__":
    unittest.main()
Author	SHA1	Message	Date
ergosteur	fe6ebdd75b	tooling: scheduled profiles/full-sweep list posts+reels by browser (chrome-sync per profile/tab, early stop for weekly); GDL_LISTING=api fallback; reels-tab sweep note in MANUAL-RUN	2026-10-23 00:49:45 -04:00
ergosteur	1b67765d8b	tooling: instagram-chrome-scrape.py (was reels-scrape.py) gains --tab posts with early stop; chrome-sync.sh + reels/posts-sync.sh wrappers; MANUAL-RUN.md; atomic-deploy note	2026-10-22 23:53:35 -04:00
ergosteur	30c9fbe611	tooling: auto-upgrade gallery-dl + yt-dlp before every run and weekly; document 2026-10-22 empty posts listing	2026-10-22 23:24:15 -04:00
ergosteur	556eab1169	docs: video upgrade pass — findings, decisions, open items (not built)	2026-10-14 21:39:28 -04:00
ergosteur	99719d164b	docs: scraping account replaced by erg05teur on 2026-10-14; warm-up rules apply	2026-10-14 21:34:14 -04:00
ergosteur	5c3307685a	docs: profiles timer now weekly on the host; Persistent catch-up on restart	2026-10-14 21:25:57 -04:00
ergosteur	832bbdb391	docs: 2026-10-14 session-death incident, guards, resolution findings Document the new exit codes (76/77), the wrapper/unit fix, recovery steps, the verbose output, and the archive-resolution check (images match a logged-in fetch; 1080p VP9 exists for newer videos; iOS plays VP9+Opus webm but not VP9+AAC mp4). Correct the stale "timers are disabled" line, and flag that the repo's weekly profiles timer was never installed on the host.	2026-10-14 21:02:58 -04:00
ergosteur	4a2bd7aa19	feat: guard against a dead Instagram session, show progress, fix missing pings - check-session.py: preflight in gdl-cron.sh. No unexpired sessionid in the Chrome profile gdl reads -> abort before any request, ping /76 (red check). - stories-streak.py: postflight. Every fetched stories source empty for 3 consecutive days -> exit 77, for a session Instagram stopped honouring while the cookie is still present. The 2026-10-07 session death went unnoticed for a week because every run still exited 0. - gdl-sync.py: [n/N] source headers, a 60s heartbeat with new-file counts, and per-source timing, so a healthy paced run no longer looks hung. - gdl-sync@.service now runs run-%i.sh instead of gdl-cron.sh: HC_URL is set in the wrappers, so scheduled runs had never pinged Healthchecks. - Tests for both guards (offline) and the progress helpers.	2026-10-14 21:02:58 -04:00
ergosteurandClaude Sonnet 5	fcfb8e5fd3	chore: update profiles cadence from monthly to weekly, randomized day Matches the instagram-archiver-posts healthchecks.io check's Simple Schedule (period + grace, day-agnostic). The disabled systemd timer now spreads across the whole week (OnCalendar=weekly + 6d randomized delay) rather than pinning to a fixed day, same "don't look like a machine" reasoning as the stories timer's 45m jitter -- just applied to the day instead of the minute. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011qAds5qr7nZRq5R4yAuxUk	2026-09-14 18:26:07 -04:00
ergosteurandClaude Sonnet 5	9b2c33766f	feat: add healthchecks.io pings to run-stories.sh and run-profiles.sh Start/finish pings wired through gdl-cron.sh via HC_URL, finish ping carries the run's log as its body (visible in the dashboard, no SSH needed to diagnose a failure). Exit code is the trailing URL segment by convention, so the existing "stories: 0 to sync" -> exit 75 remap already reaches the dashboard as a failure too. Also fixes the actual bug that would have made this useless: the routine chown-permission rsync warning (the archive share is never owned by the ssh user) made gdl-sync.py return 1 on every clean run, not just failing ones -- harmless when a human reads the log, but it would have pinged every successful run as a failure. publish() now only masks that one specific, verified-benign error shape; a real transfer failure sharing the same exit code (23) still fails loudly. 6 new tests. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011qAds5qr7nZRq5R4yAuxUk	2026-09-14 14:01:17 -04:00
ergosteurandClaude Sonnet 5	8275416b6e	chore: migrate HeeJin's archive from 0ct0ber19 to heejin Her Instagram handle changed and the old one is now permanently dead (no redirect, "Sorry, this page isn't available"). Verified @heejin is her before touching anything. Renamed 3,353 files across her archive directories on the NAS, matching only the exact username token so real collab posts physically stored there but owned by someone else were correctly left untouched. Updated both account-link files to the new URL. Full record in TOOLING.md. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011qAds5qr7nZRq5R4yAuxUk	2026-09-14 13:20:32 -04:00
ergosteurandClaude Sonnet 5	5148b677ce	docs: record that the reels/posts API block is fixed upstream gallery-dl 1.32.12 (we were on 1.32.9, three releases behind) resolved the "HTTP redirect to home page" failures on both posts and reels -- its changelog cites exactly this ("posts & reels extractors... content retrieval failures"). Verified with a full run-profiles.sh across all six profiles: zero redirect errors, first time since 2026-08-26. The browser-scrolling workaround (reels-scrape.py/reels-sync.sh/ run-reels-all.sh) is no longer needed for routine use as a result, but stays in place as a fallback. Updated the quick-reference commands to drop run-reels-all.sh from the routine list; kept the original diagnosis section intact as history, with a note at the top pointing to the fix. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011qAds5qr7nZRq5R4yAuxUk	2026-09-14 02:03:37 -04:00
ergosteurandClaude Sonnet 5	03dc023620	refactor: prefix the manual entry-point scripts with run- Makes it unambiguous at a glance that these are meant to be run by hand, not something a timer or another script calls. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011qAds5qr7nZRq5R4yAuxUk	2026-09-08 07:55:46 -04:00
ergosteurandClaude Sonnet 5	1a3f85b3aa	feat: add fixed-mode entry point scripts for the usual run modes stories.sh / profiles.sh / full-sweep.sh wrap gdl-cron.sh with the mode already chosen, so it never has to be remembered or typed. reels-all.sh runs reels-sync.sh once per profile in artms_account_links.txt (reusing gdl-sync.py's own URL-file parser, so it can't drift from what profiles.sh tracks), the scripted version of the six-profile loop run by hand on 2026-09-03. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011qAds5qr7nZRq5R4yAuxUk	2026-09-08 07:15:25 -04:00
ergosteurandClaude Sonnet 5	84dd76e11e	feat: capture coauthors and per-item width/height/tagged_users Two additions to the metadata sidecars, following up on the 2026-09-01 collab/dedup investigation already documented here: - coauthors added to the post-level JSON's include list -- a direct, native signal for collab detection instead of inferring it from the filename/directory identity mismatch also documented that day. - width, height, width_original, height_original and tagged_users get their own per-file sidecar (event: "file", one JSON per downloaded media item, named "<filename>.json"), since a carousel's items can each have different dimensions and tags -- data the existing post-level JSON has no way to represent. owner is deliberately left out, same reasoning as audio_user's existing exclusion. Verified against a live re-fetch of an already-archived carousel: correct per-item data came back, zero media re-downloaded (skip-archive still applies; only the new sidecars are new files). 3 new tests. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011qAds5qr7nZRq5R4yAuxUk	2026-09-01 23:21:22 -04:00
ergosteur	d92fed1934	docs: document collab naming mismatch, cross-profile dedup gap, and VP9 Three findings from the 2026-09-01 collab-detection/VP9 investigation in Cosmo-Live-Downloads, all rooted in gallery-dl's own scraping output: - Collab posts' JSON is filed under the true owner's identity but the media file under the scraped account's — plus coauthors is a native, uncaptured signal that already excludes the owner for us. - The shared archive-db dedups by media_id across all profiles, not per profile, so a post reachable from two profiles' listings only gets its media written into whichever profile's crawl reached it first (510 such posts found archive-wide). - Some video_versions entries are VP9; format selection is codec-blind, and Safari's VP9 decode only wires into its WebM demuxer, never MP4. Also commits the two saved raw API response examples and the missing- captions CSV generated during that investigation.	2026-09-01 23:05:19 -04:00