ergosteur/instaarchive-viewer
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs: record that the reels/posts API block is fixed upstream

Browse Source 
gallery-dl 1.32.12 (we were on 1.32.9, three releases behind) resolved
the "HTTP redirect to home page" failures on both posts and reels --
its changelog cites exactly this ("posts & reels extractors... content
retrieval failures"). Verified with a full run-profiles.sh across all
six profiles: zero redirect errors, first time since 2026-08-26.

The browser-scrolling workaround (reels-scrape.py/reels-sync.sh/
run-reels-all.sh) is no longer needed for routine use as a result, but
stays in place as a fallback. Updated the quick-reference commands to
drop run-reels-all.sh from the routine list; kept the original
diagnosis section intact as history, with a note at the top pointing
to the fix.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011qAds5qr7nZRq5R4yAuxUk
This commit is contained in:
ergosteur
2026-09-14 02:03:37 -04:00
co-authored by Claude Sonnet 5
parent 03dc023620
commit 5148b677ce
1 changed files with 35 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
TOOLING.md
+35 -17
View File
@@ -31,21 +31,21 @@ with no deploy or restart.
ssh mattellite ssh mattellite
~/gdl/run-stories.sh # ~1 min. DO THIS OFTEN - stories die in 24h ~/gdl/run-stories.sh # ~1 min. DO THIS OFTEN - stories die in 24h
~/gdl/run-profiles.sh # ~5 min. posts, highlights, stories (not reels -- see below) ~/gdl/run-profiles.sh # ~5 min. posts, reels, highlights, stories
~/gdl/run-reels-all.sh # ~15-20 min. reels, all six profiles, by scrolling
~/gdl/run-full-sweep.sh # rarely, ~420 requests. Read its own comment first. ~/gdl/run-full-sweep.sh # rarely, ~420 requests. Read its own comment first.
``` ```
Fixed-mode entry points around `gdl-cron.sh`/`reels-sync.sh`, so the mode Fixed-mode entry points around `gdl-cron.sh`, so the mode never has to be
never has to be remembered or typed — each just runs its own thing with the remembered or typed — each just runs its own thing with the right flags
right flags already set. `run-profiles.sh` covers every surface gallery-dl's API already set. `run-profiles.sh` covers every surface, so it is the one to run
still reaches, so it is the one to run if you only run one, but it is no if you only run one, but it is no substitute for `run-stories.sh`: a story
substitute for `run-stories.sh` (a story posted and expired between two runs of posted and expired between two `run-profiles.sh` runs is simply gone. Both
it is simply gone) or `run-reels-all.sh` (`/api/v1/clips/user/` is blocked -- are safe to run back-to-back; the `--min-interval` floor makes a repeat a
see "Reels: the API is blocked" below -- `run-profiles.sh` silently fails that no-op rather than a re-fetch.
one surface every time, by design, until that changes). All four are safe to
run back-to-back; the `--min-interval` floor makes a repeat a no-op rather **`run-reels-all.sh` (browser-scrolling) is no longer needed for routine use
than a re-fetch. as of 2026-09-14** — see "Reels: the API is blocked" below, now fixed
upstream. `run-profiles.sh`'s own `reels` surface covers it again.
Then check <https://instaarchive.ergosteur.com> — new posts appear without a Then check <https://instaarchive.ergosteur.com> — new posts appear without a
redeploy. The archive index re-reads a directory when its mtime changes. redeploy. The archive index re-reads a directory when its mtime changes.
@@ -540,14 +540,32 @@ before something worse.
## Reels: the API is blocked, scrape by scrolling instead ## Reels: the API is blocked, scrape by scrolling instead
As of 2026-08-26/27, `gallery-dl`'s dedicated reels extractor fails on every **Fixed upstream 2026-09-14 — read this note before the rest of the section.**
`gallery-dl` 1.32.12 (released 2026-09-12; we were on 1.32.9) resolved it: its
changelog cites "posts & reels extractors... content retrieval failures", and
upgrading (`pipx upgrade gallery-dl`) confirmed it -- a full `run-profiles.sh`
across all six profiles came back with **zero** `HTTP redirect` errors on
posts or reels, first time since 2026-08-26. `reels-scrape.py`/`reels-sync.sh`/
`run-reels-all.sh` are no longer needed for routine use; left in place as a
fallback (and because the whole-archive dedup fix they picked up along the
way, see below, is independently useful). If this regresses, check the
gallery-dl changelog for a newer release before re-reaching for the scroll
workaround -- it may already be fixed there too.
The rest of this section is kept as-is: the diagnosis, the workaround, and
what it taught along the way are all still accurate history, just no longer
the operative fix.
As of 2026-08-26/27, `gallery-dl`'s dedicated reels extractor failed on every
profile with `HTTP redirect to home page` — confirmed hours apart, with a profile with `HTTP redirect to home page` — confirmed hours apart, with a
freshly-warmed session and a correct `X-IG-WWW-Claim` header (that was the freshly-warmed session and a correct `X-IG-WWW-Claim` header (that was the
first suspect; ruled out by tracing the raw HTTP exchange). It is not a first suspect; ruled out by tracing the raw HTTP exchange). It was not a
scraping-warning interstitial — the reels tab loads completely normally in a scraping-warning interstitial — the reels tab loaded completely normally in a
real, already-signed-in browser — so this is Meta blocking the specific real, already-signed-in browser — so this was Meta blocking the specific
`/api/v1/clips/user/` endpoint gallery-dl calls, not an account-health issue. `/api/v1/clips/user/` endpoint gallery-dl calls, not an account-health issue.
Posts, highlights and stories are unaffected; only reels breaks. Posts, highlights and stories were unaffected at the time; only reels broke
(posts later broke too, some time before 2026-09-13, in the same way — same
fix resolved both).
`scripts/reels-scrape.py` works around it by never calling that endpoint: `scripts/reels-scrape.py` works around it by never calling that endpoint:
it drives the same signed-in Chrome via its loopback CDP port (`:9222`, it drives the same signed-in Chrome via its loopback CDP port (`:9222`,