feat: add fixed-mode entry point scripts for the usual run modes

stories.sh / profiles.sh / full-sweep.sh wrap gdl-cron.sh with the
mode already chosen, so it never has to be remembered or typed.
reels-all.sh runs reels-sync.sh once per profile in
artms_account_links.txt (reusing gdl-sync.py's own URL-file parser,
so it can't drift from what profiles.sh tracks), the scripted version
of the six-profile loop run by hand on 2026-09-03.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011qAds5qr7nZRq5R4yAuxUk
This commit is contained in:
2026-09-08 07:15:25 -04:00
co-authored by Claude Sonnet 5
parent 84dd76e11e
commit 1a3f85b3aa
5 changed files with 103 additions and 9 deletions
+24 -9
View File
@@ -30,14 +30,22 @@ with no deploy or restart.
```sh ```sh
ssh mattellite ssh mattellite
~/gdl/gdl-cron.sh stories # ~1 min. DO THIS OFTEN - stories die in 24h ~/gdl/stories.sh # ~1 min. DO THIS OFTEN - stories die in 24h
~/gdl/gdl-cron.sh profiles # ~5 min. posts, reels, highlights, stories ~/gdl/profiles.sh # ~5 min. posts, highlights, stories (not reels -- see below)
~/gdl/reels-all.sh # ~15-20 min. reels, all six profiles, by scrolling
~/gdl/full-sweep.sh # rarely, ~420 requests. Read its own comment first.
``` ```
`profiles` covers everything, so it is the one to run if you only run one — but Fixed-mode entry points around `gdl-cron.sh`/`reels-sync.sh`, so the mode
it is no substitute for `stories`, because a story posted and expired between never has to be remembered or typed — each just runs its own thing with the
two `profiles` runs is simply gone. Both are safe to run back-to-back; the right flags already set. `profiles.sh` covers every surface gallery-dl's API
`--min-interval` floor makes a repeat a no-op rather than a re-fetch. still reaches, so it is the one to run if you only run one, but it is no
substitute for `stories.sh` (a story posted and expired between two runs of
it is simply gone) or `reels-all.sh` (`/api/v1/clips/user/` is blocked --
see "Reels: the API is blocked" below -- `profiles.sh` silently fails that
one surface every time, by design, until that changes). All four are safe to
run back-to-back; the `--min-interval` floor makes a repeat a no-op rather
than a re-fetch.
Then check <https://instaarchive.ergosteur.com> — new posts appear without a Then check <https://instaarchive.ergosteur.com> — new posts appear without a
redeploy. The archive index re-reads a directory when its mtime changes. redeploy. The archive index re-reads a directory when its mtime changes.
@@ -54,9 +62,10 @@ redeploy. The archive index re-reads a directory when its mtime changes.
- `400 Bad Request` on `/api/v1/feed/reels_media/` — the account is behind a - `400 Bad Request` on `/api/v1/feed/reels_media/` — the account is behind a
scraping-warning interstitial; check the browser before anything else. scraping-warning interstitial; check the browser before anything else.
Long runs: `setsid nohup ~/gdl/gdl-cron.sh profiles >/dev/null 2>&1 &` and Long runs: `setsid nohup ~/gdl/profiles.sh >/dev/null 2>&1 &` (or `full-sweep.sh`
`tail -f ~/gdl/logs/$(ls -1t ~/gdl/logs | head -1)`. Nothing reaches the / `reels-all.sh`) and `tail -f ~/gdl/logs/$(ls -1t ~/gdl/logs | head -1)`.
archive until a run finishes, so killing one midway is safe. Nothing reaches the archive until a run finishes, so killing one midway is
safe.
## Safety net: a global gallery-dl config ## Safety net: a global gallery-dl config
@@ -575,6 +584,12 @@ slower by design (real scrolling, not an API call) — both good reasons not
to run this unattended without deciding that deliberately. Today it is a to run this unattended without deciding that deliberately. Today it is a
per-profile, by-hand tool only. per-profile, by-hand tool only.
For all six ARTMS profiles in one go, `~/gdl/reels-all.sh` just calls
`reels-sync.sh` once per profile in `artms_account_links.txt`, same
reasoning as above — real scrolling, so still by hand. Run 2026-09-03:
`official_artms` alone has 609 real reels and took ~14 minutes of scrolling;
the whole six-profile run took about 15 minutes total, 2 new reels found.
### Where a shared reel lands ### Where a shared reel lands
A reel found on one profile's `/reels/` page is not necessarily *owned* by A reel found on one profile's `/reels/` page is not necessarily *owned* by
+8
View File
@@ -0,0 +1,8 @@
#!/bin/bash
# Rarely: every surface, no --abort -- walks each profile to the end, ~420
# requests. The only run that notices posts EDITED after they were archived
# (e.g. a caption added later -- see TOOLING.md "Reels: the API is blocked"
# for how that showed up), and by far the most expensive thing here. Read
# TOOLING.md before running this; prefer running it by hand, not on a timer.
set -eu
exec "$(dirname "$0")/gdl-cron.sh" full-sweep "$@"
+6
View File
@@ -0,0 +1,6 @@
#!/bin/bash
# Routine run: every surface, --abort 50 (only gdl-cron.sh's "profiles" mode
# does this), ~5 min. Covers everything but is no substitute for stories.sh --
# a story posted and expired between two runs of this is simply gone.
set -eu
exec "$(dirname "$0")/gdl-cron.sh" profiles "$@"
+56
View File
@@ -0,0 +1,56 @@
#!/bin/bash
# Run reels-sync.sh for every tracked ARTMS profile, one after another.
# TOOLING.md ("Reels: the API is blocked, scrape by scrolling instead")
# covers what this actually does and why it exists; this is just the loop
# version of running reels-sync.sh six times by hand, which is what the
# 2026-09-03 run was before this script existed.
#
# Deliberately not wired into gdl-cron.sh or a timer -- same reasoning as
# reels-sync.sh itself: it drives your actual browser session and is slower
# by design, both good reasons to run it deliberately, not unattended.
#
# Usage: ./reels-all.sh
# GDL_ARTMS_LINKS=/path/to/list ./reels-all.sh # override the profile list
set -eu
GDL_HOME="${GDL_HOME:-$HOME/gdl}"
LINKS="${GDL_ARTMS_LINKS:-$GDL_HOME/artms_account_links.txt}"
GDL_PYTHON="${GDL_PYTHON:-$HOME/.local/share/pipx/venvs/gallery-dl/bin/python3}"
if [ ! -f "$LINKS" ]; then
echo "profile list not found: $LINKS" >&2
exit 2
fi
# Same parser gdl-sync.py uses for --urls-file (URLs or bare usernames, #
# comments, blank lines, duplicates dropped), reused so this never drifts
# from what a routine profiles.sh run actually tracks.
mapfile -t PROFILES < <("$GDL_PYTHON" -c "
import sys, importlib.util
from pathlib import Path
spec = importlib.util.spec_from_file_location('gdl_sync', Path('$GDL_HOME/gdl-sync.py'))
gdl = importlib.util.module_from_spec(spec)
sys.modules['gdl_sync'] = gdl
spec.loader.exec_module(gdl)
for p in gdl.read_urls_file(Path('$LINKS')):
print(p)
")
if [ "${#PROFILES[@]}" -eq 0 ]; then
echo "no usable profiles in $LINKS" >&2
exit 2
fi
echo "=== reels-all: ${#PROFILES[@]} profile(s) $(date -Is) ==="
status=0
for p in "${PROFILES[@]}"; do
echo "=== starting $p ==="
if ! "$(dirname "$0")/reels-sync.sh" "$p"; then
status=1
echo "=== $p FAILED ===" >&2
fi
echo "=== finished $p ==="
done
echo "=== reels-all: exit $status at $(date -Is) ==="
exit "$status"
+9
View File
@@ -0,0 +1,9 @@
#!/bin/bash
# Daily run: stories only, ~1 min. The one surface that cannot be backfilled
# -- do this often. See TOOLING.md "Quick reference" for the full picture.
#
# Just a fixed-mode entry point so you don't have to remember gdl-cron.sh's
# mode argument; override pacing/paths the same way gdl-cron.sh does, with
# GDL_* env vars (see gdl-cron.sh's own comments for what's available).
set -eu
exec "$(dirname "$0")/gdl-cron.sh" stories "$@"