feat: add fixed-mode entry point scripts for the usual run modes
stories.sh / profiles.sh / full-sweep.sh wrap gdl-cron.sh with the mode already chosen, so it never has to be remembered or typed. reels-all.sh runs reels-sync.sh once per profile in artms_account_links.txt (reusing gdl-sync.py's own URL-file parser, so it can't drift from what profiles.sh tracks), the scripted version of the six-profile loop run by hand on 2026-09-03. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011qAds5qr7nZRq5R4yAuxUk
This commit is contained in:
+24
-9
@@ -30,14 +30,22 @@ with no deploy or restart.
|
||||
```sh
|
||||
ssh mattellite
|
||||
|
||||
~/gdl/gdl-cron.sh stories # ~1 min. DO THIS OFTEN - stories die in 24h
|
||||
~/gdl/gdl-cron.sh profiles # ~5 min. posts, reels, highlights, stories
|
||||
~/gdl/stories.sh # ~1 min. DO THIS OFTEN - stories die in 24h
|
||||
~/gdl/profiles.sh # ~5 min. posts, highlights, stories (not reels -- see below)
|
||||
~/gdl/reels-all.sh # ~15-20 min. reels, all six profiles, by scrolling
|
||||
~/gdl/full-sweep.sh # rarely, ~420 requests. Read its own comment first.
|
||||
```
|
||||
|
||||
`profiles` covers everything, so it is the one to run if you only run one — but
|
||||
it is no substitute for `stories`, because a story posted and expired between
|
||||
two `profiles` runs is simply gone. Both are safe to run back-to-back; the
|
||||
`--min-interval` floor makes a repeat a no-op rather than a re-fetch.
|
||||
Fixed-mode entry points around `gdl-cron.sh`/`reels-sync.sh`, so the mode
|
||||
never has to be remembered or typed — each just runs its own thing with the
|
||||
right flags already set. `profiles.sh` covers every surface gallery-dl's API
|
||||
still reaches, so it is the one to run if you only run one, but it is no
|
||||
substitute for `stories.sh` (a story posted and expired between two runs of
|
||||
it is simply gone) or `reels-all.sh` (`/api/v1/clips/user/` is blocked --
|
||||
see "Reels: the API is blocked" below -- `profiles.sh` silently fails that
|
||||
one surface every time, by design, until that changes). All four are safe to
|
||||
run back-to-back; the `--min-interval` floor makes a repeat a no-op rather
|
||||
than a re-fetch.
|
||||
|
||||
Then check <https://instaarchive.ergosteur.com> — new posts appear without a
|
||||
redeploy. The archive index re-reads a directory when its mtime changes.
|
||||
@@ -54,9 +62,10 @@ redeploy. The archive index re-reads a directory when its mtime changes.
|
||||
- `400 Bad Request` on `/api/v1/feed/reels_media/` — the account is behind a
|
||||
scraping-warning interstitial; check the browser before anything else.
|
||||
|
||||
Long runs: `setsid nohup ~/gdl/gdl-cron.sh profiles >/dev/null 2>&1 &` and
|
||||
`tail -f ~/gdl/logs/$(ls -1t ~/gdl/logs | head -1)`. Nothing reaches the
|
||||
archive until a run finishes, so killing one midway is safe.
|
||||
Long runs: `setsid nohup ~/gdl/profiles.sh >/dev/null 2>&1 &` (or `full-sweep.sh`
|
||||
/ `reels-all.sh`) and `tail -f ~/gdl/logs/$(ls -1t ~/gdl/logs | head -1)`.
|
||||
Nothing reaches the archive until a run finishes, so killing one midway is
|
||||
safe.
|
||||
|
||||
## Safety net: a global gallery-dl config
|
||||
|
||||
@@ -575,6 +584,12 @@ slower by design (real scrolling, not an API call) — both good reasons not
|
||||
to run this unattended without deciding that deliberately. Today it is a
|
||||
per-profile, by-hand tool only.
|
||||
|
||||
For all six ARTMS profiles in one go, `~/gdl/reels-all.sh` just calls
|
||||
`reels-sync.sh` once per profile in `artms_account_links.txt`, same
|
||||
reasoning as above — real scrolling, so still by hand. Run 2026-09-03:
|
||||
`official_artms` alone has 609 real reels and took ~14 minutes of scrolling;
|
||||
the whole six-profile run took about 15 minutes total, 2 new reels found.
|
||||
|
||||
### Where a shared reel lands
|
||||
|
||||
A reel found on one profile's `/reels/` page is not necessarily *owned* by
|
||||
|
||||
Executable
+8
@@ -0,0 +1,8 @@
|
||||
#!/bin/bash
|
||||
# Rarely: every surface, no --abort -- walks each profile to the end, ~420
|
||||
# requests. The only run that notices posts EDITED after they were archived
|
||||
# (e.g. a caption added later -- see TOOLING.md "Reels: the API is blocked"
|
||||
# for how that showed up), and by far the most expensive thing here. Read
|
||||
# TOOLING.md before running this; prefer running it by hand, not on a timer.
|
||||
set -eu
|
||||
exec "$(dirname "$0")/gdl-cron.sh" full-sweep "$@"
|
||||
Executable
+6
@@ -0,0 +1,6 @@
|
||||
#!/bin/bash
|
||||
# Routine run: every surface, --abort 50 (only gdl-cron.sh's "profiles" mode
|
||||
# does this), ~5 min. Covers everything but is no substitute for stories.sh --
|
||||
# a story posted and expired between two runs of this is simply gone.
|
||||
set -eu
|
||||
exec "$(dirname "$0")/gdl-cron.sh" profiles "$@"
|
||||
Executable
+56
@@ -0,0 +1,56 @@
|
||||
#!/bin/bash
|
||||
# Run reels-sync.sh for every tracked ARTMS profile, one after another.
|
||||
# TOOLING.md ("Reels: the API is blocked, scrape by scrolling instead")
|
||||
# covers what this actually does and why it exists; this is just the loop
|
||||
# version of running reels-sync.sh six times by hand, which is what the
|
||||
# 2026-09-03 run was before this script existed.
|
||||
#
|
||||
# Deliberately not wired into gdl-cron.sh or a timer -- same reasoning as
|
||||
# reels-sync.sh itself: it drives your actual browser session and is slower
|
||||
# by design, both good reasons to run it deliberately, not unattended.
|
||||
#
|
||||
# Usage: ./reels-all.sh
|
||||
# GDL_ARTMS_LINKS=/path/to/list ./reels-all.sh # override the profile list
|
||||
set -eu
|
||||
|
||||
GDL_HOME="${GDL_HOME:-$HOME/gdl}"
|
||||
LINKS="${GDL_ARTMS_LINKS:-$GDL_HOME/artms_account_links.txt}"
|
||||
GDL_PYTHON="${GDL_PYTHON:-$HOME/.local/share/pipx/venvs/gallery-dl/bin/python3}"
|
||||
|
||||
if [ ! -f "$LINKS" ]; then
|
||||
echo "profile list not found: $LINKS" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# Same parser gdl-sync.py uses for --urls-file (URLs or bare usernames, #
|
||||
# comments, blank lines, duplicates dropped), reused so this never drifts
|
||||
# from what a routine profiles.sh run actually tracks.
|
||||
mapfile -t PROFILES < <("$GDL_PYTHON" -c "
|
||||
import sys, importlib.util
|
||||
from pathlib import Path
|
||||
spec = importlib.util.spec_from_file_location('gdl_sync', Path('$GDL_HOME/gdl-sync.py'))
|
||||
gdl = importlib.util.module_from_spec(spec)
|
||||
sys.modules['gdl_sync'] = gdl
|
||||
spec.loader.exec_module(gdl)
|
||||
for p in gdl.read_urls_file(Path('$LINKS')):
|
||||
print(p)
|
||||
")
|
||||
|
||||
if [ "${#PROFILES[@]}" -eq 0 ]; then
|
||||
echo "no usable profiles in $LINKS" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
echo "=== reels-all: ${#PROFILES[@]} profile(s) $(date -Is) ==="
|
||||
status=0
|
||||
for p in "${PROFILES[@]}"; do
|
||||
echo "=== starting $p ==="
|
||||
if ! "$(dirname "$0")/reels-sync.sh" "$p"; then
|
||||
status=1
|
||||
echo "=== $p FAILED ===" >&2
|
||||
fi
|
||||
echo "=== finished $p ==="
|
||||
done
|
||||
|
||||
echo "=== reels-all: exit $status at $(date -Is) ==="
|
||||
exit "$status"
|
||||
Executable
+9
@@ -0,0 +1,9 @@
|
||||
#!/bin/bash
|
||||
# Daily run: stories only, ~1 min. The one surface that cannot be backfilled
|
||||
# -- do this often. See TOOLING.md "Quick reference" for the full picture.
|
||||
#
|
||||
# Just a fixed-mode entry point so you don't have to remember gdl-cron.sh's
|
||||
# mode argument; override pacing/paths the same way gdl-cron.sh does, with
|
||||
# GDL_* env vars (see gdl-cron.sh's own comments for what's available).
|
||||
set -eu
|
||||
exec "$(dirname "$0")/gdl-cron.sh" stories "$@"
|
||||
Reference in New Issue
Block a user