Compare commits
24
Commits
6b8410da09
...
tooling
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
84dd76e11e | ||
|
|
d92fed1934 | ||
|
|
7638fa4a9a | ||
|
|
e0566f06ac | ||
|
|
784ba43bbc | ||
|
|
2f46123022 | ||
|
|
eceee6ec02 | ||
|
|
0753395391 | ||
|
|
aa062849cb | ||
|
|
b2eae177b6 | ||
|
|
7c7ce70ff9 | ||
|
|
9c574e02eb | ||
|
|
3a7dc749c7 | ||
|
|
627cedd3a4 | ||
|
|
99fb24ad21 | ||
|
|
db0b97dcec | ||
|
|
652747e0f1 | ||
|
|
ef99389cf4 | ||
|
|
ff0d0f9f78 | ||
|
|
f53c47a5ef | ||
|
|
a54249e79d | ||
|
|
84c573b3ed | ||
|
|
26d2d3e379 | ||
|
|
61c2b62141 |
+751
@@ -0,0 +1,751 @@
|
||||
# Tooling branch
|
||||
|
||||
> [!CAUTION]
|
||||
> ## This branch must not be pushed to GitHub
|
||||
>
|
||||
> `tooling` is the only branch that still contains the archive-fetching
|
||||
> scripts and their docs, and those name things `main` was rewritten to
|
||||
> remove:
|
||||
>
|
||||
> - the fetch host's **public IP** (`docs/gallery-dl.md`)
|
||||
> - the browser profile the session cookie is read from
|
||||
> - the NAS archive path
|
||||
> - the **list of Instagram accounts being archived**
|
||||
>
|
||||
> On 2026-08-20 `main`'s entire history was rewritten with `git filter-repo`,
|
||||
> the GitHub repo was deleted and recreated, and 22 container images were
|
||||
> pruned from ghcr — all to get exactly this material out of public view.
|
||||
> **One push of this branch to GitHub undoes all of it.**
|
||||
>
|
||||
> A second cleanup would be harder than the first: after a force-push the old
|
||||
> commits stayed reachable by raw SHA, and only deleting the repository
|
||||
> outright removed them.
|
||||
|
||||
## Quick reference — getting new content onto the site
|
||||
|
||||
Timers are **disabled**, so this is manual today. Everything runs on
|
||||
`mattellite`; it fetches, publishes to the NAS itself, and the site picks it up
|
||||
with no deploy or restart.
|
||||
|
||||
```sh
|
||||
ssh mattellite
|
||||
|
||||
~/gdl/gdl-cron.sh stories # ~1 min. DO THIS OFTEN - stories die in 24h
|
||||
~/gdl/gdl-cron.sh profiles # ~5 min. posts, reels, highlights, stories
|
||||
```
|
||||
|
||||
`profiles` covers everything, so it is the one to run if you only run one — but
|
||||
it is no substitute for `stories`, because a story posted and expired between
|
||||
two `profiles` runs is simply gone. Both are safe to run back-to-back; the
|
||||
`--min-interval` floor makes a repeat a no-op rather than a re-fetch.
|
||||
|
||||
Then check <https://instaarchive.ergosteur.com> — new posts appear without a
|
||||
redeploy. The archive index re-reads a directory when its mtime changes.
|
||||
|
||||
**Reading the output.** These two look alarming and are fine:
|
||||
|
||||
- `rsync error: ... (code 23)` / `done; 1 step(s) failed` — the `chown` to
|
||||
`rslsync` failing because the ssh user is not root. Files landed.
|
||||
- `No results` for a profile — it has no active story right now.
|
||||
|
||||
**These mean stop**, and are covered in "When a run fails":
|
||||
|
||||
- `429` from `scontent-*.cdninstagram.com`
|
||||
- `400 Bad Request` on `/api/v1/feed/reels_media/` — the account is behind a
|
||||
scraping-warning interstitial; check the browser before anything else.
|
||||
|
||||
Long runs: `setsid nohup ~/gdl/gdl-cron.sh profiles >/dev/null 2>&1 &` and
|
||||
`tail -f ~/gdl/logs/$(ls -1t ~/gdl/logs | head -1)`. Nothing reaches the
|
||||
archive until a run finishes, so killing one midway is safe.
|
||||
|
||||
## Safety net: a global gallery-dl config
|
||||
|
||||
`~/.config/gallery-dl/config.json` on `mattellite` exists so that a **plain
|
||||
`gallery-dl <url>` typed by hand** — for a quick manual check, outside
|
||||
`gdl-sync.py` entirely — still gets the hand-paced caution settings instead of
|
||||
gallery-dl's own faster defaults. It is loaded automatically; nothing needs to
|
||||
reference it. `gdl-sync.py`'s own `--sleep-request`/`--sleep`/`--rate` flags
|
||||
still override it as normal — this is only a floor for when nobody passed any.
|
||||
|
||||
```json
|
||||
{
|
||||
"extractor": {
|
||||
"instagram": {
|
||||
"api": "rest",
|
||||
"cookies": ["chrome", "/home/matt/.config/google-chrome-devtools"],
|
||||
"sleep-request": [12.0, 20.0],
|
||||
"sleep": [5.0, 10.0],
|
||||
"sleep-429": 120.0,
|
||||
"retries": 8,
|
||||
"videos": true
|
||||
}
|
||||
},
|
||||
"downloader": {
|
||||
"http": {
|
||||
"rate": "500K",
|
||||
"retries": 8
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Every value here mirrors `gdl-cron.sh`'s own hand-paced defaults (see its
|
||||
`SLEEP_REQUEST`/`SLEEP`/`RATE` comments) and `build_config()` in
|
||||
`gdl-sync.py` — `api: rest` matters most: the graphql backend issues one
|
||||
request PER POST for every video and carousel, the pattern that got this
|
||||
account banned once already. `cookies` here is the config-file equivalent of
|
||||
`--cookies-from-browser`, so a bare `gallery-dl <url>` is already
|
||||
authenticated as the archive account, not anonymous.
|
||||
|
||||
Verified with `gallery_dl.config.load()` + `config.get(...)` (zero live
|
||||
requests) — every value above loads correctly with no `--config` flag passed.
|
||||
|
||||
This file is **not tracked in the repo** — like `artms.db` and the state
|
||||
files, it is host-local runtime config, and it embeds the same real
|
||||
Chrome-profile path already documented above. Recreate it by hand (or from
|
||||
this section) after a fresh `mattellite` setup.
|
||||
|
||||
## Remotes
|
||||
|
||||
| remote | what goes there |
|
||||
|---|---|
|
||||
| `origin` → gitea | everything: `main`, `tooling`, tags, backups |
|
||||
| `github` | **`main` and the current release tag only** — it exists to run the CI/CD image build |
|
||||
|
||||
The other 22 release tags stay on gitea. Pushing them all to GitHub triggers
|
||||
one container build per tag, because each tag carries its own workflow file.
|
||||
|
||||
## Guards — recreate these after a fresh clone
|
||||
|
||||
Neither guard is versioned, so a new clone has **no protection at all**:
|
||||
|
||||
```sh
|
||||
git config remote.github.push refs/heads/main:refs/heads/main
|
||||
|
||||
cat > .git/hooks/pre-push <<'HOOK'
|
||||
#!/bin/sh
|
||||
remote_url="$2"
|
||||
case "$remote_url" in *github.com*) ;; *) exit 0 ;; esac
|
||||
while read -r _ _ remote_ref _; do
|
||||
[ -z "$remote_ref" ] && continue
|
||||
case "$remote_ref" in
|
||||
refs/heads/main|refs/tags/*) ;;
|
||||
*) echo "pre-push: refusing to push '$remote_ref' to GitHub." >&2; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
exit 0
|
||||
HOOK
|
||||
chmod +x .git/hooks/pre-push
|
||||
```
|
||||
|
||||
Decide on the **remote** ref, not the local one: a delete push sends
|
||||
`(delete)` as the local ref, and an earlier version of this hook rejected
|
||||
every deletion because of it.
|
||||
|
||||
## What lives here
|
||||
|
||||
| path | what it is |
|
||||
|---|---|
|
||||
| `scripts/gdl-sync.py` | the gallery-dl fetcher; replaced JD2 for the ARTMS profiles |
|
||||
| `scripts/gdl-cron.sh` | unattended wrapper: `stories` \| `profiles` \| `full-sweep` |
|
||||
| `scripts/systemd/` | the timers actually installed on the fetch host |
|
||||
| `scripts/test_gdl_sync.py` | its tests |
|
||||
| `scripts/jd2-sync.ts` | JDownloader `.crawljob` generator, still used elsewhere |
|
||||
| `docs/gallery-dl.md` | the measurements behind every option in the fetcher — **read before changing pacing** |
|
||||
| `docs/jdownloader.md` | the older JD2 flow |
|
||||
| `docs/artms-instagram-accounts.txt` | the profile list passed to `--urls-file` |
|
||||
|
||||
## Commands
|
||||
|
||||
```sh
|
||||
# fetch: always --dry-run first; it prints the plan and the publish step
|
||||
./scripts/gdl-sync.py --index https://instaarchive.ergosteur.com \
|
||||
--staging <dir> --publish <user>@<nas>:<archives> \
|
||||
--archive-db <db> --urls-file artms_account_links.txt --abort 50 --dry-run
|
||||
|
||||
# crawljobs (no npm script — package.json is kept identical to main)
|
||||
npx tsx scripts/jd2-sync.ts --archives <dir> --dry-run
|
||||
```
|
||||
|
||||
Run the fetcher from the host whose public IP matches the browser the cookie
|
||||
came from. `--abort 50` is the routine setting; omit it for a full sweep that
|
||||
also catches edited carousels.
|
||||
|
||||
## Why there is no CLAUDE.md entry for any of this
|
||||
|
||||
`CLAUDE.md`, `README.md` and `package.json` are kept **byte-identical** to
|
||||
`main` so that merging `main` into `tooling` never conflicts. The earlier
|
||||
attempt put tooling notes in `CLAUDE.md` and a `jd2` script in `package.json`;
|
||||
because `main` had *deleted* those lines, every merge re-applied the deletion.
|
||||
Keep branch-specific documentation in this file, which `main` does not have.
|
||||
|
||||
## Running it by hand
|
||||
|
||||
Everything happens on **`mattellite`** — the fetch host whose public IP matches
|
||||
the browser the cookie came from. Running it anywhere else is what
|
||||
session-hijack detection looks for.
|
||||
|
||||
```sh
|
||||
ssh mattellite
|
||||
~/gdl/gdl-cron.sh profiles # or: stories | full-sweep
|
||||
```
|
||||
|
||||
That is the whole thing: it wipes staging, fetches, and publishes straight to
|
||||
the NAS. To drive `gdl-sync.py` directly instead — always `--dry-run` first,
|
||||
which prints the plan and the exact rsync that would touch the archive:
|
||||
|
||||
```sh
|
||||
cd ~/gdl
|
||||
PATH=$HOME/.local/bin:$PATH ./gdl-sync.py \
|
||||
--index https://instaarchive.ergosteur.com \
|
||||
--staging ~/gdl/staging-manual \
|
||||
--publish agentapi@10.20.28.200:/volume1/rslsync/sync/Instagram-archive/archives/ \
|
||||
--archive-db ~/gdl/artms.db \
|
||||
--urls-file ~/gdl/artms_account_links.txt \
|
||||
--sleep-request 12 20 --sleep 5 10 --rate 500K \
|
||||
--abort 50 --dry-run # swap for --execute when the plan looks right
|
||||
```
|
||||
|
||||
**Do not omit the pacing flags.** `gdl-sync.py`'s own defaults are 6-10s / 3-6s
|
||||
/ 1M, which is roughly half the caution the wrapper applies. A hand-run that
|
||||
leaves them off is *less* careful than the automation — which is backwards, and
|
||||
was true of this very example until 2026-08-22. Prefer `gdl-cron.sh`; it
|
||||
carries them for you.
|
||||
|
||||
`PATH` matters: `gallery-dl` is a pipx install in `~/.local/bin`, which is not
|
||||
on cron's PATH and not on a non-login shell's either.
|
||||
|
||||
Useful variations:
|
||||
|
||||
```sh
|
||||
--profile 0ct0ber19 --profile kimxxlip # instead of --urls-file
|
||||
--only stories # one surface
|
||||
--max-sources 4 # hard ceiling on what one run touches
|
||||
--force # ignore --min-interval AND the probe
|
||||
# cache; almost never what you want
|
||||
```
|
||||
|
||||
### Long runs
|
||||
|
||||
A sync runs for minutes to hours at this pacing, so detach it rather than
|
||||
holding an ssh session open:
|
||||
|
||||
```sh
|
||||
cd ~/gdl && setsid nohup ./gdl-cron.sh profiles > /dev/null 2>&1 &
|
||||
tail -f ~/gdl/logs/$(ls -1t ~/gdl/logs | head -1)
|
||||
```
|
||||
|
||||
**Do not kill it with `pkill -f <pattern>` over ssh.** The pattern matches your
|
||||
own `ssh` command line, so you kill your own shell and the sync survives — this
|
||||
happened twice on 2026-08-22. Use `pkill -x chrome` style exact-name matches,
|
||||
or kill the pid: `pgrep -f 'only posts,reels' | tail -1`.
|
||||
|
||||
Nothing reaches the archive until the run finishes: `gdl-sync.py` publishes
|
||||
once, at the end, so a run killed midway leaves the archive untouched.
|
||||
|
||||
### The three modes
|
||||
|
||||
Renamed on 2026-08-22. `full` was misleading — it is the abort-*limited* run —
|
||||
and `sweep` did not convey that it is the exhaustive one. The wrapper rejects
|
||||
the old names with a pointer rather than a bare error.
|
||||
|
||||
| mode | cadence | cost | why |
|
||||
|---|---|---|---|
|
||||
| `stories` | daily | ~6 requests | stories expire in 24h and **cannot be backfilled**; this is the only run that loses content if skipped |
|
||||
| `profiles` | monthly | ~40-60 requests | every surface, `--abort 50` — stops enumerating a profile once it reaches content already held. Catches everything **new** |
|
||||
| `full-sweep` | rarely, by hand | **~420 requests** | no abort; walks every profile to the end. The only run that notices posts **edited** after we archived them (test case 15) |
|
||||
|
||||
The skip-archive means an infrequent `profiles` run costs barely more than a frequent
|
||||
one — it only fetches what is new. Frequency buys freshness, not completeness,
|
||||
except for stories.
|
||||
|
||||
**Pacing is deliberately slower than `gdl-sync.py`'s own defaults.** All three
|
||||
modes run at `--sleep-request 12 20 --sleep 5 10 --rate 500K`, against defaults
|
||||
of 6-10 / 3-6 / 1M. These are the values the 2026-08-22 runs used by hand after
|
||||
the scraping warning, and they produced 0 400s and 0 429s. An archive sync has
|
||||
no deadline: being slow is free, being restricted is not. Override with
|
||||
`GDL_SLEEP_REQUEST`, `GDL_SLEEP`, `GDL_RATE` — to raise them, not lower them.
|
||||
|
||||
`stories` also runs at `--min-interval 8` rather than the 20h default, because
|
||||
at 20h the daily timer silently did nothing whenever a manual run had happened
|
||||
the previous afternoon. The floor exists to stop an *aborted restart*
|
||||
re-enumerating profiles, which is a minutes-to-hours concern; a stories fetch
|
||||
is one request per profile, so 8h permits at worst about twelve requests in a
|
||||
day instead of six. **A stories run that skips every source now exits 75 and
|
||||
prints a warning** rather than reporting success.
|
||||
|
||||
## Scheduling — installed on `mattellite`
|
||||
|
||||
systemd **user** timers, running as `matt`, with lingering enabled so they fire
|
||||
without a login session:
|
||||
|
||||
```sh
|
||||
loginctl show-user matt --property=Linger # Linger=yes
|
||||
systemctl --user list-timers 'gdl-sync@*'
|
||||
```
|
||||
|
||||
| unit | schedule | next fire (as installed) |
|
||||
|---|---|---|
|
||||
| `gdl-sync@stories.timer` | daily 09:00 | 09:36:45 — the delay is the randomisation working |
|
||||
| `gdl-sync@profiles.timer` | 3rd of each month, 04:00 | 04:37:44 |
|
||||
| `gdl-sync@full-sweep.timer` | 7th of Jan/Apr/Jul/Oct, 04:00 | 04:42:39 |
|
||||
|
||||
Unit files are version-controlled in `scripts/systemd/` and installed to
|
||||
`~/.config/systemd/user/`. One templated service, `gdl-sync@.service`, takes
|
||||
the mode as its instance name and runs `gdl-cron.sh %i`.
|
||||
|
||||
Three settings are load-bearing:
|
||||
|
||||
- **`RandomizedDelaySec=45m`** — a job firing at exactly 09:00 daily is
|
||||
obviously a machine, and the entire safety model is about not looking like
|
||||
one. This is why the table above shows 09:36 rather than 09:00.
|
||||
- **`Persistent=true`** — catch up a run missed because the host was off.
|
||||
cron silently skips, and a skipped `stories` run is content gone for good.
|
||||
- **`TimeoutStartSec=infinity`** — a sweep can run for hours at this pacing.
|
||||
The default 90s would kill it mid-fetch.
|
||||
|
||||
Operating them:
|
||||
|
||||
```sh
|
||||
export XDG_RUNTIME_DIR=/run/user/$(id -u) # needed over non-interactive ssh
|
||||
systemctl --user start gdl-sync@stories.service # run one now
|
||||
systemctl --user status gdl-sync@profiles.timer
|
||||
journalctl --user -u 'gdl-sync@*' -n 50
|
||||
systemctl --user disable --now gdl-sync@full-sweep.timer # stop one
|
||||
```
|
||||
|
||||
`systemctl --user` fails with "Failed to connect to bus" over ssh unless
|
||||
`XDG_RUNTIME_DIR` is set. Note also that **month names are not valid in
|
||||
`OnCalendar`'s date field** — `Jan,Apr,Jul,Oct-07` is rejected outright, hence
|
||||
`*-01,04,07,10-07`. Check any change with `systemd-analyze calendar '<expr>'`
|
||||
before installing it.
|
||||
|
||||
### cron, if you ever prefer it
|
||||
|
||||
```cron
|
||||
17 9 * * * sleep $(shuf -i 0-2700 -n1); $HOME/gdl/gdl-cron.sh stories
|
||||
43 4 3 * * sleep $(shuf -i 0-2700 -n1); $HOME/gdl/gdl-cron.sh profiles
|
||||
11 4 7 1,4,7,10 * sleep $(shuf -i 0-2700 -n1); $HOME/gdl/gdl-cron.sh full-sweep
|
||||
```
|
||||
|
||||
cron runs `/bin/sh`, so `$RANDOM` does not exist — hence `shuf`. And `%` in a
|
||||
crontab line means newline unless escaped, so avoid it entirely. cron has no
|
||||
equivalent of `Persistent=true`.
|
||||
|
||||
## Verifying a run
|
||||
|
||||
The first unattended run is **2026-08-21, around 09:36** (09:00 plus the
|
||||
randomised delay). Nothing below costs an Instagram request — every check is
|
||||
against the journal, local logs, or our own viewer's API.
|
||||
|
||||
```sh
|
||||
# 1. did it run, and did it exit 0?
|
||||
ssh mattellite
|
||||
export XDG_RUNTIME_DIR=/run/user/$(id -u)
|
||||
systemctl --user list-timers 'gdl-sync@*' # LAST/PASSED columns
|
||||
journalctl --user -u 'gdl-sync@stories.service' --since yesterday --no-pager
|
||||
|
||||
# 2. what did it actually fetch? (sidecars vastly outnumber media -- count media)
|
||||
ls -1t ~/gdl/logs | head -3
|
||||
grep -E '^(==>| FAILED|done;)' ~/gdl/logs/stories-*.log | tail -20
|
||||
grep -ic 429 ~/gdl/logs/stories-*.log # MUST be 0 -- see below
|
||||
```
|
||||
|
||||
**A `429` from `scontent-*.cdninstagram.com` ends the session, it is not a
|
||||
pacing knob to tune.** The warning order last time was CDN 429 → `400` on the
|
||||
highlights endpoint → suspension. If a run logs one, disable the timers and
|
||||
stop for the day:
|
||||
|
||||
```sh
|
||||
systemctl --user disable --now gdl-sync@stories.timer gdl-sync@profiles.timer gdl-sync@full-sweep.timer
|
||||
```
|
||||
|
||||
Then confirm the archive actually grew, from the workstation:
|
||||
|
||||
```sh
|
||||
# 3. did the publish land? compare against yesterday's counts
|
||||
for u in 0ct0ber19 kimxxlip withaseul cher_ryppo zindoriyam official_artms; do
|
||||
n=$(curl -s "https://instaarchive.ergosteur.com/api/archives/$u/files" | python3 -c 'import json,sys; d=json.load(sys.stdin); print(len(d if isinstance(d,list) else d["files"]))')
|
||||
printf '%-18s %s\n' "$u" "$n"
|
||||
done
|
||||
```
|
||||
|
||||
Counts after the 2026-08-22 runs, to diff against:
|
||||
|
||||
| profile | files |
|
||||
|---|---:|
|
||||
| 0ct0ber19 | 3160 |
|
||||
| official_artms | 6760 |
|
||||
| cher_ryppo | 3056 |
|
||||
| kimxxlip | 3021 |
|
||||
| zindoriyam | 2256 |
|
||||
| withaseul | 1760 |
|
||||
|
||||
A stories-only run adds few files and often **none** — profiles frequently have
|
||||
no active story. "0 new" is a normal result, not a failure. `fileCount` in
|
||||
`/api/archives` is stale by design; use the per-profile `/files` listing.
|
||||
|
||||
## 2026-08-21 — scraping warning, automation stopped
|
||||
|
||||
**Instagram flagged the account.** Not suspended: an interstitial at
|
||||
`/accounts/scraping_warning/` reading *"We suspect automated behaviour on your
|
||||
account"*. It was dismissed in the browser and the account is healthy — feed
|
||||
loads, still signed in. **All three timers are disabled.** Do not re-enable
|
||||
them without deciding the cadence question below.
|
||||
|
||||
How it unfolded, because each step misled in a different way:
|
||||
|
||||
1. **09:13** the daily timer fired, exited 0 in one second, logged
|
||||
`done; 0 step(s) failed` — and fetched nothing. Yesterday's manual run was
|
||||
18.9–19.2h earlier, just under the `--min-interval 20` floor, so all six
|
||||
sources were skipped. **A silent no-op on the one surface that cannot be
|
||||
backfilled, reported as success.**
|
||||
2. **20:28** `chrome-devtools.service` was OOM-killed (5.1 GB peak, ~1w3d CPU).
|
||||
Unrelated to the above, and it does not break fetching — gallery-dl reads
|
||||
the cookie *file*, not a live browser — but it meant no browser was running
|
||||
to notice anything was wrong.
|
||||
3. **23:19** a manual recovery run passed the floor (33h) and every source
|
||||
failed with `400 Bad Request` on
|
||||
`/api/v1/feed/reels_media/?reel_ids=…`. Six identical failures across six
|
||||
profiles is not a per-profile fault.
|
||||
4. Cookies were **exported and checked before assuming a block**:
|
||||
`sessionid` 77 chars, printable, colon-delimited, 360 days to expiry;
|
||||
`ds_user_id` present. Decryption was fine, so the fault was server-side.
|
||||
This check costs no Instagram requests and should always come first.
|
||||
5. The browser then showed the interstitial. The 400s were the challenge
|
||||
state, not a ban.
|
||||
|
||||
### What has to change before automation is re-enabled
|
||||
|
||||
- **The `--min-interval` floor silently defeats the daily job.** Any manual run
|
||||
in the preceding 20h makes the scheduled one a no-op. The floor exists to
|
||||
stop an *aborted restart* re-enumerating profiles — a minutes-to-hours
|
||||
concern — and a stories fetch is one request per profile. `stories` should
|
||||
use something like `--min-interval 8`, not 20.
|
||||
- **A skipped stories run must be loud.** `0 to sync, 6 skipped` currently
|
||||
exits 0 and looks identical to success. On this surface a skip is a real
|
||||
loss, and it should be visible in the journal without reading the log.
|
||||
- **Reconsider the daily cadence itself.** A job hitting story endpoints for
|
||||
six profiles every morning is the most machine-like thing here, randomised
|
||||
delay or not, and it is what was flagged. Every-few-days, or on-demand, may
|
||||
be the honest answer even though stories will be missed.
|
||||
- **`chrome-devtools.service` has `Restart=no`** and died silently for three
|
||||
hours. It needs `Restart=on-failure` and probably a `MemoryMax=`, or it will
|
||||
be dead the next time the cookie needs refreshing.
|
||||
|
||||
### 2026-08-22 — caught up by hand, cleanly
|
||||
|
||||
Both surfaces were fetched manually the next day, on the owner's call, at
|
||||
**roughly double the configured caution**: `--sleep-request 12 20`,
|
||||
`--sleep 5 10`, `--rate 500K`, versus the defaults of 6-10 / 3-6 / 1M.
|
||||
|
||||
| run | result |
|
||||
|---|---|
|
||||
| stories, 6 profiles | 16 media, +21 files, **0 400s, 0 429s** |
|
||||
| posts+reels, 12 sources, `--abort 50` | 26 media, +64 files, **0 400s, 0 429s** |
|
||||
|
||||
So the 400s really were the challenge state and nothing more: once the
|
||||
interstitial was dismissed in the browser, the same endpoints served normally.
|
||||
The stories that looked lost — `official_artms`, `0ct0ber19`, `cher_ryppo`,
|
||||
`kimxxlip`, `zindoriyam` — were all captured before expiry.
|
||||
|
||||
**This does not retire the warning.** Two hand-paced runs a day later are not
|
||||
evidence that the previous cadence was safe; they are evidence that the
|
||||
account still works. What actually changed the request cost is `--abort 50`:
|
||||
twelve sources across six profiles, `official_artms` included at 1829 posts
|
||||
and 781 reels, finished in minutes for a few dozen requests where the old
|
||||
behaviour would have spent ~400.
|
||||
|
||||
Note the gap this leaves: **the scheduled `profiles` mode still uses the default
|
||||
6-10s pacing**, not the 12-20s used here. Reconcile that before re-enabling
|
||||
the timers, or the automation will be less careful than the hand runs that
|
||||
followed a warning.
|
||||
|
||||
## When a run fails
|
||||
|
||||
Worked through on 2026-08-22. Do these **in order** — the first two cost no
|
||||
Instagram requests, and the third costs one page view.
|
||||
|
||||
### 1. Read the error, not the exit code
|
||||
|
||||
```sh
|
||||
L=~/gdl/logs/$(ls -1t ~/gdl/logs | head -1); grep -vE '^(profiles|surfaces|sources|pacing|staging|publish|read )' "$L" | head -40
|
||||
```
|
||||
|
||||
Two results are **not** failures, despite how they look:
|
||||
|
||||
| looks like | actually |
|
||||
|---|---|
|
||||
| `rsync error: some files/attrs were not transferred (code 23)` and `done; 1 step(s) failed` | the `chown` to `rslsync` failing because the ssh user is not root. Data landed. Confirm by re-running the rsync with `--dry-run`: an empty file list means everything arrived |
|
||||
| a source reporting `No results` | that profile simply has no active story / no highlights |
|
||||
|
||||
### 2. Check the cookies before concluding you are blocked
|
||||
|
||||
Free, and it separates a local fault from a server-side one:
|
||||
|
||||
```sh
|
||||
PATH=$HOME/.local/bin:$PATH gallery-dl \
|
||||
--cookies-from-browser "chrome:/home/matt/.config/google-chrome-devtools" \
|
||||
--cookies-export /tmp/ck.txt
|
||||
grep instagram /tmp/ck.txt | awk '{print $6, length($7)}' # names + value lengths
|
||||
rm -f /tmp/ck.txt
|
||||
```
|
||||
|
||||
A healthy `sessionid` is ~77 chars, printable ASCII, colon-delimited, and
|
||||
unexpired; `ds_user_id` should be an 11-digit number. Garbage or non-printable
|
||||
values mean Chrome's cookie decryption failed locally — not that the account is
|
||||
in trouble. **Never print the values into a transcript or a bug report.**
|
||||
|
||||
### 3. Look at the account in the browser
|
||||
|
||||
The session lives in a dedicated Chrome on `mattellite`, on display `:1` with
|
||||
CDP on 9222, run by a systemd unit:
|
||||
|
||||
```sh
|
||||
export XDG_RUNTIME_DIR=/run/user/$(id -u)
|
||||
systemctl --user status chrome-devtools.service
|
||||
systemctl --user start chrome-devtools.service # it has Restart=no
|
||||
```
|
||||
|
||||
Then VNC to `:1` and open `instagram.com`. What you are looking for:
|
||||
|
||||
- **`/accounts/scraping_warning/`** — *"We suspect automated behaviour on your
|
||||
account"*. This is what `400 Bad Request` on `/api/v1/feed/reels_media/`
|
||||
actually means; the session is in a challenge state, not banned. Dismissing
|
||||
it in the browser restores API access immediately, verified 2026-08-22.
|
||||
- a checkpoint or login page — the session is gone; re-log in the browser.
|
||||
- a normal feed — the fault is elsewhere.
|
||||
|
||||
**Do not dismiss a warning and immediately resume.** The dismissal fixes the
|
||||
symptom. The behaviour that caused it is the thing to change.
|
||||
|
||||
### 4. Stop, if it was a 429 or a repeated 400
|
||||
|
||||
```sh
|
||||
systemctl --user disable --now gdl-sync@stories.timer gdl-sync@profiles.timer gdl-sync@full-sweep.timer
|
||||
```
|
||||
|
||||
The documented escalation is CDN 429 → 400 on a stories/highlights endpoint →
|
||||
suspension. It has now run twice, and both times the 400 was the last warning
|
||||
before something worse.
|
||||
|
||||
## Reels: the API is blocked, scrape by scrolling instead
|
||||
|
||||
As of 2026-08-26/27, `gallery-dl`'s dedicated reels extractor fails on every
|
||||
profile with `HTTP redirect to home page` — confirmed hours apart, with a
|
||||
freshly-warmed session and a correct `X-IG-WWW-Claim` header (that was the
|
||||
first suspect; ruled out by tracing the raw HTTP exchange). It is not a
|
||||
scraping-warning interstitial — the reels tab loads completely normally in a
|
||||
real, already-signed-in browser — so this is Meta blocking the specific
|
||||
`/api/v1/clips/user/` endpoint gallery-dl calls, not an account-health issue.
|
||||
Posts, highlights and stories are unaffected; only reels breaks.
|
||||
|
||||
`scripts/reels-scrape.py` works around it by never calling that endpoint:
|
||||
it drives the same signed-in Chrome via its loopback CDP port (`:9222`,
|
||||
already exposed for MCP automation), scrolls the reels tab like a person
|
||||
would, and scrapes `/reel/<code>/` links out of the rendered page. It only
|
||||
finds shortcodes — nothing is downloaded until the fetch step — and dedupes
|
||||
against the **whole archive**, not just the profile being scraped (see why
|
||||
below), so re-running it costs nothing for reels already held anywhere.
|
||||
|
||||
### The easy way: `reels-sync.sh`
|
||||
|
||||
```sh
|
||||
ssh mattellite
|
||||
~/gdl/reels-sync.sh zindoriyam
|
||||
# or a full URL: ~/gdl/reels-sync.sh https://www.instagram.com/zindoriyam/
|
||||
```
|
||||
|
||||
Runs both steps (scrape, then fetch + publish whatever's new) with the same
|
||||
hand-paced settings `gdl-cron.sh` uses, logs to `~/gdl/logs/reels-<profile>-*`,
|
||||
and exits 0 with "no new reels" printed when a profile is already caught up
|
||||
— `gdl-sync.py` never even gets invoked in that case. Override pacing the
|
||||
same way as `gdl-cron.sh` (`GDL_SLEEP_REQUEST`, `GDL_SLEEP`, `GDL_RATE`), plus
|
||||
`GDL_SCROLL_PAUSE` and `GDL_MAX_IDLE_ROUNDS` for the scrape step. Staging
|
||||
(`~/gdl/staging-reels-<profile>`) and the scraped URL list
|
||||
(`~/gdl/<profile>-reels.txt`) are wiped at the START of the next run, not
|
||||
after — left behind for inspection, same as `gdl-cron.sh`'s `staging-*`.
|
||||
|
||||
Verified end to end against `zindoriyam` on 2026-08-27: 26 reels found on the
|
||||
page, 10 new ones fetched and published cleanly on the first run.
|
||||
|
||||
**Not wired into `gdl-cron.sh` or the timers.** Both scripts drive your
|
||||
actual browser session rather than firing a background request, and are
|
||||
slower by design (real scrolling, not an API call) — both good reasons not
|
||||
to run this unattended without deciding that deliberately. Today it is a
|
||||
per-profile, by-hand tool only.
|
||||
|
||||
### Where a shared reel lands
|
||||
|
||||
A reel found on one profile's `/reels/` page is not necessarily *owned* by
|
||||
that profile — reposts and collabs between tracked accounts show up there
|
||||
too. It always gets filed under its **true owner**, per Instagram's own
|
||||
metadata on the post, never under whichever profile's page you happened to
|
||||
scrape it from — `gdl-sync.py`'s directory template for a scraped reel is
|
||||
`{username}` filled in from that metadata, the same mechanism highlights
|
||||
already used for their own directory. So:
|
||||
|
||||
- Scrape order doesn't matter. Run `reels-sync.sh` on `0ct0ber19` or
|
||||
`zindoriyam` first, whichever — a reel they share lands in the same place
|
||||
either way, and running it on the other one afterward just sees that
|
||||
shortcode as already archived (dedup checks every profile, not only the
|
||||
one being scraped) and skips it.
|
||||
- It never gets duplicated into both accounts' directories, and it never
|
||||
gets misattributed to the profile you scraped instead of who actually
|
||||
posted it.
|
||||
|
||||
This is also literally why the whole-archive dedup fix above exists: a
|
||||
zindoriyam-scraped shortcode that turned out to belong to `0ct0ber19` was
|
||||
filed under `0ct0ber19/`, not `zindoriyam/` — dedup that only checked
|
||||
`zindoriyam`'s own directory would never have found it there.
|
||||
|
||||
### What it's doing, if you want to run the two steps separately
|
||||
|
||||
```sh
|
||||
PROFILE=someuser
|
||||
|
||||
# 1. Scrape by scrolling; dedupe against the archive; write new URLs to a file.
|
||||
# Needs gallery-dl's own pipx venv python -- that's where websocket-client
|
||||
# (the one extra dependency this needs) got injected.
|
||||
~/.local/share/pipx/venvs/gallery-dl/bin/python3 ~/gdl/reels-scrape.py \
|
||||
--profile "$PROFILE" \
|
||||
--index https://instaarchive.ergosteur.com \
|
||||
--out ~/gdl/"$PROFILE"-reels.txt
|
||||
|
||||
# 2. Fetch whatever's new -- dry run first, same as any other gdl-sync.py call.
|
||||
cd ~/gdl && PATH=$HOME/.local/bin:$PATH ./gdl-sync.py \
|
||||
--publish agentapi@10.20.28.200:/volume1/rslsync/sync/Instagram-archive/archives/ \
|
||||
--staging ~/gdl/staging-reels-scraped \
|
||||
--sleep-request 12 20 --sleep 5 10 --rate 500K \
|
||||
--post-urls-file ~/gdl/"$PROFILE"-reels.txt --dry-run
|
||||
# ...then swap --dry-run for --execute once the plan looks right.
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- `reels-scrape.py` fails fast if Chrome/CDP isn't up
|
||||
(`curl -s http://localhost:9222/json/version` to check first).
|
||||
- If it finds nothing new, `gdl-sync.py --post-urls-file` refuses to run
|
||||
("no usable URLs") rather than doing nothing quietly — expected when a
|
||||
profile is already caught up.
|
||||
- `--max-idle-rounds` (default 3) and `--scroll-pause MIN MAX` (default
|
||||
`2.0 3.5`) are worth raising for an unusually large or slow-loading reels
|
||||
tab; the default stops once 3 consecutive scrolls find nothing new.
|
||||
|
||||
### Why dedup checks the whole archive, not just the scraped profile
|
||||
|
||||
First version deduped only against the scraped profile's own directories.
|
||||
Re-running it on `zindoriyam` minutes after a successful fetch found "9 new"
|
||||
reels again — all reposts/collabs originally by `0ct0ber19` and
|
||||
`official_artms` (also tracked profiles). The *old*, broken direct-reels-tab
|
||||
fetch had left orphaned sidecar-only remnants (`.json`/`.txt`, no media) for
|
||||
them, misfiled under `zindoriyam - reels/` with the true owner's name baked
|
||||
into the filename stem — `index_existing()` correctly does not count a
|
||||
sidecar-only entry as "held", so they looked new. `gdl-sync.py` re-fetched
|
||||
them, filed them correctly under `{username}` (the post's *true* owner, per
|
||||
its own metadata) — where they already existed from that profile's own
|
||||
regular sync — and `rsync --ignore-existing` silently skipped every one, so
|
||||
nothing was lost or duplicated. But 9 Instagram requests were spent finding
|
||||
that out. Since a shortcode is globally unique, `reels-scrape.py` now checks
|
||||
every archived profile's listing, not just the one being scraped — all local
|
||||
requests to the viewer's own API, never to `instagram.com`, so checking all
|
||||
16 profiles costs nothing on the budget that actually matters. Confirmed
|
||||
fixed the same day: re-running against `zindoriyam` immediately afterward
|
||||
found "26 already archived, 0 new" and exited clean.
|
||||
|
||||
## What changed on 2026-08-20
|
||||
|
||||
One session, three separate pieces of work. Recorded because the reasons are
|
||||
not recoverable from the diffs.
|
||||
|
||||
**The sync run.** First incremental fetch in four days: 184 new media, 299
|
||||
files published, 0 failures, 0 CDN 429s. 20 story items, which are the part
|
||||
that could not have been recovered later. Cost about half what it would have,
|
||||
because the archive DB was already seeded and the state file was primed by hand
|
||||
so no probe passes ran.
|
||||
|
||||
**`--abort 50`.** The skip-archive suppresses *downloads*, which spends the
|
||||
CDN; it does nothing about the *listing pass*, which spends `instagram.com` and
|
||||
scales with how big a profile is rather than how much is new. Measured from
|
||||
sidecar write times mid-run: 3 new posts took ~100s each, the other 2272 were
|
||||
written in one second. Enumerating `cher_ryppo` fell from 2151 posts to 7.
|
||||
|
||||
**The repo split.** `main` is public and now carries none of the fetching
|
||||
tooling, no host details, and no real account names — its entire history was
|
||||
rewritten, the GitHub repo deleted and recreated to clear force-push residue,
|
||||
and 22 container images pruned from ghcr because the server bundle had been
|
||||
shipping source comments naming real accounts. This branch holds everything
|
||||
that was removed. See the caution at the top.
|
||||
|
||||
**Automation.** mattellite got a key on the NAS, closing the last manual step,
|
||||
and three systemd timers now run the sync unattended.
|
||||
|
||||
## Outstanding
|
||||
|
||||
State as of 2026-08-20, after the sync run and the repo split. Nothing here is
|
||||
broken; these are decisions not yet made and cleanups not yet done.
|
||||
|
||||
### Fetching
|
||||
|
||||
- **The timers are DISABLED** after the 2026-08-21 scraping warning; see that
|
||||
section. Their one unattended firing did the wrong thing — it skipped every
|
||||
source on the 20h floor and reported success — so re-enabling should wait on
|
||||
the four fixes listed there, not just on the account settling.
|
||||
- `mattellite`'s `~/.ssh/id_ed25519.pub` is in the NAS's `authorized_keys` for
|
||||
`agentapi` (added 2026-08-20, alongside the workstation's existing key), so
|
||||
the fetch host publishes straight to the archive and no `sshpass` step is
|
||||
needed. **That key is what makes the timers work** — remove it and every
|
||||
scheduled run will fetch successfully and then fail at publish.
|
||||
|
||||
- **5.4 GB of stale staging on `mattellite`** (`~/gdl`, 44 GB free): `staging`
|
||||
and `out` at 2.2 GB each from 2026-08-17, `staging-0820` / `out-0820` at
|
||||
446 MB each, plus `staging-full` (78 MB) and `staging-stories` (66 MB) from
|
||||
2026-08-22. All of it was verified published, so all of it is safe to delete.
|
||||
Staging is wiped per-run by `gdl-cron.sh`, but the `out-*` publish targets and
|
||||
anything created by a direct `gdl-sync.py` call are never cleaned up.
|
||||
- **Stories currently depend on someone remembering.** The daily timer exists
|
||||
but is disabled, so the one surface that cannot be backfilled has no
|
||||
automation behind it. Every day nobody runs `gdl-cron.sh stories` is a day
|
||||
of stories gone. That is the central unresolved tension: the cadence that
|
||||
protects stories is also the most machine-like pattern here.
|
||||
- ~~The scheduled modes have not been reconciled with the pacing used by
|
||||
hand.~~ **Done 2026-08-22**: all modes now pass `--sleep-request 12 20
|
||||
--sleep 5 10 --rate 500K`, `stories` uses `--min-interval 8`, and a
|
||||
fully-skipped stories run exits 75 with a warning instead of looking like a
|
||||
success. The timers are still **disabled** — enabling them is a separate
|
||||
decision about cadence, not about pacing.
|
||||
- **`--abort 50` is opt-in.** `gdl-cron.sh profiles` passes it and the manual
|
||||
runs used it; `full-sweep` deliberately does not. It stops noticing **edited
|
||||
carousels** (test case 15), which only a full enumeration finds — which is
|
||||
what `full-sweep` is for. No cadence for it was ever agreed, and at ~420
|
||||
requests it is the riskiest thing on the schedule; prefer running it by hand.
|
||||
- **The `seeded` flags in `<db>.state.json` were hand-written**, reconstructed
|
||||
from the 2026-08-17 log rather than derived from the archive DB. They assert
|
||||
"the skip-archive already knows this source". If `artms.db` is ever rebuilt,
|
||||
moved or lost, **clear the state file too** — otherwise those sources will
|
||||
never re-seed and a fetch into empty staging re-downloads everything.
|
||||
- **`~/gdl/gdl-sync.py` on the fetch host is a copy, not a checkout.** It
|
||||
currently matches this branch (`96e5694e…`), but nothing keeps them in sync;
|
||||
`scp` it after any change and re-check the hash.
|
||||
- The 2026-08-20 run is split across two logs — `artms-run3.log` (12 sources,
|
||||
no abort) and `artms-run4.log` (12 sources, `--abort 50`) — because it was
|
||||
stopped midway to pick up the new flag.
|
||||
|
||||
### Repo and infrastructure
|
||||
|
||||
- **The `pre-rewrite-*` branches on gitea hold the unredacted history** — real
|
||||
account names, the fetch host's IP, and the tooling, as it was before the
|
||||
rewrite. They are deliberate backups. Decide whether they expire; the
|
||||
`pre-push` hook does cover them (it allows only `main` and tags to GitHub).
|
||||
- **The `pre-rewrite-full.bundle` backup is in a session scratchpad** and will
|
||||
be deleted with it. If a durable backup outside gitea is wanted, move it now.
|
||||
- **Only one container image exists.** 22 versions were pruned, so rolling back
|
||||
to an older release means checking out its tag from gitea and pushing that
|
||||
tag to GitHub to rebuild it — the old images are gone, not archived.
|
||||
- **CI warns that the Node 20 actions are deprecated.** `actions/checkout@v4`,
|
||||
`docker/login-action@v3`, `docker/metadata-action@v5` and
|
||||
`docker/build-push-action@v5` are being forced onto Node 24. They work today;
|
||||
bump when convenient.
|
||||
- **`review-fixes`** on gitea is a stale v1.3.0-era branch, never merged,
|
||||
published only because the whole local repo was pushed. Probably deletable.
|
||||
- GitHub Actions run history was lost when the repo was recreated. Cosmetic.
|
||||
@@ -0,0 +1,221 @@
|
||||
[
|
||||
[
|
||||
2,
|
||||
{
|
||||
"category": "instagram",
|
||||
"coauthors": [
|
||||
{
|
||||
"full_name": "\ucd5c\ub9ac \ucd5c\uc608\ub9bc Choerry",
|
||||
"id": "57668363710",
|
||||
"username": "cher_ryppo"
|
||||
},
|
||||
{
|
||||
"full_name": "\uc9c4\uc194 \ud835\udc09\ud835\udc08\ud835\udc0d\ud835\udc12\ud835\udc0e\ud835\udc14\ud835\udc0b",
|
||||
"id": "55465048543",
|
||||
"username": "zindoriyam"
|
||||
},
|
||||
{
|
||||
"full_name": "Official ARTMS",
|
||||
"id": "58524253183",
|
||||
"username": "official_artms"
|
||||
},
|
||||
{
|
||||
"full_name": "HEEJIN",
|
||||
"id": "57723176039",
|
||||
"username": "0ct0ber19"
|
||||
},
|
||||
{
|
||||
"full_name": "\uae40\ub9bd KIMLIP",
|
||||
"id": "57506288936",
|
||||
"username": "kimxxlip"
|
||||
}
|
||||
],
|
||||
"count": 1,
|
||||
"date": "2026-08-18 07:18:54",
|
||||
"description": "#\uc81c\uc791\uc9c0\uc6d0 ARTMS(\uc544\ub974\ud14c\ubbf8\uc2a4) \u2018Blue Blood Night\u2019 \ub9ac\uc2a4\ub2dd \ud30c\ud2f0 \ud83c\udf19\n\n\uc131\uc218\ub3d9\uc5d0\uc11c \ud3bc\uccd0\uc9c4 ARTMS\uc640 300\uba85 \ud32c\ub4e4\uc758 \uc5ed\ub300\uae09 \ubc24!\n\nARTMS\uac00 \uc57d 300\uba85\uc758 \ud32c\ub4e4\uacfc \ud568\uaed8 \ud074\ub7fd \ud615\ud0dc\uc758\n\ub3c5\ud2b9\ud558\uace0 \ubabd\ud658\uc801\uc778 \uacf5\uac04\uc5d0\uc11c \uc624\ud504\ub77c\uc778 \ud32c \uc774\ubca4\ud2b8\n\u2018Blue Blood Night\u2019 \ub9ac\uc2a4\ub2dd \ud30c\ud2f0\ub97c \uac1c\ucd5c\ud588\uc2b5\ub2c8\ub2e4 \u2728\n\n\ud558\uc774 \ub370\ud328\ub274\ub3c4 \uc678\uce58\uace0 ARTMS \uc0ac\ub791\ud574\uc694\ub3c4 \uc678\ucce4\ub358\n\ub728\uac70\uc6e0\ub358 \ud604\uc7a5\uc5d0 \ub370\ud328\ub274\uac00 \ub2e4\ub140\uc654\uc2b5\ub2c8\ub2e4 \ud83d\ude0e\n\n\ud83d\udccc\uc544\ub974\ud14c\ubbf8\uc2a4 (ARTMS) \n@official_artms \n\n\u25aa\ufe0f2024\ub144\uc5d0 \ub370\ubdd4\ud55c \ubaa8\ub4dc\ud558\uc6b0\uc2a4(MODHAUS) \uc18c\uc18d\uc758 5\uc778\uc870 \uac78\uadf8\ub8f9 \n\u25aa\ufe0f\uba64\ubc84: \ud76c\uc9c4, \ud558\uc2ac, \uae40\ub9bd, \uc9c4\uc194, \ucd5c\ub9ac \n\u25aa\ufe0f\uc774\ub2ec\uc758 \uc18c\ub140 \ucd9c\uc2e0 \uba64\ubc84\ub4e4\uc774 \ub73b\uc744 \ubaa8\uc544 \uacb0\uc131\ud55c \ud504\ub85c\uc81d\ud2b8\uc774\uc790 \uc815\uc2dd \uac78\uadf8\ub8f9 \n\u25aa\ufe0f\uccab \uc815\uaddc \uc568\ubc94\uacfc \ud0c0\uc774\ud2c0\uace1 \u2018Virtual Angel\u2019 \ub4f1\uc744 \uc120\ubcf4\uc784\n\n#ARTMS #\uc544\ub974\ud14c\ubbf8\uc2a4 #BlueBloodNight\n\n\ud83c\udfa5 @dailyfashion_news",
|
||||
"fullname": "\u270d\ud83c\udffb\ud83e\udd13\ud328\uc158 \ub274\uc2a4 \uc694\uc57d \u2b50\ufe0f \ub370\ud328\ub274\ud83d\ude0e\ud83d\udc85\ud83c\udffb",
|
||||
"liked": false,
|
||||
"likes": 7219,
|
||||
"owner_id": "38980901318",
|
||||
"pinned": [],
|
||||
"post_date": "2026-08-18 07:18:54",
|
||||
"post_id": "3966279735525149864",
|
||||
"post_shortcode": "DcLDme7zoyo",
|
||||
"post_url": "https://www.instagram.com/reel/DcLDme7zoyo/",
|
||||
"subcategory": "reel",
|
||||
"tags": [
|
||||
"#ARTMS",
|
||||
"#BlueBloodNight",
|
||||
"#\uc544\ub974\ud14c\ubbf8\uc2a4",
|
||||
"#\uc81c\uc791\uc9c0\uc6d0"
|
||||
],
|
||||
"type": "reel",
|
||||
"username": "dailyfashion_news"
|
||||
}
|
||||
],
|
||||
[
|
||||
3,
|
||||
"ytdl:https://www.instagram.com/reel/DcLDme7zoyo/1.mp4",
|
||||
{
|
||||
"category": "instagram",
|
||||
"coauthors": [
|
||||
{
|
||||
"full_name": "\ucd5c\ub9ac \ucd5c\uc608\ub9bc Choerry",
|
||||
"id": "57668363710",
|
||||
"username": "cher_ryppo"
|
||||
},
|
||||
{
|
||||
"full_name": "\uc9c4\uc194 \ud835\udc09\ud835\udc08\ud835\udc0d\ud835\udc12\ud835\udc0e\ud835\udc14\ud835\udc0b",
|
||||
"id": "55465048543",
|
||||
"username": "zindoriyam"
|
||||
},
|
||||
{
|
||||
"full_name": "Official ARTMS",
|
||||
"id": "58524253183",
|
||||
"username": "official_artms"
|
||||
},
|
||||
{
|
||||
"full_name": "HEEJIN",
|
||||
"id": "57723176039",
|
||||
"username": "0ct0ber19"
|
||||
},
|
||||
{
|
||||
"full_name": "\uae40\ub9bd KIMLIP",
|
||||
"id": "57506288936",
|
||||
"username": "kimxxlip"
|
||||
}
|
||||
],
|
||||
"count": 1,
|
||||
"date": "2026-08-18 07:18:54",
|
||||
"description": "#\uc81c\uc791\uc9c0\uc6d0 ARTMS(\uc544\ub974\ud14c\ubbf8\uc2a4) \u2018Blue Blood Night\u2019 \ub9ac\uc2a4\ub2dd \ud30c\ud2f0 \ud83c\udf19\n\n\uc131\uc218\ub3d9\uc5d0\uc11c \ud3bc\uccd0\uc9c4 ARTMS\uc640 300\uba85 \ud32c\ub4e4\uc758 \uc5ed\ub300\uae09 \ubc24!\n\nARTMS\uac00 \uc57d 300\uba85\uc758 \ud32c\ub4e4\uacfc \ud568\uaed8 \ud074\ub7fd \ud615\ud0dc\uc758\n\ub3c5\ud2b9\ud558\uace0 \ubabd\ud658\uc801\uc778 \uacf5\uac04\uc5d0\uc11c \uc624\ud504\ub77c\uc778 \ud32c \uc774\ubca4\ud2b8\n\u2018Blue Blood Night\u2019 \ub9ac\uc2a4\ub2dd \ud30c\ud2f0\ub97c \uac1c\ucd5c\ud588\uc2b5\ub2c8\ub2e4 \u2728\n\n\ud558\uc774 \ub370\ud328\ub274\ub3c4 \uc678\uce58\uace0 ARTMS \uc0ac\ub791\ud574\uc694\ub3c4 \uc678\ucce4\ub358\n\ub728\uac70\uc6e0\ub358 \ud604\uc7a5\uc5d0 \ub370\ud328\ub274\uac00 \ub2e4\ub140\uc654\uc2b5\ub2c8\ub2e4 \ud83d\ude0e\n\n\ud83d\udccc\uc544\ub974\ud14c\ubbf8\uc2a4 (ARTMS) \n@official_artms \n\n\u25aa\ufe0f2024\ub144\uc5d0 \ub370\ubdd4\ud55c \ubaa8\ub4dc\ud558\uc6b0\uc2a4(MODHAUS) \uc18c\uc18d\uc758 5\uc778\uc870 \uac78\uadf8\ub8f9 \n\u25aa\ufe0f\uba64\ubc84: \ud76c\uc9c4, \ud558\uc2ac, \uae40\ub9bd, \uc9c4\uc194, \ucd5c\ub9ac \n\u25aa\ufe0f\uc774\ub2ec\uc758 \uc18c\ub140 \ucd9c\uc2e0 \uba64\ubc84\ub4e4\uc774 \ub73b\uc744 \ubaa8\uc544 \uacb0\uc131\ud55c \ud504\ub85c\uc81d\ud2b8\uc774\uc790 \uc815\uc2dd \uac78\uadf8\ub8f9 \n\u25aa\ufe0f\uccab \uc815\uaddc \uc568\ubc94\uacfc \ud0c0\uc774\ud2c0\uace1 \u2018Virtual Angel\u2019 \ub4f1\uc744 \uc120\ubcf4\uc784\n\n#ARTMS #\uc544\ub974\ud14c\ubbf8\uc2a4 #BlueBloodNight\n\n\ud83c\udfa5 @dailyfashion_news",
|
||||
"display_url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.82787-15/777982919_18124593284301319_6754878398807305029_n.jpg?stp=dst-jpg_e15_tt6&_nc_cat=107&ig_cache_key=Mzk2NjI3OTczNTUyNTE0OTg2NDE4MTI0NTkzMjgxMzAxMzE5.3-ccb7-5&ccb=7-5&_nc_sid=58cdad&efg=eyJ2ZW5jb2RlX3RhZyI6IkNMSVBTLnhwaWRzLjEzMjAuc2RyLnZpZGVvX2RlZmF1bHRfY292ZXJfZnJhbWUuQzMifQ%3D%3D&_nc_ohc=VTiIkvHCoBMQ7kNvwFnD08V&_nc_oc=Adp2K1e-pF18avq4GpMbXRgiZv_T6ySFEqTFdXPRtemyWnFeG8p6cgvxaXxZtyDDw-U&_nc_ad=z-m&_nc_cid=0&_nc_zt=23&_nc_ht=instagram.fyto3-1.fna&_nc_gid=-Kd8mdFfght8nFvwkJmRPA&_nc_ss=7a22e&oh=00_AQIKaJEGeU5iyrLIjnfXozkpbabHpemHWlDoSEEx11SMFQ&oe=6A9D5761",
|
||||
"extension": "mp4",
|
||||
"filename": "AQNbHAasCyJwuQWdYi7vWw3w6iCiMdBAXZQ58cnOKhanFZgw_IAhVC_f380GVo23sNeeCD559rVhmUSr0bmruolOjAgXMVwrmoRu88A",
|
||||
"fullname": "\u270d\ud83c\udffb\ud83e\udd13\ud328\uc158 \ub274\uc2a4 \uc694\uc57d \u2b50\ufe0f \ub370\ud328\ub274\ud83d\ude0e\ud83d\udc85\ud83c\udffb",
|
||||
"height": 1920,
|
||||
"height_original": 1920,
|
||||
"liked": false,
|
||||
"likes": 7219,
|
||||
"media_id": "3966279735525149864",
|
||||
"num": 1,
|
||||
"owner": {
|
||||
"account_badges": [],
|
||||
"account_type": 3,
|
||||
"can_see_quiet_post_attribution": true,
|
||||
"eligible_for_text_app_activation_badge": false,
|
||||
"fan_club_info": {
|
||||
"autosave_to_exclusive_highlight": null,
|
||||
"connected_member_count": null,
|
||||
"fan_club_id": null,
|
||||
"fan_club_name": null,
|
||||
"fan_consideration_page_revamp_eligiblity": null,
|
||||
"has_created_ssc": null,
|
||||
"has_enough_subscribers_for_ssc": null,
|
||||
"is_fan_club_gifting_eligible": null,
|
||||
"is_fan_club_referral_eligible": null,
|
||||
"is_free_trial_eligible": null,
|
||||
"largest_public_bc_id": null,
|
||||
"should_show_playlists_in_profile_tab": null,
|
||||
"subscriber_count": null
|
||||
},
|
||||
"fbid_v2": "17841439039552612",
|
||||
"feed_post_reshare_disabled": false,
|
||||
"friendship_status": {
|
||||
"followed_by": false,
|
||||
"following": false,
|
||||
"is_bestie": false,
|
||||
"is_feed_favorite": false,
|
||||
"is_muting_reel": false,
|
||||
"is_private": false,
|
||||
"is_restricted": false
|
||||
},
|
||||
"full_name": "\u270d\ud83c\udffb\ud83e\udd13\ud328\uc158 \ub274\uc2a4 \uc694\uc57d \u2b50\ufe0f \ub370\ud328\ub274\ud83d\ude0e\ud83d\udc85\ud83c\udffb",
|
||||
"has_anonymous_profile_picture": false,
|
||||
"hd_profile_pic_url_info": {
|
||||
"height": 1080,
|
||||
"url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.2885-19/283399158_1201757910664534_1998021056617019217_n.jpg?efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gE8DUM4skVKFnTxJ_6S8QN3RWaHsgKT9sigiWMRE8sqvl9r488ADsJFU6ErFHln2eg&_nc_ohc=LjOAkQaNPtMQ7kNvwHhtSrv&_nc_gid=-Kd8mdFfght8nFvwkJmRPA&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQLdX6t5krhp725dzvNOvQbGOFpVdvwgZWqDkFvv1AeQPA&oe=6A9D6555&_nc_sid=fc8dfb",
|
||||
"width": 1080
|
||||
},
|
||||
"hd_profile_pic_versions": [
|
||||
{
|
||||
"height": 320,
|
||||
"url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.2885-19/283399158_1201757910664534_1998021056617019217_n.jpg?stp=dst-jpg_s320x320_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gE8DUM4skVKFnTxJ_6S8QN3RWaHsgKT9sigiWMRE8sqvl9r488ADsJFU6ErFHln2eg&_nc_ohc=LjOAkQaNPtMQ7kNvwHhtSrv&_nc_gid=-Kd8mdFfght8nFvwkJmRPA&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQKux7g7JpZsP_Q7z6rX3bFlcdVCUCwwDAQfmRRWbB6ePQ&oe=6A9D6555&_nc_sid=fc8dfb",
|
||||
"width": 320
|
||||
},
|
||||
{
|
||||
"height": 640,
|
||||
"url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.2885-19/283399158_1201757910664534_1998021056617019217_n.jpg?stp=dst-jpg_s640x640_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gE8DUM4skVKFnTxJ_6S8QN3RWaHsgKT9sigiWMRE8sqvl9r488ADsJFU6ErFHln2eg&_nc_ohc=LjOAkQaNPtMQ7kNvwHhtSrv&_nc_gid=-Kd8mdFfght8nFvwkJmRPA&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQJJz0fXTbLsfMqkiG7KHQtZbaHwHAU8nS1eTzB0sQqRSA&oe=6A9D6555&_nc_sid=fc8dfb",
|
||||
"width": 640
|
||||
}
|
||||
],
|
||||
"id": "38980901318",
|
||||
"is_active_on_text_post_app": true,
|
||||
"is_embeds_disabled": false,
|
||||
"is_favorite": false,
|
||||
"is_private": false,
|
||||
"is_ring_creator": false,
|
||||
"is_unpublished": false,
|
||||
"is_verified": true,
|
||||
"latest_reel_media": 1788306389,
|
||||
"pk": "38980901318",
|
||||
"pk_id": "38980901318",
|
||||
"profile_pic_id": "2844231369895585868_38980901318",
|
||||
"profile_pic_url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.2885-19/283399158_1201757910664534_1998021056617019217_n.jpg?stp=dst-jpg_s150x150_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gE8DUM4skVKFnTxJ_6S8QN3RWaHsgKT9sigiWMRE8sqvl9r488ADsJFU6ErFHln2eg&_nc_ohc=LjOAkQaNPtMQ7kNvwHhtSrv&_nc_gid=-Kd8mdFfght8nFvwkJmRPA&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQI28hrlvg9XAWGFkTjCyv96HLuMEZb5q6AlNbUUqw2WVQ&oe=6A9D6555&_nc_sid=fc8dfb",
|
||||
"show_account_transparency_details": true,
|
||||
"show_ring_award": false,
|
||||
"strong_id__": "38980901318",
|
||||
"text_post_app_is_private": false,
|
||||
"third_party_downloads_enabled": 2,
|
||||
"transparency_product_enabled": false,
|
||||
"user_activation_info": {},
|
||||
"username": "dailyfashion_news"
|
||||
},
|
||||
"owner_id": "38980901318",
|
||||
"pinned": [],
|
||||
"post_date": "2026-08-18 07:18:54",
|
||||
"post_id": "3966279735525149864",
|
||||
"post_shortcode": "DcLDme7zoyo",
|
||||
"post_url": "https://www.instagram.com/reel/DcLDme7zoyo/",
|
||||
"shortcode": "DcLDme7zoyo",
|
||||
"subcategory": "reel",
|
||||
"tagged_users": [
|
||||
{
|
||||
"full_name": "\uc9c4\uc194 \ud835\udc09\ud835\udc08\ud835\udc0d\ud835\udc12\ud835\udc0e\ud835\udc14\ud835\udc0b",
|
||||
"id": "55465048543",
|
||||
"username": "zindoriyam"
|
||||
},
|
||||
{
|
||||
"full_name": "\uae40\ub9bd KIMLIP",
|
||||
"id": "57506288936",
|
||||
"username": "kimxxlip"
|
||||
},
|
||||
{
|
||||
"full_name": "\ucd5c\ub9ac \ucd5c\uc608\ub9bc Choerry",
|
||||
"id": "57668363710",
|
||||
"username": "cher_ryppo"
|
||||
},
|
||||
{
|
||||
"full_name": "HEEJIN",
|
||||
"id": "57723176039",
|
||||
"username": "0ct0ber19"
|
||||
},
|
||||
{
|
||||
"full_name": "Official ARTMS",
|
||||
"id": "58524253183",
|
||||
"username": "official_artms"
|
||||
}
|
||||
],
|
||||
"tags": [
|
||||
"#ARTMS",
|
||||
"#BlueBloodNight",
|
||||
"#\uc544\ub974\ud14c\ubbf8\uc2a4",
|
||||
"#\uc81c\uc791\uc9c0\uc6d0"
|
||||
],
|
||||
"type": "reel",
|
||||
"username": "dailyfashion_news",
|
||||
"video_url": "https://instagram.fyto3-1.fna.fbcdn.net/o1/v/t2/f2/m86/AQNbHAasCyJwuQWdYi7vWw3w6iCiMdBAXZQ58cnOKhanFZgw_IAhVC_f380GVo23sNeeCD559rVhmUSr0bmruolOjAgXMVwrmoRu88A.mp4?_nc_cat=105&_nc_oc=AdomTMuitkVGoe5U3qTaa9WVmrQNaUmC-hpxQMgtcKNDrT2Htj3cu6iPiBDfh9M0Hyk&_nc_sid=5e9851&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_ohc=O3edipmWdLQQ7kNvwFP9r4s&efg=eyJ2ZW5jb2RlX3RhZyI6Inhwdl9wcm9ncmVzc2l2ZS5JTlNUQUdSQU0uQ0xJUFMuQzMuNzIwLmRhc2hfYmFzZWxpbmVfMV92MSIsInhwdl9hc3NldF9pZCI6MTgxMTkyNTgxMzEyMzA0OSwiYXNzZXRfYWdlX2RheXMiOjE0LCJ2aV91c2VjYXNlX2lkIjoxMDA5OSwiZHVyYXRpb25fcyI6NTQsInVybGdlbl9zb3VyY2UiOiJ3d3cifQ%3D%3D&ccb=17-1&vs=69de96aad46e550e&_nc_vs=HBksFQIYUmlnX3hwdl9yZWVsc19wZXJtYW5lbnRfc3JfcHJvZC83RTRFMjc3MDhFMjVCMzUxNjA5MURDQUM2QTQ3MTlBQV92aWRlb19kYXNoaW5pdC5tcDQVAALIARIAFQIYUWlnX3hwdl9wbGFjZW1lbnRfcGVybWFuZW50X3YyL0I3NDdFMUM4NTRCODk2NDRENDM2NTIwQzVBODBCOTk5X2F1ZGlvX2Rhc2hpbml0Lm1wNBUCAsgBEgAoABgAGwKIB3VzZV9vaWwBMRJwcm9ncmVzc2l2ZV9yZWNpcGUBMRUAACbS35qn-_u3BhUCKAJDMywXQEtRBiTdLxsYEmRhc2hfYmFzZWxpbmVfMV92MREAdf4HZeadAQA&_nc_gid=-Kd8mdFfght8nFvwkJmRPA&_nc_zt=28&_nc_ss=7a22e&oh=00_AQLoXUFyQixJHvIcp3UE-6sEsjKuOlmhiPWmQphL7U7o5g&oe=6A996A42",
|
||||
"width": 1080,
|
||||
"width_original": 1080
|
||||
}
|
||||
]
|
||||
]
|
||||
@@ -0,0 +1,181 @@
|
||||
[
|
||||
[
|
||||
2,
|
||||
{
|
||||
"audio_artist": null,
|
||||
"audio_duration": 15.0,
|
||||
"audio_timestamps": null,
|
||||
"audio_title": "Original audio",
|
||||
"audio_user": {
|
||||
"full_name": "\uc2e0\ubaa8\ucc0c \ud835\udde0\ud835\uddfc\ud835\uddf0\ud835\uddf5\ud835\uddf6 \ud83c\udf58",
|
||||
"id": "61396137942",
|
||||
"is_private": false,
|
||||
"is_verified": false,
|
||||
"pk": "61396137942",
|
||||
"pk_id": "61396137942",
|
||||
"profile_pic_id": "3506506412954824214_61396137942",
|
||||
"profile_pic_url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.2885-19/468037362_1017459960091671_5806837525855946557_n.jpg?stp=dst-jpg_s150x150_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby42MDAuYzIifQ&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=103&_nc_oc=Q6cZ2gEBkafhMKkLe-YbSz6FVQiMLEw4juIKWvM8PNmZ2nTcMpNyjdl0kV_-fWna0tytdLA&_nc_ohc=sPpx_bESe4cQ7kNvwFvI70J&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQIS_52Qhu7n6Fo7XNtQaIKXGEhkwhgz-C9vpWo4yb6dnw&oe=6A9D5107&_nc_sid=fc8dfb",
|
||||
"strong_id__": "61396137942",
|
||||
"username": "mochi.053"
|
||||
},
|
||||
"category": "instagram",
|
||||
"coauthors": [
|
||||
{
|
||||
"full_name": "HEEJIN",
|
||||
"id": "57723176039",
|
||||
"username": "0ct0ber19"
|
||||
}
|
||||
],
|
||||
"count": 2,
|
||||
"date": "2026-08-22 02:48:05",
|
||||
"description": "\u3060\u3044\u3058\u3087\u270c\ufe0f\u3067\u3057\u3087",
|
||||
"fullname": "\uce04",
|
||||
"liked": true,
|
||||
"likes": 178797,
|
||||
"owner_id": "54301371254",
|
||||
"pinned": [],
|
||||
"post_date": "2026-08-22 02:48:05",
|
||||
"post_id": "3969041070976751617",
|
||||
"post_shortcode": "DcU3dM-gVgB",
|
||||
"post_url": "https://www.instagram.com/p/DcU3dM-gVgB/",
|
||||
"subcategory": "post",
|
||||
"type": "post",
|
||||
"username": "chuuo3o"
|
||||
}
|
||||
],
|
||||
[
|
||||
3,
|
||||
"ytdl:https://www.instagram.com/p/DcU3dM-gVgB/1.mp4",
|
||||
{
|
||||
"audio_artist": null,
|
||||
"audio_duration": 15.0,
|
||||
"audio_timestamps": null,
|
||||
"audio_title": "Original audio",
|
||||
"audio_user": {
|
||||
"full_name": "\uc2e0\ubaa8\ucc0c \ud835\udde0\ud835\uddfc\ud835\uddf0\ud835\uddf5\ud835\uddf6 \ud83c\udf58",
|
||||
"id": "61396137942",
|
||||
"is_private": false,
|
||||
"is_verified": false,
|
||||
"pk": "61396137942",
|
||||
"pk_id": "61396137942",
|
||||
"profile_pic_id": "3506506412954824214_61396137942",
|
||||
"profile_pic_url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.2885-19/468037362_1017459960091671_5806837525855946557_n.jpg?stp=dst-jpg_s150x150_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby42MDAuYzIifQ&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=103&_nc_oc=Q6cZ2gEBkafhMKkLe-YbSz6FVQiMLEw4juIKWvM8PNmZ2nTcMpNyjdl0kV_-fWna0tytdLA&_nc_ohc=sPpx_bESe4cQ7kNvwFvI70J&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQIS_52Qhu7n6Fo7XNtQaIKXGEhkwhgz-C9vpWo4yb6dnw&oe=6A9D5107&_nc_sid=fc8dfb",
|
||||
"strong_id__": "61396137942",
|
||||
"username": "mochi.053"
|
||||
},
|
||||
"category": "instagram",
|
||||
"coauthors": [
|
||||
{
|
||||
"full_name": "HEEJIN",
|
||||
"id": "57723176039",
|
||||
"username": "0ct0ber19"
|
||||
}
|
||||
],
|
||||
"count": 2,
|
||||
"date": "2026-08-22 02:48:05",
|
||||
"description": "\u3060\u3044\u3058\u3087\u270c\ufe0f\u3067\u3057\u3087",
|
||||
"display_url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.82787-15/783856831_18043412798811255_1724949997749570795_n.jpg?stp=dst-jpg_e15_tt6&_nc_cat=108&ig_cache_key=Mzk2OTA0MTA3MDk3Njc1MTYxNzE4MDQzNDEyNzkyODExMjU1.3-ccb7-5&ccb=7-5&_nc_sid=58cdad&efg=eyJ2ZW5jb2RlX3RhZyI6IkNMSVBTLnhwaWRzLjcyMC5zZHIudmlkZW9fZGVmYXVsdF9jb3Zlcl9mcmFtZS5DMyJ9&_nc_ohc=JWzEND3gUxAQ7kNvwGDGCLf&_nc_oc=AdpwpFkh8Wj84hyF4zWcsaNB5zUnYJh2bJijNe9AxrOR1WKa08Uy35khb5BpFnG9XhM&_nc_ad=z-m&_nc_cid=0&_nc_zt=23&_nc_ht=instagram.fyto3-1.fna&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&_nc_ss=7a22e&oh=00_AQLQlu6EItD2G6QrlOmQSas2IHDOpucLIhxVWcCPFE1BOw&oe=6A9D4B4F",
|
||||
"extension": "mp4",
|
||||
"filename": "AQN8nhc_L3hdDSQVVRF2OXm2gKwDpsdh_jw81ZnA9Uh4nqOwukKke8QXrGCe1jrMtOXzPT5Bkuipu8RShT651VV7q6EVbjvxC-fxgUE",
|
||||
"fullname": "\uce04",
|
||||
"height": 1920,
|
||||
"height_original": 1920,
|
||||
"liked": true,
|
||||
"likes": 178797,
|
||||
"media_id": "3969041070976751617",
|
||||
"num": 1,
|
||||
"owner": {
|
||||
"account_badges": [],
|
||||
"account_type": 3,
|
||||
"can_see_quiet_post_attribution": true,
|
||||
"eligible_for_text_app_activation_badge": false,
|
||||
"fan_club_info": {
|
||||
"autosave_to_exclusive_highlight": null,
|
||||
"connected_member_count": null,
|
||||
"fan_club_id": null,
|
||||
"fan_club_name": null,
|
||||
"fan_consideration_page_revamp_eligiblity": null,
|
||||
"has_created_ssc": null,
|
||||
"has_enough_subscribers_for_ssc": null,
|
||||
"is_fan_club_gifting_eligible": null,
|
||||
"is_fan_club_referral_eligible": null,
|
||||
"is_free_trial_eligible": null,
|
||||
"largest_public_bc_id": null,
|
||||
"should_show_playlists_in_profile_tab": null,
|
||||
"subscriber_count": null
|
||||
},
|
||||
"fbid_v2": "17841454337627671",
|
||||
"feed_post_reshare_disabled": false,
|
||||
"friendship_status": {
|
||||
"followed_by": false,
|
||||
"following": true,
|
||||
"is_bestie": false,
|
||||
"is_feed_favorite": false,
|
||||
"is_muting_reel": false,
|
||||
"is_private": false,
|
||||
"is_restricted": false
|
||||
},
|
||||
"full_name": "\uce04",
|
||||
"has_anonymous_profile_picture": false,
|
||||
"hd_profile_pic_url_info": {
|
||||
"height": 1080,
|
||||
"url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.82787-19/714930835_18032224961811255_6816644959905468206_n.jpg?efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gEBkafhMKkLe-YbSz6FVQiMLEw4juIKWvM8PNmZ2nTcMpNyjdl0kV_-fWna0tytdLA&_nc_ohc=hpoiFipmAqsQ7kNvwE3y5Ow&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQJiPho8ymPf1bC2B5dAlKCbdXW8d4KK-9qqsVrcOytVGQ&oe=6A9D4CB4&_nc_sid=fc8dfb",
|
||||
"width": 1080
|
||||
},
|
||||
"hd_profile_pic_versions": [
|
||||
{
|
||||
"height": 320,
|
||||
"url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.82787-19/714930835_18032224961811255_6816644959905468206_n.jpg?stp=dst-jpg_s320x320_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gEBkafhMKkLe-YbSz6FVQiMLEw4juIKWvM8PNmZ2nTcMpNyjdl0kV_-fWna0tytdLA&_nc_ohc=hpoiFipmAqsQ7kNvwE3y5Ow&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQIn43F6SprbhopBuJxmswE2wd4C39mDZoL9Fbqs8Ih-cw&oe=6A9D4CB4&_nc_sid=fc8dfb",
|
||||
"width": 320
|
||||
},
|
||||
{
|
||||
"height": 640,
|
||||
"url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.82787-19/714930835_18032224961811255_6816644959905468206_n.jpg?stp=dst-jpg_s640x640_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gEBkafhMKkLe-YbSz6FVQiMLEw4juIKWvM8PNmZ2nTcMpNyjdl0kV_-fWna0tytdLA&_nc_ohc=hpoiFipmAqsQ7kNvwE3y5Ow&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQJF1vYqhGCDkV-v6WbrTb_PoVC4GA3QCbEGJU0y0F96BA&oe=6A9D4CB4&_nc_sid=fc8dfb",
|
||||
"width": 640
|
||||
}
|
||||
],
|
||||
"id": "54301371254",
|
||||
"is_active_on_text_post_app": false,
|
||||
"is_embeds_disabled": false,
|
||||
"is_favorite": false,
|
||||
"is_private": false,
|
||||
"is_ring_creator": false,
|
||||
"is_unpublished": false,
|
||||
"is_verified": true,
|
||||
"latest_reel_media": 0,
|
||||
"pk": "54301371254",
|
||||
"pk_id": "54301371254",
|
||||
"profile_pic_id": "3912796961274695044_54301371254",
|
||||
"profile_pic_url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.82787-19/714930835_18032224961811255_6816644959905468206_n.jpg?stp=dst-jpg_s150x150_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gEBkafhMKkLe-YbSz6FVQiMLEw4juIKWvM8PNmZ2nTcMpNyjdl0kV_-fWna0tytdLA&_nc_ohc=hpoiFipmAqsQ7kNvwE3y5Ow&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQJu85hrHiCGK3enXnfDQExy_ZbgzCV_AGYJ4yOy7mi_QQ&oe=6A9D4CB4&_nc_sid=fc8dfb",
|
||||
"show_account_transparency_details": true,
|
||||
"show_ring_award": false,
|
||||
"strong_id__": "54301371254",
|
||||
"text_post_app_is_private": true,
|
||||
"third_party_downloads_enabled": 1,
|
||||
"transparency_product_enabled": false,
|
||||
"user_activation_info": {},
|
||||
"username": "chuuo3o"
|
||||
},
|
||||
"owner_id": "54301371254",
|
||||
"pinned": [],
|
||||
"post_date": "2026-08-22 02:48:05",
|
||||
"post_id": "3969041070976751617",
|
||||
"post_shortcode": "DcU3dM-gVgB",
|
||||
"post_url": "https://www.instagram.com/p/DcU3dM-gVgB/",
|
||||
"shortcode": "DcU3dM-gVgB",
|
||||
"subcategory": "post",
|
||||
"tagged_users": [
|
||||
{
|
||||
"full_name": "HEEJIN",
|
||||
"id": "57723176039",
|
||||
"username": "0ct0ber19"
|
||||
}
|
||||
],
|
||||
"type": "post",
|
||||
"username": "chuuo3o",
|
||||
"video_url": "https://instagram.fyto3-1.fna.fbcdn.net/o1/v/t2/f2/m86/AQN8nhc_L3hdDSQVVRF2OXm2gKwDpsdh_jw81ZnA9Uh4nqOwukKke8QXrGCe1jrMtOXzPT5Bkuipu8RShT651VV7q6EVbjvxC-fxgUE.mp4?_nc_cat=102&_nc_oc=AdqR865Bo8jW90mx5U9WR94Q0Eh1YI3ycT_ZhhW71XQNFHxUR3aBwhQEgcQ9Z2Hnrzs&_nc_sid=5e9851&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_ohc=2F0lG8gquXwQ7kNvwGIIkUT&efg=eyJ2ZW5jb2RlX3RhZyI6Inhwdl9wcm9ncmVzc2l2ZS5JTlNUQUdSQU0uQ0xJUFMuQzMuNzIwLmRhc2hfYmFzZWxpbmVfMV92MSIsInhwdl9hc3NldF9pZCI6MTYxMDAzNjE0Mzk3NzkxMywiYXNzZXRfYWdlX2RheXMiOjExLCJ2aV91c2VjYXNlX2lkIjoxMDA5OSwiZHVyYXRpb25fcyI6MTUsInVybGdlbl9zb3VyY2UiOiJ3d3cifQ%3D%3D&ccb=17-1&vs=93986893873e15f7&_nc_vs=HBksFQIYUmlnX3hwdl9yZWVsc19wZXJtYW5lbnRfc3JfcHJvZC80ODQ5NDdDRjM0NzcyNkI2RkE4RDZDRkFGQjQzQTlCQl92aWRlb19kYXNoaW5pdC5tcDQVAALIARIAFQIYUWlnX3hwdl9wbGFjZW1lbnRfcGVybWFuZW50X3YyLzkwNDc3OEIwRTY2MDYxNEY5RjA4MUMxQzNDMDVBMkFDX2F1ZGlvX2Rhc2hpbml0Lm1wNBUCAsgBEgAoABgAGwKIB3VzZV9vaWwBMRJwcm9ncmVzc2l2ZV9yZWNpcGUBMRUAACby1v-GuJTcBRUCKAJDMywXQC6qfvnbItEYEmRhc2hfYmFzZWxpbmVfMV92MREAdf4HZeadAQA&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&_nc_zt=28&_nc_ss=7a22e&oh=00_AQIrUM_ZU8b3LTxlgefSs0x6vjvCX9BV4piQfWrs74dXIg&oe=6A99835F",
|
||||
"width": 1080,
|
||||
"width_original": 1080
|
||||
}
|
||||
]
|
||||
]
|
||||
+103
-9
@@ -276,6 +276,79 @@ viewer retire the lone-video heuristic in `src/lib/post-tabs.ts` — see
|
||||
`/p/<shortcode>/` URL leaves it `null`. Sync always uses listing URLs, so this
|
||||
only matters when testing by hand.
|
||||
|
||||
### Collab posts: the JSON and the media disagree about whose post it is
|
||||
|
||||
Verified directly against two saved raw API responses (`docs/example-api-
|
||||
response-DcU3dM-gVgB.json`, a 2-way collab, and `docs/example-api-response-
|
||||
DcLDme7zoyo.json`, a 5-way collab owned by an external account), 2026-09-01.
|
||||
|
||||
For an Instagram Collab, `username`/`fullname`/`owner_id` in the listing
|
||||
response are always the **original poster's**, never the scraped account's —
|
||||
even when the scraped account is one of the collaborators, not the owner. The
|
||||
metadata `.json` sidecar (built from `include`, above) is filed under that
|
||||
same original-poster identity, since its filename template uses
|
||||
`{username}`. But the **media file** for that same post is written into
|
||||
whichever profile's own crawl directory triggered the download, and its
|
||||
filename's `{username}` slot took the *scraped* account's name, not the true
|
||||
owner's. Net effect: one physical post produces a JSON named for the real
|
||||
owner and a media file (or files, for a carousel) named for whoever we were
|
||||
crawling — two different identities for one post, in the same directory.
|
||||
|
||||
Downstream (`cosmo_normalize_instagram.py` in `Cosmo-Live-Downloads`) had to
|
||||
stop matching media to its JSON by username and match on shortcode/`{num}`
|
||||
only, and separately built a cross-profile `collab_with` pass so every
|
||||
participant's page shows the post, not just the one whose directory JD2/
|
||||
gallery-dl happened to land the JSON in. See that repo's `NOTES.md`,
|
||||
2026-09-01 entries, for the full fix.
|
||||
|
||||
**`coauthors` is a native, richer signal for this, and is now captured.**
|
||||
The raw API response carries a `coauthors` array (`{"full_name", "id",
|
||||
"username"}` per collaborator) that **excludes the post's own owner** —
|
||||
confirmed on both saved examples, including one where the owner
|
||||
(`dailyfashion_news`) is an external account with no ARTMS members in her
|
||||
own name, present only via `coauthors`. Added to the post-level metadata
|
||||
JSON's `include` list on 2026-09-01: a direct field instead of inferring a
|
||||
collab from filename/directory identity mismatches, and it never needs the
|
||||
"is this the owner" branch `coauthors` already excludes for us.
|
||||
|
||||
**Per-carousel-item fields got their own sidecar, added the same day.**
|
||||
`width`, `height`, `width_original`, `height_original` and `tagged_users`
|
||||
live on the per-FILE kwdict, not the per-post one the metadata JSON above
|
||||
reads — a carousel's items can each have different dimensions and tags,
|
||||
which one post-level JSON can't represent. `gdl-sync.py`'s `media_pp` is a
|
||||
second `metadata` postprocessor, `event: "file"` (gallery-dl's default when
|
||||
omitted), so it runs once per downloaded file and writes `<filename>.json`
|
||||
alongside it — e.g. `... - 01.jpg.json` next to `... - 01.jpg`, never
|
||||
colliding with the post-level `....json`, which has no per-item number.
|
||||
`owner` — a full user object (profile pic URLs, privacy flags) for whoever
|
||||
posted that specific item — is deliberately left out, the same reasoning as
|
||||
`audio_user` above. Verified against the same two saved examples: correct
|
||||
per-item `width`/`height` and `tagged_users` came back on a live re-fetch of
|
||||
an already-archived carousel, with zero media re-downloaded (the existing
|
||||
skip-archive still applies; only the new sidecars are new files).
|
||||
|
||||
### The shared archive-db dedups media across profiles too, not just within one
|
||||
|
||||
The skip-archive DB (see "Incremental sync" above) keys purely on
|
||||
`instagram_<media_id>`, with no per-profile scoping. When the *same* media_id
|
||||
is reachable from more than one profile's listing — a Collab post, or a
|
||||
repost — whichever profile's crawl reaches it **first** downloads the file;
|
||||
every other profile that later lists the same media_id sees it as
|
||||
already-in-the-archive-db and skips the download, even though that file has
|
||||
never actually landed in *that profile's own* directory tree. The metadata
|
||||
`.json`/`.txt` sidecars are written regardless (they aren't gated by the
|
||||
download-archive), so the symptom is a JSON with zero matching media files
|
||||
in its own directory — 510 such posts were found across the real archive on
|
||||
2026-09-01, entirely from this mechanism, not from anything actually missing
|
||||
from Instagram.
|
||||
|
||||
This isn't fixable on the gallery-dl side without per-profile archive DBs
|
||||
(which would defeat the point of skip-archive — re-downloading anything a
|
||||
sibling profile already fetched). The fix lives downstream instead: `cosmo_
|
||||
normalize_instagram.py` builds one archive-wide `shortcode → path` index once
|
||||
and falls back to it when a post's own directory has no matching media. Worth
|
||||
knowing before assuming a JSON-with-no-media post reflects a real scrape gap.
|
||||
|
||||
## Cadence, and the budget that enforces it
|
||||
|
||||
**Monthly for everything, daily for stories only.** Stories expire in 24h and
|
||||
@@ -559,19 +632,40 @@ Verify by asking gallery-dl's own interpreter, not the shell:
|
||||
- **Highlights already have two naming styles on disk**, and every undated file
|
||||
has a dated twin. The scanner dedupes by index so they render once; it is
|
||||
wasted disk, not a display bug.
|
||||
- **Some `video_versions` entries are VP9, and format selection is
|
||||
codec-blind.** The extractor picks `max(video_versions, key=lambda x:
|
||||
(x["width"], x["height"], x["type"]))` — resolution only, no codec check
|
||||
(`instagram.py`). Instagram appears to have started serving VP9-encoded
|
||||
highest-resolution variants for some posts around when the new gdl-based
|
||||
workflow started (2026-08); 87 such files were found archive-wide on
|
||||
2026-09-01. VP9-in-MP4 plays fine everywhere gallery-dl was tested from
|
||||
except **Safari/WebKit**, which wires VP9 decode only into its WebM
|
||||
demuxer, never its MP4/ISOBMFF path — confirmed via WebKit bug trackers,
|
||||
not guessed. Fetching a lower-resolution non-VP9 variant instead was
|
||||
considered and rejected (quality loss); the fix is downstream, a one-time
|
||||
`-c:v copy -c:a libopus` remux to `.webm` (`cosmo_remux_instagram_vp9.py`
|
||||
in `Cosmo-Live-Downloads`) that keeps VP9 losslessly and only re-encodes
|
||||
audio (WebM disallows AAC). Confirmed live via gallery-dl/yt-dlp that
|
||||
Instagram never offers a native WebM option to request instead — this has
|
||||
to be done locally, there's no source-side fix.
|
||||
|
||||
## Scanner work (not done yet)
|
||||
## Scanner work — done
|
||||
|
||||
`useArchiveScanner` currently treats any `.json` in the tree as a possible
|
||||
manifest. Adding gallery-dl sidecars needs it to distinguish three things:
|
||||
Shipped in `53b1f80` ("read gallery-dl sidecars for reel type and post dates").
|
||||
`useArchiveScanner` tells the three `.json` shapes apart **structurally**, not
|
||||
by filename, in `src/lib/gallery-dl-sidecar.ts`:
|
||||
|
||||
1. Instagram export manifests (`posts_1.json`) — existing path.
|
||||
2. Instaloader `.json.xz` — existing path, GraphQL node shape.
|
||||
3. gallery-dl `.json` — new, flat shape, identified by having
|
||||
`post_shortcode` + `type` at the top level.
|
||||
1. Instagram export manifests (`posts_1.json`) — top-level `media` array.
|
||||
2. Instaloader `.json.xz` — GraphQL node under `node` / `__typename`.
|
||||
3. gallery-dl `.json` — flat, `post_shortcode` + `type`, none of the above.
|
||||
|
||||
Once (3) is read, `source`/`isStory` and the reel flag should come from `type`
|
||||
rather than from the directory and the lone-video heuristic.
|
||||
`post.isReel` now comes from the sidecar's `type`, which is Instagram's own
|
||||
classification, and beats every fallback in `post-tabs.ts`. Post dates are
|
||||
ranked rather than last-write-wins (`src/lib/post-dates.ts`): sidecar beats
|
||||
filename beats mtime.
|
||||
|
||||
Note those files live on **`main`** — they parse the archive at display time
|
||||
and are viewer code, not fetching tooling.
|
||||
|
||||
## Test cases
|
||||
|
||||
|
||||
@@ -0,0 +1,184 @@
|
||||
slug,username,archive_dir,shortcode,type,date,post_url,json_file
|
||||
choerry,cher_ryppo,cher_ryppo,C5SBC--Syik,post,2024-04-03 01:50:25,https://www.instagram.com/p/C5SBC--Syik/,2024-04-02_cher_ryppo - C5SBC--Syik.json
|
||||
choerry,cher_ryppo,cher_ryppo,C811ib_ys7D,post,2024-06-30 13:16:41,https://www.instagram.com/p/C811ib_ys7D/,2024-06-30_cher_ryppo - C811ib_ys7D.json
|
||||
choerry,cher_ryppo,cher_ryppo,C9JTMgnSCHF,post,2024-07-08 02:41:25,https://www.instagram.com/p/C9JTMgnSCHF/,2024-07-07_cher_ryppo - C9JTMgnSCHF.json
|
||||
choerry,cher_ryppo,cher_ryppo,C-K6kxZy6qq,post,2024-08-02 14:16:09,https://www.instagram.com/p/C-K6kxZy6qq/,2024-08-02_cher_ryppo - C-K6kxZy6qq.json
|
||||
choerry,cher_ryppo,cher_ryppo,DBXI0zVs_Iv,post,2024-10-20 21:45:44,https://www.instagram.com/p/DBXI0zVs_Iv/,2024-10-20_cher_ryppo - DBXI0zVs_Iv.json
|
||||
choerry,cher_ryppo,cher_ryppo,DRRDiezjdyV,post,2025-11-20 05:26:24,https://www.instagram.com/p/DRRDiezjdyV/,2025-11-20_cher_ryppo - DRRDiezjdyV.json
|
||||
haseul,withaseul,withaseul,Cqps3ZWBlav,post,2023-04-05 10:44:56,https://www.instagram.com/p/Cqps3ZWBlav/,2023-04-05_withaseul - Cqps3ZWBlav.json
|
||||
haseul,withaseul,withaseul,CqsjfUzSelq,post,2023-04-06 13:20:43,https://www.instagram.com/p/CqsjfUzSelq/,2023-04-06_withaseul - CqsjfUzSelq.json
|
||||
haseul,withaseul,withaseul,CqxiTfjhNJ2,post,2023-04-08 11:46:34,https://www.instagram.com/p/CqxiTfjhNJ2/,2023-04-08_withaseul - CqxiTfjhNJ2.json
|
||||
haseul,withaseul,withaseul,Cq4qymBhftL,post,2023-04-11 06:15:24,https://www.instagram.com/p/Cq4qymBhftL/,2023-04-11_withaseul - Cq4qymBhftL.json
|
||||
haseul,withaseul,withaseul,CrnhJtrhoZi,post,2023-04-29 10:55:29,https://www.instagram.com/p/CrnhJtrhoZi/,2023-04-29_withaseul - CrnhJtrhoZi.json
|
||||
haseul,withaseul,withaseul,Cr7fZZyB4Sp,post,2023-05-07 05:04:58,https://www.instagram.com/p/Cr7fZZyB4Sp/,2023-05-07_withaseul - Cr7fZZyB4Sp.json
|
||||
haseul,withaseul,withaseul,Cr8QeaFhFsq,post,2023-05-07 12:13:49,https://www.instagram.com/p/Cr8QeaFhFsq/,2023-05-07_withaseul - Cr8QeaFhFsq.json
|
||||
haseul,withaseul,withaseul,CsBIA4HBUOX,post,2023-05-09 09:36:05,https://www.instagram.com/p/CsBIA4HBUOX/,2023-05-09_withaseul - CsBIA4HBUOX.json
|
||||
haseul,withaseul,withaseul,CsgbxFDhNmj,post,2023-05-21 13:25:08,https://www.instagram.com/p/CsgbxFDhNmj/,2023-05-21_withaseul - CsgbxFDhNmj.json
|
||||
haseul,withaseul,withaseul,Csn9aO4B1r-,post,2023-05-24 11:33:48,https://www.instagram.com/p/Csn9aO4B1r-/,2023-05-24_withaseul - Csn9aO4B1r-.json
|
||||
haseul,withaseul,withaseul,CsvY2jrBjMc,post,2023-05-27 08:48:17,https://www.instagram.com/p/CsvY2jrBjMc/,2023-05-27_withaseul - CsvY2jrBjMc.json
|
||||
haseul,withaseul,withaseul,CtEHmRDPesc,post,2023-06-04 10:01:34,https://www.instagram.com/p/CtEHmRDPesc/,2023-06-04_withaseul - CtEHmRDPesc.json
|
||||
haseul,withaseul,withaseul,CtRmUjsBUoc,post,2023-06-09 15:40:09,https://www.instagram.com/p/CtRmUjsBUoc/,2023-06-09_withaseul - CtRmUjsBUoc.json
|
||||
haseul,withaseul,withaseul,Cuzar_3LqAA,post,2023-07-17 15:24:04,https://www.instagram.com/p/Cuzar_3LqAA/,2023-07-17_withaseul - Cuzar_3LqAA.json
|
||||
haseul,withaseul,withaseul,CvHe1VMr7C-,post,2023-07-25 10:25:06,https://www.instagram.com/p/CvHe1VMr7C-/,2023-07-25_withaseul - CvHe1VMr7C-.json
|
||||
haseul,withaseul,withaseul,CvhRFq7rnHc,post,2023-08-04 10:45:15,https://www.instagram.com/p/CvhRFq7rnHc/,2023-08-04_withaseul - CvhRFq7rnHc.json
|
||||
haseul,withaseul,withaseul,CwIr3qEBPgF,post,2023-08-19 18:09:39,https://www.instagram.com/p/CwIr3qEBPgF/,2023-08-19_withaseul - CwIr3qEBPgF.json
|
||||
haseul,withaseul,withaseul,CwzPlMThh5m,post,2023-09-05 06:49:48,https://www.instagram.com/p/CwzPlMThh5m/,2023-09-05_withaseul - CwzPlMThh5m.json
|
||||
haseul,withaseul,withaseul,Cw9s3LNBhsq,post,2023-09-09 08:18:04,https://www.instagram.com/p/Cw9s3LNBhsq/,2023-09-09_withaseul - Cw9s3LNBhsq.json
|
||||
haseul,withaseul,withaseul,CxIma4nBlaP,post,2023-09-13 13:53:26,https://www.instagram.com/p/CxIma4nBlaP/,2023-09-13_withaseul - CxIma4nBlaP.json
|
||||
haseul,withaseul,withaseul,CxLcZ2jBME5,post,2023-09-14 16:23:38,https://www.instagram.com/p/CxLcZ2jBME5/,2023-09-14_withaseul - CxLcZ2jBME5.json
|
||||
haseul,withaseul,withaseul,CxN3g95hVDH,post,2023-09-15 14:59:00,https://www.instagram.com/p/CxN3g95hVDH/,2023-09-15_withaseul - CxN3g95hVDH.json
|
||||
haseul,withaseul,withaseul,CxTGBb_BAYw,post,2023-09-17 15:41:59,https://www.instagram.com/p/CxTGBb_BAYw/,2023-09-17_withaseul - CxTGBb_BAYw.json
|
||||
haseul,withaseul,withaseul,CxnTyJIhUc0,post,2023-09-25 12:07:02,https://www.instagram.com/p/CxnTyJIhUc0/,2023-09-25_withaseul - CxnTyJIhUc0.json
|
||||
haseul,withaseul,withaseul,CxsPZAur6Xv,post,2023-09-27 10:04:51,https://www.instagram.com/p/CxsPZAur6Xv/,2023-09-27_withaseul - CxsPZAur6Xv.json
|
||||
haseul,withaseul,withaseul,Cx7mOFqhU7R,post,2023-10-03 09:12:57,https://www.instagram.com/p/Cx7mOFqhU7R/,2023-10-03_withaseul - Cx7mOFqhU7R.json
|
||||
haseul,withaseul,withaseul,CyqV0lfruX-,post,2023-10-21 12:53:58,https://www.instagram.com/p/CyqV0lfruX-/,2023-10-21_withaseul - CyqV0lfruX-.json
|
||||
haseul,withaseul,withaseul,Cy7RByYh2BH,post,2023-10-28 02:39:10,https://www.instagram.com/p/Cy7RByYh2BH/,2023-10-27_withaseul - Cy7RByYh2BH.json
|
||||
haseul,withaseul,withaseul,Cz7nefxBJT0,post,2023-11-22 02:26:43,https://www.instagram.com/p/Cz7nefxBJT0/,2023-11-21_withaseul - Cz7nefxBJT0.json
|
||||
haseul,withaseul,withaseul,C0YxsROhG4W,post,2023-12-03 10:13:57,https://www.instagram.com/p/C0YxsROhG4W/,2023-12-03_withaseul - C0YxsROhG4W.json
|
||||
haseul,withaseul,withaseul,C0zOk2JBXTY,post,2023-12-13 16:46:36,https://www.instagram.com/p/C0zOk2JBXTY/,2023-12-13_withaseul - C0zOk2JBXTY.json
|
||||
haseul,withaseul,withaseul,C1q_9TzBrdt,post,2024-01-04 08:36:20,https://www.instagram.com/p/C1q_9TzBrdt/,2024-01-04_withaseul - C1q_9TzBrdt.json
|
||||
haseul,withaseul,withaseul,C1t-aeHBCef,post,2024-01-05 12:20:34,https://www.instagram.com/p/C1t-aeHBCef/,2024-01-05_withaseul - C1t-aeHBCef.json
|
||||
haseul,withaseul,withaseul,C1yrgmMhsmz,post,2024-01-07 08:11:35,https://www.instagram.com/p/C1yrgmMhsmz/,2024-01-07_withaseul - C1yrgmMhsmz.json
|
||||
haseul,withaseul,withaseul,C1yrxqIhGBZ,post,2024-01-07 08:13:54,https://www.instagram.com/p/C1yrxqIhGBZ/,2024-01-07_withaseul - C1yrxqIhGBZ.json
|
||||
haseul,withaseul,withaseul,C10p70xh-6x,post,2024-01-08 02:36:18,https://www.instagram.com/p/C10p70xh-6x/,2024-01-07_withaseul - C10p70xh-6x.json
|
||||
haseul,withaseul,withaseul,C2APujnBSF-,post,2024-01-12 14:38:11,https://www.instagram.com/p/C2APujnBSF-/,2024-01-12_withaseul - C2APujnBSF-.json
|
||||
haseul,withaseul,withaseul,C3enxknhzeF,post,2024-02-18 06:16:55,https://www.instagram.com/p/C3enxknhzeF/,2024-02-18_withaseul - C3enxknhzeF.json
|
||||
haseul,withaseul,withaseul,C4f8UQIBZAe,post,2024-03-14 15:07:03,https://www.instagram.com/p/C4f8UQIBZAe/,2024-03-14_withaseul - C4f8UQIBZAe.json
|
||||
haseul,withaseul,withaseul,C4qIrJTh3po,post,2024-03-18 14:07:26,https://www.instagram.com/p/C4qIrJTh3po/,2024-03-18_withaseul - C4qIrJTh3po.json
|
||||
haseul,withaseul,withaseul,C4rjDNhhJu8,post,2024-03-19 03:17:09,https://www.instagram.com/p/C4rjDNhhJu8/,2024-03-18_withaseul - C4rjDNhhJu8.json
|
||||
haseul,withaseul,withaseul,C4xSiGGhP40,post,2024-03-21 08:48:16,https://www.instagram.com/p/C4xSiGGhP40/,2024-03-21_withaseul - C4xSiGGhP40.json
|
||||
haseul,withaseul,withaseul,C5GBz4FB-kE,post,2024-03-29 10:06:12,https://www.instagram.com/p/C5GBz4FB-kE/,2024-03-29_withaseul - C5GBz4FB-kE.json
|
||||
haseul,withaseul,withaseul,C5GJmlABYug,post,2024-03-29 11:14:17,https://www.instagram.com/p/C5GJmlABYug/,2024-03-29_withaseul - C5GJmlABYug.json
|
||||
haseul,withaseul,withaseul,C5LgFFtB8ri,post,2024-03-31 13:06:54,https://www.instagram.com/p/C5LgFFtB8ri/,2024-03-31_withaseul - C5LgFFtB8ri.json
|
||||
haseul,withaseul,withaseul,C5VZGPDB-Qo,post,2024-04-04 09:18:17,https://www.instagram.com/p/C5VZGPDB-Qo/,2024-04-04_withaseul - C5VZGPDB-Qo.json
|
||||
haseul,withaseul,withaseul - reels,C5bWrT3BaGy,post,2024-04-06 16:52:50,https://www.instagram.com/p/C5bWrT3BaGy/,2024-04-06_withaseul - C5bWrT3BaGy.json
|
||||
haseul,withaseul,withaseul,C6LAH8vBs_a,post,2024-04-25 04:59:04,https://www.instagram.com/p/C6LAH8vBs_a/,2024-04-25_withaseul - C6LAH8vBs_a.json
|
||||
haseul,withaseul,withaseul,C6ypIzVBmiL,post,2024-05-10 14:27:49,https://www.instagram.com/p/C6ypIzVBmiL/,2024-05-10_withaseul - C6ypIzVBmiL.json
|
||||
haseul,withaseul,withaseul,C68cTpzhUzQ,post,2024-05-14 09:48:07,https://www.instagram.com/p/C68cTpzhUzQ/,2024-05-14_withaseul - C68cTpzhUzQ.json
|
||||
haseul,withaseul,withaseul,C68oiiahwZd,post,2024-05-14 11:35:00,https://www.instagram.com/p/C68oiiahwZd/,2024-05-14_withaseul - C68oiiahwZd.json
|
||||
haseul,withaseul,withaseul,C7Oh6mYB7oF,post,2024-05-21 10:23:27,https://www.instagram.com/p/C7Oh6mYB7oF/,2024-05-21_withaseul - C7Oh6mYB7oF.json
|
||||
haseul,withaseul,withaseul - reels,C7RnCv6BZT3,post,2024-05-22 15:06:48,https://www.instagram.com/p/C7RnCv6BZT3/,2024-05-22_withaseul - C7RnCv6BZT3.json
|
||||
haseul,withaseul,withaseul,C7R7owoBd4n,post,2024-05-22 18:05:56,https://www.instagram.com/p/C7R7owoBd4n/,2024-05-22_withaseul - C7R7owoBd4n.json
|
||||
haseul,withaseul,withaseul,C7mPzLRhCRz,post,2024-05-30 15:26:55,https://www.instagram.com/p/C7mPzLRhCRz/,2024-05-30_withaseul - C7mPzLRhCRz.json
|
||||
haseul,withaseul,withaseul,C7tNIlPh7NK,post,2024-06-02 08:18:19,https://www.instagram.com/p/C7tNIlPh7NK/,2024-06-02_withaseul - C7tNIlPh7NK.json
|
||||
haseul,withaseul,withaseul,C70yRTmBgng,post,2024-06-05 06:57:30,https://www.instagram.com/p/C70yRTmBgng/,2024-06-05_withaseul - C70yRTmBgng.json
|
||||
haseul,withaseul,withaseul,C71aPMWBfqs,post,2024-06-05 12:46:44,https://www.instagram.com/p/C71aPMWBfqs/,2024-06-05_withaseul - C71aPMWBfqs.json
|
||||
haseul,withaseul,withaseul,C8NBuqRhCJv,post,2024-06-14 16:54:21,https://www.instagram.com/p/C8NBuqRhCJv/,2024-06-14_withaseul - C8NBuqRhCJv.json
|
||||
haseul,withaseul,withaseul,C8PPpUKhyLf,post,2024-06-15 13:34:26,https://www.instagram.com/p/C8PPpUKhyLf/,2024-06-15_withaseul - C8PPpUKhyLf.json
|
||||
haseul,withaseul,withaseul,C8UjYG8B74s,post,2024-06-17 15:03:03,https://www.instagram.com/p/C8UjYG8B74s/,2024-06-17_withaseul - C8UjYG8B74s.json
|
||||
haseul,withaseul,withaseul,C87PlKyBTU6,post,2024-07-02 15:40:27,https://www.instagram.com/p/C87PlKyBTU6/,2024-07-02_withaseul - C87PlKyBTU6.json
|
||||
haseul,withaseul,withaseul,C9FqomJhky4,post,2024-07-06 16:49:16,https://www.instagram.com/p/C9FqomJhky4/,2024-07-06_withaseul - C9FqomJhky4.json
|
||||
haseul,withaseul,withaseul,C9ICMZ5BUfr,post,2024-07-07 14:53:36,https://www.instagram.com/p/C9ICMZ5BUfr/,2024-07-07_withaseul - C9ICMZ5BUfr.json
|
||||
haseul,withaseul,withaseul,C9V8bjJBPdk,post,2024-07-13 00:32:37,https://www.instagram.com/p/C9V8bjJBPdk/,2024-07-12_withaseul - C9V8bjJBPdk.json
|
||||
haseul,withaseul,withaseul,C9xK8b1SZAY,post,2024-07-23 14:18:56,https://www.instagram.com/p/C9xK8b1SZAY/,2024-07-23_withaseul - C9xK8b1SZAY.json
|
||||
haseul,withaseul,withaseul,C-DLjJ6BE0y,post,2024-07-30 14:10:33,https://www.instagram.com/p/C-DLjJ6BE0y/,2024-07-30_withaseul - C-DLjJ6BE0y.json
|
||||
haseul,withaseul,withaseul,C-Ncp1MyhiD,post,2024-08-03 13:52:25,https://www.instagram.com/p/C-Ncp1MyhiD/,2024-08-03_withaseul - C-Ncp1MyhiD.json
|
||||
haseul,withaseul,withaseul,C_A4QkApuLO,post,2024-08-23 13:14:54,https://www.instagram.com/p/C_A4QkApuLO/,2024-08-23_withaseul - C_A4QkApuLO.json
|
||||
haseul,withaseul,withaseul,C_CcGblt8X9,post,2024-08-24 03:47:20,https://www.instagram.com/p/C_CcGblt8X9/,2024-08-23_withaseul - C_CcGblt8X9.json
|
||||
haseul,withaseul,withaseul,C_ZCyXPylSu,post,2024-09-01 22:28:40,https://www.instagram.com/p/C_ZCyXPylSu/,2024-09-01_withaseul - C_ZCyXPylSu.json
|
||||
haseul,withaseul,withaseul,C_gRp4gPuY3,post,2024-09-04 17:53:16,https://www.instagram.com/p/C_gRp4gPuY3/,2024-09-04_withaseul - C_gRp4gPuY3.json
|
||||
haseul,withaseul,withaseul,C_7XyvJMmm7,post,2024-09-15 06:26:24,https://www.instagram.com/p/C_7XyvJMmm7/,2024-09-15_withaseul - C_7XyvJMmm7.json
|
||||
haseul,withaseul,withaseul,DACAVOGyOEc,post,2024-09-17 20:16:04,https://www.instagram.com/p/DACAVOGyOEc/,2024-09-17_withaseul - DACAVOGyOEc.json
|
||||
haseul,withaseul,withaseul,DAqwdZIBjY5,post,2024-10-03 16:06:14,https://www.instagram.com/p/DAqwdZIBjY5/,2024-10-03_withaseul - DAqwdZIBjY5.json
|
||||
haseul,withaseul,withaseul,DAv8DxvyGbr,post,2024-10-05 16:23:48,https://www.instagram.com/p/DAv8DxvyGbr/,2024-10-05_withaseul - DAv8DxvyGbr.json
|
||||
haseul,withaseul,withaseul,DBhdsL1NIvC,post,2024-10-24 22:00:28,https://www.instagram.com/p/DBhdsL1NIvC/,2024-10-24_withaseul - DBhdsL1NIvC.json
|
||||
haseul,withaseul,withaseul,DBrP-pYSDEn,post,2024-10-28 17:13:03,https://www.instagram.com/p/DBrP-pYSDEn/,2024-10-28_withaseul - DBrP-pYSDEn.json
|
||||
haseul,withaseul,withaseul,DBu5zCHBgrT,post,2024-10-30 03:16:12,https://www.instagram.com/p/DBu5zCHBgrT/,2024-10-29_withaseul - DBu5zCHBgrT.json
|
||||
haseul,withaseul,withaseul,DByVUvOBSQy,post,2024-10-31 11:14:27,https://www.instagram.com/p/DByVUvOBSQy/,2024-10-31_withaseul - DByVUvOBSQy.json
|
||||
haseul,withaseul,withaseul,DB5re5DhW8a,post,2024-11-03 07:42:45,https://www.instagram.com/p/DB5re5DhW8a/,2024-11-03_withaseul - DB5re5DhW8a.json
|
||||
haseul,withaseul,withaseul,DCBsfwGhFYd,post,2024-11-06 10:25:32,https://www.instagram.com/p/DCBsfwGhFYd/,2024-11-06_withaseul - DCBsfwGhFYd.json
|
||||
haseul,withaseul,withaseul,DCObB0SR64m,post,2024-11-11 09:03:02,https://www.instagram.com/p/DCObB0SR64m/,2024-11-11_withaseul - DCObB0SR64m.json
|
||||
haseul,withaseul,withaseul,DCRhOPphLWU,post,2024-11-12 13:54:52,https://www.instagram.com/p/DCRhOPphLWU/,2024-11-12_withaseul - DCRhOPphLWU.json
|
||||
haseul,withaseul,withaseul,DCbgyVghiT-,post,2024-11-16 11:03:28,https://www.instagram.com/p/DCbgyVghiT-/,2024-11-16_withaseul - DCbgyVghiT-.json
|
||||
haseul,withaseul,withaseul,DCe4qpXhgBV,post,2024-11-17 18:29:51,https://www.instagram.com/p/DCe4qpXhgBV/,2024-11-17_withaseul - DCe4qpXhgBV.json
|
||||
haseul,withaseul,withaseul,DC_4KddBmo_,post,2024-11-30 14:00:24,https://www.instagram.com/p/DC_4KddBmo_/,2024-11-30_withaseul - DC_4KddBmo_.json
|
||||
haseul,withaseul,withaseul,DDRNi9Bhicn,post,2024-12-07 07:34:20,https://www.instagram.com/p/DDRNi9Bhicn/,2024-12-07_withaseul - DDRNi9Bhicn.json
|
||||
haseul,withaseul,withaseul,DDW2JdFBUMs,post,2024-12-09 12:05:19,https://www.instagram.com/p/DDW2JdFBUMs/,2024-12-09_withaseul - DDW2JdFBUMs.json
|
||||
haseul,withaseul,withaseul,DDjWETEBWw2,post,2024-12-14 08:35:07,https://www.instagram.com/p/DDjWETEBWw2/,2024-12-14_withaseul - DDjWETEBWw2.json
|
||||
haseul,withaseul,withaseul,DDpkLTyhUdt,post,2024-12-16 18:33:51,https://www.instagram.com/p/DDpkLTyhUdt/,2024-12-16_withaseul - DDpkLTyhUdt.json
|
||||
haseul,withaseul,withaseul,DE1NVFihF2g,post,2025-01-15 03:36:30,https://www.instagram.com/p/DE1NVFihF2g/,2025-01-14_withaseul - DE1NVFihF2g.json
|
||||
haseul,withaseul,withaseul,DE5GO2-S5PB,post,2025-01-16 15:51:26,https://www.instagram.com/p/DE5GO2-S5PB/,2025-01-16_withaseul - DE5GO2-S5PB.json
|
||||
haseul,withaseul,withaseul - reels,DE5H7LcB5aX,post,2025-01-16 16:07:31,https://www.instagram.com/p/DE5H7LcB5aX/,2025-01-16_withaseul - DE5H7LcB5aX.json
|
||||
haseul,withaseul,withaseul,DFFtju9hTdv,post,2025-01-21 13:25:58,https://www.instagram.com/p/DFFtju9hTdv/,2025-01-21_withaseul - DFFtju9hTdv.json
|
||||
haseul,withaseul,withaseul,DFsLxphhwqE,post,2025-02-05 12:01:09,https://www.instagram.com/p/DFsLxphhwqE/,2025-02-05_withaseul - DFsLxphhwqE.json
|
||||
haseul,withaseul,withaseul,DHvlOgAxE_j,post,2025-03-28 12:44:03,https://www.instagram.com/p/DHvlOgAxE_j/,2025-03-28_withaseul - DHvlOgAxE_j.json
|
||||
haseul,withaseul,withaseul,DH96B0QtxfT,post,2025-04-03 02:15:11,https://www.instagram.com/p/DH96B0QtxfT/,2025-04-02_withaseul - DH96B0QtxfT.json
|
||||
haseul,withaseul,withaseul,DIL6wNXRGDC,post,2025-04-08 12:50:53,https://www.instagram.com/p/DIL6wNXRGDC/,2025-04-08_withaseul - DIL6wNXRGDC.json
|
||||
haseul,withaseul,withaseul,DIP1mift6QV,post,2025-04-10 01:22:50,https://www.instagram.com/p/DIP1mift6QV/,2025-04-09_withaseul - DIP1mift6QV.json
|
||||
haseul,withaseul,withaseul,DIfvDIKxwuT,post,2025-04-16 05:33:25,https://www.instagram.com/p/DIfvDIKxwuT/,2025-04-16_withaseul - DIfvDIKxwuT.json
|
||||
haseul,withaseul,withaseul,DIm-MshvCJy,post,2025-04-19 01:00:03,https://www.instagram.com/p/DIm-MshvCJy/,2025-04-18_withaseul - DIm-MshvCJy.json
|
||||
haseul,withaseul,withaseul,DIuQyfphWnR,post,2025-04-21 20:57:37,https://www.instagram.com/p/DIuQyfphWnR/,2025-04-21_withaseul - DIuQyfphWnR.json
|
||||
haseul,withaseul,withaseul,DI3hOzrBLs-,post,2025-04-25 11:14:27,https://www.instagram.com/p/DI3hOzrBLs-/,2025-04-25_withaseul - DI3hOzrBLs-.json
|
||||
haseul,withaseul,withaseul,DK0lklBhrH6,post,2025-06-13 00:57:27,https://www.instagram.com/p/DK0lklBhrH6/,2025-06-12_withaseul - DK0lklBhrH6.json
|
||||
haseul,withaseul,withaseul,DK1EdUZBJtr,post,2025-06-13 05:27:20,https://www.instagram.com/p/DK1EdUZBJtr/,2025-06-13_withaseul - DK1EdUZBJtr.json
|
||||
haseul,withaseul,withaseul,DLJiMzrhM83,post,2025-06-21 04:12:02,https://www.instagram.com/p/DLJiMzrhM83/,2025-06-21_withaseul - DLJiMzrhM83.json
|
||||
haseul,withaseul,withaseul,DLL1iePBcwm,post,2025-06-22 01:39:30,https://www.instagram.com/p/DLL1iePBcwm/,2025-06-21_withaseul - DLL1iePBcwm.json
|
||||
haseul,withaseul,withaseul,DLM-sUIhK5P,post,2025-06-22 12:18:44,https://www.instagram.com/p/DLM-sUIhK5P/,2025-06-22_withaseul - DLM-sUIhK5P.json
|
||||
haseul,withaseul,withaseul,DLU66yCvWZh,post,2025-06-25 14:19:41,https://www.instagram.com/p/DLU66yCvWZh/,2025-06-25_withaseul - DLU66yCvWZh.json
|
||||
haseul,withaseul,withaseul,DLxAZg5hpnW,post,2025-07-06 12:06:18,https://www.instagram.com/p/DLxAZg5hpnW/,2025-07-06_withaseul - DLxAZg5hpnW.json
|
||||
haseul,withaseul,withaseul,DMFfDqABJgM,post,2025-07-14 10:59:00,https://www.instagram.com/p/DMFfDqABJgM/,2025-07-14_withaseul - DMFfDqABJgM.json
|
||||
haseul,withaseul,withaseul,DM0NH9yhkzT,post,2025-08-01 14:26:37,https://www.instagram.com/p/DM0NH9yhkzT/,2025-08-01_withaseul - DM0NH9yhkzT.json
|
||||
haseul,withaseul,withaseul,DM9_Ol7SzmK,post,2025-08-05 09:37:35,https://www.instagram.com/p/DM9_Ol7SzmK/,2025-08-05_withaseul - DM9_Ol7SzmK.json
|
||||
haseul,withaseul,withaseul,DNnih4pBd_M,post,2025-08-21 12:54:55,https://www.instagram.com/p/DNnih4pBd_M/,2025-08-21_withaseul - DNnih4pBd_M.json
|
||||
haseul,withaseul,withaseul,DOIxptrgXgv,post,2025-09-03 10:42:00,https://www.instagram.com/p/DOIxptrgXgv/,2025-09-03_withaseul - DOIxptrgXgv.json
|
||||
haseul,withaseul,withaseul,DOa8a_ugc2Z,post,2025-09-10 12:02:26,https://www.instagram.com/p/DOa8a_ugc2Z/,2025-09-10_withaseul - DOa8a_ugc2Z.json
|
||||
haseul,withaseul,withaseul,DPMLxsmgUJJ,post,2025-09-29 14:59:24,https://www.instagram.com/p/DPMLxsmgUJJ/,2025-09-29_withaseul - DPMLxsmgUJJ.json
|
||||
haseul,withaseul,withaseul,DPRez1QgUxE,post,2025-10-01 16:21:55,https://www.instagram.com/p/DPRez1QgUxE/,2025-10-01_withaseul - DPRez1QgUxE.json
|
||||
haseul,withaseul,withaseul,DQg9WOOgaKD,post,2025-11-01 13:08:45,https://www.instagram.com/p/DQg9WOOgaKD/,2025-11-01_withaseul - DQg9WOOgaKD.json
|
||||
haseul,withaseul,withaseul,DQ_JrMdDciC,post,2025-11-13 06:33:42,https://www.instagram.com/p/DQ_JrMdDciC/,2025-11-13_withaseul - DQ_JrMdDciC.json
|
||||
haseul,withaseul,withaseul,DROhLjLjVqY,post,2025-11-19 05:47:42,https://www.instagram.com/p/DROhLjLjVqY/,2025-11-19_withaseul - DROhLjLjVqY.json
|
||||
haseul,withaseul,withaseul,DRwiTEDDRHR,post,2025-12-02 10:51:38,https://www.instagram.com/p/DRwiTEDDRHR/,2025-12-02_withaseul - DRwiTEDDRHR.json
|
||||
haseul,withaseul,withaseul,DTw-4d9gRUx,post,2026-01-21 08:04:12,https://www.instagram.com/p/DTw-4d9gRUx/,2026-01-21_withaseul - DTw-4d9gRUx.json
|
||||
haseul,withaseul,withaseul,DT0rHDCjPpE,post,2026-01-22 18:28:24,https://www.instagram.com/p/DT0rHDCjPpE/,2026-01-22_withaseul - DT0rHDCjPpE.json
|
||||
haseul,withaseul,withaseul,DT6HqajDJfF,post,2026-01-24 21:14:05,https://www.instagram.com/p/DT6HqajDJfF/,2026-01-24_withaseul - DT6HqajDJfF.json
|
||||
haseul,withaseul,withaseul,DU0oU25gWso,post,2026-02-16 14:19:54,https://www.instagram.com/p/DU0oU25gWso/,2026-02-16_withaseul - DU0oU25gWso.json
|
||||
haseul,withaseul,withaseul,DWZPNAKAZcl,post,2026-03-27 16:19:14,https://www.instagram.com/p/DWZPNAKAZcl/,2026-03-27_withaseul - DWZPNAKAZcl.json
|
||||
haseul,withaseul,withaseul,DW3yBm2gctK,post,2026-04-08 13:00:00,https://www.instagram.com/p/DW3yBm2gctK/,2026-04-08_withaseul - DW3yBm2gctK.json
|
||||
haseul,withaseul,withaseul,DZj6uVWhV8M,post,2026-06-14 09:26:10,https://www.instagram.com/p/DZj6uVWhV8M/,2026-06-14_withaseul - DZj6uVWhV8M.json
|
||||
haseul,withaseul,withaseul,DaZm_xLAZeL,post,2026-07-05 05:52:43,https://www.instagram.com/p/DaZm_xLAZeL/,2026-07-05_withaseul - DaZm_xLAZeL.json
|
||||
haseul,withaseul,withaseul,Dbnq4k9gb5U,post,2026-08-04 13:27:27,https://www.instagram.com/p/Dbnq4k9gb5U/,2026-08-04_withaseul - Dbnq4k9gb5U.json
|
||||
heejin,0ct0ber19,0ct0ber19,CrdsY5CrSsO,post,2023-04-25 15:21:16,https://www.instagram.com/p/CrdsY5CrSsO/,2023-04-25_0ct0ber19 - CrdsY5CrSsO.json
|
||||
heejin,0ct0ber19,0ct0ber19,CtohvHxLnWO,post,2023-06-18 13:22:37,https://www.instagram.com/p/CtohvHxLnWO/,2023-06-18_0ct0ber19 - CtohvHxLnWO.json
|
||||
heejin,0ct0ber19,0ct0ber19,CuEOMWppp5S,post,2023-06-29 07:30:35,https://www.instagram.com/p/CuEOMWppp5S/,2023-06-29_0ct0ber19 - CuEOMWppp5S.json
|
||||
heejin,0ct0ber19,0ct0ber19,CwC0Y-prGoB,post,2023-08-17 11:28:40,https://www.instagram.com/p/CwC0Y-prGoB/,2023-08-17_0ct0ber19 - CwC0Y-prGoB.json
|
||||
heejin,0ct0ber19,0ct0ber19,CxsKPWgpzU4,post,2023-09-27 09:19:51,https://www.instagram.com/p/CxsKPWgpzU4/,2023-09-27_0ct0ber19 - CxsKPWgpzU4.json
|
||||
heejin,0ct0ber19,0ct0ber19,CzJf78xryub,post,2023-11-02 15:18:48,https://www.instagram.com/p/CzJf78xryub/,2023-11-02_0ct0ber19 - CzJf78xryub.json
|
||||
heejin,0ct0ber19,0ct0ber19,CzOVVzFLTnD,post,2023-11-04 12:22:25,https://www.instagram.com/p/CzOVVzFLTnD/,2023-11-04_0ct0ber19 - CzOVVzFLTnD.json
|
||||
heejin,0ct0ber19,0ct0ber19,C14VKVmpdtC,post,2024-01-09 12:51:44,https://www.instagram.com/p/C14VKVmpdtC/,2024-01-09_0ct0ber19 - C14VKVmpdtC.json
|
||||
heejin,0ct0ber19,0ct0ber19,C2FnIelpue9,post,2024-01-14 16:38:54,https://www.instagram.com/p/C2FnIelpue9/,2024-01-14_0ct0ber19 - C2FnIelpue9.json
|
||||
heejin,0ct0ber19,0ct0ber19,C5SDY_UJIn6,post,2024-04-03 02:10:53,https://www.instagram.com/p/C5SDY_UJIn6/,2024-04-02_0ct0ber19 - C5SDY_UJIn6.json
|
||||
heejin,0ct0ber19,0ct0ber19,C5WB6zwp_rX,post,2024-04-04 15:15:00,https://www.instagram.com/p/C5WB6zwp_rX/,2024-04-04_0ct0ber19 - C5WB6zwp_rX.json
|
||||
heejin,0ct0ber19,0ct0ber19,C5dJ0xXpJpX,post,2024-04-07 09:38:46,https://www.instagram.com/p/C5dJ0xXpJpX/,2024-04-07_0ct0ber19 - C5dJ0xXpJpX.json
|
||||
heejin,0ct0ber19,0ct0ber19,C5oJdT-pQBI,post,2024-04-11 16:07:12,https://www.instagram.com/p/C5oJdT-pQBI/,2024-04-11_0ct0ber19 - C5oJdT-pQBI.json
|
||||
heejin,0ct0ber19,0ct0ber19,C53YPQzp7Wj,post,2024-04-17 14:04:58,https://www.instagram.com/p/C53YPQzp7Wj/,2024-04-17_0ct0ber19 - C53YPQzp7Wj.json
|
||||
heejin,0ct0ber19,0ct0ber19,C5--hdUJKZH,post,2024-04-20 12:54:11,https://www.instagram.com/p/C5--hdUJKZH/,2024-04-20_0ct0ber19 - C5--hdUJKZH.json
|
||||
heejin,0ct0ber19,0ct0ber19,C71iAmbpekA,post,2024-06-05 13:54:39,https://www.instagram.com/p/C71iAmbpekA/,2024-06-05_0ct0ber19 - C71iAmbpekA.json
|
||||
heejin,0ct0ber19,0ct0ber19,C8M8TxnSOKI,post,2024-06-14 16:07:00,https://www.instagram.com/p/C8M8TxnSOKI/,2024-06-14_0ct0ber19 - C8M8TxnSOKI.json
|
||||
heejin,0ct0ber19,0ct0ber19,C8hxpuYyWGv,post,2024-06-22 18:17:55,https://www.instagram.com/p/C8hxpuYyWGv/,2024-06-22_0ct0ber19 - C8hxpuYyWGv.json
|
||||
heejin,0ct0ber19,0ct0ber19,C-AbtWSJbHB,post,2024-07-29 12:34:02,https://www.instagram.com/p/C-AbtWSJbHB/,2024-07-29_0ct0ber19 - C-AbtWSJbHB.json
|
||||
heejin,0ct0ber19,0ct0ber19,C_WxFtuvqXK,post,2024-09-01 01:15:32,https://www.instagram.com/p/C_WxFtuvqXK/,2024-08-31_0ct0ber19 - C_WxFtuvqXK.json
|
||||
heejin,0ct0ber19,0ct0ber19,DcgTbz-iTAH,post,2026-08-26 13:19:15,https://www.instagram.com/p/DcgTbz-iTAH/,2026-08-26_0ct0ber19 - DcgTbz-iTAH.json
|
||||
jinsoul,zindoriyam,zindoriyam,CndmboMBHeB,post,2023-01-16 04:23:38,https://www.instagram.com/p/CndmboMBHeB/,2023-01-15_zindoriyam - CndmboMBHeB.json
|
||||
jinsoul,zindoriyam,zindoriyam,Ctb5AnZxCwh,post,2023-06-13 15:35:51,https://www.instagram.com/p/Ctb5AnZxCwh/,2023-06-13_zindoriyam - Ctb5AnZxCwh.json
|
||||
jinsoul,zindoriyam,zindoriyam,Cu6ZRz8hj_T,post,2023-07-20 08:26:26,https://www.instagram.com/p/Cu6ZRz8hj_T/,2023-07-20_zindoriyam - Cu6ZRz8hj_T.json
|
||||
jinsoul,zindoriyam,zindoriyam,Cwrydt6OXmo,post,2023-09-02 09:20:42,https://www.instagram.com/p/Cwrydt6OXmo/,2023-09-02_zindoriyam - Cwrydt6OXmo.json
|
||||
jinsoul,zindoriyam,zindoriyam,Cxsj3_IhwNv,post,2023-09-27 13:03:51,https://www.instagram.com/p/Cxsj3_IhwNv/,2023-09-27_zindoriyam - Cxsj3_IhwNv.json
|
||||
jinsoul,zindoriyam,zindoriyam,Cza0C4gBkb5,post,2023-11-09 08:41:36,https://www.instagram.com/p/Cza0C4gBkb5/,2023-11-09_zindoriyam - Cza0C4gBkb5.json
|
||||
jinsoul,zindoriyam,zindoriyam,C5LqGJUBTIK,post,2024-03-31 14:34:25,https://www.instagram.com/p/C5LqGJUBTIK/,2024-03-31_zindoriyam - C5LqGJUBTIK.json
|
||||
jinsoul,zindoriyam,zindoriyam,C5QZg9BBc12,post,2024-04-02 10:45:44,https://www.instagram.com/p/C5QZg9BBc12/,2024-04-02_zindoriyam - C5QZg9BBc12.json
|
||||
jinsoul,zindoriyam,zindoriyam,C8NAliShdHm,post,2024-06-14 16:44:22,https://www.instagram.com/p/C8NAliShdHm/,2024-06-14_zindoriyam - C8NAliShdHm.json
|
||||
jinsoul,zindoriyam,zindoriyam,C9-Ot__hyGY,post,2024-07-28 16:02:02,https://www.instagram.com/p/C9-Ot__hyGY/,2024-07-28_zindoriyam - C9-Ot__hyGY.json
|
||||
jinsoul,zindoriyam,zindoriyam,DANQU24um-N,post,2024-09-22 05:07:29,https://www.instagram.com/p/DANQU24um-N/,2024-09-22_zindoriyam - DANQU24um-N.json
|
||||
jinsoul,zindoriyam,zindoriyam,DHz88e2sS_J,post,2025-03-30 05:28:16,https://www.instagram.com/p/DHz88e2sS_J/,2025-03-30_zindoriyam - DHz88e2sS_J.json
|
||||
jinsoul,zindoriyam,zindoriyam,DJXAVrANWfp,post,2025-05-07 16:42:44,https://www.instagram.com/p/DJXAVrANWfp/,2025-05-07_zindoriyam - DJXAVrANWfp.json
|
||||
jinsoul,zindoriyam,zindoriyam,DLhdKkZJ2_5,post,2025-06-30 11:09:49,https://www.instagram.com/p/DLhdKkZJ2_5/,2025-06-30_zindoriyam - DLhdKkZJ2_5.json
|
||||
jinsoul,zindoriyam,zindoriyam,DMiZJbaJZXv,post,2025-07-25 16:25:21,https://www.instagram.com/p/DMiZJbaJZXv/,2025-07-25_zindoriyam - DMiZJbaJZXv.json
|
||||
jinsoul,zindoriyam,zindoriyam,DNJCd9jpT-J,post,2025-08-09 16:37:33,https://www.instagram.com/p/DNJCd9jpT-J/,2025-08-09_zindoriyam - DNJCd9jpT-J.json
|
||||
jinsoul,zindoriyam,zindoriyam,DRl8jAxFnwE,post,2025-11-28 08:09:22,https://www.instagram.com/p/DRl8jAxFnwE/,2025-11-28_zindoriyam - DRl8jAxFnwE.json
|
||||
jinsoul,zindoriyam,zindoriyam,DTQR5cDk6kY,post,2026-01-08 15:15:25,https://www.instagram.com/p/DTQR5cDk6kY/,2026-01-08_zindoriyam - DTQR5cDk6kY.json
|
||||
jinsoul,zindoriyam,zindoriyam,DVv1oRSEyIy,post,2026-03-11 14:26:54,https://www.instagram.com/p/DVv1oRSEyIy/,2026-03-11_zindoriyam - DVv1oRSEyIy.json
|
||||
jinsoul,zindoriyam,zindoriyam,DW_6oxSkz8u,post,2026-04-11 16:49:54,https://www.instagram.com/p/DW_6oxSkz8u/,2026-04-11_zindoriyam - DW_6oxSkz8u.json
|
||||
jinsoul,zindoriyam,zindoriyam,DcRLAVJEwlG,post,2026-08-20 16:16:59,https://www.instagram.com/p/DcRLAVJEwlG/,2026-08-20_zindoriyam - DcRLAVJEwlG.json
|
||||
kimlip,kimxxlip,kimxxlip,CndmBanNyrR,post,2023-01-16 04:20:04,https://www.instagram.com/p/CndmBanNyrR/,2023-01-15_kimxxlip - CndmBanNyrR.json
|
||||
kimlip,kimxxlip,kimxxlip,CwXG1L3vQFH,post,2023-08-25 08:34:37,https://www.instagram.com/p/CwXG1L3vQFH/,2023-08-25_kimxxlip - CwXG1L3vQFH.json
|
||||
kimlip,kimxxlip,kimxxlip,CxsL_FQvHna,post,2023-09-27 09:35:06,https://www.instagram.com/p/CxsL_FQvHna/,2023-09-27_kimxxlip - CxsL_FQvHna.json
|
||||
|
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "instaarchive-viewer",
|
||||
"version": "1.8.0",
|
||||
"version": "1.8.1",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "instaarchive-viewer",
|
||||
"version": "1.8.0",
|
||||
"version": "1.8.1",
|
||||
"dependencies": {
|
||||
"@tailwindcss/vite": "^4.1.14",
|
||||
"@vitejs/plugin-react": "^5.0.4",
|
||||
|
||||
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "instaarchive-viewer",
|
||||
"private": true,
|
||||
"version": "1.8.0",
|
||||
"version": "1.8.1",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "vite --port=3000 --host=0.0.0.0",
|
||||
"build": "vite build && npm run build:server",
|
||||
"build:server": "tsc server.ts --esModuleInterop --module ESNext --target ES2022 --moduleResolution bundler --outDir dist-server",
|
||||
"build:server": "tsc server.ts --esModuleInterop --module ESNext --target ES2022 --moduleResolution bundler --removeComments --outDir dist-server",
|
||||
"preview": "vite preview",
|
||||
"server": "tsx server.ts",
|
||||
"clean": "rm -rf dist",
|
||||
|
||||
Executable
+112
@@ -0,0 +1,112 @@
|
||||
#!/bin/bash
|
||||
# Unattended wrapper around gdl-sync.py. One argument: the run mode.
|
||||
#
|
||||
# stories daily ~6 requests; the only surface that cannot be backfilled
|
||||
# profiles monthly every surface, --abort 50: stops enumerating a profile
|
||||
# once it reaches content already held, so it costs
|
||||
# ~40-60 requests and catches everything NEW
|
||||
# full-sweep rarely every surface, no abort: walks each profile to the end
|
||||
# for ~420 requests. The only run that notices posts
|
||||
# EDITED after we archived them, and by far the most
|
||||
# expensive thing here -- see TOOLING.md before running.
|
||||
#
|
||||
# Exits non-zero if the sync does, so cron mails you. Everything is logged.
|
||||
set -eu
|
||||
|
||||
MODE="${1:?usage: gdl-cron.sh stories|profiles|full-sweep}"
|
||||
|
||||
GDL_HOME="${GDL_HOME:-$HOME/gdl}"
|
||||
INDEX="${GDL_INDEX:-https://instaarchive.ergosteur.com}"
|
||||
PUBLISH="${GDL_PUBLISH:-agentapi@10.20.28.200:/volume1/rslsync/sync/Instagram-archive/archives/}"
|
||||
STAGING="$GDL_HOME/staging-$MODE"
|
||||
# $$ in the name because two runs in the same SECOND would otherwise share a
|
||||
# log file, and `tee -a` appends -- which made a test see the previous run.
|
||||
LOG="$GDL_HOME/logs/$MODE-$(date +%Y%m%d-%H%M%S)-$$.log"
|
||||
|
||||
PATH="$HOME/.local/bin:$PATH"; export PATH
|
||||
|
||||
# Pacing. These are the values the 2026-08-22 runs used by hand, after the
|
||||
# scraping warning -- roughly double the caution of gdl-sync.py's own defaults
|
||||
# (6-10s / 3-6s / 1M). An archive sync has no deadline; being slow is free and
|
||||
# being restricted is not. Override per-run with GDL_SLEEP_REQUEST etc. if you
|
||||
# ever need to, but raise them rather than lower them.
|
||||
SLEEP_REQUEST="${GDL_SLEEP_REQUEST:-12 20}"
|
||||
SLEEP="${GDL_SLEEP:-5 10}"
|
||||
RATE="${GDL_RATE:-500K}"
|
||||
|
||||
case "$MODE" in
|
||||
# --min-interval 8, not the 20h default. The floor exists to stop an ABORTED
|
||||
# RESTART re-enumerating profiles -- a minutes-to-hours concern. At 20h the
|
||||
# daily timer silently did nothing whenever a manual run had happened the
|
||||
# previous afternoon, which is exactly what happened on 2026-08-21: it fired,
|
||||
# skipped all six sources and reported success. A stories fetch is one
|
||||
# request per profile, so the worst case 8h permits is roughly twelve
|
||||
# requests in a day instead of six.
|
||||
stories) ARGS="--only stories --min-interval 8" ;;
|
||||
profiles) ARGS="--only posts,reels,stories,highlights --abort 50" ;;
|
||||
# No --abort: the whole point of full-sweep is enumerating to the end, so it
|
||||
# is the only run that notices carousels edited after we archived them.
|
||||
full-sweep) ARGS="--only posts,reels,stories,highlights" ;;
|
||||
# Renamed 2026-08-22: "full" was misleading (it is the abort-LIMITED run) and
|
||||
# "sweep" did not say it was the exhaustive one. Catch the old names rather
|
||||
# than failing with a bare error, in case something still passes them.
|
||||
full) echo "mode 'full' was renamed to 'profiles'" >&2; exit 2 ;;
|
||||
sweep) echo "mode 'sweep' was renamed to 'full-sweep'" >&2; exit 2 ;;
|
||||
*) echo "unknown mode: $MODE (want stories|profiles|full-sweep)" >&2; exit 2 ;;
|
||||
esac
|
||||
|
||||
mkdir -p "$GDL_HOME/logs"
|
||||
|
||||
# Staging is wiped every run ON PURPOSE. What we already hold is decided by the
|
||||
# skip-archive (--download-archive), never by which files happen to be sitting
|
||||
# in staging, so starting empty is correct -- and it keeps the publish rsync
|
||||
# to just the new files instead of re-walking gigabytes each time.
|
||||
rm -rf "$STAGING"
|
||||
|
||||
echo "=== $MODE run $(date -Is) ===" | tee -a "$LOG"
|
||||
|
||||
# The exit status has to survive the pipe into tee. The left-hand side of a
|
||||
# pipeline runs in a SUBSHELL, so an `exit` in there sets the subshell's status
|
||||
# and the script goes on to return tee's, which is always 0. An earlier version
|
||||
# of this file did exactly that and reported success no matter what the sync
|
||||
# did -- which is why the shebang is bash: PIPESTATUS is the fix.
|
||||
# This run's output only. The check below must never see a previous run's
|
||||
# lines, so it reads this rather than the (appended-to) log.
|
||||
RUNOUT=$(mktemp)
|
||||
trap 'rm -f "$RUNOUT"' EXIT
|
||||
|
||||
set +e
|
||||
# shellcheck disable=SC2086
|
||||
"$GDL_HOME/gdl-sync.py" \
|
||||
--index "$INDEX" \
|
||||
--staging "$STAGING" \
|
||||
--publish "$PUBLISH" \
|
||||
--archive-db "$GDL_HOME/artms.db" \
|
||||
--urls-file "$GDL_HOME/artms_account_links.txt" \
|
||||
--sleep-request $SLEEP_REQUEST \
|
||||
--sleep $SLEEP \
|
||||
--rate "$RATE" \
|
||||
$ARGS --execute 2>&1 | tee -a "$LOG" "$RUNOUT"
|
||||
status=${PIPESTATUS[0]}
|
||||
set -e
|
||||
|
||||
# A stories run that skipped every source is NOT a success. It means the
|
||||
# min-interval floor blocked the one surface that cannot be backfilled, and
|
||||
# without this it looks identical to a clean run: exit 0, "0 step(s) failed".
|
||||
# Note this is not the same as "no stories today" -- that shows up as sources
|
||||
# being fetched and returning no results, which is normal and stays quiet.
|
||||
if [ "$MODE" = "stories" ] && grep -q "sources : 0 to sync" "$RUNOUT"; then
|
||||
echo "WARNING: every stories source was skipped by --min-interval." | tee -a "$LOG" >&2
|
||||
echo " Nothing was fetched. Stories expire in 24h and cannot be" | tee -a "$LOG" >&2
|
||||
echo " backfilled, so this is a real loss, not a quiet no-op." | tee -a "$LOG" >&2
|
||||
[ "$status" -eq 0 ] && status=75
|
||||
fi
|
||||
|
||||
echo "=== exit $status at $(date -Is) ===" | tee -a "$LOG"
|
||||
|
||||
# Keep the log directory from growing without bound.
|
||||
ls -1t "$GDL_HOME/logs" | tail -n +30 | while read -r old; do
|
||||
rm -f "$GDL_HOME/logs/$old"
|
||||
done
|
||||
|
||||
exit $status
|
||||
+201
-6
@@ -23,18 +23,78 @@ the one that would touch the archive.
|
||||
|
||||
Run it from the host whose public IP matches the browser the cookie came from;
|
||||
using the cookie from elsewhere is what session-hijack detection looks for.
|
||||
|
||||
See `--help` for every flag, with worked examples for each way of selecting
|
||||
what to fetch.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
EXAMPLES = """\
|
||||
examples:
|
||||
|
||||
Routine incremental sync of the tracked profiles (what gdl-cron.sh runs) --
|
||||
--abort 50 stops enumerating each profile once it reaches content already
|
||||
held, so a run that has seeded once costs ~40-60 requests, not a full walk:
|
||||
|
||||
./scripts/gdl-sync.py --index https://instaarchive.ergosteur.com \\
|
||||
--staging /var/tmp/gdl --publish user@host:/path/to/archives \\
|
||||
--urls-file artms_account_links.txt --archive-db /var/tmp/gdl.db \\
|
||||
--abort 50 --dry-run
|
||||
# ...then swap --dry-run for --execute once the plan looks right.
|
||||
|
||||
Cheapest possible run -- stories only, the one surface that expires in 24h
|
||||
and cannot be backfilled, so it is worth doing often:
|
||||
|
||||
./scripts/gdl-sync.py --index https://instaarchive.ergosteur.com \\
|
||||
--staging /var/tmp/gdl --publish user@host:/path/to/archives \\
|
||||
--urls-file artms_account_links.txt --only stories --execute
|
||||
|
||||
One profile, by name, without a urls-file:
|
||||
|
||||
./scripts/gdl-sync.py --index /path/to/archives --staging /var/tmp/gdl \\
|
||||
--publish /path/to/archives --profile some_account --execute
|
||||
|
||||
Every profile the archive already knows about (no urls-file, no --profile):
|
||||
|
||||
./scripts/gdl-sync.py --index /path/to/archives --staging /var/tmp/gdl \\
|
||||
--publish /path/to/archives --all --execute
|
||||
|
||||
An arbitrary single post or reel from an account NOT otherwise tracked --
|
||||
e.g. a link someone shared. Filed under its owner like any other post; no
|
||||
--index needed, since there is no profile list to plan against:
|
||||
|
||||
./scripts/gdl-sync.py --staging /var/tmp/gdl --publish user@host:/path \\
|
||||
--post-url https://www.instagram.com/p/SHORTCODE/ --execute
|
||||
|
||||
Full sweep -- no --abort, walks every profile to the end. The only run that
|
||||
notices a carousel edited after it was archived, and by far the most
|
||||
expensive thing here (~420 requests for six profiles). Read docs/gallery-dl.md
|
||||
and TOOLING.md before running this one:
|
||||
|
||||
./scripts/gdl-sync.py --index https://instaarchive.ergosteur.com \\
|
||||
--staging /var/tmp/gdl --publish user@host:/path/to/archives \\
|
||||
--urls-file artms_account_links.txt --execute
|
||||
|
||||
Hand-paced caution after a scraping warning (roughly double the defaults;
|
||||
see docs/gallery-dl.md for where these numbers come from):
|
||||
|
||||
./scripts/gdl-sync.py ... --sleep-request 12 20 --sleep 5 10 --rate 500K
|
||||
|
||||
Always --dry-run first (the default): it prints the plan and the rsync
|
||||
command that would publish, without spending a single Instagram request.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import datetime as dt
|
||||
import json
|
||||
import os
|
||||
import random
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
@@ -161,6 +221,23 @@ def read_urls_file(path: Path) -> list[str]:
|
||||
return users
|
||||
|
||||
|
||||
def read_post_urls_file(path: Path) -> list[str]:
|
||||
"""
|
||||
Read individual post/reel URLs from a file, one per line -- the output
|
||||
format `reels-scrape.py` writes. Blank lines and `#` comments are skipped;
|
||||
order is kept and duplicates dropped, same conventions as read_urls_file.
|
||||
"""
|
||||
urls: list[str] = []
|
||||
seen: set[str] = set()
|
||||
for raw in path.read_text().splitlines():
|
||||
line = raw.split("#", 1)[0].strip()
|
||||
if not line or line in seen:
|
||||
continue
|
||||
seen.add(line)
|
||||
urls.append(line)
|
||||
return urls
|
||||
|
||||
|
||||
class ArchiveIndex:
|
||||
"""
|
||||
What the archive already holds, as filenames only.
|
||||
@@ -248,6 +325,32 @@ def build_config(rate: str, sleep_request: list[float],
|
||||
"post_shortcode", "post_id", "type", "date", "post_date",
|
||||
"username", "fullname", "owner_id", "description", "count",
|
||||
"likes", "post_url", "sidecar_shortcode",
|
||||
# A Collab's `username`/`fullname`/`owner_id` above are always the
|
||||
# original poster's, never the scraped account's -- see
|
||||
# docs/gallery-dl.md. `coauthors` is the API's own list of every
|
||||
# OTHER collaborator (it excludes the post's owner), a direct
|
||||
# signal instead of inferring a collab from identity mismatches.
|
||||
"coauthors",
|
||||
],
|
||||
}
|
||||
# Per-carousel-item fields (`width`/`height`/`tagged_users`/`owner`) live
|
||||
# on the per-FILE kwdict, not the per-post one `meta_pp` reads -- a
|
||||
# carousel's items can each have different dimensions and tags, which one
|
||||
# post-level JSON can't represent. `owner` is deliberately left out: it is
|
||||
# a full user object (profile pic URLs, privacy flags) for whoever posted
|
||||
# that specific item, the same reason `audio_user` is excluded above.
|
||||
media_pp = {
|
||||
"name": "metadata",
|
||||
"mode": "json",
|
||||
# event defaults to "file" when omitted -- runs once per downloaded
|
||||
# file, appending ".json" to that file's own full name, e.g.
|
||||
# "... - 01.jpg" gets "... - 01.jpg.json" alongside it. Never
|
||||
# collides with the post-level "....json" above, which has no
|
||||
# per-item number.
|
||||
"include": [
|
||||
"media_id", "shortcode", "num",
|
||||
"width", "height", "width_original", "height_original",
|
||||
"tagged_users",
|
||||
],
|
||||
}
|
||||
|
||||
@@ -286,6 +389,7 @@ def build_config(rate: str, sleep_request: list[float],
|
||||
"postprocessors": [
|
||||
{**caption_pp, "filename": stem + ".txt"},
|
||||
{**meta_pp, "filename": stem + ".json"},
|
||||
media_pp,
|
||||
],
|
||||
}
|
||||
|
||||
@@ -331,6 +435,12 @@ def build_config(rate: str, sleep_request: list[float],
|
||||
"directory": [],
|
||||
"posts": post_like(POST_STEM),
|
||||
"reels": post_like(POST_STEM),
|
||||
# Ad hoc single-post fetches (--post-url) go through gallery-dl's
|
||||
# own post/reel extractor instead of a profile listing, so the
|
||||
# owning account is never known ahead of time -- only mid-
|
||||
# extraction, same reasoning as highlights below.
|
||||
"post": {**post_like(POST_STEM), "directory": ["{username}"]},
|
||||
"reel": {**post_like(POST_STEM), "directory": ["{username}"]},
|
||||
"stories": item_like(ITEM_STEM),
|
||||
"highlights": {
|
||||
**item_like(ITEM_STEM),
|
||||
@@ -363,9 +473,11 @@ def gdl_command(src: Source, staging: Path, config: Path, cookies: str,
|
||||
cmd += ["--download-archive", str(archive_db)]
|
||||
# Forced destination -- never `{username}` -- because a reels tab returns
|
||||
# collab reels owned by other accounts, which would otherwise be filed
|
||||
# under the wrong profile. Highlights are the exception: their directory
|
||||
# embeds a title only known mid-extraction, so the config formats it.
|
||||
dest = staging if src.subcategory == "highlights" else staging / src.directory
|
||||
# under the wrong profile. Highlights and ad hoc single-post fetches are
|
||||
# the exception: their directory is only known mid-extraction (a title, or
|
||||
# the post's owner), so the config formats it instead.
|
||||
dest = (staging if src.subcategory in ("highlights", "post", "reel")
|
||||
else staging / src.directory)
|
||||
cmd += ["--destination", str(dest)]
|
||||
cmd.append(src.url)
|
||||
return cmd
|
||||
@@ -639,6 +751,61 @@ def publish(staging: Path, dest: str, dry_run: bool) -> int:
|
||||
return subprocess.run(cmd).returncode
|
||||
|
||||
|
||||
def run_post_urls(args) -> int:
|
||||
"""
|
||||
Fetch one or more individual posts/reels by URL -- an ad hoc pull outside
|
||||
the tracked profile list, e.g. a link shared from some other account. Each
|
||||
is a single request, not a recurring surface, so there is no archive-db
|
||||
seeding and no --min-interval floor to plan around.
|
||||
"""
|
||||
config = build_config(args.rate, list(args.sleep_request),
|
||||
list(args.sleep), abort=0)
|
||||
args.staging.mkdir(parents=True, exist_ok=True)
|
||||
config_path = args.staging.parent / f"{args.staging.name}.gdl-config.json"
|
||||
|
||||
sources = [Source("post", url, "", "post") for url in args.post_url]
|
||||
|
||||
print(f"post-url : {len(sources)} to fetch")
|
||||
print(f"pacing : {args.sleep_request[0]}-{args.sleep_request[1]}s between "
|
||||
f"requests, rate cap {args.rate}")
|
||||
print(f"staging : {args.staging}")
|
||||
print(f"publish : {args.publish}")
|
||||
print()
|
||||
|
||||
if not args.execute:
|
||||
for src in sources:
|
||||
print(f" {src.url}")
|
||||
print()
|
||||
print(" " + " ".join(rsync_command(args.staging, args.publish, True)))
|
||||
print("\ndry run; nothing fetched. pass --execute to run.")
|
||||
return 0
|
||||
|
||||
config_path.write_text(json.dumps(config, indent=2))
|
||||
failures = 0
|
||||
for i, src in enumerate(sources):
|
||||
# gallery-dl's own sleep-request only paces requests INSIDE one
|
||||
# invocation; each URL here is its own subprocess, so back-to-back
|
||||
# items would otherwise fire with no gap at all -- the same pacing
|
||||
# applied here as between requests within a single fetch.
|
||||
if i:
|
||||
pause = random.uniform(*args.sleep_request)
|
||||
time.sleep(pause)
|
||||
print(f"==> {src.url}")
|
||||
cmd = gdl_command(src, args.staging, config_path, args.cookies,
|
||||
args.archive_db)
|
||||
result = subprocess.run(cmd)
|
||||
if result.returncode != 0:
|
||||
failures += 1
|
||||
print(f" FAILED (exit {result.returncode})", file=sys.stderr)
|
||||
|
||||
print("\n==> publish")
|
||||
if publish(args.staging, args.publish, dry_run=False) != 0:
|
||||
failures += 1
|
||||
|
||||
print(f"\ndone; {failures} step(s) failed")
|
||||
return 1 if failures else 0
|
||||
|
||||
|
||||
def main() -> int:
|
||||
# A sync runs for hours and is normally watched through a redirected log,
|
||||
# where Python's block buffering would withhold progress until it happened
|
||||
@@ -647,12 +814,13 @@ def main() -> int:
|
||||
sys.stdout.reconfigure(line_buffering=True)
|
||||
sys.stderr.reconfigure(line_buffering=True)
|
||||
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
ap = argparse.ArgumentParser(description=__doc__, epilog=EXAMPLES,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--index", required=True,
|
||||
ap.add_argument("--index",
|
||||
help="existing archive listing: a local root, or the "
|
||||
"viewer's base URL (only a FILE LISTING is needed, "
|
||||
"never the contents)")
|
||||
"never the contents). Required unless --post-url/"
|
||||
"--post-urls-file")
|
||||
ap.add_argument("--publish", required=True,
|
||||
help="rsync destination for fetched files; a local path or "
|
||||
"user@host:/path")
|
||||
@@ -665,6 +833,17 @@ def main() -> int:
|
||||
g.add_argument("--urls-file", type=Path,
|
||||
help="file of Instagram profile URLs or usernames, one per "
|
||||
"line; # comments and blank lines allowed")
|
||||
g.add_argument("--post-url", action="append", default=[],
|
||||
help="fetch one post or reel by URL (e.g. "
|
||||
"https://www.instagram.com/p/SHORTCODE/), filed under "
|
||||
"its owner's account like any other post; repeatable. "
|
||||
"A one-off fetch outside the tracked profile list: no "
|
||||
"archive-db seeding, no --min-interval floor")
|
||||
g.add_argument("--post-urls-file", type=Path,
|
||||
help="file of post/reel URLs, one per line; # comments and "
|
||||
"blank lines allowed. Same handling as --post-url, "
|
||||
"paced with --sleep-request between items. This is "
|
||||
"the format reels-scrape.py writes")
|
||||
ap.add_argument("--cookies", default="chrome:/home/matt/.config/google-chrome-devtools",
|
||||
help="gallery-dl --cookies-from-browser value")
|
||||
ap.add_argument("--archive-db", type=Path, default=None,
|
||||
@@ -709,6 +888,22 @@ def main() -> int:
|
||||
print("rsync not on PATH", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
if args.post_urls_file:
|
||||
if not args.post_urls_file.is_file():
|
||||
print(f"post-urls file not found: {args.post_urls_file}", file=sys.stderr)
|
||||
return 2
|
||||
args.post_url = read_post_urls_file(args.post_urls_file)
|
||||
if not args.post_url:
|
||||
print(f"no usable URLs in {args.post_urls_file}", file=sys.stderr)
|
||||
return 2
|
||||
if args.post_url:
|
||||
return run_post_urls(args)
|
||||
|
||||
if not args.index:
|
||||
print("--index is required unless --post-url/--post-urls-file is given",
|
||||
file=sys.stderr)
|
||||
return 2
|
||||
|
||||
index = ArchiveIndex(args.index)
|
||||
names = index.profiles()
|
||||
if args.urls_file:
|
||||
|
||||
@@ -0,0 +1,237 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Find a profile's reels by scrolling the real page, not calling the API.
|
||||
|
||||
gallery-dl's dedicated reels extractor POSTs to /api/v1/clips/user/, which
|
||||
Instagram now 302-redirects to the home page for this account -- confirmed on
|
||||
2026-08-26/27 across multiple profiles, hours apart, with a freshly-warmed
|
||||
session and a correct X-IG-WWW-Claim header. The reels tab itself loads fine
|
||||
in a real browser, so this drives the SAME logged-in Chrome (already exposed
|
||||
on loopback CDP for MCP automation -- see TOOLING.md) via the DevTools
|
||||
protocol, scrolls it like a person would, and scrapes `/reel/<code>/` links
|
||||
out of the rendered page instead.
|
||||
|
||||
This only finds shortcodes; it never downloads anything itself. What comes
|
||||
out the other end is deduped against the archive (via the same --index used
|
||||
elsewhere) and printed as plain post URLs -- feed them to gdl-sync.py:
|
||||
|
||||
./scripts/reels-scrape.py --profile someuser \\
|
||||
--index https://instaarchive.ergosteur.com > /tmp/someuser-reels.txt
|
||||
|
||||
./scripts/gdl-sync.py --publish user@host:/path --staging /var/tmp/gdl \\
|
||||
--post-urls-file /tmp/someuser-reels.txt \\
|
||||
--sleep-request 12 20 --sleep 5 10 --rate 500K --execute
|
||||
|
||||
Requires the `websocket-client` package (only imported inside scrape_reels,
|
||||
so everything else here stays importable -- and testable -- without it).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import itertools
|
||||
import json
|
||||
import random
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
from urllib.request import urlopen
|
||||
|
||||
RE_REEL_HREF = re.compile(r"/reel/([^/?#]+)")
|
||||
|
||||
# The reels tab has finished loading once a scroll round finds no NEW
|
||||
# shortcodes this many times in a row -- lazy-loaded feeds sometimes stall
|
||||
# for a round or two before producing more, so one dry round is not enough.
|
||||
DEFAULT_MAX_IDLE_ROUNDS = 3
|
||||
DEFAULT_MAX_SCROLLS = 200
|
||||
DEFAULT_SCROLL_PAUSE = (2.0, 3.5)
|
||||
|
||||
SCRAPE_JS = (
|
||||
"(() => {"
|
||||
"window.scrollTo(0, document.body.scrollHeight);"
|
||||
"return Array.from(document.querySelectorAll('a[href*=\"/reel/\"]'))"
|
||||
".map(a => a.getAttribute('href'));"
|
||||
"})()"
|
||||
)
|
||||
|
||||
|
||||
def extract_shortcodes(hrefs: list[str]) -> list[str]:
|
||||
codes = []
|
||||
for href in hrefs:
|
||||
if m := RE_REEL_HREF.search(href):
|
||||
codes.append(m.group(1))
|
||||
return codes
|
||||
|
||||
|
||||
class CDPError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
class CDP:
|
||||
"""
|
||||
A deliberately minimal synchronous DevTools Protocol client: one request
|
||||
in flight at a time, which is all a linear scroll-and-scrape loop needs.
|
||||
Anything fancier (concurrent requests, event subscriptions) is scope this
|
||||
script has no reason to carry.
|
||||
"""
|
||||
|
||||
def __init__(self, ws_url: str, timeout: float = 30.0):
|
||||
import websocket # local: keep this importable without the package
|
||||
self.ws = websocket.create_connection(ws_url, timeout=timeout)
|
||||
self._ids = itertools.count(1)
|
||||
|
||||
def send(self, method: str, params: dict | None = None,
|
||||
session_id: str | None = None) -> dict:
|
||||
msg_id = next(self._ids)
|
||||
payload = {"id": msg_id, "method": method, "params": params or {}}
|
||||
if session_id:
|
||||
payload["sessionId"] = session_id
|
||||
self.ws.send(json.dumps(payload))
|
||||
while True:
|
||||
msg = json.loads(self.ws.recv())
|
||||
if msg.get("id") != msg_id:
|
||||
continue # an event notification, not our reply -- ignore
|
||||
if "error" in msg:
|
||||
raise CDPError(f"{method}: {msg['error']}")
|
||||
return msg.get("result", {})
|
||||
|
||||
def close(self) -> None:
|
||||
self.ws.close()
|
||||
|
||||
|
||||
def scrape_reels(user: str, cdp_port: int = 9222,
|
||||
scroll_pause: tuple[float, float] = DEFAULT_SCROLL_PAUSE,
|
||||
max_idle_rounds: int = DEFAULT_MAX_IDLE_ROUNDS,
|
||||
max_scrolls: int = DEFAULT_MAX_SCROLLS,
|
||||
log=lambda msg: None) -> list[str]:
|
||||
"""
|
||||
Open the profile's reels tab in a NEW tab of the already-signed-in Chrome,
|
||||
scroll it to the bottom repeatedly, and collect every unique `/reel/`
|
||||
shortcode that appears. Closes the tab when done either way.
|
||||
"""
|
||||
version = json.loads(urlopen(f"http://localhost:{cdp_port}/json/version",
|
||||
timeout=10).read())
|
||||
browser = CDP(version["webSocketDebuggerUrl"])
|
||||
target_id = None
|
||||
try:
|
||||
target = browser.send("Target.createTarget", {
|
||||
"url": f"https://www.instagram.com/{user}/reels/"})
|
||||
target_id = target["targetId"]
|
||||
attach = browser.send("Target.attachToTarget", {
|
||||
"targetId": target_id, "flatten": True})
|
||||
session_id = attach["sessionId"]
|
||||
browser.send("Page.enable", session_id=session_id)
|
||||
browser.send("Runtime.enable", session_id=session_id)
|
||||
|
||||
time.sleep(4.0) # initial page load, before the first scroll
|
||||
|
||||
seen: set[str] = set()
|
||||
idle = 0
|
||||
for i in range(max_scrolls):
|
||||
result = browser.send(
|
||||
"Runtime.evaluate",
|
||||
{"expression": SCRAPE_JS, "returnByValue": True},
|
||||
session_id=session_id)
|
||||
hrefs = result.get("result", {}).get("value") or []
|
||||
codes = extract_shortcodes(hrefs)
|
||||
new = [c for c in codes if c not in seen]
|
||||
seen.update(new)
|
||||
log(f" scroll {i + 1}: {len(seen)} unique reels so far (+{len(new)})")
|
||||
|
||||
if new:
|
||||
idle = 0
|
||||
else:
|
||||
idle += 1
|
||||
if idle >= max_idle_rounds:
|
||||
break
|
||||
|
||||
time.sleep(random.uniform(*scroll_pause))
|
||||
|
||||
return sorted(seen)
|
||||
finally:
|
||||
if target_id:
|
||||
try:
|
||||
browser.send("Target.closeTarget", {"targetId": target_id})
|
||||
except CDPError:
|
||||
pass # best-effort cleanup; a leftover tab is harmless
|
||||
browser.close()
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--profile", required=True,
|
||||
help="Instagram username to scrape reels for")
|
||||
ap.add_argument("--index", required=True,
|
||||
help="existing archive listing, same as gdl-sync.py's "
|
||||
"--index: a local root, or the viewer's base URL. "
|
||||
"Used only to dedupe -- already-archived shortcodes "
|
||||
"are dropped before anything is printed")
|
||||
ap.add_argument("--cdp-port", type=int, default=9222,
|
||||
help="Chrome's loopback DevTools port (see TOOLING.md)")
|
||||
ap.add_argument("--scroll-pause", nargs=2, type=float,
|
||||
default=list(DEFAULT_SCROLL_PAUSE), metavar=("MIN", "MAX"),
|
||||
help="random pause between scrolls, seconds")
|
||||
ap.add_argument("--max-idle-rounds", type=int, default=DEFAULT_MAX_IDLE_ROUNDS,
|
||||
help="stop after this many consecutive scrolls with no "
|
||||
"new reels")
|
||||
ap.add_argument("--max-scrolls", type=int, default=DEFAULT_MAX_SCROLLS,
|
||||
help="hard ceiling on scroll rounds, in case a page never "
|
||||
"goes idle")
|
||||
ap.add_argument("--out", type=Path,
|
||||
help="write new reel URLs here, one per line (default: "
|
||||
"stdout)")
|
||||
args = ap.parse_args()
|
||||
|
||||
import importlib.util
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
"gdl_sync", Path(__file__).with_name("gdl-sync.py"))
|
||||
gdl = importlib.util.module_from_spec(spec)
|
||||
sys.modules["gdl_sync"] = gdl
|
||||
spec.loader.exec_module(gdl)
|
||||
|
||||
index = gdl.ArchiveIndex(args.index)
|
||||
# Deduped against the WHOLE archive, not just this profile's own
|
||||
# directories: a shortcode is globally unique, and a reels tab commonly
|
||||
# surfaces reposts/collabs by OTHER tracked accounts. Missing that let a
|
||||
# 2026-08-27 run re-fetch 9 reels already held under their true owner's
|
||||
# directory -- gallery-dl filed them correctly there (by the post's real
|
||||
# `username`, not the scraped profile), so nothing was lost, but it spent
|
||||
# 9 avoidable Instagram requests to find that out. All of this is local
|
||||
# requests to our own viewer, never to instagram.com, so checking every
|
||||
# profile costs nothing on the budget that actually matters.
|
||||
profiles = index.profiles()
|
||||
have: set[str] = set()
|
||||
for profile in profiles:
|
||||
have.update(code for code, _ in gdl.index_existing(index.listing(profile)))
|
||||
print(f"archive already holds {len(have)} shortcode(s) across "
|
||||
f"{len(profiles)} profile(s)", file=sys.stderr)
|
||||
|
||||
print(f"scraping https://www.instagram.com/{args.profile}/reels/ ...",
|
||||
file=sys.stderr)
|
||||
codes = scrape_reels(args.profile, cdp_port=args.cdp_port,
|
||||
scroll_pause=tuple(args.scroll_pause),
|
||||
max_idle_rounds=args.max_idle_rounds,
|
||||
max_scrolls=args.max_scrolls,
|
||||
log=lambda msg: print(msg, file=sys.stderr))
|
||||
|
||||
new_codes = [c for c in codes if c not in have]
|
||||
print(f"found {len(codes)} reel(s) on the page, {len(codes) - len(new_codes)} "
|
||||
f"already archived, {len(new_codes)} new", file=sys.stderr)
|
||||
|
||||
urls = [f"https://www.instagram.com/{args.profile}/reel/{c}/"
|
||||
for c in new_codes]
|
||||
text = "\n".join(urls)
|
||||
if args.out:
|
||||
args.out.write_text(text + ("\n" if text else ""))
|
||||
print(f"wrote {len(urls)} URL(s) to {args.out}", file=sys.stderr)
|
||||
else:
|
||||
if text:
|
||||
print(text)
|
||||
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Executable
+114
@@ -0,0 +1,114 @@
|
||||
#!/bin/bash
|
||||
# The full reels pipeline for one profile: scrape by scrolling the real page,
|
||||
# then fetch and publish whatever's new. See TOOLING.md ("Reels: the API is
|
||||
# blocked, scrape by scrolling instead") for why this exists at all -- the
|
||||
# dedicated reels API is blocked, this drives the actual signed-in browser
|
||||
# session instead, and is slower by design.
|
||||
#
|
||||
# Usage: ./reels-sync.sh <username-or-profile-url>
|
||||
# ./reels-sync.sh zindoriyam
|
||||
# ./reels-sync.sh https://www.instagram.com/zindoriyam/
|
||||
#
|
||||
# Deliberately NOT wired into gdl-cron.sh or the timers -- see TOOLING.md.
|
||||
# Exits non-zero if either step does. Everything is logged.
|
||||
set -eu
|
||||
|
||||
RAW="${1:?usage: reels-sync.sh <username-or-profile-url>}"
|
||||
|
||||
GDL_HOME="${GDL_HOME:-$HOME/gdl}"
|
||||
GDL_PYTHON="${GDL_PYTHON:-$HOME/.local/share/pipx/venvs/gallery-dl/bin/python3}"
|
||||
INDEX="${GDL_INDEX:-https://instaarchive.ergosteur.com}"
|
||||
PUBLISH="${GDL_PUBLISH:-agentapi@10.20.28.200:/volume1/rslsync/sync/Instagram-archive/archives/}"
|
||||
|
||||
# Same hand-paced pacing gdl-cron.sh uses -- see its comment for why. Override
|
||||
# per-run with GDL_SLEEP_REQUEST etc. if you ever need to, but raise them
|
||||
# rather than lower them.
|
||||
SLEEP_REQUEST="${GDL_SLEEP_REQUEST:-12 20}"
|
||||
SLEEP="${GDL_SLEEP:-5 10}"
|
||||
RATE="${GDL_RATE:-500K}"
|
||||
SCROLL_PAUSE="${GDL_SCROLL_PAUSE:-2.0 3.5}"
|
||||
MAX_IDLE_ROUNDS="${GDL_MAX_IDLE_ROUNDS:-3}"
|
||||
|
||||
PATH="$HOME/.local/bin:$PATH"; export PATH
|
||||
|
||||
if [ ! -x "$GDL_PYTHON" ]; then
|
||||
echo "reels-scrape.py needs gallery-dl's own pipx venv python (websocket-client" >&2
|
||||
echo "was injected there, not into the system python): $GDL_PYTHON not found" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# Same username-from-URL parsing gdl-sync.py already does for --urls-file,
|
||||
# reused rather than re-implemented so the two never drift apart.
|
||||
PROFILE=$("$GDL_PYTHON" -c "
|
||||
import re, sys, importlib.util
|
||||
from pathlib import Path
|
||||
spec = importlib.util.spec_from_file_location('gdl_sync', Path('$GDL_HOME/gdl-sync.py'))
|
||||
gdl = importlib.util.module_from_spec(spec)
|
||||
sys.modules['gdl_sync'] = gdl
|
||||
spec.loader.exec_module(gdl)
|
||||
raw = '$RAW'
|
||||
m = gdl.RE_PROFILE_URL.match(raw)
|
||||
user = m.group('user') if m else raw.strip('/')
|
||||
if not user or '/' in user or ' ' in user:
|
||||
print(f'cannot read a username from {raw!r}', file=sys.stderr)
|
||||
sys.exit(1)
|
||||
print(user)
|
||||
")
|
||||
|
||||
mkdir -p "$GDL_HOME/logs"
|
||||
LOG="$GDL_HOME/logs/reels-$PROFILE-$(date +%Y%m%d-%H%M%S)-$$.log"
|
||||
URLS_FILE="$GDL_HOME/$PROFILE-reels.txt"
|
||||
STAGING="$GDL_HOME/staging-reels-$PROFILE"
|
||||
|
||||
echo "=== reels-sync $PROFILE $(date -Is) ===" | tee -a "$LOG"
|
||||
|
||||
# The exit status has to survive the pipe into tee -- see gdl-cron.sh's
|
||||
# comment on PIPESTATUS for why this needs to be bash, not sh.
|
||||
set +e
|
||||
"$GDL_PYTHON" "$GDL_HOME/reels-scrape.py" \
|
||||
--profile "$PROFILE" \
|
||||
--index "$INDEX" \
|
||||
--scroll-pause $SCROLL_PAUSE \
|
||||
--max-idle-rounds "$MAX_IDLE_ROUNDS" \
|
||||
--out "$URLS_FILE" 2>&1 | tee -a "$LOG"
|
||||
scrape_status=${PIPESTATUS[0]}
|
||||
set -e
|
||||
|
||||
if [ "$scrape_status" -ne 0 ]; then
|
||||
echo "=== exit $scrape_status (scrape failed) at $(date -Is) ===" | tee -a "$LOG"
|
||||
exit "$scrape_status"
|
||||
fi
|
||||
|
||||
if [ ! -s "$URLS_FILE" ]; then
|
||||
echo "no new reels for $PROFILE; nothing to fetch" | tee -a "$LOG"
|
||||
echo "=== exit 0 at $(date -Is) ===" | tee -a "$LOG"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Staging is wiped every run on purpose -- same reasoning as gdl-cron.sh: what
|
||||
# we already hold is decided by the archive dedupe in reels-scrape.py, not by
|
||||
# what happens to be sitting in staging.
|
||||
rm -rf "$STAGING"
|
||||
|
||||
set +e
|
||||
# shellcheck disable=SC2086
|
||||
"$GDL_HOME/gdl-sync.py" \
|
||||
--publish "$PUBLISH" \
|
||||
--staging "$STAGING" \
|
||||
--post-urls-file "$URLS_FILE" \
|
||||
--sleep-request $SLEEP_REQUEST \
|
||||
--sleep $SLEEP \
|
||||
--rate "$RATE" \
|
||||
--execute 2>&1 | tee -a "$LOG"
|
||||
status=${PIPESTATUS[0]}
|
||||
set -e
|
||||
|
||||
echo "=== exit $status at $(date -Is) ===" | tee -a "$LOG"
|
||||
|
||||
# Keep the log directory from growing without bound -- scoped to this
|
||||
# script's own logs so it never touches gdl-cron.sh's rotation.
|
||||
ls -1t "$GDL_HOME/logs" | grep '^reels-' | tail -n +30 | while read -r old; do
|
||||
rm -f "$GDL_HOME/logs/$old"
|
||||
done
|
||||
|
||||
exit "$status"
|
||||
@@ -0,0 +1,16 @@
|
||||
[Unit]
|
||||
# One templated service for all three modes; the instance name (%i) is the
|
||||
# mode: stories, full or sweep.
|
||||
Description=Instagram archive sync (%i)
|
||||
Documentation=file:%h/gdl/TOOLING.md
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=%h/gdl/gdl-cron.sh %i
|
||||
# A sync has no deadline and the pacing is deliberately slow; a full sweep can
|
||||
# run for hours. Never let systemd kill one midway -- a half-published run is
|
||||
# the one state the publish step is designed to avoid.
|
||||
TimeoutStartSec=infinity
|
||||
Nice=10
|
||||
@@ -0,0 +1,18 @@
|
||||
[Unit]
|
||||
Description=Full Instagram archive sweep, no abort (~420 requests)
|
||||
# The only run that enumerates every profile to the end, and so the only one
|
||||
# that notices carousels edited after we archived them (test case 15).
|
||||
#
|
||||
# It is also by far the most expensive thing here: ~420 requests to
|
||||
# instagram.com, the same order as the run that preceded the 2026-08-21
|
||||
# scraping warning, spent to catch a handful of retroactively edited posts.
|
||||
# Consider running it by hand when you mean to, rather than on a timer.
|
||||
|
||||
[Timer]
|
||||
# Month names are not valid in OnCalendar's date field -- numeric only.
|
||||
OnCalendar=*-01,04,07,10-07 04:00:00
|
||||
RandomizedDelaySec=45m
|
||||
Persistent=true
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
@@ -0,0 +1,10 @@
|
||||
[Unit]
|
||||
Description=Monthly Instagram profile sync (all surfaces, --abort 50)
|
||||
|
||||
[Timer]
|
||||
OnCalendar=*-*-03 04:00:00
|
||||
RandomizedDelaySec=45m
|
||||
Persistent=true
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
@@ -0,0 +1,15 @@
|
||||
[Unit]
|
||||
Description=Daily Instagram stories sync
|
||||
# Stories expire in 24h and cannot be backfilled. This is the only timer whose
|
||||
# missed run means content is gone for good, which is what Persistent= is for.
|
||||
|
||||
[Timer]
|
||||
OnCalendar=*-*-* 09:00:00
|
||||
# Not a fixed time: a job firing at exactly 09:00 every day is obviously a
|
||||
# machine, and the whole safety model is about not looking like one.
|
||||
RandomizedDelaySec=45m
|
||||
# Catch up after the host was asleep or off. cron would silently skip.
|
||||
Persistent=true
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
@@ -209,6 +209,60 @@ class Publishing(unittest.TestCase):
|
||||
self.assertIn("--dry-run", cmd)
|
||||
|
||||
|
||||
class PostUrl(unittest.TestCase):
|
||||
"""--post-url: a one-off fetch outside the tracked profile list, whose
|
||||
owning account is only known mid-extraction -- same reasoning as
|
||||
highlights, so it must be exempted from the same forced-destination rule."""
|
||||
|
||||
def test_config_keys_a_username_directory(self):
|
||||
config = gdl.build_config("1M", [6.0, 10.0], [3.0, 6.0])
|
||||
for kind in ("post", "reel"):
|
||||
self.assertEqual(
|
||||
config["extractor"]["instagram"][kind]["directory"],
|
||||
["{username}"])
|
||||
|
||||
def test_destination_is_not_forced_like_posts_and_reels(self):
|
||||
staging = Path("/stage")
|
||||
for subcategory in ("post", "reel"):
|
||||
src = gdl.Source(subcategory, "https://www.instagram.com/p/ABC/",
|
||||
"", subcategory)
|
||||
cmd = gdl.gdl_command(src, staging, Path("/cfg.json"), "chrome:x",
|
||||
None)
|
||||
self.assertIn(str(staging), cmd)
|
||||
self.assertNotIn(str(staging / subcategory), cmd)
|
||||
|
||||
|
||||
class MetadataFields(unittest.TestCase):
|
||||
"""Extra fields captured from the raw API response, verified against two
|
||||
saved real examples on 2026-09-01 (see docs/gallery-dl.md)."""
|
||||
|
||||
def test_post_level_json_captures_coauthors(self):
|
||||
config = gdl.build_config("1M", [6.0, 10.0], [3.0, 6.0])
|
||||
pps = config["extractor"]["instagram"]["posts"]["postprocessors"]
|
||||
post_json = next(pp for pp in pps if pp.get("filename", "").endswith(".json"))
|
||||
self.assertIn("coauthors", post_json["include"])
|
||||
|
||||
def test_per_item_dimensions_and_tags_get_their_own_sidecar(self):
|
||||
config = gdl.build_config("1M", [6.0, 10.0], [3.0, 6.0])
|
||||
pps = config["extractor"]["instagram"]["posts"]["postprocessors"]
|
||||
# No "filename" of its own -- unlike the other two, which are keyed
|
||||
# by POST_STEM -- because it must vary per carousel item, not per post.
|
||||
media_pp = next(pp for pp in pps if "filename" not in pp)
|
||||
for field in ("width", "height", "width_original", "height_original",
|
||||
"tagged_users", "shortcode", "num"):
|
||||
self.assertIn(field, media_pp["include"])
|
||||
# `owner` is a full user object (profile pic URLs, privacy flags) for
|
||||
# whoever posted that item -- deliberately excluded, same reasoning
|
||||
# as `audio_user` on the post-level json.
|
||||
self.assertNotIn("owner", media_pp["include"])
|
||||
|
||||
def test_reels_get_the_same_capture_as_posts(self):
|
||||
config = gdl.build_config("1M", [6.0, 10.0], [3.0, 6.0])
|
||||
posts_pps = config["extractor"]["instagram"]["posts"]["postprocessors"]
|
||||
reels_pps = config["extractor"]["instagram"]["reels"]["postprocessors"]
|
||||
self.assertEqual(posts_pps, reels_pps)
|
||||
|
||||
|
||||
class UrlsFile(unittest.TestCase):
|
||||
def test_reads_every_form_a_person_might_paste(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
@@ -227,5 +281,23 @@ class UrlsFile(unittest.TestCase):
|
||||
self.assertEqual(gdl.read_urls_file(p), ["a", "b", "c", "d", "e"])
|
||||
|
||||
|
||||
class PostUrlsFile(unittest.TestCase):
|
||||
"""The format reels-scrape.py writes: whole URLs, not usernames."""
|
||||
|
||||
def test_skips_comments_blanks_and_duplicates(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
p = Path(d) / "reels.txt"
|
||||
p.write_text(
|
||||
"# scraped 2026-08-27\n"
|
||||
"https://www.instagram.com/u/reel/AAA/\n"
|
||||
"\n"
|
||||
"https://www.instagram.com/u/reel/BBB/\n"
|
||||
"https://www.instagram.com/u/reel/AAA/\n") # duplicate
|
||||
self.assertEqual(gdl.read_post_urls_file(p), [
|
||||
"https://www.instagram.com/u/reel/AAA/",
|
||||
"https://www.instagram.com/u/reel/BBB/",
|
||||
])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main(verbosity=2)
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Tests for the pure parts of reels-scrape.py -- everything except the actual
|
||||
CDP session, which needs a live signed-in Chrome and is exercised by hand.
|
||||
|
||||
python3 -m unittest discover -s scripts -p 'test_*.py'
|
||||
"""
|
||||
|
||||
import importlib.util
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
_spec = importlib.util.spec_from_file_location(
|
||||
"reels_scrape", Path(__file__).with_name("reels-scrape.py"))
|
||||
scrape = importlib.util.module_from_spec(_spec)
|
||||
sys.modules["reels_scrape"] = scrape
|
||||
_spec.loader.exec_module(scrape)
|
||||
|
||||
|
||||
class ExtractShortcodes(unittest.TestCase):
|
||||
def test_reads_relative_and_absolute_hrefs(self):
|
||||
hrefs = [
|
||||
"/someuser/reel/ABC123/",
|
||||
"https://www.instagram.com/someuser/reel/DEF456/",
|
||||
"/reel/GHI789/?img_index=1",
|
||||
]
|
||||
self.assertEqual(scrape.extract_shortcodes(hrefs),
|
||||
["ABC123", "DEF456", "GHI789"])
|
||||
|
||||
def test_ignores_non_reel_links(self):
|
||||
hrefs = ["/someuser/", "/someuser/p/ABC123/", "/explore/tags/foo/"]
|
||||
self.assertEqual(scrape.extract_shortcodes(hrefs), [])
|
||||
|
||||
def test_empty_input(self):
|
||||
self.assertEqual(scrape.extract_shortcodes([]), [])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main(verbosity=2)
|
||||
Reference in New Issue
Block a user