docs: record the first full incremental sync, and what made it cheap

All six ARTMS profiles, four days after the previous run: 184 new media, 299
files published, 0 failures and 0 CDN 429s. The header still claimed the
script was a skeleton with nothing migrated, which stopped being true a while
ago.

Notes the two things that made it cheap — an already-seeded archive DB, so no
probe passes ran at all, and --abort 50 — and that the run was deliberately
stopped and resumed midway to pick up the new flag. That is only safe because
the state file had already marked the finished sources as fetched, so the 20h
floor skipped them. Stopping a run used to mean repeating it.

Also records that rsync exit 23 is the NORMAL outcome of a publish: the SSH
user cannot chown to rslsync, so attrs fail while data arrives intact. The
check is a --dry-run re-run returning an empty file list, which is what
verified this one (+299 files, matching rsync exactly).

The account was restored; noting that without pretending it licenses more
traffic.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UXfdJu7QhSJLr47K7koTDF
This commit is contained in:
2026-08-20 14:36:32 -04:00
co-authored by Claude Opus 5
parent a83da461c1
commit 71cfd29f36
+46 -9
View File
@@ -1,10 +1,10 @@
# gallery-dl — a CLI replacement for JDownloader2 # gallery-dl — a CLI replacement for JDownloader2
Status: **design + verified config.** `scripts/gdl-sync.py` is a skeleton; no Status: **in production.** All six ARTMS profiles are synced with
profile has been migrated yet. `scripts/gdl-sync.py`; JD2 is no longer used for them.
Everything below was measured against the live site and the real archive on Everything below was measured against the live site and the real archive on
2026-08-16, not inferred from documentation. 2026-08-16 and 2026-08-20, not inferred from documentation.
## Why gallery-dl and not a hand-rolled script ## Why gallery-dl and not a hand-rolled script
@@ -20,6 +20,12 @@ The account used for all of the below was suspended the same day this tooling
was built, for "activity that doesn't follow our Community Standards on spam". was built, for "activity that doesn't follow our Community Standards on spam".
The fetching was not the expensive part. **Verification was.** The fetching was not the expensive part. **Verification was.**
**It was restored, and synced normally again on 2026-08-20** — a full run
across all six profiles with 0 failures and 0 CDN 429s. That is not evidence
the limits were imagined; it is one data point on a restored account that has
been treated carefully since. Everything below still applies, and the budget is
still per session rather than per command.
What was actually spent against `instagram.com` in a few hours, from one What was actually spent against `instagram.com` in a few hours, from one
session and one IP: session and one IP:
@@ -437,15 +443,46 @@ never reaches the archive half-written.
## Status ## Status
In use. `withaseul` has been fetched and published to the live archive — 322 In use for all six ARTMS profiles.
files added (74 media, 241 `.json`, 7 `.txt`), nothing overwritten or deleted.
Of the 74 new media files, **zero** duplicated media already held under a `withaseul` first — 322 files added (74 media, 241 `.json`, 7 `.txt`), nothing
different name, which is the check that says JD2 and gallery-dl naming really overwritten or deleted. Of the 74 new media, **zero** duplicated media already
do converge. held under a different name, which is the check that says JD2 and gallery-dl
naming really do converge.
**2026-08-20**, the first full incremental sync, four days after the previous
one. 184 new media, 299 files published, 0 failures and **0 CDN 429s**:
| profile | posts | reels | stories | files added |
|---|---:|---:|---:|---:|
| 0ct0ber19 | 58 | 2 | 4 | +77 |
| official_artms | 12 | — | 2 | +85 |
| cher_ryppo | 41 | 1 | 8 | +63 |
| zindoriyam | 23 | — | 4 | +35 |
| kimxxlip | 16 | — | 2 | +23 |
| withaseul | 10 | — | — | +16 |
The 20 story items are the part that could not have been recovered later.
Two things made it cheap, and both are worth keeping:
- The archive DB was already seeded from the previous run, so `--min-interval`
and the recorded `seeded` state meant **no probe passes at all**. A state
file has to exist for this; if one is missing after a manual run, write it
rather than letting the tool re-seed 24 sources.
- `--abort 50` (see above) cut the remaining listing cost by roughly 85%.
The run was deliberately **stopped and resumed** halfway to pick up `--abort`.
That is safe precisely because of the state file: the 12 finished sources were
already marked `fetched`, so the 20h floor skipped them and only the remaining
12 re-ran. Stopping a run is cheap now; it was not before.
Published files land owned by the SSH user rather than `rslsync`. The viewer Published files land owned by the SSH user rather than `rslsync`. The viewer
reads them fine (world-readable), but Resilio does not own what it syncs; worth reads them fine (world-readable), but Resilio does not own what it syncs; worth
a `chown` if that ever matters. a `chown` if that ever matters. This also makes **`rsync` exit 23**
("some files/attrs were not transferred") the *normal* outcome of a publish —
it is the failed `chown`, not lost data. Confirm by re-running the same rsync
with `--dry-run`: an empty file list means everything arrived.
The profiles to fetch live in `artms_account_links.txt` at the archive root, The profiles to fetch live in `artms_account_links.txt` at the archive root,
passed with `--urls-file`. passed with `--urls-file`.