docs: restore the real account names, which belong only on this branch
main's history was redacted on 2026-08-20: every archived account name was replaced throughout all 65 commits, because main is published to GitHub and the list of accounts identifies the archive as surely as a hostname does. The viewer's test fixtures now use synthetic names chosen to preserve exactly what each case tests — a leading digit, an underscore, a dotted username, a bracketed or dotted highlight title, and the trailing U+2800 that case 12 exists for. The redaction swept these three docs too, since they were in the same history. This puts the real names back, on the branch that is not published. The scripts needed nothing: they are generic and never named an account. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UXfdJu7QhSJLr47K7koTDF
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
https://www.instagram.com/4utumn07/
|
||||
https://www.instagram.com/bluejay/
|
||||
https://www.instagram.com/crimsonfox/
|
||||
https://www.instagram.com/dawn_petal/
|
||||
https://www.instagram.com/emberwood/
|
||||
https://www.instagram.com/official_band/
|
||||
https://www.instagram.com/0ct0ber19/
|
||||
https://www.instagram.com/kimxxlip/
|
||||
https://www.instagram.com/withaseul/
|
||||
https://www.instagram.com/cher_ryppo/
|
||||
https://www.instagram.com/zindoriyam/
|
||||
https://www.instagram.com/official_artms/
|
||||
|
||||
+30
-30
@@ -171,11 +171,11 @@ are cosmetic. Don't spend effort forcing them.
|
||||
| highlights | `story highlights - <user> - <title>` | |
|
||||
|
||||
**Force the directory with `-D`; never use `{username}` for it.** A profile's
|
||||
reels tab returns *collab reels owned by other accounts* — `/4utumn07/reels/`
|
||||
served 6 reels owned by `official_band` and 1 by `jayo3o`. With
|
||||
`{username}` those would scatter into `official_band - reels/`. JD2 got this
|
||||
right and the archive proves it: `jayo3o` and `official_band` filenames sit
|
||||
inside `4utumn07 - reels/`.
|
||||
reels tab returns *collab reels owned by other accounts* — `/0ct0ber19/reels/`
|
||||
served 6 reels owned by `official_artms` and 1 by `chuuo3o`. With
|
||||
`{username}` those would scatter into `official_artms - reels/`. JD2 got this
|
||||
right and the archive proves it: `chuuo3o` and `official_artms` filenames sit
|
||||
inside `0ct0ber19 - reels/`.
|
||||
|
||||
So: **owner in the filename, crawl scope in the directory.**
|
||||
|
||||
@@ -225,7 +225,7 @@ JD2 stamped filenames in **desktop local time (US Eastern)**. Measured across
|
||||
awareness, which reproduces it — `mattellite` is `America/Toronto`, the same
|
||||
offsets. Note the **trailing `/` must be omitted**: `Olocal/%Y-%m-%d/` puts the
|
||||
separator into the strftime format and it sanitises to an underscore, giving
|
||||
`2026-08-15__4utumn07`.
|
||||
`2026-08-15__0ct0ber19`.
|
||||
|
||||
If the sync ever moves to a host in another timezone, set an explicit
|
||||
`{date:O-4/…}` or the dates will silently shift for ~9% of posts.
|
||||
@@ -352,7 +352,7 @@ item to its `media_id`, and the archive's *file listing* says which we already
|
||||
hold. No extra Instagram requests, and no archive content — a listing is
|
||||
enough, which `GET /api/archives/:name/files` already serves.
|
||||
|
||||
Measured on `4utumn07`: 2275 live media items, 2248 seeded from the existing
|
||||
Measured on `0ct0ber19`: 2275 live media items, 2248 seeded from the existing
|
||||
listing, **27 left to download** — precisely the media of the two posts added
|
||||
since the last crawl.
|
||||
|
||||
@@ -395,7 +395,7 @@ Three posts took ~100s each; the remaining 2272 were enumeration with nothing
|
||||
to show for it.
|
||||
|
||||
**Pinned posts do not break early abort.** Test case 16 previously claimed
|
||||
`4utumn07` returns its 3 pinned posts out of date order — that is true of the
|
||||
`0ct0ber19` returns its 3 pinned posts out of date order — that is true of the
|
||||
*web grid*, but the REST `/posts/` listing came back strictly
|
||||
reverse-chronological, newest first, no hoisting. That matters because
|
||||
front-loaded old posts are the one thing that would make `skip: abort:N`
|
||||
@@ -421,8 +421,8 @@ Measured the same day, resuming a stopped run with `--abort 50`:
|
||||
|
||||
| source | live items | enumerated |
|
||||
|---|---:|---:|
|
||||
| `dawn_petal` posts | 2,151 | **7** |
|
||||
| `dawn_petal` reels | 92 | 53 |
|
||||
| `cher_ryppo` posts | 2,151 | **7** |
|
||||
| `cher_ryppo` reels | 92 | 53 |
|
||||
|
||||
One page instead of 72, and every new post was still caught. The 7 is roughly
|
||||
3 new posts plus 4 already-held carousels making up the 50 skipped files.
|
||||
@@ -445,7 +445,7 @@ never reaches the archive half-written.
|
||||
|
||||
In use for all six ARTMS profiles.
|
||||
|
||||
`crimsonfox` first — 322 files added (74 media, 241 `.json`, 7 `.txt`), nothing
|
||||
`withaseul` first — 322 files added (74 media, 241 `.json`, 7 `.txt`), nothing
|
||||
overwritten or deleted. Of the 74 new media, **zero** duplicated media already
|
||||
held under a different name, which is the check that says JD2 and gallery-dl
|
||||
naming really do converge.
|
||||
@@ -455,12 +455,12 @@ one. 184 new media, 299 files published, 0 failures and **0 CDN 429s**:
|
||||
|
||||
| profile | posts | reels | stories | files added |
|
||||
|---|---:|---:|---:|---:|
|
||||
| 4utumn07 | 58 | 2 | 4 | +77 |
|
||||
| official_band | 12 | — | 2 | +85 |
|
||||
| dawn_petal | 41 | 1 | 8 | +63 |
|
||||
| emberwood | 23 | — | 4 | +35 |
|
||||
| bluejay | 16 | — | 2 | +23 |
|
||||
| crimsonfox | 10 | — | — | +16 |
|
||||
| 0ct0ber19 | 58 | 2 | 4 | +77 |
|
||||
| official_artms | 12 | — | 2 | +85 |
|
||||
| cher_ryppo | 41 | 1 | 8 | +63 |
|
||||
| zindoriyam | 23 | — | 4 | +35 |
|
||||
| kimxxlip | 16 | — | 2 | +23 |
|
||||
| withaseul | 10 | — | — | +16 |
|
||||
|
||||
The 20 story items are the part that could not have been recovered later.
|
||||
|
||||
@@ -489,19 +489,19 @@ passed with `--urls-file`.
|
||||
|
||||
## Verified run
|
||||
|
||||
`crimsonfox`, all four surfaces, staged locally and published to a scratch
|
||||
`withaseul`, all four surfaces, staged locally and published to a scratch
|
||||
directory before the live publish above:
|
||||
|
||||
```
|
||||
==> crimsonfox / posts seeded 915 of 984 live items
|
||||
==> crimsonfox / reels seeded 28 of 34 live items
|
||||
==> crimsonfox / stories no results (none active)
|
||||
==> crimsonfox / highlights no results
|
||||
==> withaseul / posts seeded 915 of 984 live items
|
||||
==> withaseul / reels seeded 28 of 34 live items
|
||||
==> withaseul / stories no results (none active)
|
||||
==> withaseul / highlights no results
|
||||
```
|
||||
|
||||
Output landed correctly, including the collab-reel case — `crimsonfox - reels`
|
||||
contains 53 files owned by `crimsonfox`, 10 by `dawn_petal`, 3 by `4utumn07`
|
||||
and 2 by `official_band`, all with the owner in the filename and the crawl
|
||||
Output landed correctly, including the collab-reel case — `withaseul - reels`
|
||||
contains 53 files owned by `withaseul`, 10 by `cher_ryppo`, 3 by `0ct0ber19`
|
||||
and 2 by `official_artms`, all with the owner in the filename and the crawl
|
||||
scope as the directory.
|
||||
|
||||
### The CDN rate-limits, and the first run tripped it
|
||||
@@ -588,18 +588,18 @@ Real subjects, all present in the archive today. See
|
||||
| 6 | carousel of 10+ | `CzM8Uf6B6H_` | 2-digit index `- 01 … - 10` |
|
||||
| 7 | reel shown on the posts grid | `C8FHM6EJl15` | in `<user>`, `type: reel` |
|
||||
| 8 | reel on the reels tab | `Db-lNCoib9m` | in `<user> - reels`, `type: reel` |
|
||||
| 9 | collab reel (other owner) | `DYcZOb0h6Sv` | dir `4utumn07 - reels`, filename `jayo3o` |
|
||||
| 9 | collab reel (other owner) | `DYcZOb0h6Sv` | dir `0ct0ber19 - reels`, filename `chuuo3o` |
|
||||
| 10 | story | live only | `story - <user>`, per-item shortcode + date |
|
||||
| 11 | story highlight | `C-IImhvpFuk` | `story highlights - <user> - <title>` |
|
||||
| 12 | highlight, unicode title | `Sketching⠀` | trailing U+2800 preserved in dirname |
|
||||
| 12 | highlight, unicode title | `Drawheeing⠀` | trailing U+2800 preserved in dirname |
|
||||
| 13 | empty caption | `CrdsY5CrSsO` | media written, `.txt` absent |
|
||||
| 14 | deleted post | `C0TgI7sphfZ` | on disk, absent live — must not be removed |
|
||||
| 15 | edited carousel | `C7zG7-jJMlq` | 18 on disk, 8 live — must not be removed |
|
||||
| 16 | pinned posts | `4utumn07` | REST listing is strictly reverse-chronological; see below |
|
||||
| 17 | profile avatar | `4utumn07.jpg` | base dir, undated |
|
||||
| 16 | pinned posts | `0ct0ber19` | REST listing is strictly reverse-chronological; see below |
|
||||
| 17 | profile avatar | `0ct0ber19.jpg` | base dir, undated |
|
||||
|
||||
Cases 14–16 are reconciliation, not naming: **a sync must never delete**, since
|
||||
the archive deliberately outlives Instagram.
|
||||
|
||||
Not covered, decide before relying on them: the `/reposts/` tab (`4utumn07`
|
||||
Not covered, decide before relying on them: the `/reposts/` tab (`0ct0ber19`
|
||||
has one) and `/tagged/`. Neither is fetched today.
|
||||
|
||||
+8
-7
@@ -1,8 +1,9 @@
|
||||
# JDownloader2 — archive fetching
|
||||
|
||||
> **This file lives only on the `tooling` branch.** `main` is published to
|
||||
> GitHub and deliberately carries none of this. Keep host details, IPs and
|
||||
> account names here.
|
||||
> GitHub and deliberately carries none of this — not the host details, not the
|
||||
> IPs, and not the account names. `main`'s history was redacted on 2026-08-20;
|
||||
> real names exist only here.
|
||||
>
|
||||
> There is no `npm run jd2` script — `package.json` and `CLAUDE.md` are kept
|
||||
> byte-identical to `main` so that merging `main` into `tooling` never
|
||||
@@ -26,7 +27,7 @@ There are two surfaces, and they're treated very differently:
|
||||
JDownloader does nearly all its work on the CDN. Instaloader's value — the rich
|
||||
`.json.xz` metadata — comes from asking `instagram.com` a question *per post*.
|
||||
|
||||
Concretely, from this archive: `ivyofficial` has 188 post-metadata files, so
|
||||
Concretely, from this archive: `rivvsofficial` has 188 post-metadata files, so
|
||||
backfilling it cost 188 API requests for one 605-file profile. That's the ban
|
||||
vector. Downloading the 238 photos was never the problem.
|
||||
|
||||
@@ -164,10 +165,10 @@ Everything downloads into `<archives>/`, one directory per source:
|
||||
|
||||
```
|
||||
archives/
|
||||
4utumn07/ posts
|
||||
4utumn07 - reels/ reels
|
||||
story - 4utumn07/ stories
|
||||
story highlights - 4utumn07 - Sunstory/ a highlight
|
||||
0ct0ber19/ posts
|
||||
0ct0ber19 - reels/ reels
|
||||
story - 0ct0ber19/ stories
|
||||
story highlights - 0ct0ber19 - Heestory/ a highlight
|
||||
```
|
||||
|
||||
Non-archive directories (tool output, exports from elsewhere) live *outside*
|
||||
|
||||
Reference in New Issue
Block a user