docs: add a JDownloader quick reference

Covers why fetching goes through JDownloader rather than Instaloader (the
instagram.com vs CDN split, and what the metadata gap actually costs), the
settings that matter, cookie handling, the two-URL workflow, jd2-sync usage,
the expected on-disk layout, and what to do when something breaks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011uBWhwV3wFQ5MBCcMHHem7

CLAUDE.md

@@ -16,7 +16,8 @@ InstaArchive Viewer is a React 19 + Vite 6 PWA for browsing archived Instagram d
 - `npm test` / `npm run test:watch` — vitest
 - `npx vitest run src/lib/archive-patterns.test.ts` — a single test file
 - `npm run jd2 -- --archives <dir> --dry-run` — generate JDownloader `.crawljob`
-  files for every profile on disk (see `scripts/jd2-sync.ts`)
+  files for every profile on disk (see `scripts/jd2-sync.ts` and
+  `docs/jdownloader.md`)
 
 Local development usually needs both `npm run dev` and `npm run server`. Local-folder mode works without the backend; server-mode archives do not.
 

docs/jdownloader.md

@@ -0,0 +1,165 @@
+# JDownloader2 — archive fetching quick reference
+
+How content gets into this archive, and why the setup is shaped the way it is.
+
+## Why JDownloader and not Instaloader
+
+There are two surfaces, and they're treated very differently:
+
+| Surface | What hits it | Risk |
+|---|---|---|
+| `instagram.com` | profile pages, GraphQL/API metadata | Tied to your session, heavily rate-limited. **This is where bans come from.** |
+| `scontent*.cdninstagram.com` | the actual media | Signed URLs, CDN-served, tolerant. Mostly a bandwidth question. |
+
+JDownloader does nearly all its work on the CDN. Instaloader's value — the rich
+`.json.xz` metadata — comes from asking `instagram.com` a question *per post*.
+
+Concretely, from this archive: `rivvsofficial` has 188 post-metadata files, so
+backfilling it cost 188 API requests for one 605-file profile. That's the ban
+vector. Downloading the 238 photos was never the problem.
+
+Instaloader got this account banned once. JDownloader with throttling has not.
+
+### What the metadata gap actually costs
+
+Comparing a JDownloader profile against an Instaloader one:
+
+| | JDownloader | Instaloader |
+|---|---|---|
+| Media | ✅ | ✅ |
+| Captions (`.txt`) | ✅ | ✅ |
+| Dates (from filenames) | ✅ | ✅ |
+| Bio / full name | ❌ | ✅ |
+| Follower counts | ❌ | ✅ |
+| External URL | ❌ | ✅ |
+
+Captions already work — the viewer reads the `.txt` sidecars. Everything missing
+lives in a *single* profile-level record, not the per-post ones. That's why
+JDownloader-sourced profiles show "0 followers" and a placeholder bio.
+
+Not worth extra requests. If you ever want it, the zero-request option is a
+hand-written `profile.json` sidecar (not implemented yet — ask).
+
+## Settings that matter
+
+**Chunks per download → 1.** The single most important one. JDownloader splits
+each file into multiple ranged requests by default; that `Range` pattern looks
+nothing like a browser or the app. One chunk = one sequential GET per file.
+`jd2-sync` sets `chunks=1` per job, so no global change is needed — but set it
+globally too if you ever add links by hand.
+
+**Max simultaneous downloads → 2–3**, connections-per-host low. Concurrency is
+what turns "a user" into a statistic.
+
+**Leave reconnect / IP-change features off.** A mid-session IP change on a live
+cookie is a *stronger* anomaly signal than the request rate you'd be avoiding.
+
+## The cookie
+
+Exported manually from a real browser session. This is the right approach — no
+programmatic login anywhere, which is the thing that actually gets flagged.
+
+- Use it from the **same public IP** as the browser it came from. A cookie used
+  from a different network is what session-hijack detection looks for.
+- When it expires, **re-export from the browser**. Never add a login step to a tool.
+- It's a full account credential. Keep it off the NAS share and out of the repo.
+
+## Workflow
+
+Two URLs per profile, because the profile grid misses some reels:
+
+```
+https://www.instagram.com/<user>/
+https://www.instagram.com/<user>/reels/
+```
+
+They overlap slightly — a reel caught by both lands in each directory and shows
+up twice in the viewer. That's correct and matches Instagram, which also shows
+reels in the profile grid *and* the Reels tab.
+
+## Generating jobs
+
+Instead of pasting URLs and setting output folders by hand:
+
+```bash
+npm run jd2 -- --archives /volume1/rslsync/sync/Instagram-archive/archives --dry-run
+```
+
+Review, then write it into JDownloader's folder-watch directory:
+
+```bash
+npm run jd2 -- --archives /volume1/rslsync/sync/Instagram-archive/archives \
+               --out ~/.jd2/folderwatch
+```
+
+JDownloader runs on the desktop while the archive lives on the NAS, so tell it
+the path *it* sees:
+
+```bash
+npm run jd2 -- --archives /mnt/nas/Instagram-archive/archives \
+               --download-base 'Z:\Instagram-archive\archives' \
+               --out ~/.jd2/folderwatch
+```
+
+| Flag | Purpose |
+|---|---|
+| `--archives <dir>` | Archive root to scan (or `$ARCHIVES_DIR`) |
+| `--out <dir>` | JDownloader folder-watch directory |
+| `--download-base <dir>` | Root path as JDownloader sees it (Windows paths fine) |
+| `--user <name>` | Just this profile (repeatable) |
+| `--skip <name>` | Never emit jobs for this directory (repeatable) |
+| `--chunks <n>` | Connections per file (default 1) |
+| `--auto-start` | Start immediately instead of parking in LinkGrabber |
+| `--all-reels` | Emit a reels job even where no reels directory exists |
+| `--dry-run` | Print instead of writing |
+
+Defaults are deliberately conservative: `chunks=1`, and links park in the
+LinkGrabber for review rather than auto-starting.
+
+Only posts and reels are emitted. Highlight URLs need a numeric id and story
+URLs expire, so those stay manual.
+
+Directories that aren't Instagram profiles are skipped by username shape
+(letters, digits, dots, underscores, ≤30 chars) — pointing a crawl at those
+spends `instagram.com` requests to be told the profile doesn't exist. For names
+that *look* like usernames but aren't, use `--skip` or a `.jd2ignore` file in
+the archive root, one name per line.
+
+Format reference: `src/org/jdownloader/extensions/folderwatchV2/explain.txt`.
+JDownloader develops on SVN — read it via the daily mirror at
+<https://github.com/mycodedoesnotcompile2/jdownloader_mirror> (`svn_trunk/`),
+not one of the abandoned GitHub copies.
+
+## Expected layout
+
+Everything downloads into `<archives>/`, one directory per source:
+
+```
+archives/
+  0ct0ber19/                                 posts
+  0ct0ber19 - reels/                         reels
+  story - 0ct0ber19/                         stories
+  story highlights - 0ct0ber19 - Heestory/   a highlight
+```
+
+Non-archive directories (tool output, exports from elsewhere) live *outside*
+`archives/` so they never reach the viewer.
+
+The server picks up changes automatically — its index is keyed on directory
+mtime, so a new file invalidates only that directory.
+
+## If something goes wrong
+
+**429 / rate limited** — stop for hours, not seconds. Retrying into a limit is
+what converts a soft throttle into something worse.
+
+**Cookie stops working** — re-export from the browser. Don't add a login step.
+
+**Files land in the wrong folder** — a Packagizer rule is overriding the job.
+Generated jobs set `overwritePackagizerEnabled=TRUE` to prevent this; check that
+rules aren't set to run after it.
+
+**Viewer doesn't show new posts** — check the file is in the right directory and
+matches the naming pattern (`YYYY-MM-DD_<user> - <shortcode>[ - NN].<ext>`).
+The index refreshes on directory mtime, so a genuinely new file is picked up on
+the next request.