diff --git a/docs/gallery-dl.md b/docs/gallery-dl.md new file mode 100644 index 0000000..915b157 --- /dev/null +++ b/docs/gallery-dl.md @@ -0,0 +1,263 @@ +# gallery-dl — a CLI replacement for JDownloader2 + +Status: **design + verified config.** `scripts/gdl-sync.py` is a skeleton; no +profile has been migrated yet. + +Everything below was measured against the live site and the real archive on +2026-08-16, not inferred from documentation. + +## Why gallery-dl and not a hand-rolled script + +The hard parts of fetching Instagram are pagination, cookie handling, CDN URL +expiry and resumption. gallery-dl already has all of them, plus extractors that +map 1:1 onto our sidecar directory layout (`posts`, `reels`, `stories`, +`highlights`). Rolling our own would mean reimplementing the ban-sensitive part +by hand. + +## The safety model — read this before changing any option + +The ban vector is **requests to `instagram.com`**, not bandwidth. See +`docs/jdownloader.md` for the history; Instaloader got this account banned by +asking `instagram.com` a question *per post*. + +gallery-dl has two API backends and the difference is exactly that vector: + +```python +if self.config("api") == "graphql": + self.api = InstagramGraphqlAPI(self) # per-post api.media() for every +else: # video and every carousel + self.api = InstagramRestAPI(self) # <- default, listing-only +``` + +The REST backend paginates at `count: 30` (feed) / `page_size: 50` (clips), and +those responses already carry `carousel_media`, `image_versions2`, +`video_versions` and `product_type`. **No per-post request.** A 300-post +profile costs roughly 10 requests to `instagram.com`. + +Rules, in order of importance: + +1. **`"api": "rest"` always.** Never `graphql`. This is the whole ballgame. +2. **Never enable `metadata`-style options that trigger extra calls.** If a + field is not already in the listing response, it is not worth a request. +3. **Pace it.** `"sleep-request": [4.0, 7.0]` — a randomised gap, not a fixed + one. Also `"sleep": [1.0, 3.0]` between downloads. +4. **Cap the download rate** (`downloader.http.rate`) so the CDN side looks like + a person, not a mirror. +5. **Run from the same public IP as the browser the cookie came from.** At time + of writing that is `mattellite` (`66.23.52.196`); the dev workstation is a + *different* public IP and using the cookie from there is precisely what + session-hijack detection looks for. +6. **No programmatic login, ever.** gallery-dl's username/password path is + disabled upstream anyway; use `--cookies-from-browser`. + +Do not add proxy rotation, fingerprint spoofing or account rotation. Throttling +and request-avoidance are welcome; evasion is not. + +### Cookies + +The logged-in Chrome on `mattellite` runs with a non-default profile: + +``` +--user-data-dir=/home/matt/.config/google-chrome-devtools +``` + +so the cookie flag is: + +``` +--cookies-from-browser "chrome:/home/matt/.config/google-chrome-devtools" +``` + +Plain `--cookies-from-browser chrome` fails with "Unable to find chrome cookies +database" because it looks in `~/.config/google-chrome/`. + +Anonymous access is **not** a viable fallback: it serves lower-resolution media, +caps profile pagination at 12 posts, and returns `AuthRequired` for stories and +highlights. + +## Output format + +The viewer's parser is the contract, not JD2's exact bytes. `EXPORT_RE` in +`src/lib/archive-patterns.ts` accepts all of these, and normalises the index +with `parseInt`, so **JD2 and gallery-dl naming interoperate**: + +``` +"… - CrORBIcJJbM.mp4" -> postId=CrORBIcJJbM index=1 +"… - CrORBIcJJbM - 1.mp4" -> postId=CrORBIcJJbM index=1 +"… - C53YPQzp7Wj - 09.jpg" -> postId=C53YPQzp7Wj index=9 +``` + +That means zero-padding and the presence/absence of ` - N` on single-media posts +are cosmetic. Don't spend effort forcing them. + +### Directory layout + +| kind | directory | note | +|---|---|---| +| posts | `` | | +| reels | ` - reels` | | +| stories | `story - ` | | +| highlights | `story highlights - - ` | | + +**Force the directory with `-D`; never use `{username}` for it.** A profile's +reels tab returns *collab reels owned by other accounts* — `/0ct0ber19/reels/` +served 6 reels owned by `official_artms` and 1 by `chuuo3o`. With +`{username}` those would scatter into `official_artms - reels/`. JD2 got this +right and the archive proves it: `chuuo3o` and `official_artms` filenames sit +inside `0ct0ber19 - reels/`. + +So: **owner in the filename, crawl scope in the directory.** + +### Filenames + +```jsonc +"filename": { + "sidecar_shortcode and count >= 10": + "{date:Olocal/%Y-%m-%d}_{username} - {post_shortcode} - {num:02}.{extension}", + "sidecar_shortcode": + "{date:Olocal/%Y-%m-%d}_{username} - {post_shortcode} - {num}.{extension}", + "": + "{date:Olocal/%Y-%m-%d}_{username} - {post_shortcode}.{extension}" +} +``` + +`sidecar_shortcode` is set only when the post is a carousel, so it is the +carousel discriminator. Conditions are evaluated in order, first match wins +(`path.py:265`). + +Stories and highlights use the per-item `{shortcode}`, not `{post_shortcode}` +(which is the *reel's* id, shared by every item in it): + +``` +"{date:Olocal/%Y-%m-%d}_{username} - {shortcode}.{extension}" +``` + +`{date}` on a story/highlight file is the **per-item** `taken_at` +(`instagram.py:337` prefers `item["taken_at"]`), verified on a 154-item +highlight whose items carried distinct times while `post_date` stayed pinned to +the reel. Highlights therefore gain real dates — today they fall back to +directory mtime. + +### The timezone is not UTC + +JD2 stamped filenames in **desktop local time (US Eastern)**. Measured across +212 comparable posts: + +| model | mismatches | +|---|---:| +| UTC | 19 | +| UTC−5 (EST) | 10 | +| UTC−4 (EDT) | **0** | +| America/New_York (DST-aware) | **0** | + +`{date:Olocal/%Y-%m-%d}` uses the machine's local zone with per-timestamp DST +awareness, which reproduces it — `mattellite` is `America/Toronto`, the same +offsets. Note the **trailing `/` must be omitted**: `Olocal/%Y-%m-%d/` puts the +separator into the strftime format and it sanitises to an underscore, giving +`2026-08-15__0ct0ber19`. + +If the sync ever moves to a host in another timezone, set an explicit +`{date:O-4/…}` or the dates will silently shift for ~9% of posts. + +### Caption sidecars + +JD2 writes one `.txt` per post, named without the index, containing the caption +with **no trailing newline**, and writes nothing when the caption is empty +(measured: 197 of 217 posts, 86 of 86 reels, 0 of 10 stories, 0 of 16 +highlights). gallery-dl reproduces this exactly with the default +`"empty": false`: + +```jsonc +{ "name": "metadata", "event": "post", "mode": "custom", + "content-format": "{description}", "extension": "txt", + "filename": "{date:Olocal/%Y-%m-%d}_{username} - {post_shortcode}.txt" } +``` + +`"event": "post"` is what makes it one file per post rather than per media file. + +### Metadata sidecar (new — JD2 had no equivalent) + +```jsonc +{ "name": "metadata", "event": "post", "mode": "json", + "filename": "{date:Olocal/%Y-%m-%d}_{username} - {post_shortcode}.json", + "include": ["post_shortcode","post_id","type","date","post_date","username", + "fullname","owner_id","description","count","likes","post_url", + "sidecar_shortcode"] } +``` + +Use **`include`**, not `fields` — `fields` is for `mode: custom` and silently +does nothing here, leaving `audio_user` blobs (including another user's profile +picture URL) in the output. + +The payoff is `type`, which is Instagram's own classification: + +```json +{ "post_shortcode": "DbdG9L9jU4m", "type": "post", "count": 2 } // feed video +{ "post_shortcode": "Db-lNCoib9m", "type": "reel", "count": 1 } // real reel +``` + +This is the `product_type: "clips"` signal, delivered free in the listing +response. It is the authoritative answer to "is this a reel", and would let the +viewer retire the lone-video heuristic in `src/lib/post-tabs.ts` — see +"Scanner work" below. + +**`type` is only populated by listing extractors.** Extracting a single +`/p/<shortcode>/` URL leaves it `null`. Sync always uses listing URLs, so this +only matters when testing by hand. + +## Known quirks + +- **`count` is not the emitted file count.** For 135 of 214 posts it was exactly + one higher than the number of files written. This makes the `count >= 10` + padding condition mis-pad a handful of 9-item posts (10 of 214 measured). Since + the parser normalises the index, this is cosmetic — but it means a re-fetch + over an existing JD2 tree writes `- 01.jpg` beside an existing `- 1.jpg`. +- **Carousels get edited.** Two posts had a different media count live than on + disk. Padding width follows the count *at download time*, so a grown carousel + produces mixed widths — the archive already contains one such post from JD2. +- **Highlights already have two naming styles on disk**, and every undated file + has a dated twin. The scanner dedupes by index so they render once; it is + wasted disk, not a display bug. + +## Scanner work (not done yet) + +`useArchiveScanner` currently treats any `.json` in the tree as a possible +manifest. Adding gallery-dl sidecars needs it to distinguish three things: + +1. Instagram export manifests (`posts_1.json`) — existing path. +2. Instaloader `.json.xz` — existing path, GraphQL node shape. +3. gallery-dl `.json` — new, flat shape, identified by having + `post_shortcode` + `type` at the top level. + +Once (3) is read, `source`/`isStory` and the reel flag should come from `type` +rather than from the directory and the lone-video heuristic. + +## Test cases + +Real subjects, all present in the archive today. See +`scripts/gdl-sync.py --selftest` for the harness. + +| # | case | shortcode | expected | +|---|---|---|---| +| 1 | single image | `CwcXnQhOqFG` | one `.jpg`, no index | +| 2 | single feed video | `DbdG9L9jU4m` | one `.mp4`, `type: post` | +| 3 | carousel, images only | `Cq8LrxSJAJE` | `- 1 … - 3` | +| 4 | carousel, image + video | `CtohvHxLnWO` | `- 1.jpg … - 4.mp4`, **no `.txt`** | +| 5 | carousel of exactly 9 | `Cv2Hb_brx_N` | 1-digit index | +| 6 | carousel of 10+ | `CzM8Uf6B6H_` | 2-digit index `- 01 … - 10` | +| 7 | reel shown on the posts grid | `C8FHM6EJl15` | in `<user>`, `type: reel` | +| 8 | reel on the reels tab | `Db-lNCoib9m` | in `<user> - reels`, `type: reel` | +| 9 | collab reel (other owner) | `DYcZOb0h6Sv` | dir `0ct0ber19 - reels`, filename `chuuo3o` | +| 10 | story | live only | `story - <user>`, per-item shortcode + date | +| 11 | story highlight | `C-IImhvpFuk` | `story highlights - <user> - <title>` | +| 12 | highlight, unicode title | `Drawheeing⠀` | trailing U+2800 preserved in dirname | +| 13 | empty caption | `CrdsY5CrSsO` | media written, `.txt` absent | +| 14 | deleted post | `C0TgI7sphfZ` | on disk, absent live — must not be removed | +| 15 | edited carousel | `C7zG7-jJMlq` | 18 on disk, 8 live — must not be removed | +| 16 | pinned posts | `0ct0ber19` | 3 pinned, returned out of date order | +| 17 | profile avatar | `0ct0ber19.jpg` | base dir, undated | + +Cases 14–16 are reconciliation, not naming: **a sync must never delete**, since +the archive deliberately outlives Instagram. + +Not covered, decide before relying on them: the `/reposts/` tab (`0ct0ber19` +has one) and `/tagged/`. Neither is fetched today. diff --git a/scripts/gdl-sync.py b/scripts/gdl-sync.py new file mode 100755 index 0000000..b624624 --- /dev/null +++ b/scripts/gdl-sync.py @@ -0,0 +1,333 @@ +#!/usr/bin/env python3 +""" +Fetch Instagram profiles into the archive layout using gallery-dl. + +The CLI replacement for the JDownloader2 workflow. See docs/gallery-dl.md for +the measurements behind every choice here — especially the safety model, which +is the reason this script exists in this shape rather than a simpler one. + +STATUS: skeleton. The config generation and planning are complete and tested; +`--execute` is deliberately gated behind an explicit flag and has not been run +against the live archive. + +Usage: + ./scripts/gdl-sync.py --archives <dir> --profile 0ct0ber19 --dry-run + ./scripts/gdl-sync.py --archives <dir> --all --dry-run + ./scripts/gdl-sync.py --archives <dir> --profile 0ct0ber19 --execute + +Run it from the host whose public IP matches the browser the cookie came from. +""" + +from __future__ import annotations + +import argparse +import json +import os +import re +import shutil +import subprocess +import sys +from dataclasses import dataclass, field +from pathlib import Path + +# -------------------------------------------------------------------------- +# Archive layout +# -------------------------------------------------------------------------- + +# Mirrors src/lib/archive-grouping.ts. Instagram usernames cannot contain +# spaces, which is what makes the username separable from a highlight title. +RE_HIGHLIGHT = re.compile(r"^story highlights - ([^ ]+) - (.+)$") +RE_STORIES = re.compile(r"^story - ([^ ]+)$") +RE_REELS = re.compile(r"^([^ ]+) - reels$") + +DATE_FMT = "{date:Olocal/%Y-%m-%d}" +"""Local-time date. JD2 stamped US Eastern, NOT UTC (0/212 mismatches vs 19 for +UTC). `Olocal` is DST-aware per timestamp. The trailing separator must be +omitted or it lands in the strftime format and sanitises to an underscore.""" + +POST_STEM = DATE_FMT + "_{username} - {post_shortcode}" +ITEM_STEM = DATE_FMT + "_{username} - {shortcode}" + + +@dataclass +class Source: + """One gallery-dl invocation: a URL fetched into a specific directory.""" + + kind: str # posts | reels | stories | highlights + url: str + directory: str # relative to the archives root + subcategory: str # gallery-dl config key + title: str | None = None # highlight title, when known + + +@dataclass +class Profile: + user: str + existing: dict[str, str] = field(default_factory=dict) # kind -> dirname + + def sources(self, include_stories: bool) -> list[Source]: + u = self.user + base = f"https://www.instagram.com/{u}" + out = [ + Source("posts", f"{base}/posts/", u, "posts"), + Source("reels", f"{base}/reels/", f"{u} - reels", "reels"), + ] + if include_stories: + # Stories expire after 24h, so these can only ever be captured + # live. There is no backfill and no re-fetch. + out.append(Source( + "stories", f"https://www.instagram.com/stories/{u}/", + f"story - {u}", "stories")) + # Highlight directories embed the title, which gallery-dl only + # learns mid-extraction -- so this one source fans out into many + # directories and is handled with a directory format string. + out.append(Source( + "highlights", f"{base}/highlights", "", "highlights")) + return out + + +def scan_archives(root: Path) -> dict[str, Profile]: + """Group existing directories into profiles, as the server does.""" + profiles: dict[str, Profile] = {} + + def get(user: str) -> Profile: + return profiles.setdefault(user, Profile(user)) + + for entry in sorted(os.listdir(root)): + if not (root / entry).is_dir() or entry.startswith("."): + continue + if m := RE_HIGHLIGHT.match(entry): + get(m.group(1)).existing.setdefault("highlights", entry) + elif m := RE_STORIES.match(entry): + get(m.group(1)).existing["stories"] = entry + elif m := RE_REELS.match(entry): + get(m.group(1)).existing["reels"] = entry + else: + get(entry).existing["posts"] = entry + return profiles + + +# -------------------------------------------------------------------------- +# gallery-dl configuration +# -------------------------------------------------------------------------- + +def build_config(rate: str, sleep_request: list[float], + sleep: list[float]) -> dict: + """ + The config is generated rather than checked in so the safety-critical + options cannot drift out of sync with the docs. + + `api: rest` is the single most important line in this file. The graphql + backend issues one request PER POST for every video and carousel, which is + the pattern that got this account banned once already. + """ + caption_pp = { + "name": "metadata", + "event": "post", + "mode": "custom", + "content-format": "{description}", + "extension": "txt", + # JD2 wrote no .txt when the caption was empty; "empty": false (the + # default) reproduces that. + } + meta_pp = { + "name": "metadata", + "event": "post", + "mode": "json", + # `include`, NOT `fields` -- `fields` applies to mode:custom and + # silently does nothing here, dumping audio_user blobs that contain + # unrelated users' profile picture URLs. + "include": [ + "post_shortcode", "post_id", "type", "date", "post_date", + "username", "fullname", "owner_id", "description", "count", + "likes", "post_url", "sidecar_shortcode", + ], + } + + def post_like(stem: str) -> dict: + """Naming for surfaces whose unit is a post (posts, reels).""" + return { + # `sidecar_shortcode` is set only for carousels, so it is the + # carousel discriminator. First matching condition wins. + "filename": { + "sidecar_shortcode and count >= 10": + stem + " - {num:02}.{extension}", + "sidecar_shortcode": + stem + " - {num}.{extension}", + "": + stem + ".{extension}", + }, + "postprocessors": [ + {**caption_pp, "filename": stem + ".txt"}, + {**meta_pp, "filename": stem + ".json"}, + ], + } + + def item_like(stem: str) -> dict: + """ + Naming for surfaces whose unit is an item inside a reel (stories, + highlights). `{shortcode}` is per item; `{post_shortcode}` is the + reel's id and is shared by every item in it. + """ + return { + "filename": stem + ".{extension}", + "postprocessors": [{**meta_pp, "filename": stem + ".json"}], + } + + return { + "extractor": { + "base-directory": ".", + "instagram": { + "api": "rest", # never "graphql" -- see docstring + "sleep-request": sleep_request, + "sleep": sleep, + "videos": True, + "include": "", # never "all"; sources are explicit + # Directory is forced per-invocation with -D, because a reels + # tab returns collab reels owned by OTHER accounts and + # {username} would scatter them into the wrong profile. + "directory": [], + "posts": post_like(POST_STEM), + "reels": post_like(POST_STEM), + "stories": item_like(ITEM_STEM), + "highlights": { + **item_like(ITEM_STEM), + # The only surface that must derive its own directory, + # since the title is not known until extraction. + "directory": ["story highlights - {username} - {highlight_title}"], + }, + }, + }, + "downloader": {"http": {"rate": rate}}, + "output": {"mode": "null"}, + } + + +# -------------------------------------------------------------------------- +# Planning and execution +# -------------------------------------------------------------------------- + +def gdl_command(src: Source, root: Path, config: Path, cookies: str, + archive_db: Path | None) -> list[str]: + cmd = [ + "gallery-dl", + "--config", str(config), + "--cookies-from-browser", cookies, + ] + if archive_db: + # gallery-dl's skip-archive. Seed it before the first real run or the + # whole 110k-file tree is re-downloaded; see --seed-archive. + cmd += ["--download-archive", str(archive_db)] + if src.subcategory != "highlights": + cmd += ["--destination", str(root / src.directory)] + else: + cmd += ["--destination", str(root)] + cmd.append(src.url) + return cmd + + +def seed_archive_db(root: Path, profiles: dict[str, Profile], + db: Path) -> int: + """ + TODO: populate the skip-archive from filenames already on disk. + + gallery-dl keys its archive on an extractor-specific id string, so this + needs the exact format `InstagramExtractor.archive_fmt` produces. Until + it is implemented, the first run of any profile re-downloads everything — + bandwidth on the CDN (the tolerant surface), but hours of it. + """ + raise NotImplementedError( + "seed_archive_db is unimplemented; run without --archive-db and accept " + "a full re-download, or implement this first") + + +def main() -> int: + ap = argparse.ArgumentParser(description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter) + ap.add_argument("--archives", required=True, type=Path, + help="archive root (one directory per profile plus sidecars)") + g = ap.add_mutually_exclusive_group(required=True) + g.add_argument("--profile", action="append", default=[], + help="profile to sync; repeatable") + g.add_argument("--all", action="store_true", help="every profile on disk") + ap.add_argument("--cookies", default="chrome:/home/matt/.config/google-chrome-devtools", + help="gallery-dl --cookies-from-browser value") + ap.add_argument("--archive-db", type=Path, default=None, + help="gallery-dl skip-archive sqlite path") + ap.add_argument("--rate", default="3M", help="per-download rate cap") + ap.add_argument("--sleep-request", nargs=2, type=float, default=[4.0, 7.0], + metavar=("MIN", "MAX")) + ap.add_argument("--sleep", nargs=2, type=float, default=[1.0, 3.0], + metavar=("MIN", "MAX")) + ap.add_argument("--no-stories", action="store_true", + help="skip stories and highlights (posts and reels only)") + mode = ap.add_mutually_exclusive_group() + mode.add_argument("--dry-run", action="store_true", default=True, + help="print the plan and the config; default") + mode.add_argument("--execute", action="store_true", + help="actually run gallery-dl") + args = ap.parse_args() + + if not args.archives.is_dir(): + print(f"archives root not found: {args.archives}", file=sys.stderr) + return 2 + if not shutil.which("gallery-dl"): + print("gallery-dl not on PATH", file=sys.stderr) + return 2 + + profiles = scan_archives(args.archives) + if args.profile: + missing = [p for p in args.profile if p not in profiles] + for p in missing: + print(f"note: {p} has no directory yet; it will be created") + profiles.setdefault(p, Profile(p)) + selected = [profiles[p] for p in args.profile] + else: + selected = list(profiles.values()) + + config = build_config(args.rate, list(args.sleep_request), list(args.sleep)) + config_path = args.archives / ".gdl-sync.config.json" + + plan: list[tuple[Profile, Source]] = [ + (prof, src) + for prof in selected + for src in prof.sources(include_stories=not args.no_stories) + ] + + print(f"profiles : {len(selected)}") + print(f"sources : {len(plan)}") + print(f"pacing : {args.sleep_request[0]}-{args.sleep_request[1]}s between " + f"requests, rate cap {args.rate}") + print() + + if not args.execute: + print(json.dumps(config, indent=2)) + print() + for prof, src in plan: + dest = src.directory or "(per-highlight)" + print(f" {prof.user:<20} {src.kind:<11} -> {dest}") + print("\ndry run; nothing fetched. pass --execute to run.") + return 0 + + config_path.write_text(json.dumps(config, indent=2)) + failures = 0 + for prof, src in plan: + if src.subcategory != "highlights": + (args.archives / src.directory).mkdir(parents=True, exist_ok=True) + cmd = gdl_command(src, args.archives, config_path, args.cookies, + args.archive_db) + print(f"==> {prof.user} / {src.kind}") + result = subprocess.run(cmd) + if result.returncode != 0: + failures += 1 + # Keep going: one private/renamed profile must not abort the run. + print(f" FAILED (exit {result.returncode})", file=sys.stderr) + + # A sync NEVER deletes. The archive deliberately outlives Instagram -- + # 2 posts in 0ct0ber19 exist only here now. + print(f"\ndone; {failures} source(s) failed") + return 1 if failures else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/src/lib/archive-patterns.test.ts b/src/lib/archive-patterns.test.ts index 6d4e32e..76e0b5a 100644 --- a/src/lib/archive-patterns.test.ts +++ b/src/lib/archive-patterns.test.ts @@ -116,3 +116,43 @@ describe('scopedPostId', () => { expect(inPosts).not.toBe(inHighlight); }); }); + +/** + * gallery-dl is replacing JDownloader as the fetcher (docs/gallery-dl.md). + * Its naming differs cosmetically, and these cases pin down that the two + * interoperate so a mixed archive parses identically. + */ +describe('gallery-dl / JDownloader naming interop', () => { + it('treats a single-media post the same with or without an index', () => { + const jd2 = parseArchiveFilename('2023-04-19_0ct0ber19 - CrORBIcJJbM.mp4')!; + const gdl = parseArchiveFilename('2023-04-19_0ct0ber19 - CrORBIcJJbM - 1.mp4')!; + expect(jd2.postId).toBe(gdl.postId); + expect(jd2.index).toBe(gdl.index); + expect(jd2.index).toBe(1); + }); + + it('normalises zero-padded carousel indices', () => { + // JD2 pads to the width of the media count (10+ items -> "01"), and + // gallery-dl's count can be one higher, so the same post may be padded + // by one tool and not the other. + expect(parseArchiveFilename('2024-04-17_0ct0ber19 - C53YPQzp7Wj - 09.jpg')!.index).toBe(9); + expect(parseArchiveFilename('2024-04-17_0ct0ber19 - C53YPQzp7Wj - 9.jpg')!.index).toBe(9); + expect(parseArchiveFilename('2023-11-03_0ct0ber19 - CzM8Uf6B6H_ - 01.jpg')!.index).toBe(1); + }); + + it('reads a gallery-dl story name, which carries a per-item shortcode', () => { + const p = parseArchiveFilename('2026-08-16_official_artms - DcF9OyhBJ1H.jpg', 'stories')!; + expect(p.postId).toBe('DcF9OyhBJ1H'); + expect(p.date).toBe('2026-08-16'); + }); + + it('gives a dated highlight a real date instead of the mtime fallback', () => { + const mtime = Date.parse('2026-08-17T00:00:00Z'); + const undated = parseArchiveFilename('0ct0ber19 - C-IImhvpFuk.jpg', 'highlight', mtime)!; + const dated = parseArchiveFilename('2024-08-04_0ct0ber19 - C-IImhvpFuk.jpg', 'highlight', mtime)!; + // Same item either way, so re-fetching cannot split it into two posts. + expect(dated.postId).toBe(undated.postId); + expect(undated.date).toBe('2026-08-17'); + expect(dated.date).toBe('2024-08-04'); + }); +});