Two additions to the metadata sidecars, following up on the 2026-09-01
collab/dedup investigation already documented here:
- coauthors added to the post-level JSON's include list -- a direct,
native signal for collab detection instead of inferring it from the
filename/directory identity mismatch also documented that day.
- width, height, width_original, height_original and tagged_users get
their own per-file sidecar (event: "file", one JSON per downloaded
media item, named "<filename>.json"), since a carousel's items can
each have different dimensions and tags -- data the existing
post-level JSON has no way to represent. owner is deliberately left
out, same reasoning as audio_user's existing exclusion.
Verified against a live re-fetch of an already-archived carousel:
correct per-item data came back, zero media re-downloaded (skip-archive
still applies; only the new sidecars are new files). 3 new tests.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011qAds5qr7nZRq5R4yAuxUk
Three findings from the 2026-09-01 collab-detection/VP9 investigation in
Cosmo-Live-Downloads, all rooted in gallery-dl's own scraping output:
- Collab posts' JSON is filed under the true owner's identity but the
media file under the scraped account's — plus coauthors is a native,
uncaptured signal that already excludes the owner for us.
- The shared archive-db dedups by media_id across all profiles, not per
profile, so a post reachable from two profiles' listings only gets its
media written into whichever profile's crawl reached it first (510 such
posts found archive-wide).
- Some video_versions entries are VP9; format selection is codec-blind,
and Safari's VP9 decode only wires into its WebM demuxer, never MP4.
Also commits the two saved raw API response examples and the missing-
captions CSV generated during that investigation.
Nothing here is broken — these are decisions not made and cleanups not done,
written down before the session's context is lost.
The two that can actually cost something: no daily stories run is scheduled,
and stories are the one surface that cannot be backfilled; and the `seeded`
flags in the state file were reconstructed by hand from a log rather than
derived from the archive DB, so losing artms.db without also clearing the
state file would leave those sources permanently unseeded and re-download
everything.
Also corrects "Scanner work (not done yet)", which shipped in 53b1f80 —
the three .json shapes are told apart structurally in gallery-dl-sidecar.ts
and isReel comes from the sidecar's type. That file lives on main: it parses
archives at display time and is viewer code, not tooling.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UXfdJu7QhSJLr47K7koTDF
main's history was redacted on 2026-08-20: every archived account name was
replaced throughout all 65 commits, because main is published to GitHub and
the list of accounts identifies the archive as surely as a hostname does. The
viewer's test fixtures now use synthetic names chosen to preserve exactly what
each case tests — a leading digit, an underscore, a dotted username, a
bracketed or dotted highlight title, and the trailing U+2800 that case 12
exists for.
The redaction swept these three docs too, since they were in the same history.
This puts the real names back, on the branch that is not published. The
scripts needed nothing: they are generic and never named an account.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UXfdJu7QhSJLr47K7koTDF
The scripts and docs for fetching from Instagram now live here rather than on
main, which is the branch published to GitHub. They carry things that do not
belong in a public repo: the fetch host's public IP, the browser profile path
the cookie is read from, the NAS archive path, and the list of accounts being
archived.
This branch is a superset of main — the viewer plus the tooling — so it can
take main's changes by merging, and the npm script and CLAUDE.md entries that
reference the tooling live here where the files actually exist.
Restored with the sync work from the 2026-08-20 run already in place: the
--abort flag, the corrected yt-dlp install advice, and the measurements behind
both.
Note that main's history was rewritten to strip these paths, so the tooling's
own per-file history does not exist on this branch. It is preserved on gitea
as pre-rewrite-20260820 and pre-rewrite-tooling-20260820.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UXfdJu7QhSJLr47K7koTDF
ergosteurandClaude Sonnet 5