main's history was redacted on 2026-08-20: every archived account name was
replaced throughout all 65 commits, because main is published to GitHub and
the list of accounts identifies the archive as surely as a hostname does. The
viewer's test fixtures now use synthetic names chosen to preserve exactly what
each case tests — a leading digit, an underscore, a dotted username, a
bracketed or dotted highlight title, and the trailing U+2800 that case 12
exists for.
The redaction swept these three docs too, since they were in the same history.
This puts the real names back, on the branch that is not published. The
scripts needed nothing: they are generic and never named an account.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UXfdJu7QhSJLr47K7koTDF
A superset branch only stays cheap if the files it shares with main are
untouched. Both diverged: package.json carried a jd2 script and CLAUDE.md
described the tooling, so every merge from main would have conflicted on them
forever, and main's removal commit would have kept trying to delete the
tooling entries here.
The tooling files themselves do not exist on main, so they can hold whatever
they like. The jd2 invocation moves into docs/jdownloader.md as a direct npx
call instead of an npm script.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UXfdJu7QhSJLr47K7koTDF
The scripts and docs for fetching from Instagram now live here rather than on
main, which is the branch published to GitHub. They carry things that do not
belong in a public repo: the fetch host's public IP, the browser profile path
the cookie is read from, the NAS archive path, and the list of accounts being
archived.
This branch is a superset of main — the viewer plus the tooling — so it can
take main's changes by merging, and the npm script and CLAUDE.md entries that
reference the tooling live here where the files actually exist.
Restored with the sync work from the 2026-08-20 run already in place: the
--abort flag, the corrected yt-dlp install advice, and the measurements behind
both.
Note that main's history was rewritten to strip these paths, so the tooling's
own per-file history does not exist on this branch. It is preserved on gitea
as pre-rewrite-20260820 and pre-rewrite-tooling-20260820.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UXfdJu7QhSJLr47K7koTDF
The fetching scripts and their docs now live on the `tooling` branch, which
is not published to GitHub. This removes the two references that would
otherwise dangle here: the `jd2` npm script and the CLAUDE.md bullet
describing it.
The viewer's own gallery-dl support is untouched and stays here —
src/lib/gallery-dl-sidecar.ts and friends parse sidecars at display time and
are app code, not tooling.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UXfdJu7QhSJLr47K7koTDF
Two .pyc files under scripts/__pycache__ were committed at some point and have
been churning ever since — merely importing gdl-sync.py to check a config
rewrites them and dirties the tree, which is how they surfaced.
.gitignore had no Python entries at all, only Node ones. The files stay on
disk; this just untracks them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UXfdJu7QhSJLr47K7koTDF
Docker Build and Publish / build-and-push (push) Failing after 10s
Ships the gallery-dl sidecar work to the viewer. The visible change is
reel classification: official_artms' Reels tab drops from 781 items to
360, because the sidecars say the other 421 are ordinary feed videos the
clips endpoint returns via include_feed_video. Directory-based
classification counted them all as reels.
Also in this release: dates ranked by source rather than scan order, and
highlight items no longer appearing twice when the archive holds them
under both JDownloader naming conventions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
JDownloader wrote story-shaped names for highlights during one period of
its life, so the same item exists on disk as both
0ct0ber19 - C5dQPEYpd9W.mp4
2024-04-07_0ct0ber19 - 01 - C5dQPEYpd9W.mp4
which parsed to the ids "C5dQPEYpd9W" and "01 - C5dQPEYpd9W" -- two posts
for one item. The leading ordinal is a position within a day's stories
and carries nothing the shortcode does not, so story and highlight ids
drop it. Post ids are untouched, since those are permalinks.
Measured on the two real files, same archive, cache cleared between:
without the fix the profile reads "Heestory - 2 items", with it
"Heestory - 1 item".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The previous commit had this backwards: the sidecar date was only
consulted when the existing date came from an mtime, so a filename date
silently outranked what Instagram itself reported.
The order is sidecar, then filename, then mtime -- metadata first,
mtime last, since mtime is when the file hit disk and says nothing about
when the post was made. Ties keep the incumbent so two equally
authoritative files cannot flip a post's date by scan order.
Extracted to src/lib/post-dates.ts rather than left inline, because the
rule is easy to state and easy to get wrong -- the tests include an
order-independence case that would have caught the original mistake.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The .json sidecars published with the ARTMS fetch were inert: the scanner
fed them through the Instaloader path, where `node.edge_media_to_caption`
and `checkIsStory`'s `product_type` are both absent, so nothing happened.
They are now recognised structurally -- flat, with post_shortcode and
type, and none of the markers the other two JSON shapes carry -- and used
for three things:
- `type` sets post.isReel, which post-tabs prefers over every fallback.
This is Instagram's own classification and it disagrees with ours a
lot: of 781 items in "official_artms - reels", the sidecars say only
360 are reels. The other 421 are feed videos the clips endpoint returns
via include_feed_video, and the directory-based rule counted them all.
- `description` fills the caption where no .txt exists.
- `date` dates a post whose filename could not.
Also fixes date precedence. Only JDownloader highlights lack a date in
the filename, so parseArchiveFilename now marks those as mtime-derived
and the scanner lets any real date replace them -- previously the date
depended on which file the scan reached first.
Verified against real published files: a directory of three type=post and
three type=reel renders 6 in the grid and exactly the 3 reels in the
Reels tab.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every claim in docs/gallery-dl.md was measured against the live site and
the archive rather than taken from documentation, because two of the
assumptions turned out to be wrong.
The safety model is the reason the config looks the way it does.
gallery-dl has two API backends: the graphql one issues a request PER
POST for every video and carousel -- the pattern that got this account
banned via Instaloader -- while the default rest one paginates listings
at 30-50 items and carries carousel_media, video_versions and
product_type inline. A 300-post profile costs ~10 requests.
Findings worth recording:
- JD2 stamped filenames in desktop LOCAL time (US Eastern), not UTC.
Across 212 comparable posts: UTC 19 mismatches, UTC-5 10, UTC-4 zero.
{date:Olocal/%Y-%m-%d} reproduces it; the trailing separator must be
omitted or it lands in the strftime format.
- A profile's reels tab returns collab reels owned by OTHER accounts, so
the directory must be forced with -D. JD2 did the same: chuuo3o and
official_artms filenames sit inside "0ct0ber19 - reels".
- Stories and highlights need per-item {shortcode}; {post_shortcode} is
the reel's id and is shared by every item. {date} is per-item, verified
on a 154-item highlight with distinct times.
- gallery-dl reproduces JD2's caption .txt exactly, including writing
nothing for an empty caption and omitting the trailing newline.
- The json sidecar needs `include`, not `fields`; `fields` silently does
nothing in mode:json and leaks audio_user blobs. It yields `type`
(post/reel) -- Instagram's own flag, which can retire the lone-video
heuristic once the scanner reads it.
Naming differences between the two tools are cosmetic: EXPORT_RE already
makes the index optional and parseInt normalises zero-padding, so a mixed
archive parses identically. Tests pin that down.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Docker Build and Publish / build-and-push (push) Failing after 10s
The header rendered allPosts.length, which is pre-dedupe — 0ct0ber19
showed "303 posts" over a 300-tile grid. Instagram's counter equals its
grid, so count the grid.
Also correct CLAUDE.md. v1.7.0 claimed the grid holds everything "as on
Instagram"; Instagram actually includes a reel in the grid only when the
creator shared it to feed, per post. Measured live: official_artms has
21 reels in its first 34 grid tiles, 0ct0ber19 has 1 in 214. Archives
carry no such flag, so showing everything approximates the behaviour
rather than reproducing it.
Records the DOM trap that caused the wrong reading in the first place:
grid reels link to /reel/<code>/, not /<user>/p/<code>/.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Docker Build and Publish / build-and-push (push) Failing after 9s
The Posts tab filtered reels out, so the grid was not the archive — it
was the archive minus its videos. For `for.heejin` that hid 533 of 1225
posts; for `loonatheworld`, 1100 of 3813. On Instagram the grid holds
everything and the Reels tab is a filtered view of that same set.
Extract the tab logic to src/lib/post-tabs.ts so the reel heuristic is
testable outside the component, and add dedupePostCopies: the jd2 flow
crawls the profile URL and the /reels URL separately because the profile
page misses some reels, so the two overlap and a reel can land on disk
twice. Those are two posts with distinct directory-scoped ids, which the
grid would now render side by side; the reels-source copy wins so the
survivor is still recognised as a reel.
Deciding what *is* a reel stays a guess for most archives. Instagram
marks it with product_type ("clips" vs "feed" vs "igtv" — all three are
GraphVideo, and aspect ratio does not separate them), but only newer
Instaloader captures carry it: 1101 of gibiofficial's 5919 sidecars, and
only 2 marked clips. JDownloader archives carry none, so those still
fall back to treating a lone video as a reel.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ergosteurandClaude Opus 5