2 Commits

Include renames

Author	SHA1	Message	Date
ergosteurandClaude Opus 5	7b63ba6a76	docs: design a gallery-dl replacement for the JDownloader fetcher ... Every claim in docs/gallery-dl.md was measured against the live site and the archive rather than taken from documentation, because two of the assumptions turned out to be wrong. The safety model is the reason the config looks the way it does. gallery-dl has two API backends: the graphql one issues a request PER POST for every video and carousel -- the pattern that got this account banned via Instaloader -- while the default rest one paginates listings at 30-50 items and carries carousel_media, video_versions and product_type inline. A 300-post profile costs ~10 requests. Findings worth recording: - JD2 stamped filenames in desktop LOCAL time (US Eastern), not UTC. Across 212 comparable posts: UTC 19 mismatches, UTC-5 10, UTC-4 zero. {date:Olocal/%Y-%m-%d} reproduces it; the trailing separator must be omitted or it lands in the strftime format. - A profile's reels tab returns collab reels owned by OTHER accounts, so the directory must be forced with -D. JD2 did the same: chuuo3o and official_artms filenames sit inside "0ct0ber19 - reels". - Stories and highlights need per-item {shortcode}; {post_shortcode} is the reel's id and is shared by every item. {date} is per-item, verified on a 154-item highlight with distinct times. - gallery-dl reproduces JD2's caption .txt exactly, including writing nothing for an empty caption and omitting the trailing newline. - The json sidecar needs `include`, not `fields`; `fields` silently does nothing in mode:json and leaks audio_user blobs. It yields `type` (post/reel) -- Instagram's own flag, which can retire the lone-video heuristic once the scanner reads it. Naming differences between the two tools are cosmetic: EXPORT_RE already makes the index optional and parseInt normalises zero-padding, so a mixed archive parses identically. Tests pin that down. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-16 21:04:39 -04:00
ergosteurandClaude Opus 5	1b4aba54d6	fix: security, performance and correctness pass; add sidecar archive support ... Security - Fix path traversal in GET /api/archives/:name/files. Express decodes route params after segment matching, so `..%2f..%2fetc` escaped ARCHIVES_DIR and returned a recursive listing of arbitrary directories. - Add CSP and baseline security headers; disable x-powered-by. - Stop baking GEMINI_API_KEY into the client bundle (the SDK was unused). - Run the container as `node` instead of root. Performance - Add a directory-mtime-keyed archive index, warmed in the background and persisted. Listing 110k files went from ~52s to ~0.1s; the largest archive (24k files) serves in ~0.3s. Per-file stat over CIFS costs ~1.4ms and does not parallelise, so it is now done once rather than per request. - Build media URLs from the File directly instead of `new Blob([await file.arrayBuffer()])`, which read every media file fully into memory (a 20GB archive tried to become 20GB of resident blobs). - Track and revoke object URLs; previously none were ever revoked. - Give `requestThumbnail` a stable identity so a completed thumbnail stops re-running the effect in every mounted thumbnail. - Namespace IndexedDB keys so listing archives no longer deserializes every cached thumbnail blob, and thumbnails no longer collide across archives. - Serve real file sizes: RemoteArchiveFile was constructed with size 0, which silently disabled high-res thumbnailing for every server archive. Correctness - Local archives cached media as blob: URLs, which die with the document, so a cached local archive restored as an archive of broken images. Media now carries a stable path and is rehydrated from a persisted directory handle (File System Access API), falling back to re-prompting for the folder. - Fix permalinks: the URL-writing effect erased ?a= on mount before the archive list arrived to consume it, so deep links never resolved. - Make cache invalidation detect nested changes via a directory signature. - Add an error boundary and tolerate unparseable dates, which previously threw a RangeError and blanked the app. - Default video to muted so autoplay is not blocked by Safari/Firefox. Features - Fold sidecar directories into their base profile: `<user> - reels`, `story - <user>` and `story highlights - <user> - <title>` now appear as reels, the story ring and Instagram-style highlight circles rather than as separate archives. Housekeeping - Add @types/react; React was previously type-checked against its JavaScript source, so `npm run lint` gave almost no type safety on components. - Vendor fonts and PWA icons locally; the app made third-party CDN requests despite advertising offline support and local-only processing. - Drop unused better-sqlite3 (a native module that broke `npm install`). - Add vitest with 36 tests over the filename and directory-naming rules. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011uBWhwV3wFQ5MBCcMHHem7	2026-08-14 01:59:19 -04:00