diff --git a/CLAUDE.md b/CLAUDE.md index 30e4d57..d057716 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,63 +4,96 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## Project Overview -InstaArchive Viewer is a React 19 + Vite 6 PWA for browsing archived Instagram data (both official Instagram exports and Instaloader archives). All archive parsing and media processing happens client-side in the browser — the Express backend only lists/serves files from disk, it never parses archive contents. +InstaArchive Viewer is a React 19 + Vite 6 PWA for browsing archived Instagram data (official Instagram exports and Instaloader archives). All archive *parsing* happens client-side in the browser; the Express backend only indexes and serves files from disk, and never parses archive contents. ## Commands - `npm install` — install dependencies -- `npm run dev` — start Vite dev server on port 3000 (proxies `/api` and `/archives` to `http://localhost:3001`) -- `npm run server` — start the Express backend (`tsx server.ts`) on port 3001, serving archives from `ARCHIVES_DIR` (defaults to `./_sample-archives`) -- `npm run build` — build frontend to `dist/` (`vite build`) and backend to `dist-server/` (`tsc server.ts ...`) -- `npm run lint` — type-check only (`tsc --noEmit`); there is no separate test suite or linter config -- `npm run clean` — remove `dist/` +- `npm run dev` — Vite dev server on port 3000 (proxies `/api` and `/archives` to `http://localhost:3001`) +- `npm run server` — Express backend (`tsx server.ts`) on port 3001, serving `ARCHIVES_DIR` (defaults to `./_sample-archives`) +- `npm run build` — frontend to `dist/`, backend to `dist-server/` +- `npm run lint` — type-check only (`tsc --noEmit`) +- `npm test` / `npm run test:watch` — vitest +- `npx vitest run src/lib/archive-patterns.test.ts` — a single test file -For local development you typically need both `npm run dev` and `npm run server` running concurrently — the frontend alone has nothing to talk to for server-mode archives (local-folder mode works without the backend). +Local development usually needs both `npm run dev` and `npm run server`. Local-folder mode works without the backend; server-mode archives do not. ## Architecture ### Two archive sources, one data model -The app supports loading archives two ways, unified behind the `ArchiveFile` interface (`src/types/index.ts`, implementations in `src/lib/archive-files.ts`): +Loading is unified behind the `ArchiveFile` interface (`src/types/index.ts`, implementations in `src/lib/archive-files.ts`): -- **`LocalArchiveFile`** — wraps a browser `File` from a local folder picker (`webkitdirectory`). Fully offline, media is never uploaded anywhere. -- **`RemoteArchiveFile`** — wraps a file served from the Express backend's `/archives/:name/...` static route, fetched on demand. +- **`LocalArchiveFile`** — wraps a browser `File`. `createObjectUrl()` mints a **disk-backed** blob URL directly from the File; never route media through `arrayBuffer()`, which pulls whole files into memory. +- **`RemoteArchiveFile`** — wraps a file served from `/archives/...`, fetched on demand. -All downstream parsing code (`useArchiveScanner`) operates only on `ArchiveFile[]` and doesn't care which backing implementation it got. +`revocable` tells callers whether the returned URL must be revoked. The scanner tracks every minted URL and releases them on archive teardown. + +### Sidecar directories + +An archive root holds one directory per profile plus *sidecars* that belong to it: + +``` +4utumn07 -> posts (base) +4utumn07 - reels -> reels +story - 4utumn07 -> stories +story highlights - 4utumn07 - Sunstory -> highlight "Sunstory" +``` + +`src/lib/archive-grouping.ts` (shared by server and tests) folds these into a single profile with a `sources` list. Sidecars never appear as standalone archives. Each file the server returns carries its `kind`, so the client routes posts / reels / story ring / highlight circles without re-deriving naming rules. + +### Server-side archive index (`src/lib/archive-index.ts`) + +**Do not reintroduce per-request filesystem walks.** Archives typically live on network storage where per-file `stat` costs ~1.4ms and does not parallelise; a naive walk of a 110k-file root took ~52s per listing. Instead: + +- Each source directory is indexed once and cached, keyed by its **directory mtime** (a directory `stat` is effectively free). +- The index is warmed in the background at startup and persisted to `CACHE_DIR` (mount a volume at `/cache`). +- `GET /api/archives` does no file walking at all — it returns directory-mtime `signature`s, which the client uses for cache invalidation instead of a file count. +- Only media files are stat'd (for `size`, which gates thumbnailing) and only highlights need `mtime` (their filenames carry no date). ### Scanning pipeline (`src/hooks/useArchiveScanner.ts`) -This is the core of the app — a single large `handleFiles` function that: -1. **Indexes** all files, detecting archive format by filename regex: Instagram "export" format (`YYYY-MM-DD_user - post_id[- idx][- story].ext`), Instaloader format (`YYYY-MM-DD_HH-MM-SS_UTC[_idx][_story].ext`), or a generic JSON-manifest format (`posts_1.json`, `reels_1.json`, `stories_1.json`, possibly `.json.xz`-compressed via `xz-decompress`). -2. **Parses** according to detected format, building a `Map>`. For JSON-manifest format, media files are matched to JSON entries by URI substring match, then by ID substring match, then by filename-derived heuristic — in that fallback order. -3. **Falls back** to generic filename-prefix grouping when no posts were found via regex/JSON matching (treats files sharing a common basename as one carousel post, chunked into groups of 20). -4. Detects a **profile picture** from `*_profile_pic.jpg` / `.jpg` files, or falls back to the oldest image in the archive by filename sort ("Smart Fallback"). -5. **Caches** the final `{ posts, stories, profileMetadata, ... }` result to IndexedDB via `idb-keyval`, keyed by archive name (or `local_archive` for unnamed local folders) — this is what makes repeat visits load instantly. Both server and local archives are cached; the cache shape is documented inline in `useArchiveScanner`'s state (mirrors the `CacheData` interface in `GEMINI.md`). +`handleFiles` indexes files, detects format, then parses via one of three largely independent paths that all write into a shared `postsMap`: -When modifying format-detection or media-matching logic, be aware the three code paths (JSON-manifest, filename-regex export/instaloader, generic fallback) are largely independent and a change to one rarely needs to touch the others — but all three write into the same `postsMap`. +1. **JSON manifest** (`posts_1.json`, possibly `.json.xz` via `xz-decompress`) — media matched to entries by URI, then ID, then filename heuristic, in that fallback order. +2. **Filename patterns** — see `src/lib/archive-patterns.ts` for the export / Instaloader / highlight regexes, extracted as pure functions and covered by tests. Prefer changing them there. +3. **Generic grouping fallback** — when nothing else matched, files sharing a basename become one carousel. -### Thumbnail generation (`src/hooks/useThumbnailQueue.ts` + `src/lib/thumbnail-worker.ts`) +Results are cached to IndexedDB. Media records store a stable `path`; **`url` is not persistable** for local archives because blob URLs die with the document. -High-res images (>1MiB) are downscaled off the main thread: -- `useThumbnailQueue` maintains a **serial** (one-at-a-time) queue — this is deliberate, not a bug: decoding multiple 50MP+ images concurrently causes OOM crashes in the browser. -- Actual resizing happens in `thumbnail-worker.ts` using `OffscreenCanvas`/`createImageBitmap` inside a Web Worker. -- Results are cached in IndexedDB under a `thumb_` key, checked before falling back to the worker, so thumbnails persist across sessions. +### Cache and local-archive persistence (`src/lib/archive-cache.ts`) -### URL state sync (`src/App.tsx`) +IndexedDB keys are namespaced (`archive:`, `thumb:`, `handle:`) so listing archives does not deserialize every cached thumbnail blob, and thumbnails are scoped per archive to avoid cross-archive collisions. -App state (selected archive, active tab, selected post) is synchronized with URL query params (`?a=`, `?t=`, `?p=`) via `URLSearchParams` + `window.history.replaceState` in a cluster of `useEffect` hooks — this is what enables permalinks/deep-linking. When adding new shareable state, follow this pattern rather than introducing a router. +Restoring an archive **rehydrates URLs from `path`**: server archives rebuild HTTP URLs; local archives re-open a persisted `FileSystemDirectoryHandle` and mint fresh blob URLs. If the folder is unreachable (permission lapsed, or the browser lacks `showDirectoryPicker` — Firefox/Safari), the app re-prompts rather than rendering broken images. + +### Thumbnails (`src/hooks/useThumbnailQueue.ts` + `src/lib/thumbnail-worker.ts`) + +Images over 1MiB are downscaled in a Web Worker via `OffscreenCanvas`. The queue is **serial on purpose** — decoding several 50MP+ images at once OOMs the tab. `requestThumbnail` must keep a stable identity (it reads cache state through a ref), or every completed thumbnail re-runs the effect in all mounted thumbnails. + +### URL state (`src/App.tsx`) + +App state syncs to `?a=` / `?t=` / `?p=`. Two rules, both learned from real bugs: + +- The initial query string is captured into a ref on first render; the URL is rewritten from state as soon as anything loads, so reading `window.location` later sees the rewrite, not the user's link. +- URL writing is gated on `hasInitialLoaded`, otherwise it erases the deep link before the loader consumes it. + +Deep-link resolution waits on the archive fetch having *settled*, not on `isServerMode` (which is still false while in flight). ### Backend (`server.ts`) -Minimal Express server, three responsibilities only: -- `GET /api/archives` — lists subdirectories of `ARCHIVES_DIR` (skipping dotfiles/`@`/`_`-prefixed dirs) as `ServerArchive[]`, guessing a thumbnail per archive. -- `GET /api/archives/:name/files` — recursively lists all files in one archive directory. -- Static-serves `ARCHIVES_DIR` under `/archives` and, in production, serves the built `dist/` frontend with an SPA fallback. +Serves `/api/archives`, `/api/archives/:name/files`, static `/archives`, and the built SPA. Notes: -It does no parsing of archive/JSON contents — that's entirely client-side in `useArchiveScanner`. `ARCHIVES_DIR` is resolved from the `ARCHIVES_DIR` env var (see `.env` / Docker volume mount at `/archives`). +- Express decodes route params **after** segment matching, so `..%2f` reaches the handler as `../`. All user-supplied archive names go through `resolveArchivePath`. +- Sets CSP and related security headers. The CSP allows `blob:`/`data:` for media and `unsafe-inline` styles (the animation library sets inline styles); scripts stay same-origin only. +- `os.userInfo()` throws for a UID with no `/etc/passwd` entry, which is what `--user 1234:1234` produces — use `describeUser()`. + +### Deployment + +Container runs as non-root. The image defaults to `node`, but the archive share must be *listable* by that UID — a mode-711 share owned by another account needs `user: ":"` in compose. Mount a volume at `/cache` so the index survives restarts. ### PWA / build quirks -- `vite.config.ts` sets `hmr: process.env.DISABLE_HMR !== 'true'` — this is intentionally left alone; it exists to disable file-watch flicker when running under AI Studio-style agent editing. Don't "clean up" or remove it. -- `workbox.navigateFallbackDenylist` excludes `/api` and `/archives` from the SPA fallback so those routes hit the real server/static files instead of `index.html` (needed for "open original file in new tab"). -- Service worker uses `registerType: 'autoUpdate'` with hourly periodic checks — new deployments propagate to open clients automatically. +- `vite.config.ts` sets `hmr: process.env.DISABLE_HMR !== 'true'` — intentional, leave it. +- `workbox.navigateFallbackDenylist` excludes `/api` and `/archives` so those hit the real server. +- Fonts and icons are vendored in `public/` — do not reintroduce CDN references; the app advertises offline support and local-only processing.