Compare commits

...
5 Commits
Author SHA1 Message Date
ergosteurandClaude Opus 5 ab7099220a chore: bump version to 1.3.1
Docker Build and Publish / build-and-push (push) Failing after 10s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011uBWhwV3wFQ5MBCcMHHem7
2026-08-14 02:08:35 -04:00
ergosteurandClaude Opus 5 e6663657b0 fix: don't crash at boot when running under a UID with no passwd entry
os.userInfo() throws ERR_SYSTEM_ERROR (uv_os_get_passwd) for a UID that has no
/etc/passwd entry, which is exactly what `docker run --user 1234:1234` produces
— the very workaround the README recommends. Combined with the switch to a
non-root image user, this crashed the server on startup for any deployment that
needed a custom UID to read its archives.

Also document the non-root default and the /cache index volume.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011uBWhwV3wFQ5MBCcMHHem7
2026-08-14 02:08:35 -04:00
ergosteurandClaude Opus 5 24ff2727c8 Merge branch 'review-fixes': security, performance and sidecar archive support
Docker Build and Publish / build-and-push (push) Failing after 10s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011uBWhwV3wFQ5MBCcMHHem7
2026-08-14 01:59:41 -04:00
ergosteurandClaude Opus 5 f089e49e84 chore: bump version to 1.3.0
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011uBWhwV3wFQ5MBCcMHHem7
2026-08-14 01:59:41 -04:00
ergosteurandClaude Opus 5 1d86fa3583 fix: security, performance and correctness pass; add sidecar archive support
Security
- Fix path traversal in GET /api/archives/:name/files. Express decodes route
  params after segment matching, so `..%2f..%2fetc` escaped ARCHIVES_DIR and
  returned a recursive listing of arbitrary directories.
- Add CSP and baseline security headers; disable x-powered-by.
- Stop baking GEMINI_API_KEY into the client bundle (the SDK was unused).
- Run the container as `node` instead of root.

Performance
- Add a directory-mtime-keyed archive index, warmed in the background and
  persisted. Listing 110k files went from ~52s to ~0.1s; the largest archive
  (24k files) serves in ~0.3s. Per-file stat over CIFS costs ~1.4ms and does
  not parallelise, so it is now done once rather than per request.
- Build media URLs from the File directly instead of
  `new Blob([await file.arrayBuffer()])`, which read every media file fully
  into memory (a 20GB archive tried to become 20GB of resident blobs).
- Track and revoke object URLs; previously none were ever revoked.
- Give `requestThumbnail` a stable identity so a completed thumbnail stops
  re-running the effect in every mounted thumbnail.
- Namespace IndexedDB keys so listing archives no longer deserializes every
  cached thumbnail blob, and thumbnails no longer collide across archives.
- Serve real file sizes: RemoteArchiveFile was constructed with size 0, which
  silently disabled high-res thumbnailing for every server archive.

Correctness
- Local archives cached media as blob: URLs, which die with the document, so
  a cached local archive restored as an archive of broken images. Media now
  carries a stable path and is rehydrated from a persisted directory handle
  (File System Access API), falling back to re-prompting for the folder.
- Fix permalinks: the URL-writing effect erased ?a= on mount before the
  archive list arrived to consume it, so deep links never resolved.
- Make cache invalidation detect nested changes via a directory signature.
- Add an error boundary and tolerate unparseable dates, which previously
  threw a RangeError and blanked the app.
- Default video to muted so autoplay is not blocked by Safari/Firefox.

Features
- Fold sidecar directories into their base profile: `<user> - reels`,
  `story - <user>` and `story highlights - <user> - <title>` now appear as
  reels, the story ring and Instagram-style highlight circles rather than as
  separate archives.

Housekeeping
- Add @types/react; React was previously type-checked against its JavaScript
  source, so `npm run lint` gave almost no type safety on components.
- Vendor fonts and PWA icons locally; the app made third-party CDN requests
  despite advertising offline support and local-only processing.
- Drop unused better-sqlite3 (a native module that broke `npm install`).
- Add vitest with 36 tests over the filename and directory-naming rules.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011uBWhwV3wFQ5MBCcMHHem7
2026-08-14 01:59:19 -04:00
39 changed files with 2329 additions and 1643 deletions
+66
View File
@@ -0,0 +1,66 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
InstaArchive Viewer is a React 19 + Vite 6 PWA for browsing archived Instagram data (both official Instagram exports and Instaloader archives). All archive parsing and media processing happens client-side in the browser — the Express backend only lists/serves files from disk, it never parses archive contents.
## Commands
- `npm install` — install dependencies
- `npm run dev` — start Vite dev server on port 3000 (proxies `/api` and `/archives` to `http://localhost:3001`)
- `npm run server` — start the Express backend (`tsx server.ts`) on port 3001, serving archives from `ARCHIVES_DIR` (defaults to `./_sample-archives`)
- `npm run build` — build frontend to `dist/` (`vite build`) and backend to `dist-server/` (`tsc server.ts ...`)
- `npm run lint` — type-check only (`tsc --noEmit`); there is no separate test suite or linter config
- `npm run clean` — remove `dist/`
For local development you typically need both `npm run dev` and `npm run server` running concurrently — the frontend alone has nothing to talk to for server-mode archives (local-folder mode works without the backend).
## Architecture
### Two archive sources, one data model
The app supports loading archives two ways, unified behind the `ArchiveFile` interface (`src/types/index.ts`, implementations in `src/lib/archive-files.ts`):
- **`LocalArchiveFile`** — wraps a browser `File` from a local folder picker (`webkitdirectory`). Fully offline, media is never uploaded anywhere.
- **`RemoteArchiveFile`** — wraps a file served from the Express backend's `/archives/:name/...` static route, fetched on demand.
All downstream parsing code (`useArchiveScanner`) operates only on `ArchiveFile[]` and doesn't care which backing implementation it got.
### Scanning pipeline (`src/hooks/useArchiveScanner.ts`)
This is the core of the app — a single large `handleFiles` function that:
1. **Indexes** all files, detecting archive format by filename regex: Instagram "export" format (`YYYY-MM-DD_user - post_id[- idx][- story].ext`), Instaloader format (`YYYY-MM-DD_HH-MM-SS_UTC[_idx][_story].ext`), or a generic JSON-manifest format (`posts_1.json`, `reels_1.json`, `stories_1.json`, possibly `.json.xz`-compressed via `xz-decompress`).
2. **Parses** according to detected format, building a `Map<postId, Partial<Post>>`. For JSON-manifest format, media files are matched to JSON entries by URI substring match, then by ID substring match, then by filename-derived heuristic — in that fallback order.
3. **Falls back** to generic filename-prefix grouping when no posts were found via regex/JSON matching (treats files sharing a common basename as one carousel post, chunked into groups of 20).
4. Detects a **profile picture** from `*_profile_pic.jpg` / `<username>.jpg` files, or falls back to the oldest image in the archive by filename sort ("Smart Fallback").
5. **Caches** the final `{ posts, stories, profileMetadata, ... }` result to IndexedDB via `idb-keyval`, keyed by archive name (or `local_archive` for unnamed local folders) — this is what makes repeat visits load instantly. Both server and local archives are cached; the cache shape is documented inline in `useArchiveScanner`'s state (mirrors the `CacheData` interface in `GEMINI.md`).
When modifying format-detection or media-matching logic, be aware the three code paths (JSON-manifest, filename-regex export/instaloader, generic fallback) are largely independent and a change to one rarely needs to touch the others — but all three write into the same `postsMap`.
### Thumbnail generation (`src/hooks/useThumbnailQueue.ts` + `src/lib/thumbnail-worker.ts`)
High-res images (>1MiB) are downscaled off the main thread:
- `useThumbnailQueue` maintains a **serial** (one-at-a-time) queue — this is deliberate, not a bug: decoding multiple 50MP+ images concurrently causes OOM crashes in the browser.
- Actual resizing happens in `thumbnail-worker.ts` using `OffscreenCanvas`/`createImageBitmap` inside a Web Worker.
- Results are cached in IndexedDB under a `thumb_<id>` key, checked before falling back to the worker, so thumbnails persist across sessions.
### URL state sync (`src/App.tsx`)
App state (selected archive, active tab, selected post) is synchronized with URL query params (`?a=`, `?t=`, `?p=`) via `URLSearchParams` + `window.history.replaceState` in a cluster of `useEffect` hooks — this is what enables permalinks/deep-linking. When adding new shareable state, follow this pattern rather than introducing a router.
### Backend (`server.ts`)
Minimal Express server, three responsibilities only:
- `GET /api/archives` — lists subdirectories of `ARCHIVES_DIR` (skipping dotfiles/`@`/`_`-prefixed dirs) as `ServerArchive[]`, guessing a thumbnail per archive.
- `GET /api/archives/:name/files` — recursively lists all files in one archive directory.
- Static-serves `ARCHIVES_DIR` under `/archives` and, in production, serves the built `dist/` frontend with an SPA fallback.
It does no parsing of archive/JSON contents — that's entirely client-side in `useArchiveScanner`. `ARCHIVES_DIR` is resolved from the `ARCHIVES_DIR` env var (see `.env` / Docker volume mount at `/archives`).
### PWA / build quirks
- `vite.config.ts` sets `hmr: process.env.DISABLE_HMR !== 'true'` — this is intentionally left alone; it exists to disable file-watch flicker when running under AI Studio-style agent editing. Don't "clean up" or remove it.
- `workbox.navigateFallbackDenylist` excludes `/api` and `/archives` from the SPA fallback so those routes hit the real server/static files instead of `index.html` (needed for "open original file in new tab").
- Service worker uses `registerType: 'autoUpdate'` with hourly periodic checks — new deployments propagate to open clients automatically.
+11 -4
View File
@@ -18,6 +18,9 @@ FROM node:20-slim AS runtime
ENV NODE_ENV=production ENV NODE_ENV=production
ENV PORT=3000 ENV PORT=3000
ENV ARCHIVES_DIR=/archives ENV ARCHIVES_DIR=/archives
# Where the archive index is persisted; mount a volume here so a restart does
# not have to re-walk the whole archive root.
ENV CACHE_DIR=/cache
WORKDIR /app WORKDIR /app
@@ -25,12 +28,16 @@ WORKDIR /app
COPY package*.json ./ COPY package*.json ./
RUN npm ci --omit=dev RUN npm ci --omit=dev
# Copy built assets and server # Copy built assets and server. The whole dist-server tree is needed: the
# server imports shared archive-grouping logic emitted alongside it.
COPY --from=build /app/dist ./dist COPY --from=build /app/dist ./dist
COPY --from=build /app/dist-server/server.js ./server.js COPY --from=build /app/dist-server/ ./
# Ensure archives directory exists # Ensure archives and cache directories exist, writable by the runtime user.
RUN mkdir -p /archives RUN mkdir -p /archives /cache && chown node:node /cache
# Drop root: the server reads the archives volume and writes only its index.
USER node
EXPOSE 3000 EXPOSE 3000
+19 -2
View File
@@ -54,10 +54,27 @@ If the app shows "No Archives Found" and logs `EACCES: permission denied`:
chmod -R 755 /path/to/archives chmod -R 755 /path/to/archives
``` ```
2. **SELinux (Fedora/RHEL/CentOS)**: Use the `:z` flag in your volume mount as shown above. 2. **SELinux (Fedora/RHEL/CentOS)**: Use the `:z` flag in your volume mount as shown above.
3. **User Mapping**: You can force the container to run as your host user: 3. **User Mapping**: The container runs as the non-root `node` user (UID 1000).
If your archives are readable only by another account, run as that user
instead — the container needs to *list* the archive directory, so `--x`
(traverse-only) permissions are not enough:
```bash ```bash
docker run --user $(id -u):$(id -g) ... docker run --user $(stat -c '%u:%g' /path/to/archives) ...
``` ```
In Compose:
```yaml
services:
instaarchive:
user: "1234:1234" # a UID that can read your archives
```
### Archive Index
On first start the server walks the archive root once and caches the result,
keyed by directory mtime. This matters on network storage: for a 110k-file
archive root, listing went from ~52s per request to ~0.1s. Mount a volume at
`/cache` (or set `CACHE_DIR`) so the index survives restarts, otherwise it is
rebuilt on every start.
## Supported Archive Structure ## Supported Archive Structure
+6
View File
@@ -5,7 +5,13 @@ services:
- "3000:3000" - "3000:3000"
volumes: volumes:
- ./archives:/archives:ro,z - ./archives:/archives:ro,z
# Persists the archive index so restarts don't re-walk every file.
- instaarchive-cache:/cache
environment: environment:
- PORT=3000 - PORT=3000
- ARCHIVES_DIR=/archives - ARCHIVES_DIR=/archives
- CACHE_DIR=/cache
restart: unless-stopped restart: unless-stopped
volumes:
instaarchive-cache:
-137
View File
@@ -1,137 +0,0 @@
import express from 'express';
import fs from 'fs';
import path from 'path';
import { fileURLToPath } from 'url';
import dotenv from 'dotenv';
import os from 'os';
dotenv.config();
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
const app = express();
const PORT = process.env.PORT || 3001;
const ARCHIVES_DIR = path.resolve(process.env.ARCHIVES_DIR || path.join(__dirname, '_sample-archives'));
console.log(`[Server] Initializing...`);
console.log(`[Server] Running as user: ${os.userInfo().username} (UID: ${os.userInfo().uid}, GID: ${os.userInfo().gid})`);
console.log(`[Server] Environment ARCHIVES_DIR: ${process.env.ARCHIVES_DIR}`);
console.log(`[Server] Resolved ARCHIVES_DIR: ${ARCHIVES_DIR}`);
// Ensure archives directory exists
if (!fs.existsSync(ARCHIVES_DIR)) {
console.warn(`[Server] Warning: Archives directory not found at ${ARCHIVES_DIR}. Creating it...`);
try {
fs.mkdirSync(ARCHIVES_DIR, { recursive: true });
}
catch (err) {
console.error(`[Server] Failed to create archives directory:`, err);
}
}
else {
console.log(`[Server] Archives directory exists.`);
}
app.use(express.json());
// API: List archives (subdirectories in ARCHIVES_DIR)
app.get('/api/archives', (req, res) => {
try {
console.log(`[API] Listing archives from ${ARCHIVES_DIR}...`);
const items = fs.readdirSync(ARCHIVES_DIR, { withFileTypes: true });
console.log(`[API] Found ${items.length} total items in archives directory.`);
const archives = items
.filter(item => {
const isDir = item.isDirectory();
const isHidden = item.name.startsWith('.') || item.name.startsWith('@') || item.name.startsWith('_');
if (!isDir)
return false;
if (isHidden) {
console.log(`[API] Skipping system/hidden directory: ${item.name}`);
return false;
}
return true;
})
.map(item => {
// Try to find a profile pic or first image for the thumbnail
const archivePath = path.join(ARCHIVES_DIR, item.name);
try {
const files = fs.readdirSync(archivePath);
console.log(`[API] Found archive: ${item.name} (${files.length} files)`);
let thumbnail = '';
const profilePic = files.find(f => f.toLowerCase().includes('_profile_pic.jpg') || f.toLowerCase() === `${item.name.toLowerCase()}.jpg`);
if (profilePic) {
thumbnail = `/archives/${item.name}/${profilePic}`;
}
else {
const firstImage = files.find(f => /\.(jpg|jpeg|png|webp)$/i.test(f));
if (firstImage)
thumbnail = `/archives/${item.name}/${firstImage}`;
}
return {
name: item.name,
thumbnail,
path: item.name,
fileCount: files.length
};
}
catch (e) {
console.error(`[API] Could not read subdirectory ${item.name}:`, e);
return null;
}
})
.filter(Boolean);
console.log(`[API] Returning ${archives.length} validated archives.`);
res.json(archives);
}
catch (err) {
if (err.code === 'EACCES') {
console.error(`[API] Permission Denied! The server (UID ${os.userInfo().uid}) cannot read ${ARCHIVES_DIR}.`);
console.error(`[API] Hint: If using Linux/Docker, check folder permissions (chmod 755) or SELinux context (append :z to your volume mount).`);
}
else {
console.error('[API] Error listing archives:', err);
}
res.status(500).json({ error: 'Permission denied or failed to list archives' });
}
});
// API: List all files in an archive (recursive)
app.get('/api/archives/:name/files', (req, res) => {
const archiveName = req.params.name;
const archivePath = path.join(ARCHIVES_DIR, archiveName);
if (!fs.existsSync(archivePath)) {
return res.status(404).json({ error: 'Archive not found' });
}
try {
const walk = (dir, base = '') => {
let results = [];
const list = fs.readdirSync(dir);
list.forEach(file => {
const filePath = path.join(dir, file);
const relativePath = path.join(base, file);
const stat = fs.statSync(filePath);
if (stat && stat.isDirectory()) {
results = results.concat(walk(filePath, relativePath));
}
else {
results.push(relativePath);
}
});
return results;
};
const files = walk(archivePath);
res.json(files);
}
catch (err) {
console.error('Error listing files:', err);
res.status(500).json({ error: 'Failed to list files' });
}
});
// Serve archive files
app.use('/archives', express.static(ARCHIVES_DIR));
// Serve production frontend
const distPath = path.join(__dirname, 'dist');
if (fs.existsSync(distPath)) {
app.use(express.static(distPath));
app.get('*', (req, res) => {
res.sendFile(path.join(distPath, 'index.html'));
});
}
app.listen(PORT, () => {
console.log(`Server running at http://localhost:${PORT}`);
console.log(`Serving archives from: ${ARCHIVES_DIR}`);
});
+472 -1171
View File
File diff suppressed because it is too large Load Diff
+8 -5
View File
@@ -1,7 +1,7 @@
{ {
"name": "instaarchive-viewer", "name": "instaarchive-viewer",
"private": true, "private": true,
"version": "1.2.0", "version": "1.3.1",
"type": "module", "type": "module",
"scripts": { "scripts": {
"dev": "vite --port=3000 --host=0.0.0.0", "dev": "vite --port=3000 --host=0.0.0.0",
@@ -10,13 +10,13 @@
"preview": "vite preview", "preview": "vite preview",
"server": "tsx server.ts", "server": "tsx server.ts",
"clean": "rm -rf dist", "clean": "rm -rf dist",
"lint": "tsc --noEmit" "lint": "tsc --noEmit",
"test": "vitest run",
"test:watch": "vitest"
}, },
"dependencies": { "dependencies": {
"@google/genai": "^1.29.0",
"@tailwindcss/vite": "^4.1.14", "@tailwindcss/vite": "^4.1.14",
"@vitejs/plugin-react": "^5.0.4", "@vitejs/plugin-react": "^5.0.4",
"better-sqlite3": "^12.4.1",
"clsx": "^2.1.1", "clsx": "^2.1.1",
"date-fns": "^4.1.0", "date-fns": "^4.1.0",
"dotenv": "^17.2.3", "dotenv": "^17.2.3",
@@ -34,10 +34,13 @@
"devDependencies": { "devDependencies": {
"@types/express": "^4.17.21", "@types/express": "^4.17.21",
"@types/node": "^22.14.0", "@types/node": "^22.14.0",
"@types/react": "^19.2.18",
"@types/react-dom": "^19.2.4",
"autoprefixer": "^10.4.21", "autoprefixer": "^10.4.21",
"tailwindcss": "^4.1.14", "tailwindcss": "^4.1.14",
"typescript": "~5.8.2", "typescript": "~5.8.2",
"vite": "^6.2.0", "vite": "^6.2.0",
"vite-plugin-pwa": "^1.2.0" "vite-plugin-pwa": "^1.2.0",
"vitest": "^3.2.7"
} }
} }
+56
View File
@@ -0,0 +1,56 @@
/* Self-hosted subset of Inter + Playfair Display.
Vendored so the PWA works offline and makes no third-party requests. */
/* latin-ext */
@font-face {
font-family: 'Inter';
font-style: normal;
font-weight: 300 700;
font-display: swap;
src: url(/fonts/inter-300_700-normal-b6db4a.woff2) format('woff2');
unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF;
}
/* latin */
@font-face {
font-family: 'Inter';
font-style: normal;
font-weight: 300 700;
font-display: swap;
src: url(/fonts/inter-300_700-normal-6ab57b.woff2) format('woff2');
unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;
}
/* latin-ext */
@font-face {
font-family: 'Playfair Display';
font-style: italic;
font-weight: 400 900;
font-display: swap;
src: url(/fonts/playfair-display-400_900-italic-2d6d99.woff2) format('woff2');
unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF;
}
/* latin */
@font-face {
font-family: 'Playfair Display';
font-style: italic;
font-weight: 400 900;
font-display: swap;
src: url(/fonts/playfair-display-400_900-italic-d14361.woff2) format('woff2');
unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;
}
/* latin-ext */
@font-face {
font-family: 'Playfair Display';
font-style: normal;
font-weight: 400 900;
font-display: swap;
src: url(/fonts/playfair-display-400_900-normal-ca7410.woff2) format('woff2');
unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF;
}
/* latin */
@font-face {
font-family: 'Playfair Display';
font-style: normal;
font-weight: 400 900;
font-display: swap;
src: url(/fonts/playfair-display-400_900-normal-61a963.woff2) format('woff2');
unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;
}
Binary file not shown.
Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 2.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

+118 -73
View File
@@ -4,6 +4,7 @@ import path from 'path';
import { fileURLToPath } from 'url'; import { fileURLToPath } from 'url';
import dotenv from 'dotenv'; import dotenv from 'dotenv';
import os from 'os'; import os from 'os';
import { ArchiveIndex } from './src/lib/archive-index.js';
dotenv.config(); dotenv.config();
@@ -15,7 +16,23 @@ const PORT = process.env.PORT || 3001;
const ARCHIVES_DIR = path.resolve(process.env.ARCHIVES_DIR || path.join(__dirname, '_sample-archives')); const ARCHIVES_DIR = path.resolve(process.env.ARCHIVES_DIR || path.join(__dirname, '_sample-archives'));
console.log(`[Server] Initializing...`); console.log(`[Server] Initializing...`);
console.log(`[Server] Running as user: ${os.userInfo().username} (UID: ${os.userInfo().uid}, GID: ${os.userInfo().gid})`); /**
* Describe the running user without assuming it exists in /etc/passwd.
*
* `os.userInfo()` throws ENOENT for a UID with no passwd entry, which is
* exactly what happens when the container is started with `--user 1234:1234`
* (as the deployment docs suggest) — previously crashing the server at boot.
*/
const describeUser = () => {
try {
const info = os.userInfo();
return `${info.username} (UID: ${info.uid}, GID: ${info.gid})`;
} catch {
return `UID: ${typeof process.getuid === 'function' ? process.getuid() : '?'}, GID: ${typeof process.getgid === 'function' ? process.getgid() : '?'} (no passwd entry)`;
}
};
console.log(`[Server] Running as user: ${describeUser()}`);
console.log(`[Server] Environment ARCHIVES_DIR: ${process.env.ARCHIVES_DIR}`); console.log(`[Server] Environment ARCHIVES_DIR: ${process.env.ARCHIVES_DIR}`);
console.log(`[Server] Resolved ARCHIVES_DIR: ${ARCHIVES_DIR}`); console.log(`[Server] Resolved ARCHIVES_DIR: ${ARCHIVES_DIR}`);
@@ -31,60 +48,89 @@ if (!fs.existsSync(ARCHIVES_DIR)) {
console.log(`[Server] Archives directory exists.`); console.log(`[Server] Archives directory exists.`);
} }
const INDEX_PATH = process.env.ARCHIVE_INDEX_PATH
|| path.join(process.env.CACHE_DIR || os.tmpdir(), 'instaarchive-index.json');
const index = new ArchiveIndex(ARCHIVES_DIR, INDEX_PATH);
// Warm in the background: the first walk of a large archive root is slow, but
// everything after it is served from directory-mtime-keyed cache.
index.load()
.then(() => index.warm())
.catch(err => console.error('[Index] Warm failed:', err));
// Don't advertise the framework.
app.disable('x-powered-by');
/**
* Baseline security headers.
*
* The CSP allows blob: and data: because archive media is rendered from object
* URLs and cached thumbnails, and 'unsafe-inline' for styles because the
* animation library sets inline styles. Scripts stay restricted to same-origin,
* and no third-party origins are permitted at all — the app bundles its own
* fonts and icons.
*/
app.use((req, res, next) => {
res.setHeader('X-Content-Type-Options', 'nosniff');
res.setHeader('X-Frame-Options', 'SAMEORIGIN');
res.setHeader('Referrer-Policy', 'no-referrer');
res.setHeader('Cross-Origin-Resource-Policy', 'same-origin');
res.setHeader('Permissions-Policy', 'geolocation=(), microphone=(), camera=(), interest-cohort=()');
res.setHeader('Content-Security-Policy', [
"default-src 'self'",
"img-src 'self' blob: data:",
"media-src 'self' blob: data:",
"script-src 'self'",
"style-src 'self' 'unsafe-inline'",
"font-src 'self'",
"connect-src 'self'",
"worker-src 'self' blob:",
"frame-ancestors 'self'",
"object-src 'none'",
"base-uri 'self'",
"form-action 'self'",
].join('; '));
next();
});
app.use(express.json()); app.use(express.json());
// API: List archives (subdirectories in ARCHIVES_DIR) /**
* Resolve a user-supplied archive name to an absolute path inside ARCHIVES_DIR.
*
* Express decodes route params *after* segment matching, so a name like
* `..%2f..%2fetc` arrives here as `../../etc` and would otherwise escape the
* archives root. Returns null for anything that resolves outside it.
*/
const resolveArchivePath = (archiveName: string): string | null => {
if (!archiveName || archiveName.includes('\0')) return null;
const resolved = path.resolve(ARCHIVES_DIR, archiveName);
if (resolved !== ARCHIVES_DIR && !resolved.startsWith(ARCHIVES_DIR + path.sep)) {
console.warn(`[Security] Rejected archive name escaping ARCHIVES_DIR: ${archiveName}`);
return null;
}
return resolved;
};
// API: List archives, grouped by profile. Costs one stat per source directory.
app.get('/api/archives', (req, res) => { app.get('/api/archives', (req, res) => {
try { try {
console.log(`[API] Listing archives from ${ARCHIVES_DIR}...`); const groups = index.groups();
const items = fs.readdirSync(ARCHIVES_DIR, { withFileTypes: true }); const archives = Array.from(groups.entries()).map(([owner, sources]) => ({
console.log(`[API] Found ${items.length} total items in archives directory.`); name: owner,
thumbnail: index.thumbnailFor(owner, sources),
const archives = items path: owner,
.filter(item => { // Null until that profile has been indexed; the client treats it as unknown.
const isDir = item.isDirectory(); fileCount: index.countFor(sources),
const isHidden = item.name.startsWith('.') || item.name.startsWith('@') || item.name.startsWith('_'); // Directory mtimes: cheap to compute and enough to invalidate a stale cache.
if (!isDir) return false; signature: index.signatureFor(sources),
if (isHidden) { sources,
console.log(`[API] Skipping system/hidden directory: ${item.name}`); }));
return false; console.log(`[API] Returning ${archives.length} archives.`);
}
return true;
})
.map(item => {
// Try to find a profile pic or first image for the thumbnail
const archivePath = path.join(ARCHIVES_DIR, item.name);
try {
const files = fs.readdirSync(archivePath);
console.log(`[API] Found archive: ${item.name} (${files.length} files)`);
let thumbnail = '';
const profilePic = files.find(f => f.toLowerCase().includes('_profile_pic.jpg') || f.toLowerCase() === `${item.name.toLowerCase()}.jpg`);
if (profilePic) {
thumbnail = `/archives/${item.name}/${profilePic}`;
} else {
const firstImage = files.find(f => /\.(jpg|jpeg|png|webp)$/i.test(f));
if (firstImage) thumbnail = `/archives/${item.name}/${firstImage}`;
}
return {
name: item.name,
thumbnail,
path: item.name,
fileCount: files.length
};
} catch (e) {
console.error(`[API] Could not read subdirectory ${item.name}:`, e);
return null;
}
})
.filter(Boolean);
console.log(`[API] Returning ${archives.length} validated archives.`);
res.json(archives); res.json(archives);
} catch (err: any) { } catch (err: any) {
if (err.code === 'EACCES') { if (err.code === 'EACCES') {
console.error(`[API] Permission Denied! The server (UID ${os.userInfo().uid}) cannot read ${ARCHIVES_DIR}.`); console.error(`[API] Permission Denied! The server (${describeUser()}) cannot read ${ARCHIVES_DIR}.`);
console.error(`[API] Hint: If using Linux/Docker, check folder permissions (chmod 755) or SELinux context (append :z to your volume mount).`); console.error(`[API] Hint: If using Linux/Docker, check folder permissions (chmod 755) or SELinux context (append :z to your volume mount).`);
} else { } else {
console.error('[API] Error listing archives:', err); console.error('[API] Error listing archives:', err);
@@ -93,33 +139,26 @@ app.get('/api/archives', (req, res) => {
} }
}); });
// API: List all files in an archive (recursive) /**
app.get('/api/archives/:name/files', (req, res) => { * List every file belonging to a profile, across its base and sidecar dirs.
*
* Paths are relative to ARCHIVES_DIR (so they include the source directory) and
* each entry carries its source kind, letting the client route posts, reels,
* stories and highlights without re-deriving the naming rules.
*
* Served from the directory index; only a directory whose mtime changed is
* re-walked.
*/
app.get('/api/archives/:name/files', async (req, res) => {
const archiveName = req.params.name; const archiveName = req.params.name;
const archivePath = path.join(ARCHIVES_DIR, archiveName); if (!resolveArchivePath(archiveName)) {
return res.status(400).json({ error: 'Invalid archive name' });
if (!fs.existsSync(archivePath)) {
return res.status(404).json({ error: 'Archive not found' });
} }
try { try {
const walk = (dir: string, base: string = ''): string[] => { const files = await index.filesFor(archiveName);
let results: string[] = []; if (!files) return res.status(404).json({ error: 'Archive not found' });
const list = fs.readdirSync(dir); void index.save();
list.forEach(file => {
const filePath = path.join(dir, file);
const relativePath = path.join(base, file);
const stat = fs.statSync(filePath);
if (stat && stat.isDirectory()) {
results = results.concat(walk(filePath, relativePath));
} else {
results.push(relativePath);
}
});
return results;
};
const files = walk(archivePath);
res.json(files); res.json(files);
} catch (err) { } catch (err) {
console.error('Error listing files:', err); console.error('Error listing files:', err);
@@ -127,8 +166,14 @@ app.get('/api/archives/:name/files', (req, res) => {
} }
}); });
// Serve archive files // Serve archive files. Archive contents are immutable in practice, so cache
app.use('/archives', express.static(ARCHIVES_DIR)); // them aggressively; the client busts its own cache via fileCount.
app.use('/archives', express.static(ARCHIVES_DIR, {
maxAge: '1y',
immutable: true,
index: false,
dotfiles: 'ignore',
}));
// Serve production frontend // Serve production frontend
const distPath = path.join(__dirname, 'dist'); const distPath = path.join(__dirname, 'dist');
+265 -95
View File
@@ -15,11 +15,24 @@ import {
Loader2, Loader2,
} from 'lucide-react'; } from 'lucide-react';
import { motion, AnimatePresence } from 'motion/react'; import { motion, AnimatePresence } from 'motion/react';
import * as idb from 'idb-keyval';
import { cn } from './lib/utils'; import { cn } from './lib/utils';
import { LocalArchiveFile, RemoteArchiveFile } from './lib/archive-files'; import { LocalArchiveFile, RemoteArchiveFile } from './lib/archive-files';
import { Post, ServerArchive } from './types'; import {
deleteCachedArchive,
getCachedArchive,
listCachedArchives,
listCachedArchiveNames,
migrateLegacyCache,
restoreArchive,
saveDirectoryHandle,
} from './lib/archive-cache';
import {
filesFromDirectory,
isDirectoryPickerSupported,
pickDirectory,
} from './lib/directory-handle';
import { CacheData, Post, ServerArchive, ServerArchiveFile } from './types';
import { ArchiveDashboard } from './components/ArchiveDashboard'; import { ArchiveDashboard } from './components/ArchiveDashboard';
import { StoryViewer } from './components/StoryViewer'; import { StoryViewer } from './components/StoryViewer';
import { PostModal } from './components/PostModal'; import { PostModal } from './components/PostModal';
@@ -29,6 +42,7 @@ import { useThumbnailQueue } from './hooks/useThumbnailQueue';
export default function App() { export default function App() {
const [showStoryViewer, setShowStoryViewer] = useState(false); const [showStoryViewer, setShowStoryViewer] = useState(false);
const [activeHighlight, setActiveHighlight] = useState<string | null>(null);
const [visiblePostsCount, setVisiblePostsCount] = useState(90); const [visiblePostsCount, setVisiblePostsCount] = useState(90);
const [selectedPost, setSelectedPost] = useState<Post | null>(null); const [selectedPost, setSelectedPost] = useState<Post | null>(null);
@@ -37,33 +51,35 @@ export default function App() {
const [activeTab, setActiveTab] = useState<'posts' | 'reels' | 'saved'>('posts'); const [activeTab, setActiveTab] = useState<'posts' | 'reels' | 'saved'>('posts');
const [serverArchives, setServerArchives] = useState<ServerArchive[]>([]); const [serverArchives, setServerArchives] = useState<ServerArchive[]>([]);
const [cachedArchives, setCachedArchives] = useState<Set<string>>(new Set()); const [cachedArchives, setCachedArchives] = useState<Set<string>>(new Set());
const [localCachedArchives, setLocalCachedArchives] = useState<any[]>([]); const [localCachedArchives, setLocalCachedArchives] = useState<CacheData[]>([]);
const [isServerMode, setIsServerMode] = useState(false); const [isServerMode, setIsServerMode] = useState(false);
/** True once GET /api/archives has settled, successfully or not. */
const [archivesFetched, setArchivesFetched] = useState(false);
const [currentArchive, setCurrentArchive] = useState<ServerArchive | null>(null); const [currentArchive, setCurrentArchive] = useState<ServerArchive | null>(null);
const [hasInitialLoaded, setHasInitialLoaded] = useState(false);
/**
* The query string as it was when the app booted.
*
* Captured during the first render because the URL is rewritten from app
* state as soon as anything loads; reading `window.location` later would see
* the rewritten value rather than the link the user actually followed.
*/
const initialParamsRef = useRef(new URLSearchParams(window.location.search));
const fileInputRef = useRef<HTMLInputElement>(null); const fileInputRef = useRef<HTMLInputElement>(null);
const profilePicInputRef = useRef<HTMLInputElement>(null); const profilePicInputRef = useRef<HTMLInputElement>(null);
const { cacheHits, requestThumbnail } = useThumbnailQueue();
const refreshCachedArchives = useCallback(async () => { const refreshCachedArchives = useCallback(async () => {
try { try {
const keys = await idb.keys(); // Names come from key prefixes, so listing no longer deserializes every
setCachedArchives(new Set(keys.map(String))); // cached thumbnail blob just to find out which entries are archives.
setCachedArchives(new Set(await listCachedArchiveNames()));
const locals: any[] = []; setLocalCachedArchives((await listCachedArchives()).filter(a => a.isLocal));
for (const key of keys) { } catch (e) {
const data: any = await idb.get(key); console.error('[Cache] Failed to list cached archives:', e);
if (data && data.isLocal) { }
// Fallback for missing allProfilePics in local cached metadata
if (!data.profileMetadata.allProfilePics) {
data.profileMetadata.allProfilePics = data.profileMetadata.profilePic ? [data.profileMetadata.profilePic] : [];
}
locals.push(data);
}
}
setLocalCachedArchives(locals);
} catch (e) {}
}, []); }, []);
const { const {
@@ -75,6 +91,8 @@ export default function App() {
currentScanningImage, currentScanningImage,
allPosts, allPosts,
allStories, allStories,
allHighlights,
setAllHighlights,
profileMetadata, profileMetadata,
handleFiles, handleFiles,
setAllPosts, setAllPosts,
@@ -82,7 +100,8 @@ export default function App() {
setProfileMetadata, setProfileMetadata,
setIsScanning, setIsScanning,
setScanningPhase, setScanningPhase,
resetScannerState resetScannerState,
registerUrl
} = useArchiveScanner('', currentArchive, refreshCachedArchives); } = useArchiveScanner('', currentArchive, refreshCachedArchives);
const [lastLoadedScanningImage, setLastLoadedScanningImage] = useState<string | null>(null); const [lastLoadedScanningImage, setLastLoadedScanningImage] = useState<string | null>(null);
@@ -98,6 +117,9 @@ export default function App() {
allProfilePics allProfilePics
} = profileMetadata; } = profileMetadata;
// Thumbnails are keyed per archive, so the queue is scoped to the open one.
const { cacheHits, requestThumbnail } = useThumbnailQueue(currentArchive?.name ?? username ?? '');
useEffect(() => { useEffect(() => {
fetch('/api/archives') fetch('/api/archives')
.then(res => { .then(res => {
@@ -107,21 +129,51 @@ export default function App() {
} }
return []; return [];
}) })
.then(data => setServerArchives(data)) .then(data => setServerArchives(Array.isArray(data) ? data : []))
.catch(() => setIsServerMode(false)); .catch(() => setIsServerMode(false))
// Deep-link resolution waits on this rather than on `isServerMode`, which
// is still false while the request is in flight.
.finally(() => setArchivesFetched(true));
}, []); }, []);
useEffect(() => { useEffect(() => {
refreshCachedArchives(); migrateLegacyCache().finally(refreshCachedArchives);
}, [refreshCachedArchives]); }, [refreshCachedArchives]);
const clearCache = async (name: string) => { await idb.del(name); await refreshCachedArchives(); }; const clearCache = async (name: string) => { await deleteCachedArchive(name); await refreshCachedArchives(); };
/**
* Archives with a `- reels` sidecar directory say outright which posts are
* reels; only fall back to the "lone video" heuristic for archives that have
* no such directory.
*/
const hasReelSource = useMemo(() => allPosts.some(p => p.source === 'reels'), [allPosts]);
const isReel = useCallback((p: Post) => (
hasReelSource ? p.source === 'reels' : p.media.length === 1 && p.media[0].type === 'video'
), [hasReelSource]);
const filteredPosts = useMemo(() => { const filteredPosts = useMemo(() => {
if (activeTab === 'reels') return allPosts.filter(p => p.media.length === 1 && p.media[0].type === 'video'); if (activeTab === 'reels') return allPosts.filter(isReel);
if (activeTab === 'posts') return allPosts.filter(p => !(p.media.length === 1 && p.media[0].type === 'video')); if (activeTab === 'posts') return allPosts.filter(p => !isReel(p));
return []; return [];
}, [allPosts, activeTab]); }, [allPosts, activeTab, isReel]);
/** Story highlights, grouped into the circles shown under the bio. */
const highlightGroups = useMemo(() => {
const groups = new Map<string, Post[]>();
for (const item of allHighlights) {
const title = item.highlightTitle?.trim() || 'Highlights';
if (!groups.has(title)) groups.set(title, []);
groups.get(title)!.push(item);
}
return Array.from(groups, ([title, items]) => ({
title,
items,
// The cover must be a still: most highlight items are videos, and a video
// URL in an <img> renders as a broken image.
cover: items.find(i => i.media[0]?.type === 'image')?.thumbnail,
}));
}, [allHighlights]);
const handleTabChange = (tab: 'posts' | 'reels' | 'saved') => { setActiveTab(tab); setVisiblePostsCount(90); }; const handleTabChange = (tab: 'posts' | 'reels' | 'saved') => { setActiveTab(tab); setVisiblePostsCount(90); };
const visiblePosts = useMemo(() => filteredPosts.slice(0, visiblePostsCount), [filteredPosts, visiblePostsCount]); const visiblePosts = useMemo(() => filteredPosts.slice(0, visiblePostsCount), [filteredPosts, visiblePostsCount]);
@@ -151,38 +203,50 @@ export default function App() {
setScanningPhase('Checking Cache'); setScanningPhase('Checking Cache');
try { try {
const cachedData = await idb.get(archive.name); const cachedData = await getCachedArchive(archive.name);
if (cachedData) { if (cachedData) {
console.log(`[Cache] Found cached data for ${archive.name}. File count: ${cachedData.fileCount} (Server has: ${archive.fileCount})`); // Invalidate on the directory-mtime signature; fall back to file count
if (cachedData.fileCount === archive.fileCount) { // for entries cached before signatures existed.
const fresh = archive.signature
? cachedData.signature === archive.signature
: cachedData.fileCount === archive.fileCount;
console.log(`[Cache] Cached ${archive.name}: signature ${cachedData.signature} vs ${archive.signature} -> ${fresh ? 'fresh' : 'stale'}`);
if (fresh) {
console.log(`[Cache] Cache hit! Restoring state...`); console.log(`[Cache] Cache hit! Restoring state...`);
setAllPosts(cachedData.posts); const restored = await restoreArchive(cachedData, registerUrl);
setAllStories(cachedData.stories); if (restored) {
setAllPosts(restored.posts);
// Handle migration from old cache schema where allProfilePics was a separate top-level key setAllStories(restored.stories);
const profileMetadata = { ...cachedData.profileMetadata }; setAllHighlights(restored.highlights);
if (!profileMetadata.allProfilePics && cachedData.allProfilePics) { setProfileMetadata({
profileMetadata.allProfilePics = cachedData.allProfilePics; ...restored.profileMetadata,
allProfilePics: restored.profileMetadata.allProfilePics
?? (restored.profileMetadata.profilePic ? [restored.profileMetadata.profilePic] : []),
});
setVisiblePostsCount(90);
setIsScanning(false);
console.log(`[Cache] Archive ${archive.name} loaded successfully from cache.`);
return;
} }
if (!profileMetadata.allProfilePics) {
profileMetadata.allProfilePics = profileMetadata.profilePic ? [profileMetadata.profilePic] : [];
}
setProfileMetadata(profileMetadata);
setVisiblePostsCount(90);
setIsScanning(false);
console.log(`[Cache] Archive ${archive.name} loaded successfully from cache.`);
return;
} }
} }
console.log(`[Scanner] Starting fresh scan from server API...`); console.log(`[Scanner] Starting fresh scan from server API...`);
const res = await fetch(`/api/archives/${archive.name}/files`); const res = await fetch(`/api/archives/${encodeURIComponent(archive.name)}/files`);
const filePaths: string[] = await res.json(); const entries: (ServerArchiveFile | string)[] = await res.json();
const archiveFiles = filePaths.map(p => { const archiveFiles = entries.map(entry => {
const name = p.split(/[/\\]/).pop() || p; // Older servers returned bare path strings relative to the archive dir.
return new RemoteArchiveFile(name, p, 0, `/archives/${archive.name}/${p}`); const legacy = typeof entry === 'string';
const filePath = legacy ? entry : entry.path;
const url = legacy
? `/archives/${encodeURI(`${archive.name}/${filePath}`)}`
: `/archives/${encodeURI(filePath)}`;
const name = filePath.split(/[/\\]/).pop() || filePath;
const source = legacy
? undefined
: { kind: entry.kind, dir: filePath.split('/')[0], title: entry.title };
return new RemoteArchiveFile(name, filePath, legacy ? 0 : entry.size, url, source, legacy ? undefined : entry.mtime);
}); });
await handleFiles(archiveFiles, archive); await handleFiles(archiveFiles, archive);
@@ -190,30 +254,43 @@ export default function App() {
console.error('[Scanner] Failed to load server archive:', err); console.error('[Scanner] Failed to load server archive:', err);
setIsScanning(false); setIsScanning(false);
} }
}, [handleFiles, setAllPosts, setAllStories, setProfileMetadata, setIsScanning, setScanningPhase]); }, [handleFiles, registerUrl, setAllPosts, setAllStories, setAllHighlights, setProfileMetadata, setIsScanning, setScanningPhase]);
const loadLocalCachedArchive = useCallback(async (archive: any) => { /**
* Open a local archive straight from cache.
*
* The cached posts carry file *paths*, not URLs — blob: URLs do not survive a
* reload — so this re-opens the stored directory handle and mints fresh URLs
* for those paths. No re-parsing happens, which is what keeps it instant.
*
* If the folder can no longer be reached (permission revoked, folder moved,
* or the browser never supported directory handles) we fall back to asking
* for the folder again rather than rendering an archive of broken images.
*/
const loadLocalCachedArchive = useCallback(async (archive: CacheData) => {
console.log(`[Cache] Loading local archive from cache: ${archive.name}`); console.log(`[Cache] Loading local archive from cache: ${archive.name}`);
setIsScanning(true); setIsScanning(true);
setCurrentArchive(null); setCurrentArchive(null);
setScanningPhase('Checking Cache'); setScanningPhase('Checking Cache');
try { try {
// Small delay for UI transition const restored = await restoreArchive(archive, registerUrl);
await new Promise(resolve => setTimeout(resolve, 300));
setAllPosts(archive.posts || []); if (!restored) {
setAllStories(archive.stories || []); console.warn(`[Cache] Folder for ${archive.name} unavailable; re-prompting.`);
setIsScanning(false);
const profileMetadata = { ...archive.profileMetadata }; await openLocalFolder(archive.name);
if (!profileMetadata.allProfilePics && archive.allProfilePics) { return;
profileMetadata.allProfilePics = archive.allProfilePics;
}
if (!profileMetadata.allProfilePics) {
profileMetadata.allProfilePics = profileMetadata.profilePic ? [profileMetadata.profilePic] : [];
} }
setProfileMetadata(profileMetadata); setAllPosts(restored.posts);
setAllStories(restored.stories);
setAllHighlights(restored.highlights);
setProfileMetadata({
...restored.profileMetadata,
allProfilePics: restored.profileMetadata.allProfilePics
?? (restored.profileMetadata.profilePic ? [restored.profileMetadata.profilePic] : []),
});
setVisiblePostsCount(90); setVisiblePostsCount(90);
setIsScanning(false); setIsScanning(false);
console.log(`[Cache] Local archive ${archive.name} restored from cache.`); console.log(`[Cache] Local archive ${archive.name} restored from cache.`);
@@ -221,13 +298,47 @@ export default function App() {
console.error('[Cache] Failed to restore local archive:', err); console.error('[Cache] Failed to restore local archive:', err);
setIsScanning(false); setIsScanning(false);
} }
}, [setAllPosts, setAllStories, setProfileMetadata, setIsScanning, setScanningPhase]); }, [registerUrl, setAllPosts, setAllStories, setAllHighlights, setProfileMetadata, setIsScanning, setScanningPhase]);
const handleLocalFiles = (files: FileList | null) => { if (!files) return; const archiveFiles = Array.from(files).map(f => new LocalArchiveFile(f)); handleFiles(archiveFiles); }; const handleLocalFiles = (files: FileList | null) => {
const triggerFileSelect = () => fileInputRef.current?.click(); if (!files) return;
handleFiles(Array.from(files).map(f => new LocalArchiveFile(f)));
};
/**
* Prefer the File System Access API so the folder can be reopened later
* without a re-prompt; fall back to <input webkitdirectory> elsewhere
* (Firefox and Safari have no showDirectoryPicker), where the archive is
* re-scanned from scratch on every visit.
*/
const openLocalFolder = useCallback(async (expectedName?: string) => {
if (!isDirectoryPickerSupported()) {
fileInputRef.current?.click();
return;
}
const handle = await pickDirectory();
if (!handle) return;
if (expectedName && handle.name !== expectedName) {
console.warn(`[Cache] Picked "${handle.name}" but expected "${expectedName}"; scanning as picked.`);
}
// Persist before scanning so the scan records that a handle exists.
await saveDirectoryHandle(handle.name, handle);
const files = await filesFromDirectory(handle);
await handleFiles(files);
}, [handleFiles]);
const triggerFileSelect = () => { void openLocalFolder(); };
const loadMore = () => setVisiblePostsCount(prev => prev + 90); const loadMore = () => setVisiblePostsCount(prev => prev + 90);
useEffect(() => { useEffect(() => {
// Hold the URL until the deep link has been consumed, otherwise this effect
// runs on mount with nothing loaded yet and erases the very parameters the
// loader below is waiting to read.
if (!hasInitialLoaded) return;
const params = new URLSearchParams(window.location.search); const params = new URLSearchParams(window.location.search);
if (currentArchive) params.set('a', currentArchive.name); if (currentArchive) params.set('a', currentArchive.name);
else if (allPosts.length > 0 && username) params.set('a', username); else if (allPosts.length > 0 && username) params.set('a', username);
@@ -246,37 +357,60 @@ export default function App() {
const newUrl = window.location.pathname + (newSearch ? `?${newSearch}` : ''); const newUrl = window.location.pathname + (newSearch ? `?${newSearch}` : '');
window.history.replaceState(null, '', newUrl); window.history.replaceState(null, '', newUrl);
} }
}, [currentArchive?.name, username, allPosts.length, activeTab, selectedPost?.id]); }, [hasInitialLoaded, currentArchive?.name, username, allPosts.length, activeTab, selectedPost?.id]);
const [hasInitialLoaded, setHasInitialLoaded] = useState(false);
useEffect(() => { useEffect(() => {
if (hasInitialLoaded || serverArchives.length === 0) return; if (hasInitialLoaded) return;
const params = new URLSearchParams(window.location.search);
const params = initialParamsRef.current;
const archiveName = params.get('a'); const archiveName = params.get('a');
const tab = params.get('t'); const tab = params.get('t');
const postId = params.get('p'); console.log('[Permalink] Initial read from URL:', {
console.log('[Permalink] Initial read from URL:', { archiveName, tab, postId }); archiveName, tab, postId: params.get('p'),
if (archiveName) { });
const archive = serverArchives.find(a => a.name === archiveName);
if (archive) { if (tab && ['posts', 'reels', 'saved'].includes(tab)) {
console.log(`[Permalink] Auto-loading archive: ?a=${archiveName}`); setActiveTab(tab as 'posts' | 'reels' | 'saved');
loadServerArchive(archive); }
if (tab && ['posts', 'reels', 'saved'].includes(tab)) {
setActiveTab(tab as any); if (!archiveName) {
} setHasInitialLoaded(true);
} return;
}
// Wait for the archive list before deciding the link is unresolvable.
if (!archivesFetched) return;
const archive = serverArchives.find(a => a.name === archiveName);
if (archive) {
console.log(`[Permalink] Auto-loading archive: ?a=${archiveName}`);
loadServerArchive(archive);
} else {
console.warn(`[Permalink] No archive named "${archiveName}".`);
} }
setHasInitialLoaded(true); setHasInitialLoaded(true);
}, [serverArchives, hasInitialLoaded, loadServerArchive]); }, [serverArchives, archivesFetched, hasInitialLoaded, loadServerArchive]);
/**
* Apply a `?p=` deep link exactly once per opened archive.
*
* Keying on the archive rather than on `selectedPost` matters: re-reading the
* URL whenever the selection changes would reopen the post the user just
* closed, and only worked before because the URL-writing effect happened to
* be declared first and had already stripped the param.
*/
const appliedPostParamRef = useRef<string | null>(null);
useEffect(() => { useEffect(() => {
const params = new URLSearchParams(window.location.search); const archiveKey = currentArchive?.name ?? username;
const postId = params.get('p'); if (!archiveKey || allPosts.length === 0) return;
if (postId && allPosts.length > 0 && !selectedPost) { if (appliedPostParamRef.current === archiveKey) return;
const post = allPosts.find(p => p.id === postId); appliedPostParamRef.current = archiveKey;
if (post) setSelectedPost(post);
} const postId = initialParamsRef.current.get('p');
}, [allPosts, selectedPost]); if (!postId) return;
const post = allPosts.find(p => p.id === postId);
if (post) setSelectedPost(post);
}, [allPosts, currentArchive?.name, username]);
return ( return (
<div className="min-h-screen bg-gray-50 text-gray-900 font-sans"> <div className="min-h-screen bg-gray-50 text-gray-900 font-sans">
@@ -367,6 +501,32 @@ export default function App() {
</div> </div>
</header> </header>
{highlightGroups.length > 0 && (
<div className="flex gap-6 md:gap-8 overflow-x-auto scrollbar-hide px-4 pb-2">
{highlightGroups.map(group => (
<button
key={group.title}
onClick={() => setActiveHighlight(group.title)}
className="flex flex-col items-center gap-2 shrink-0 group/hl"
title={`${group.title}${group.items.length} item${group.items.length === 1 ? '' : 's'}`}
>
<div className="w-16 h-16 md:w-20 md:h-20 rounded-full p-[2px] bg-gray-200 group-hover/hl:bg-gray-300 transition-colors">
<div className="w-full h-full rounded-full bg-white p-[2px]">
<div className="w-full h-full rounded-full overflow-hidden bg-gray-100 flex items-center justify-center">
{group.cover ? (
<img src={group.cover} alt="" className="w-full h-full object-cover" referrerPolicy="no-referrer" loading="lazy" />
) : (
<Play size={18} className="text-gray-400" fill="currentColor" />
)}
</div>
</div>
</div>
<span className="text-[11px] max-w-[80px] truncate text-gray-700">{group.title}</span>
</button>
))}
</div>
)}
<div className="border-t border-gray-200 flex flex-col md:flex-row items-center justify-between gap-4 text-black"> <div className="border-t border-gray-200 flex flex-col md:flex-row items-center justify-between gap-4 text-black">
<div className="flex justify-center gap-12 flex-1 text-black"> <div className="flex justify-center gap-12 flex-1 text-black">
<button onClick={() => handleTabChange('posts')} className={cn("flex items-center gap-2 py-4 border-t text-xs font-bold tracking-widest uppercase transition-all text-black", activeTab === 'posts' ? "border-black text-black" : "border-transparent text-gray-400")}><Grid3X3 size={14} />Posts</button> <button onClick={() => handleTabChange('posts')} className={cn("flex items-center gap-2 py-4 border-t text-xs font-bold tracking-widest uppercase transition-all text-black", activeTab === 'posts' ? "border-black text-black" : "border-transparent text-gray-400")}><Grid3X3 size={14} />Posts</button>
@@ -389,7 +549,7 @@ export default function App() {
</motion.div> </motion.div>
))} ))}
</div> </div>
{filteredPosts.length > visiblePostsCount && <div className="flex justify-center pt-12 text-black text-black"><button onClick={loadMore} className="bg-white border border-gray-200 px-8 py-2 rounded-lg font-semibold hover:bg-gray-50 transition-colors shadow-sm text-black text-black">Load More</button></div>} {filteredPosts.length > visiblePostsCount && <div className="flex justify-center pt-12 text-black"><button onClick={loadMore} className="bg-white border border-gray-200 px-8 py-2 rounded-lg font-semibold hover:bg-gray-50 transition-colors shadow-sm text-black">Load More</button></div>}
</div> </div>
)} )}
</main> </main>
@@ -410,10 +570,20 @@ export default function App() {
)} )}
</AnimatePresence> </AnimatePresence>
<AnimatePresence>{showStoryViewer && allStories.length > 0 && <StoryViewer stories={allStories} onClose={() => setShowStoryViewer(false)} profilePic={profilePic} />}</AnimatePresence> <AnimatePresence>{showStoryViewer && allStories.length > 0 && <StoryViewer stories={allStories} onClose={() => setShowStoryViewer(false)} profilePic={profilePic} />}</AnimatePresence>
<AnimatePresence>
{activeHighlight && (
<StoryViewer
stories={highlightGroups.find(g => g.title === activeHighlight)?.items ?? []}
title={activeHighlight}
onClose={() => setActiveHighlight(null)}
profilePic={profilePic}
/>
)}
</AnimatePresence>
{!isScanning && ( {!isScanning && (
<footer className="max-w-5xl mx-auto px-4 py-12 text-center text-xs text-gray-400 space-y-4 text-black"> <footer className="max-w-5xl mx-auto px-4 py-12 text-center text-xs text-gray-400 space-y-4 text-black">
<div className="flex flex-wrap justify-center gap-x-4 gap-y-2 uppercase tracking-tight text-black text-black"><span>Meta</span><span>About</span><span>Blog</span><span>Jobs</span><span>Help</span><span>API</span><span>Privacy</span><span>Terms</span><span>Locations</span><span>Instagram Lite</span><span>Threads</span><span>Contact Uploading & Non-Users</span><span>Meta Verified</span></div> <div className="flex flex-wrap justify-center gap-x-4 gap-y-2 uppercase tracking-tight text-black"><span>Meta</span><span>About</span><span>Blog</span><span>Jobs</span><span>Help</span><span>API</span><span>Privacy</span><span>Terms</span><span>Locations</span><span>Instagram Lite</span><span>Threads</span><span>Contact Uploading & Non-Users</span><span>Meta Verified</span></div>
<div className="text-black/40 text-black">© 2026 InstaArchive Viewer</div> <div className="text-black/40 text-black">© 2026 InstaArchive Viewer</div>
</footer> </footer>
)} )}
+8 -4
View File
@@ -6,15 +6,15 @@ import {
Trash2, Trash2,
Zap Zap
} from 'lucide-react'; } from 'lucide-react';
import { ServerArchive } from '../types'; import { CacheData, ServerArchive } from '../types';
interface ArchiveDashboardProps { interface ArchiveDashboardProps {
archives: ServerArchive[]; archives: ServerArchive[];
localArchives?: any[]; localArchives?: CacheData[];
cachedArchives: Set<string>; cachedArchives: Set<string>;
onSelect: (archive: ServerArchive) => void; onSelect: (archive: ServerArchive) => void;
onLocalSelect: () => void; onLocalSelect: () => void;
onLocalCacheSelect: (archive: any) => void; onLocalCacheSelect: (archive: CacheData) => void;
onClearCache: (name: string) => void; onClearCache: (name: string) => void;
isScanning: boolean; isScanning: boolean;
} }
@@ -85,7 +85,11 @@ export const ArchiveDashboard: React.FC<ArchiveDashboardProps> = ({
</div> </div>
<div className="p-4 space-y-1"> <div className="p-4 space-y-1">
<span className="font-bold text-sm block truncate uppercase tracking-tight text-black/80">{archive.name}</span> <span className="font-bold text-sm block truncate uppercase tracking-tight text-black/80">{archive.name}</span>
<span className="text-[10px] text-gray-400 uppercase tracking-widest">{archive.fileCount} items</span> <span className="text-[10px] text-gray-400 uppercase tracking-widest">
{archive.fileCount === null
? `${archive.sources?.length ?? 1} source${(archive.sources?.length ?? 1) === 1 ? '' : 's'}`
: `${archive.fileCount} items`}
</span>
</div> </div>
</button> </button>
+55
View File
@@ -0,0 +1,55 @@
import React from 'react';
import { AlertTriangle } from 'lucide-react';
interface Props { children: React.ReactNode }
interface State { error: Error | null }
/**
* Keeps one bad archive item from blanking the whole app.
*
* Post data is derived from filenames and arbitrary archive JSON, so a single
* malformed record used to be able to throw during render and take the entire
* tree down with it.
*/
export class ErrorBoundary extends React.Component<Props, State> {
constructor(props: Props) {
super(props);
this.state = { error: null };
}
static getDerivedStateFromError(error: Error): State {
return { error };
}
componentDidCatch(error: Error, info: React.ErrorInfo) {
console.error('[ErrorBoundary] Render failed:', error, info.componentStack);
}
render() {
if (!this.state.error) return this.props.children;
return (
<div className="min-h-screen bg-gray-50 flex items-center justify-center p-6">
<div className="max-w-md w-full bg-white rounded-2xl shadow-sm border border-gray-100 p-8 space-y-4 text-center">
<div className="w-12 h-12 mx-auto rounded-full bg-red-50 text-red-500 flex items-center justify-center">
<AlertTriangle size={24} />
</div>
<h2 className="text-xl font-semibold text-black/80">Something went wrong</h2>
<p className="text-sm text-gray-500">
This archive could not be rendered. Reloading usually clears it; if it
persists, clear the cached copy from the archive explorer.
</p>
<pre className="text-[11px] text-left text-gray-400 bg-gray-50 rounded-lg p-3 overflow-x-auto">
{this.state.error.message}
</pre>
<button
onClick={() => window.location.reload()}
className="bg-blue-500 hover:bg-blue-600 text-white px-6 py-2 rounded-lg text-sm font-semibold transition-colors"
>
Reload
</button>
</div>
</div>
);
}
}
+2 -1
View File
@@ -4,7 +4,8 @@ import { MediaFile } from '../types';
import { cn } from '../lib/utils'; import { cn } from '../lib/utils';
export const MediaRenderer = ({ file, className, isFullView }: { file: MediaFile; className?: string; isFullView?: boolean }) => { export const MediaRenderer = ({ file, className, isFullView }: { file: MediaFile; className?: string; isFullView?: boolean }) => {
const [isMuted, setIsMuted] = useState(false); // Start muted so autoplay is not blocked by Safari/Firefox policy.
const [isMuted, setIsMuted] = useState(true);
const sizingClass = isFullView ? "w-full h-auto block" : "w-full h-full object-cover"; const sizingClass = isFullView ? "w-full h-auto block" : "w-full h-full object-cover";
const mediaStyle = { transform: 'translateZ(0)' }; const mediaStyle = { transform: 'translateZ(0)' };
+2 -3
View File
@@ -10,9 +10,8 @@ import {
Bookmark Bookmark
} from 'lucide-react'; } from 'lucide-react';
import { motion, AnimatePresence } from 'motion/react'; import { motion, AnimatePresence } from 'motion/react';
import { format, parseISO } from 'date-fns';
import { Post } from '../types'; import { Post } from '../types';
import { cn } from '../lib/utils'; import { cn, formatDateSafe } from '../lib/utils';
import { MediaRenderer } from './MediaRenderer'; import { MediaRenderer } from './MediaRenderer';
interface PostModalProps { interface PostModalProps {
@@ -150,7 +149,7 @@ export const PostModal: React.FC<PostModalProps> = ({
<div className="flex-1 overflow-y-auto p-3 md:p-4 space-y-4 min-h-0 md:max-h-[60vh] text-black"> <div className="flex-1 overflow-y-auto p-3 md:p-4 space-y-4 min-h-0 md:max-h-[60vh] text-black">
<div className="flex gap-3 text-black"> <div className="flex gap-3 text-black">
<div className="w-8 h-8 rounded-full bg-gray-200 flex-shrink-0 flex items-center justify-center text-[10px] font-bold uppercase overflow-hidden text-black">{profilePic ? <img src={profilePic} alt="" className="w-full h-full object-cover" referrerPolicy="no-referrer" /> : <span className="text-black">{post.username[0]}</span>}</div> <div className="w-8 h-8 rounded-full bg-gray-200 flex-shrink-0 flex items-center justify-center text-[10px] font-bold uppercase overflow-hidden text-black">{profilePic ? <img src={profilePic} alt="" className="w-full h-full object-cover" referrerPolicy="no-referrer" /> : <span className="text-black">{post.username[0]}</span>}</div>
<div className="text-sm text-black"><span className="font-semibold mr-2 text-black">{post.username}</span><span className="whitespace-pre-wrap text-black">{post.caption}</span><div className="mt-2 text-xs text-gray-500 uppercase tracking-tight text-black">{format(parseISO(post.date), 'MMMM d, yyyy')}</div></div> <div className="text-sm text-black"><span className="font-semibold mr-2 text-black">{post.username}</span><span className="whitespace-pre-wrap text-black">{post.caption}</span><div className="mt-2 text-xs text-gray-500 uppercase tracking-tight text-black">{formatDateSafe(post.date, 'MMMM d, yyyy')}</div></div>
</div> </div>
</div> </div>
<div className="p-3 md:p-4 border-t border-gray-100 space-y-3 shrink-0 bg-white text-black"> <div className="p-3 md:p-4 border-t border-gray-100 space-y-3 shrink-0 bg-white text-black">
+21 -12
View File
@@ -7,26 +7,31 @@ import {
X X
} from 'lucide-react'; } from 'lucide-react';
import { motion } from 'motion/react'; import { motion } from 'motion/react';
import { format, parseISO } from 'date-fns';
import { Post } from '../types'; import { Post } from '../types';
import { cn } from '../lib/utils'; import { cn, formatDateSafe } from '../lib/utils';
interface StoryViewerProps { interface StoryViewerProps {
stories: Post[]; stories: Post[];
onClose: () => void; onClose: () => void;
profilePic: string | null; profilePic: string | null;
/** Highlight name, shown in place of the date when viewing a highlight. */
title?: string;
} }
export const StoryViewer: React.FC<StoryViewerProps> = ({ export const StoryViewer: React.FC<StoryViewerProps> = ({
stories, stories,
onClose, onClose,
profilePic profilePic,
title
}) => { }) => {
const [currentStoryIndex, setCurrentStoryIndex] = useState(0); const [currentStoryIndex, setCurrentStoryIndex] = useState(0);
const [progress, setProgress] = useState(0); const [progress, setProgress] = useState(0);
const [isMuted, setIsMuted] = useState(false); // Start muted: Safari and Firefox refuse to autoplay audible media, which
// would stall the reel on its first video.
const [isMuted, setIsMuted] = useState(true);
const videoRef = useRef<HTMLVideoElement>(null); const videoRef = useRef<HTMLVideoElement>(null);
const story = stories[currentStoryIndex]; const story = stories[currentStoryIndex];
const primary = story?.media?.[0];
useEffect(() => { useEffect(() => {
setProgress(0); setProgress(0);
@@ -34,7 +39,7 @@ export const StoryViewer: React.FC<StoryViewerProps> = ({
const interval = 50; const interval = 50;
const updateProgress = () => { const updateProgress = () => {
if (story.media[0].type === 'video' && videoRef.current) { if (primary?.type === 'video' && videoRef.current) {
const currentTime = videoRef.current.currentTime; const currentTime = videoRef.current.currentTime;
const totalTime = videoRef.current.duration; const totalTime = videoRef.current.duration;
if (totalTime) { if (totalTime) {
@@ -54,7 +59,7 @@ export const StoryViewer: React.FC<StoryViewerProps> = ({
}, interval); }, interval);
return () => clearInterval(timer); return () => clearInterval(timer);
}, [currentStoryIndex, story.media]); }, [currentStoryIndex, primary]);
useEffect(() => { useEffect(() => {
if (progress >= 100) { if (progress >= 100) {
@@ -80,6 +85,9 @@ export const StoryViewer: React.FC<StoryViewerProps> = ({
} }
}; };
// An empty or exhausted reel has nothing to show; bail before dereferencing.
if (!story || !primary) return null;
return ( return (
<motion.div <motion.div
initial={{ opacity: 0 }} initial={{ opacity: 0 }}
@@ -90,7 +98,7 @@ export const StoryViewer: React.FC<StoryViewerProps> = ({
> >
<div className="absolute inset-0 z-0 text-white"> <div className="absolute inset-0 z-0 text-white">
<img <img
src={story.media[0].url} src={primary.url}
alt="" alt=""
className="w-full h-full object-cover blur-3xl opacity-30" className="w-full h-full object-cover blur-3xl opacity-30"
/> />
@@ -146,12 +154,13 @@ export const StoryViewer: React.FC<StoryViewerProps> = ({
</div> </div>
<div className="flex items-center gap-2 text-white"> <div className="flex items-center gap-2 text-white">
<span className="text-xs font-semibold">{story.username}</span> <span className="text-xs font-semibold">{story.username}</span>
<span className="text-[10px] opacity-60 font-medium">{format(parseISO(story.date), 'MMM d')}</span> {title && <span className="text-[10px] opacity-80 font-medium truncate max-w-[120px]">{title}</span>}
<span className="text-[10px] opacity-60 font-medium">{formatDateSafe(story.date, 'MMM d')}</span>
</div> </div>
</div> </div>
<div className="flex items-center gap-1 text-white"> <div className="flex items-center gap-1 text-white">
{story.media[0].type === 'video' && ( {primary.type === 'video' && (
<button <button
onClick={(e) => { e.stopPropagation(); setIsMuted(!isMuted); }} onClick={(e) => { e.stopPropagation(); setIsMuted(!isMuted); }}
className="p-2 hover:bg-white/10 rounded-full transition-colors text-white" className="p-2 hover:bg-white/10 rounded-full transition-colors text-white"
@@ -166,10 +175,10 @@ export const StoryViewer: React.FC<StoryViewerProps> = ({
</div> </div>
<div className="w-full h-full flex items-center justify-center pointer-events-none text-white"> <div className="w-full h-full flex items-center justify-center pointer-events-none text-white">
{story.media[0].type === 'video' ? ( {primary.type === 'video' ? (
<video <video
ref={videoRef} ref={videoRef}
src={story.media[0].url} src={primary.url}
className="w-full h-full object-contain" className="w-full h-full object-contain"
autoPlay autoPlay
muted={isMuted} muted={isMuted}
@@ -179,7 +188,7 @@ export const StoryViewer: React.FC<StoryViewerProps> = ({
/> />
) : ( ) : (
<img <img
src={story.media[0].url} src={primary.url}
alt="" alt=""
className="w-full h-full object-contain" className="w-full h-full object-contain"
referrerPolicy="no-referrer" referrerPolicy="no-referrer"
+98 -40
View File
@@ -1,8 +1,11 @@
import { useState, useCallback, useRef } from 'react'; import { useState, useCallback, useRef, useEffect } from 'react';
// @ts-ignore // @ts-ignore
import { XzReadableStream } from 'xz-decompress'; import { XzReadableStream } from 'xz-decompress';
import * as idb from 'idb-keyval'; import { ArchiveFile, CacheData, Post, ServerArchive } from '../types';
import { ArchiveFile, Post, ServerArchive } from '../types'; import { setCachedArchive, getDirectoryHandle } from '../lib/archive-cache';
import { parseArchiveFilename, scopedPostId, EXPORT_RE, INSTALOADER_RE } from '../lib/archive-patterns';
const hasDirectoryHandle = async (name: string) => Boolean(await getDirectoryHandle(name));
export const useArchiveScanner = ( export const useArchiveScanner = (
detectedUsername: string, detectedUsername: string,
@@ -19,6 +22,7 @@ export const useArchiveScanner = (
// Result state // Result state
const [allPosts, setAllPosts] = useState<Post[]>([]); const [allPosts, setAllPosts] = useState<Post[]>([]);
const [allStories, setAllStories] = useState<Post[]>([]); const [allStories, setAllStories] = useState<Post[]>([]);
const [allHighlights, setAllHighlights] = useState<Post[]>([]);
const [profileMetadata, setProfileMetadata] = useState<{ const [profileMetadata, setProfileMetadata] = useState<{
username: string; username: string;
fullName: string; fullName: string;
@@ -39,9 +43,33 @@ export const useArchiveScanner = (
allProfilePics: [], allProfilePics: [],
}); });
/**
* Blob URLs minted for the archive currently in state. They stay alive as long
* as that archive is on screen and are released when it is torn down —
* otherwise every archive ever opened stays resident for the tab's lifetime.
*/
const createdUrlsRef = useRef<string[]>([]);
const revokeCreatedUrls = useCallback(() => {
for (const url of createdUrlsRef.current) {
try { URL.revokeObjectURL(url); } catch { /* already gone */ }
}
createdUrlsRef.current = [];
}, []);
// Release the last archive's URLs when the app unmounts.
useEffect(() => revokeCreatedUrls, [revokeCreatedUrls]);
/** Hand ownership of an externally-minted blob URL to the scanner's cleanup. */
const registerUrl = useCallback((url: string) => {
createdUrlsRef.current.push(url);
}, []);
const resetScannerState = useCallback(() => { const resetScannerState = useCallback(() => {
revokeCreatedUrls();
setAllPosts([]); setAllPosts([]);
setAllStories([]); setAllStories([]);
setAllHighlights([]);
setProfileMetadata({ setProfileMetadata({
username: '', username: '',
fullName: '', fullName: '',
@@ -52,7 +80,7 @@ export const useArchiveScanner = (
profilePic: null, profilePic: null,
allProfilePics: [], allProfilePics: [],
}); });
}, []); }, [revokeCreatedUrls]);
const handleFiles = useCallback(async (files: ArchiveFile[], archiveContext?: ServerArchive) => { const handleFiles = useCallback(async (files: ArchiveFile[], archiveContext?: ServerArchive) => {
if (!files || files.length === 0) return; if (!files || files.length === 0) return;
@@ -66,6 +94,19 @@ export const useArchiveScanner = (
console.log(`[Scanner] Starting scan of ${files.length} files...`); console.log(`[Scanner] Starting scan of ${files.length} files...`);
await new Promise(resolve => setTimeout(resolve, 100)); await new Promise(resolve => setTimeout(resolve, 100));
/**
* Mint a URL for a media file, remembering it if it needs revoking later.
* Synchronous and allocation-free: no file contents are read here.
*/
const mintUrl = (file: ArchiveFile, mimeHint?: string) => {
const url = file.createObjectUrl(mimeHint);
if (file.revocable) createdUrlsRef.current.push(url);
return url;
};
/** Stable identity for a media file, used to rehydrate URLs after a reload. */
const mediaPath = (file: ArchiveFile) => file.webkitRelativePath || file.name;
const parseXZFile = async (file: ArchiveFile) => { const parseXZFile = async (file: ArchiveFile) => {
try { try {
const stream = new XzReadableStream(file.stream()); const stream = new XzReadableStream(file.stream());
@@ -100,8 +141,6 @@ export const useArchiveScanner = (
let localFollowingCount = 0; let localFollowingCount = 0;
let localProfilePic: string | null = null; let localProfilePic: string | null = null;
const exportRegex = /^(\d{4}-\d{2}-\d{2})_(.+?) - (.+?)(?: - (\d+))?(?: - (story))?\.(.+)$/;
const instaloaderRegex = /^(\d{4}-\d{2}-\d{2}_\d{2}-\d{2}-\d{2}_UTC)(?:_(\d+))?(?:_(story))?\.(.+)$/;
const checkIsStory = (obj: any): boolean => { const checkIsStory = (obj: any): boolean => {
if (!obj) return false; if (!obj) return false;
const typeName = obj.__typename || obj.typename || ''; const typeName = obj.__typename || obj.typename || '';
@@ -140,15 +179,15 @@ export const useArchiveScanner = (
continue; continue;
} }
if (file.name.match(exportRegex)) format = 'export'; if (EXPORT_RE.test(file.name)) format = 'export';
else if (file.name.match(instaloaderRegex)) format = 'instaloader'; else if (INSTALOADER_RE.test(file.name)) format = 'instaloader';
// Highlight items match neither pattern, so a profile made up only of
// sidecar directories would otherwise never reach the filename parser.
else if (format === 'unknown' && file.source && file.source.kind !== 'posts') format = 'export';
if (lowerName.includes('_profile_pic.jpg') || (currentUsername && lowerName === `${currentUsername.toLowerCase()}.jpg`)) { if (lowerName.includes('_profile_pic.jpg') || (currentUsername && lowerName === `${currentUsername.toLowerCase()}.jpg`)) {
try { try {
const url = file.url || (await (async () => { const url = mintUrl(file, 'image/jpeg');
const blob = new Blob([await file.arrayBuffer()], { type: 'image/jpeg' });
return URL.createObjectURL(blob);
})());
discoveredProfilePics.push({ name: file.name, url }); discoveredProfilePics.push({ name: file.name, url });
if (format === 'unknown' && lowerName.includes('_profile_pic.jpg')) format = 'instaloader'; if (format === 'unknown' && lowerName.includes('_profile_pic.jpg')) format = 'instaloader';
} catch(e) {} } catch(e) {}
@@ -205,10 +244,10 @@ export const useArchiveScanner = (
if (matchedFile) { if (matchedFile) {
const type = isVideo(matchedFile.name) ? 'video' : 'image'; const type = isVideo(matchedFile.name) ? 'video' : 'image';
const url = matchedFile.url || URL.createObjectURL(new Blob([await matchedFile.arrayBuffer()], { type: type === 'video' ? 'video/mp4' : 'image/jpeg' })); const url = mintUrl(matchedFile, type === 'video' ? 'video/mp4' : 'image/jpeg');
const existingMedia = post.media!.find(media => media.index === mIdx + 1); const existingMedia = post.media!.find(media => media.index === mIdx + 1);
if (existingMedia) { if (type === 'video' && existingMedia.type === 'image') post.media = post.media!.map(media => media.index === mIdx + 1 ? { name: matchedFile!.name, url, type, index: mIdx + 1, size: matchedFile!.size } : media); } if (existingMedia) { if (type === 'video' && existingMedia.type === 'image') post.media = post.media!.map(media => media.index === mIdx + 1 ? { name: matchedFile!.name, path: mediaPath(matchedFile!), url, type, index: mIdx + 1, size: matchedFile!.size } : media); }
else post.media!.push({ name: matchedFile.name, url, type, index: mIdx + 1, size: matchedFile.size }); else post.media!.push({ name: matchedFile.name, path: mediaPath(matchedFile), url, type, index: mIdx + 1, size: matchedFile.size });
} }
} }
if (post.media!.length > 0) postsMap.set(postId, post); if (post.media!.length > 0) postsMap.set(postId, post);
@@ -228,21 +267,25 @@ export const useArchiveScanner = (
setScannedFilesLog(prev => [`Batch ${Math.floor(j_start/CHUNK_SIZE) + 1} processing...`, ...prev.slice(0, 19)]); setScannedFilesLog(prev => [`Batch ${Math.floor(j_start/CHUNK_SIZE) + 1} processing...`, ...prev.slice(0, 19)]);
for (let j = j_start; j < end; j++) { for (let j = j_start; j < end; j++) {
const file = files[j]; const lowerName = file.name.toLowerCase(); const file = files[j]; const lowerName = file.name.toLowerCase();
const expMatch = file.name.match(exportRegex); const kind = file.source?.kind ?? 'posts';
const insMatch = file.name.match(instaloaderRegex); const parsed = parseArchiveFilename(file.name, kind, file.mtime);
if (!expMatch && !insMatch) continue; if (!parsed) continue;
let postId = '', date = '', user = currentUsername || 'archived_user', index = 1, ext = '', isStory = lowerName.includes('story') || file.webkitRelativePath.toLowerCase().includes('stories'); const { date, index, ext } = parsed;
if (expMatch) { const user = parsed.username || currentUsername || 'archived_user';
const [_, dMatch, uMatch, pMatch, iStrMatch, sMatch, eMatch] = expMatch; let isStory = parsed.isStory
date = dMatch; user = uMatch; postId = pMatch; index = iStrMatch ? parseInt(iStrMatch, 10) : 1; if (sMatch) isStory = true; ext = eMatch; || lowerName.includes('story')
} else if (insMatch) { || file.webkitRelativePath.toLowerCase().includes('stories');
const [_, pMatch, iStrMatch, sMatch, eMatch] = insMatch;
postId = pMatch; date = pMatch.split('_')[0]; index = iStrMatch ? parseInt(iStrMatch, 10) : 1; if (sMatch) isStory = true; ext = eMatch; const postId = scopedPostId(parsed.postId, kind, file.source?.dir);
} if (kind === 'stories') isStory = true;
if (kind === 'highlight') isStory = false;
let post = postsMap.get(postId); let post = postsMap.get(postId);
if (!post) { post = { id: postId, date, username: user, caption: '', media: [], isStory }; postsMap.set(postId, post); } if (!post) {
post = { id: postId, date, username: user, caption: '', media: [], isStory, source: kind, highlightTitle: file.source?.title };
postsMap.set(postId, post);
}
else if (isStory) post.isStory = true; else if (isStory) post.isStory = true;
const lowerExt = ext.toLowerCase(); const lowerExt = ext.toLowerCase();
@@ -260,11 +303,11 @@ export const useArchiveScanner = (
} catch (e) {} } catch (e) {}
} else if (isMedia(file.name)) { } else if (isMedia(file.name)) {
const type = isVideo(file.name) ? 'video' : 'image'; const type = isVideo(file.name) ? 'video' : 'image';
const url = file.url || URL.createObjectURL(new Blob([await file.arrayBuffer()], { type: type === 'video' ? 'video/mp4' : 'image/jpeg' })); const url = mintUrl(file, type === 'video' ? 'video/mp4' : 'image/jpeg');
if (type === 'image') throttledSetScanningImage(url); if (type === 'image') throttledSetScanningImage(url);
const existingMedia = post.media!.find(m => m.index === index); const existingMedia = post.media!.find(m => m.index === index);
if (existingMedia) { if (type === 'video' && existingMedia.type === 'image') post.media = post.media!.map(m => m.index === index ? { name: file.name, url, type, index, size: file.size } : m); } if (existingMedia) { if (type === 'video' && existingMedia.type === 'image') post.media = post.media!.map(m => m.index === index ? { name: file.name, path: mediaPath(file), url, type, index, size: file.size } : m); }
else post.media!.push({ name: file.name, url, type, index, size: file.size }); else post.media!.push({ name: file.name, path: mediaPath(file), url, type, index, size: file.size });
} }
} }
await new Promise(resolve => setTimeout(resolve, 0)); await new Promise(resolve => setTimeout(resolve, 0));
@@ -307,9 +350,9 @@ export const useArchiveScanner = (
const post: Post = { id: postId, date: new Date().toISOString().split('T')[0], username: currentUsername || 'archived_user', caption: baseName, media: [], thumbnail: '' }; const post: Post = { id: postId, date: new Date().toISOString().split('T')[0], username: currentUsername || 'archived_user', caption: baseName, media: [], thumbnail: '' };
for (const [idx, file] of batch.entries()) { for (const [idx, file] of batch.entries()) {
const type = isVideo(file.name) ? 'video' : 'image'; const type = isVideo(file.name) ? 'video' : 'image';
const url = file.url || URL.createObjectURL(new Blob([await file.arrayBuffer()], { type: type === 'video' ? 'video/mp4' : 'image/jpeg' })); const url = mintUrl(file, type === 'video' ? 'video/mp4' : 'image/jpeg');
if (type === 'image') throttledSetScanningImage(url); if (type === 'image') throttledSetScanningImage(url);
post.media.push({ name: file.name, url, type, index: idx + 1, size: file.size }); post.media.push({ name: file.name, path: mediaPath(file), url, type, index: idx + 1, size: file.size });
} }
post.thumbnail = post.media[0].url; post.thumbnail = post.media[0].url;
postsMap.set(postId, post); postsMap.set(postId, post);
@@ -327,7 +370,7 @@ export const useArchiveScanner = (
allImageFiles.sort((a, b) => a.name.localeCompare(b.name)); allImageFiles.sort((a, b) => a.name.localeCompare(b.name));
const oldestFile = allImageFiles[0]; const oldestFile = allImageFiles[0];
try { try {
const url = oldestFile.url || URL.createObjectURL(new Blob([await oldestFile.arrayBuffer()], { type: 'image/jpeg' })); const url = mintUrl(oldestFile, 'image/jpeg');
localProfilePic = url; localProfilePic = url;
setProfileMetadata(prev => ({ ...prev, profilePic: localProfilePic, allProfilePics: [url] })); setProfileMetadata(prev => ({ ...prev, profilePic: localProfilePic, allProfilePics: [url] }));
} catch(e) {} } catch(e) {}
@@ -339,11 +382,17 @@ export const useArchiveScanner = (
return { ...p, username: (p.username === 'archived_user' || !p.username) ? finalUsername : p.username, media: sortedMedia, thumbnail: sortedMedia[0].url } as Post; return { ...p, username: (p.username === 'archived_user' || !p.username) ? finalUsername : p.username, media: sortedMedia, thumbnail: sortedMedia[0].url } as Post;
}); });
const posts = allItems.filter(p => !p.isStory).sort((a, b) => b.date.localeCompare(a.date)); const byNewest = (a: Post, b: Post) => b.date.localeCompare(a.date);
const stories = allItems.filter(p => p.isStory).sort((a, b) => b.date.localeCompare(a.date)); // Fixed bug here // Highlights are story-shaped but live behind their own circles, so they
// are kept out of both the grid and the profile-ring story reel.
const highlights = allItems.filter(p => p.source === 'highlight').sort(byNewest);
const rest = allItems.filter(p => p.source !== 'highlight');
const posts = rest.filter(p => !p.isStory).sort(byNewest);
const stories = rest.filter(p => p.isStory).sort(byNewest);
setAllPosts(posts); setAllPosts(posts);
setAllStories(stories); setAllStories(stories);
setAllHighlights(highlights);
setProfileMetadata(prev => ({ setProfileMetadata(prev => ({
...prev, ...prev,
username: finalUsername, username: finalUsername,
@@ -377,10 +426,13 @@ export const useArchiveScanner = (
} catch (e) {} } catch (e) {}
} }
const cacheData = { const cacheData: CacheData = {
name: cacheKey, isLocal, fileCount: archiveToCache ? archiveToCache.fileCount : files.length, name: cacheKey, isLocal,
posts: posts, // Enable caching for local archives fileCount: (archiveToCache ? archiveToCache.fileCount : files.length) ?? files.length,
signature: archiveToCache?.signature,
posts: posts,
stories: stories, stories: stories,
highlights: highlights,
profileMetadata: { profileMetadata: {
username: finalUsername, username: finalUsername,
fullName: localFullName, fullName: localFullName,
@@ -388,13 +440,16 @@ export const useArchiveScanner = (
followerCount: localFollowerCount, followerCount: localFollowerCount,
followingCount: localFollowingCount, followingCount: localFollowingCount,
externalUrl: localExternalUrl, externalUrl: localExternalUrl,
// Local blob: URLs die with the document, so persist a data: URL for
// the dashboard card instead. Media URLs are rehydrated from `path`.
profilePic: isLocal ? cacheThumbnail : localProfilePic, profilePic: isLocal ? cacheThumbnail : localProfilePic,
allProfilePics: isLocal ? (cacheThumbnail ? [cacheThumbnail] : []) : discoveredProfilePics.map(p => p.url) allProfilePics: isLocal ? (cacheThumbnail ? [cacheThumbnail] : []) : discoveredProfilePics.map(p => p.url)
}, },
hasDirectoryHandle: isLocal ? await hasDirectoryHandle(cacheKey) : false,
timestamp: Date.now() timestamp: Date.now()
}; };
try { try {
await idb.set(cacheKey, cacheData); await setCachedArchive(cacheData);
console.log(`[Cache] Data saved successfully.`); console.log(`[Cache] Data saved successfully.`);
await refreshCachedArchives(); await refreshCachedArchives();
} catch (e) { console.error(`[Cache] Save error:`, e); } } catch (e) { console.error(`[Cache] Save error:`, e); }
@@ -411,10 +466,12 @@ export const useArchiveScanner = (
currentScanningImage, currentScanningImage,
allPosts, allPosts,
allStories, allStories,
allHighlights,
profileMetadata, profileMetadata,
handleFiles, handleFiles,
setAllPosts, setAllPosts,
setAllStories, setAllStories,
setAllHighlights,
setProfileMetadata, setProfileMetadata,
setIsScanning, setIsScanning,
setScanningPhase, setScanningPhase,
@@ -422,6 +479,7 @@ export const useArchiveScanner = (
setTotalFiles, setTotalFiles,
setScannedFilesLog, setScannedFilesLog,
setCurrentScanningImage, setCurrentScanningImage,
resetScannerState resetScannerState,
registerUrl
}; };
}; };
+69 -47
View File
@@ -1,47 +1,38 @@
import { useState, useCallback, useRef, useEffect } from 'react'; import { useState, useCallback, useRef, useEffect } from 'react';
import * as idb from 'idb-keyval'; import * as idb from 'idb-keyval';
import { thumbKey } from '../lib/archive-cache';
interface ThumbnailRequest { interface ThumbnailRequest {
id: string; id: string;
key: string;
url: string; url: string;
blob?: Blob; blob?: Blob;
} }
const THUMBNAIL_WIDTH = 400; const THUMBNAIL_WIDTH = 400;
export const useThumbnailQueue = () => { export const useThumbnailQueue = (archiveName: string) => {
const [cacheHits, setCacheHits] = useState<Map<string, string>>(new Map()); const [cacheHits, setCacheHits] = useState<Map<string, string>>(new Map());
const queueRef = useRef<ThumbnailRequest[]>([]); const queueRef = useRef<ThumbnailRequest[]>([]);
const isProcessingRef = useRef(false); const isProcessingRef = useRef(false);
const workerRef = useRef<Worker | null>(null); const workerRef = useRef<Worker | null>(null);
useEffect(() => { /**
// Initialize worker with relative URL (vite will handle this) * Mirrors `cacheHits` for reads inside callbacks.
workerRef.current = new Worker(new URL('../lib/thumbnail-worker.ts', import.meta.url), { *
type: 'module' * `requestThumbnail` is a dependency of every PostThumbnail effect, so it must
}); * keep a stable identity closing over `cacheHits` state directly would give
* it a new identity per completed thumbnail and re-run the effect in all
* ~90 mounted thumbnails each time.
*/
const cacheHitsRef = useRef<Map<string, string>>(new Map());
/** Blob URLs handed out for thumbnails, released when the archive changes. */
const createdUrlsRef = useRef<string[]>([]);
workerRef.current.onmessage = async (e) => { const publish = useCallback((id: string, url: string) => {
const { id, blob, error } = e.data; createdUrlsRef.current.push(url);
cacheHitsRef.current.set(id, url);
if (!error && blob) { setCacheHits(new Map(cacheHitsRef.current));
const url = URL.createObjectURL(blob);
setCacheHits(prev => new Map(prev).set(id, url));
// Persist to IndexedDB (as dataURL for simple retrieval or keep as Blob)
try {
await idb.set(`thumb_${id}`, blob);
} catch (err) {}
}
// Process next in queue
isProcessingRef.current = false;
processNext();
};
return () => {
workerRef.current?.terminate();
};
}, []); }, []);
const processNext = useCallback(async () => { const processNext = useCallback(async () => {
@@ -51,53 +42,84 @@ export const useThumbnailQueue = () => {
const request = queueRef.current.shift()!; const request = queueRef.current.shift()!;
try { try {
// 1. Double check cache before expensive work // Re-check the store: the entry may have landed since being queued.
const cached = await idb.get(`thumb_${request.id}`); const cached = await idb.get(request.key);
if (cached instanceof Blob) { if (cached instanceof Blob) {
const url = URL.createObjectURL(cached); publish(request.id, URL.createObjectURL(cached));
setCacheHits(prev => new Map(prev).set(request.id, url));
isProcessingRef.current = false; isProcessingRef.current = false;
processNext(); processNext();
return; return;
} }
// 2. Fetch original if no blob provided
let blob = request.blob; let blob = request.blob;
if (!blob) { if (!blob) {
const res = await fetch(request.url); const res = await fetch(request.url);
blob = await res.blob(); blob = await res.blob();
} }
// 3. Send to worker // One image at a time: decoding several 50MP+ files concurrently is a
workerRef.current.postMessage({ // reliable way to OOM the tab.
id: request.id, workerRef.current.postMessage({ id: request.key, blob, width: THUMBNAIL_WIDTH });
blob,
width: THUMBNAIL_WIDTH
});
} catch (err) { } catch (err) {
console.error('[ThumbnailQueue] Failed to process:', request.id, err); console.error('[ThumbnailQueue] Failed to process:', request.id, err);
isProcessingRef.current = false; isProcessingRef.current = false;
processNext(); processNext();
} }
}, []); }, [publish]);
useEffect(() => {
workerRef.current = new Worker(new URL('../lib/thumbnail-worker.ts', import.meta.url), {
type: 'module'
});
workerRef.current.onmessage = async (e) => {
const { id: key, blob, error } = e.data;
if (!error && blob) {
const id = key.split(':').slice(2).join(':');
publish(id, URL.createObjectURL(blob));
try {
await idb.set(key, blob);
} catch (err) { /* quota exceeded; the in-memory hit still stands */ }
}
isProcessingRef.current = false;
processNext();
};
return () => {
workerRef.current?.terminate();
};
}, [publish, processNext]);
// Switching archives invalidates every thumbnail URL handed out so far.
useEffect(() => {
return () => {
queueRef.current = [];
for (const url of createdUrlsRef.current) {
try { URL.revokeObjectURL(url); } catch { /* already gone */ }
}
createdUrlsRef.current = [];
cacheHitsRef.current = new Map();
setCacheHits(new Map());
};
}, [archiveName]);
const requestThumbnail = useCallback(async (id: string, url: string, blob?: Blob) => { const requestThumbnail = useCallback(async (id: string, url: string, blob?: Blob) => {
// 1. Sync check state if (cacheHitsRef.current.has(id)) return;
if (cacheHits.has(id)) return;
// 2. Async check idb const key = thumbKey(archiveName, id);
const cached = await idb.get(`thumb_${id}`); const cached = await idb.get(key);
if (cached instanceof Blob) { if (cached instanceof Blob) {
setCacheHits(prev => new Map(prev).set(id, URL.createObjectURL(cached))); publish(id, URL.createObjectURL(cached));
return; return;
} }
// 3. Add to queue if not already there
if (!queueRef.current.some(r => r.id === id)) { if (!queueRef.current.some(r => r.id === id)) {
queueRef.current.push({ id, url, blob }); queueRef.current.push({ id, key, url, blob });
processNext(); processNext();
} }
}, [cacheHits, processNext]); }, [archiveName, publish, processNext]);
return { return {
cacheHits, cacheHits,
+2 -1
View File
@@ -1,4 +1,5 @@
@import url('https://fonts.googleapis.com/css2?family=Inter:wght@300;400;500;600;700&family=Playfair+Display:ital,wght@0,400..900;1,400..900&display=swap'); /* Self-hosted: no third-party font requests, works fully offline. */
@import url('/fonts/fonts.css');
@import "tailwindcss"; @import "tailwindcss";
@theme { @theme {
+170
View File
@@ -0,0 +1,170 @@
import * as idb from 'idb-keyval';
import { CacheData, Post } from '../types';
import { DirectoryHandle, ensureReadPermission, filesFromDirectory } from './directory-handle';
import { LocalArchiveFile } from './archive-files';
/**
* Persistent archive cache.
*
* Keys are namespaced so that listing archives does not require deserializing
* every thumbnail blob in the store: `archive:` entries are metadata, `thumb:`
* entries are image blobs, `handle:` entries are directory handles.
*/
const ARCHIVE_PREFIX = 'archive:';
const THUMB_PREFIX = 'thumb:';
const HANDLE_PREFIX = 'handle:';
export const archiveKey = (name: string) => `${ARCHIVE_PREFIX}${name}`;
export const handleKey = (name: string) => `${HANDLE_PREFIX}${name}`;
/** Thumbnails are scoped per archive; post IDs alone collide across archives. */
export const thumbKey = (archive: string, postId: string) => `${THUMB_PREFIX}${archive}:${postId}`;
export const getCachedArchive = (name: string): Promise<CacheData | undefined> =>
idb.get(archiveKey(name));
export const setCachedArchive = (data: CacheData) => idb.set(archiveKey(data.name), data);
/** Names of all cached archives, without loading their contents. */
export const listCachedArchiveNames = async (): Promise<string[]> =>
(await idb.keys())
.map(String)
.filter(k => k.startsWith(ARCHIVE_PREFIX))
.map(k => k.slice(ARCHIVE_PREFIX.length));
export const listCachedArchives = async (): Promise<CacheData[]> => {
const names = await listCachedArchiveNames();
const entries = await Promise.all(names.map(getCachedArchive));
return entries.filter((e): e is CacheData => Boolean(e));
};
/** Remove an archive along with its handle and every thumbnail it owns. */
export const deleteCachedArchive = async (name: string) => {
const thumbPrefix = `${THUMB_PREFIX}${name}:`;
const stale = (await idb.keys()).map(String).filter(k => k.startsWith(thumbPrefix));
await idb.delMany([archiveKey(name), handleKey(name), ...stale]);
};
export const saveDirectoryHandle = (name: string, handle: DirectoryHandle) =>
idb.set(handleKey(name), handle);
export const getDirectoryHandle = (name: string): Promise<DirectoryHandle | undefined> =>
idb.get(handleKey(name));
/**
* One-time migration from the flat key layout (archive name as a bare key,
* `thumb_<postId>` for thumbnails).
*
* Old local entries are dropped rather than migrated: their media URLs are
* dead blob: URLs, so restoring them would render an archive of broken images.
*/
export const migrateLegacyCache = async () => {
const keys = (await idb.keys()).map(String);
const legacyThumbs = keys.filter(k => k.startsWith('thumb_'));
const legacyArchives = keys.filter(
k => !k.startsWith(ARCHIVE_PREFIX) && !k.startsWith(THUMB_PREFIX) &&
!k.startsWith(HANDLE_PREFIX) && !k.startsWith('thumb_')
);
if (!legacyThumbs.length && !legacyArchives.length) return;
const drop: string[] = [...legacyThumbs];
for (const key of legacyArchives) {
const data = await idb.get(key);
drop.push(key);
if (data && typeof data === 'object' && 'posts' in data && !(data as CacheData).isLocal) {
// Server archives keep working: their URLs are plain HTTP paths.
await setCachedArchive({ ...(data as CacheData), name: key });
}
}
await idb.delMany(drop);
console.log(`[Cache] Migrated legacy cache: dropped ${drop.length} stale keys.`);
};
/**
* Rebuild a server archive's media URLs, which are stable HTTP paths.
*
* `path` is relative to the archives root and already carries the source
* directory (which may be a sidecar such as `story - user`), so it is not
* prefixed with the archive name. Entries cached before `path` existed fall
* back to their stored URL.
*/
const rehydrateRemote = (posts: Post[]): Post[] =>
posts.map(post => {
const media = post.media.map(m => ({
...m,
url: m.path ? `/archives/${encodeURI(m.path)}` : m.url,
}));
return { ...post, media, thumbnail: media[0]?.url ?? post.thumbnail };
});
/**
* Rebuild a local archive's media URLs from a live directory handle, minting
* fresh blob: URLs for the paths recorded at scan time.
*
* Returns null when the folder is no longer reachable (permission declined, or
* the handle no longer resolves), signalling the caller to re-prompt.
*/
const rehydrateLocal = async (
posts: Post[],
handle: DirectoryHandle,
onUrl: (url: string) => void,
): Promise<Post[] | null> => {
if (!(await ensureReadPermission(handle))) return null;
let files: LocalArchiveFile[];
try {
files = await filesFromDirectory(handle);
} catch (err) {
console.warn('[Cache] Directory handle no longer readable:', err);
return null;
}
const byPath = new Map(files.map(f => [f.webkitRelativePath, f]));
return posts.map(post => {
const media = post.media.map(m => {
const file = byPath.get(m.path);
if (!file) return { ...m, url: '' };
const url = file.createObjectUrl(m.type === 'video' ? 'video/mp4' : 'image/jpeg');
onUrl(url);
return { ...m, url };
});
return { ...post, media, thumbnail: media[0]?.url ?? '' };
});
};
export interface RestoredArchive {
posts: Post[];
stories: Post[];
highlights: Post[];
profileMetadata: CacheData['profileMetadata'];
}
/**
* Turn a cache entry back into displayable state.
*
* `onUrl` receives every blob: URL minted so the caller can revoke them later.
* Returns null if a local archive's folder can no longer be reached.
*/
export const restoreArchive = async (
data: CacheData,
onUrl: (url: string) => void,
): Promise<RestoredArchive | null> => {
if (!data.isLocal) {
return {
posts: rehydrateRemote(data.posts),
stories: rehydrateRemote(data.stories),
highlights: rehydrateRemote(data.highlights ?? []),
profileMetadata: data.profileMetadata,
};
}
const handle = await getDirectoryHandle(data.name);
if (!handle) return null;
const posts = await rehydrateLocal(data.posts, handle, onUrl);
if (!posts) return null;
const stories = (await rehydrateLocal(data.stories, handle, onUrl)) ?? [];
const highlights = (await rehydrateLocal(data.highlights ?? [], handle, onUrl)) ?? [];
return { posts, stories, highlights, profileMetadata: data.profileMetadata };
};
+38 -4
View File
@@ -1,21 +1,51 @@
import { ArchiveFile } from '../types'; import { ArchiveFile, ArchiveSource } from '../types';
export class LocalArchiveFile implements ArchiveFile { export class LocalArchiveFile implements ArchiveFile {
constructor(private file: File) {} /** Blob URLs minted here are revocable and must be released when done. */
readonly revocable = true;
/**
* @param explicitPath Set when the file came from the File System Access API,
* whose File objects carry an empty webkitRelativePath.
*/
constructor(private file: File, private explicitPath?: string) {}
get name() { return this.file.name; } get name() { return this.file.name; }
get webkitRelativePath() { return this.file.webkitRelativePath; } get webkitRelativePath() { return this.explicitPath ?? this.file.webkitRelativePath; }
get size() { return this.file.size; } get size() { return this.file.size; }
text() { return this.file.text(); } text() { return this.file.text(); }
arrayBuffer() { return this.file.arrayBuffer(); } arrayBuffer() { return this.file.arrayBuffer(); }
stream() { return this.file.stream(); } stream() { return this.file.stream(); }
/**
* A blob: URL backed directly by the on-disk File.
*
* Deliberately does NOT go through arrayBuffer() a File is already a Blob,
* so this hands the browser a disk-backed handle instead of pulling the whole
* file into memory. Doing otherwise means a 20GB archive tries to become 20GB
* of resident blobs.
*
* When the picker gave us no MIME type, slice() re-tags the blob with a hint.
* slice() is a zero-copy view, so this stays memory-free either way.
*/
createObjectUrl(mimeHint?: string) {
const source = this.file.type || !mimeHint
? this.file
: this.file.slice(0, this.file.size, mimeHint);
return URL.createObjectURL(source);
}
} }
export class RemoteArchiveFile implements ArchiveFile { export class RemoteArchiveFile implements ArchiveFile {
/** Served over HTTP; there is no object URL to release. */
readonly revocable = false;
constructor( constructor(
public name: string, public name: string,
public webkitRelativePath: string, public webkitRelativePath: string,
public size: number, public size: number,
public url: string public url: string,
public source?: ArchiveSource,
public mtime?: number
) {} ) {}
async text() { async text() {
const res = await fetch(this.url); const res = await fetch(this.url);
@@ -33,4 +63,8 @@ export class RemoteArchiveFile implements ArchiveFile {
}); });
return transform.readable; return transform.readable;
} }
createObjectUrl() {
return this.url;
}
} }
+89
View File
@@ -0,0 +1,89 @@
import { describe, expect, it } from 'vitest';
import { classifyDirectory, groupArchiveDirectories } from './archive-grouping';
describe('classifyDirectory', () => {
it('treats a bare profile directory as the base', () => {
expect(classifyDirectory('4utumn07')).toEqual({
owner: '4utumn07',
source: { kind: 'posts', dir: '4utumn07' },
});
});
it('recognises a reels sidecar', () => {
expect(classifyDirectory('4utumn07 - reels')).toEqual({
owner: '4utumn07',
source: { kind: 'reels', dir: '4utumn07 - reels' },
});
});
it('recognises a stories sidecar', () => {
expect(classifyDirectory('story - dawn_petal')).toEqual({
owner: 'dawn_petal',
source: { kind: 'stories', dir: 'story - dawn_petal' },
});
});
it('splits highlight owner from title', () => {
const { owner, source } = classifyDirectory('story highlights - 4utumn07 - Sunstory');
expect(owner).toBe('4utumn07');
expect(source.kind).toBe('highlight');
expect(source.title).toBe('Sunstory');
});
it.each([
['story highlights - theoldlyricmuseinsta - 💙1999-2005 era', 'theoldlyricmuseinsta', '💙1999-2005 era'],
['story highlights - member_theworld - [Bracket]', 'member_theworld', '[Bracket]'],
['story highlights - official_band - Tour Schedule', 'official_band', 'Tour Schedule'],
['story highlights - 4utumn07 - Sketching', '4utumn07', 'Sketching'],
['story highlights - official_band - A.B.C', 'official_band', 'A.B.C'],
])('handles real-world title %s', (dir, owner, title) => {
const result = classifyDirectory(dir);
expect(result.owner).toBe(owner);
expect(result.source.title).toBe(title);
});
it('keeps titles containing " - " intact', () => {
// The username is matched as a non-space run, so only the first separator
// splits owner from title.
const { owner, source } = classifyDirectory('story highlights - user - a - b');
expect(owner).toBe('user');
expect(source.title).toBe('a - b');
});
it('does not mistake a profile with spaces for a sidecar', () => {
expect(classifyDirectory('Heejin_Bubble heejinmedia').source.kind).toBe('posts');
});
});
describe('groupArchiveDirectories', () => {
const dirs = [
'4utumn07',
'4utumn07 - reels',
'story - 4utumn07',
'story highlights - 4utumn07 - Sunstory',
'story highlights - 4utumn07 - Sketching',
'kestrelsings',
];
it('folds sidecars into their base profile', () => {
const groups = groupArchiveDirectories(dirs);
expect([...groups.keys()].sort()).toEqual(['4utumn07', 'kestrelsings']);
expect(groups.get('4utumn07')).toHaveLength(5);
expect(groups.get('kestrelsings')).toHaveLength(1);
});
it('orders sources posts, reels, stories, then highlights by title', () => {
const sources = groupArchiveDirectories(dirs).get('4utumn07')!;
expect(sources.map(s => s.kind)).toEqual(['posts', 'reels', 'stories', 'highlight', 'highlight']);
expect(sources.slice(3).map(s => s.title)).toEqual(['Sketching', 'Sunstory']);
});
it('still groups a sidecar whose base profile is missing', () => {
const groups = groupArchiveDirectories(['story - orphan']);
expect(groups.get('orphan')).toEqual([{ kind: 'stories', dir: 'story - orphan' }]);
});
it('is stable for an empty archive root', () => {
expect(groupArchiveDirectories([]).size).toBe(0);
});
});
+71
View File
@@ -0,0 +1,71 @@
/**
* Archive directory naming rules.
*
* Shared by the server (to fold sidecar directories into one profile) and the
* test suite. Kept free of Node built-ins so it can be imported from either.
*/
export type SourceKind = 'posts' | 'reels' | 'stories' | 'highlight';
export interface ArchiveSource {
kind: SourceKind;
/** Directory name relative to the archives root. */
dir: string;
/** Highlight title, for kind === 'highlight'. */
title?: string;
}
/**
* Sidecar directories sit next to the profile directory they belong to:
*
* 4utumn07 -> posts (base)
* 4utumn07 - reels -> reels
* story - 4utumn07 -> stories
* story highlights - 4utumn07 - Sunstory -> highlight "Sunstory"
*
* Instagram usernames cannot contain spaces, so matching the username as a
* run of non-space characters reliably separates it from a highlight title
* (titles may themselves contain spaces, dashes and emoji).
*/
export const classifyDirectory = (dirName: string): { owner: string; source: ArchiveSource } => {
const highlight = /^story highlights - ([^ ]+) - (.+)$/.exec(dirName);
if (highlight) {
return { owner: highlight[1], source: { kind: 'highlight', dir: dirName, title: highlight[2] } };
}
const stories = /^story - ([^ ]+)$/.exec(dirName);
if (stories) {
return { owner: stories[1], source: { kind: 'stories', dir: dirName } };
}
const reels = /^([^ ]+) - reels$/.exec(dirName);
if (reels) {
return { owner: reels[1], source: { kind: 'reels', dir: dirName } };
}
return { owner: dirName, source: { kind: 'posts', dir: dirName } };
};
const RANK: Record<SourceKind, number> = { posts: 0, reels: 1, stories: 2, highlight: 3 };
/**
* Group the archive root's directories by profile.
*
* A sidecar whose owner has no base directory still forms a group of its own,
* so nothing becomes invisible just because the base profile is missing.
*/
export const groupArchiveDirectories = (dirNames: string[]): Map<string, ArchiveSource[]> => {
const groups = new Map<string, ArchiveSource[]>();
for (const dirName of dirNames) {
const { owner, source } = classifyDirectory(dirName);
if (!groups.has(owner)) groups.set(owner, []);
groups.get(owner)!.push(source);
}
for (const sources of groups.values()) {
sources.sort((a, b) => RANK[a.kind] - RANK[b.kind] || (a.title ?? '').localeCompare(b.title ?? ''));
}
return groups;
};
+232
View File
@@ -0,0 +1,232 @@
import fs from 'fs';
import fsp from 'fs/promises';
import path from 'path';
import { ArchiveSource, SourceKind, groupArchiveDirectories } from './archive-grouping.js';
/**
* On-disk archive index.
*
* Archives live on network storage where per-file `stat` costs ~1.4ms and does
* not parallelise well, so walking every file on each request is unaffordable:
* measured against a real 110k-file archive root, listing took ~52s.
*
* Directory `stat` is effectively free, so each source directory is indexed
* once and re-used until its mtime changes. The index is warmed in the
* background at startup and persisted, making steady-state requests instant.
*/
export interface IndexedFile {
path: string;
size: number;
mtime: number;
kind: SourceKind;
title?: string;
}
interface DirIndex {
dir: string;
/** Directory mtime the index was built from; the cache key. */
mtimeMs: number;
files: IndexedFile[];
}
const MEDIA_RE = /\.(jpg|jpeg|png|webp|gif|bmp|tiff|mp4|webm|ogv|mov)$/i;
const STAT_CONCURRENCY = 16;
export class ArchiveIndex {
private dirs = new Map<string, DirIndex>();
private inFlight = new Map<string, Promise<DirIndex>>();
private dirty = false;
constructor(private archivesDir: string, private cachePath: string) {}
/** Visible (non-system) directories at the archive root. */
private listRootDirs(): string[] {
return fs.readdirSync(this.archivesDir, { withFileTypes: true })
.filter(e => e.isDirectory() && !/^[.@_]/.test(e.name))
.map(e => e.name);
}
groups(): Map<string, ArchiveSource[]> {
return groupArchiveDirectories(this.listRootDirs());
}
private dirMtime(dir: string): number {
try {
return fs.statSync(path.join(this.archivesDir, dir)).mtimeMs;
} catch {
return 0;
}
}
/** Recursively list relative file paths without stat()ing them. */
private walk(absDir: string, base = ''): string[] {
let out: string[] = [];
let entries: fs.Dirent[];
try {
entries = fs.readdirSync(absDir, { withFileTypes: true });
} catch {
return out;
}
for (const entry of entries) {
const rel = base ? `${base}/${entry.name}` : entry.name;
if (entry.isDirectory()) out = out.concat(this.walk(path.join(absDir, entry.name), rel));
else if (entry.isFile()) out.push(rel);
}
return out;
}
private async buildDir(source: ArchiveSource): Promise<DirIndex> {
const started = Date.now();
const mtimeMs = this.dirMtime(source.dir);
const absDir = path.join(this.archivesDir, source.dir);
const relPaths = this.walk(absDir);
// Only media needs a size (the client thumbnails anything over 1MiB), and
// only highlights need an mtime (their filenames carry no date). Skipping
// the rest avoids thousands of pointless round trips.
const needsStat = (rel: string) => MEDIA_RE.test(rel) || source.kind === 'highlight';
const files: IndexedFile[] = relPaths.map(rel => ({
path: `${source.dir}/${rel}`,
size: 0,
mtime: 0,
kind: source.kind,
...(source.title ? { title: source.title } : {}),
}));
const targets = files.filter((_, i) => needsStat(relPaths[i]));
let cursor = 0;
const worker = async () => {
while (cursor < targets.length) {
const file = targets[cursor++];
try {
const stat = await fsp.stat(path.join(this.archivesDir, file.path));
file.size = stat.size;
file.mtime = stat.mtimeMs;
} catch { /* raced with a delete */ }
}
};
await Promise.all(Array.from({ length: STAT_CONCURRENCY }, worker));
console.log(
`[Index] ${source.dir}: ${files.length} files (${targets.length} statted) in ${((Date.now() - started) / 1000).toFixed(1)}s`
);
this.dirty = true;
return { dir: source.dir, mtimeMs, files };
}
/** Index for one source directory, rebuilding only if its mtime moved. */
private async ensureDir(source: ArchiveSource): Promise<DirIndex> {
const cached = this.dirs.get(source.dir);
const mtimeMs = this.dirMtime(source.dir);
if (cached && cached.mtimeMs === mtimeMs) return cached;
// Collapse concurrent requests for the same directory into one walk.
const existing = this.inFlight.get(source.dir);
if (existing) return existing;
const build = this.buildDir(source).then(index => {
this.dirs.set(source.dir, index);
this.inFlight.delete(source.dir);
return index;
}).catch(err => {
this.inFlight.delete(source.dir);
throw err;
});
this.inFlight.set(source.dir, build);
return build;
}
/** All files for one profile, across its base and sidecar directories. */
async filesFor(owner: string): Promise<IndexedFile[] | null> {
const sources = this.groups().get(owner);
if (!sources?.length) return null;
const indexes = await Promise.all(sources.map(s => this.ensureDir(s)));
return indexes.flatMap(i => i.files);
}
/**
* A cheap change signature for a profile, used by the client to decide
* whether its cached copy is stale. Built from directory mtimes only, so it
* costs one stat per source directory rather than a full walk.
*/
signatureFor(sources: ArchiveSource[]): string {
return sources.map(s => `${s.dir}:${this.dirMtime(s.dir)}`).join('|');
}
/** File count for a profile, if its directories are already indexed. */
countFor(sources: ArchiveSource[]): number | null {
let total = 0;
for (const source of sources) {
const cached = this.dirs.get(source.dir);
if (!cached) return null;
total += cached.files.length;
}
return total;
}
/**
* Best-effort profile picture.
*
* Probes the conventional filenames first (one stat each) and only falls back
* to the indexed listing, so an unindexed archive still gets a thumbnail
* without triggering a walk.
*/
thumbnailFor(owner: string, sources: ArchiveSource[]): string {
const base = sources.find(s => s.kind === 'posts') ?? sources[0];
if (!base) return '';
for (const candidate of [`${owner}.jpg`, `${owner}_profile_pic.jpg`, `${owner}.jpeg`, `${owner}.png`]) {
if (fs.existsSync(path.join(this.archivesDir, base.dir, candidate))) {
return `/archives/${encodeURI(`${base.dir}/${candidate}`)}`;
}
}
const cached = this.dirs.get(base.dir);
if (cached) {
const pick = cached.files.find(f => /_profile_pic\.jpg$/i.test(f.path))
?? cached.files.find(f => /\.(jpg|jpeg|png|webp)$/i.test(f.path));
if (pick) return `/archives/${encodeURI(pick.path)}`;
}
return '';
}
/** Walk every directory once, in the background, so first opens are fast. */
async warm(): Promise<void> {
const started = Date.now();
const sources = [...this.groups().values()].flat();
console.log(`[Index] Warming ${sources.length} source directories...`);
for (const source of sources) {
try {
await this.ensureDir(source);
} catch (err) {
console.error(`[Index] Failed to index ${source.dir}:`, err);
}
}
await this.save();
console.log(`[Index] Warm complete in ${((Date.now() - started) / 1000).toFixed(1)}s`);
}
async load(): Promise<void> {
try {
const raw = await fsp.readFile(this.cachePath, 'utf8');
const parsed: DirIndex[] = JSON.parse(raw);
for (const entry of parsed) this.dirs.set(entry.dir, entry);
console.log(`[Index] Loaded ${this.dirs.size} directories from ${this.cachePath}`);
} catch {
console.log('[Index] No usable index cache; will build from scratch.');
}
}
async save(): Promise<void> {
if (!this.dirty) return;
try {
await fsp.writeFile(this.cachePath, JSON.stringify([...this.dirs.values()]), 'utf8');
this.dirty = false;
console.log(`[Index] Persisted ${this.dirs.size} directories to ${this.cachePath}`);
} catch (err) {
console.warn('[Index] Could not persist index (continuing in memory):', err);
}
}
}
+118
View File
@@ -0,0 +1,118 @@
import { describe, expect, it } from 'vitest';
import { parseArchiveFilename, scopedPostId } from './archive-patterns';
describe('parseArchiveFilename — Instagram export format', () => {
it('parses a single-image post', () => {
expect(parseArchiveFilename('2023-04-19_4utumn07 - CrORBIcJJbM.mp4')).toEqual({
postId: 'CrORBIcJJbM',
date: '2023-04-19',
username: '4utumn07',
index: 1,
ext: 'mp4',
isStory: false,
});
});
it('parses a carousel slide index', () => {
const parsed = parseArchiveFilename('2023-04-12_4utumn07 - Cq8LrxSJAJE - 3.jpg');
expect(parsed).toMatchObject({ postId: 'Cq8LrxSJAJE', index: 3, ext: 'jpg' });
});
it('groups a carousel under one post id', () => {
const ids = ['1', '2', '3'].map(
n => parseArchiveFilename(`2023-04-12_user - Cq8LrxSJAJE - ${n}.jpg`)!.postId,
);
expect(new Set(ids).size).toBe(1);
});
it('parses caption sidecar files', () => {
expect(parseArchiveFilename('2023-04-12_4utumn07 - Cq8LrxSJAJE.txt')).toMatchObject({
postId: 'Cq8LrxSJAJE',
ext: 'txt',
});
});
it('flags an explicit story suffix', () => {
expect(parseArchiveFilename('2023-04-12_user - ABC - story.jpg')?.isStory).toBe(true);
});
it('parses the story sidecar layout (date_user - N - shortcode)', () => {
// Files in `story - <user>` carry a per-day ordinal before the shortcode.
const parsed = parseArchiveFilename('2025-10-26_4utumn07 - 2 - DQRuDx9iW5Q.jpg', 'stories');
expect(parsed).toMatchObject({ date: '2025-10-26', username: '4utumn07', ext: 'jpg' });
expect(parsed!.postId).toContain('DQRuDx9iW5Q');
});
it('gives each story item a distinct id', () => {
const a = parseArchiveFilename('2026-08-13_u - 1 - Db-UTJcCUUr.mp4', 'stories')!.postId;
const b = parseArchiveFilename('2026-08-13_u - 2 - Db-oNJ1CWQ4.mp4', 'stories')!.postId;
expect(a).not.toBe(b);
});
});
describe('parseArchiveFilename — Instaloader format', () => {
it('parses a timestamped filename', () => {
expect(parseArchiveFilename('2024-01-01_12-00-00_UTC.jpg')).toMatchObject({
postId: '2024-01-01_12-00-00_UTC',
date: '2024-01-01',
index: 1,
});
});
it('parses the carousel suffix', () => {
expect(parseArchiveFilename('2024-01-01_12-00-00_UTC_2.jpg')?.index).toBe(2);
});
it('flags the story suffix', () => {
expect(parseArchiveFilename('2024-01-01_12-00-00_UTC_story.jpg')?.isStory).toBe(true);
});
});
describe('parseArchiveFilename — story highlights', () => {
it('parses the dateless highlight layout', () => {
expect(parseArchiveFilename('4utumn07 - C5dQPEYpd9W.mp4', 'highlight')).toMatchObject({
postId: 'C5dQPEYpd9W',
username: '4utumn07',
ext: 'mp4',
isStory: false,
});
});
it('dates a highlight from mtime when the filename has none', () => {
const mtime = Date.UTC(2024, 4, 17, 12, 0, 0);
expect(parseArchiveFilename('user - ABC.jpg', 'highlight', mtime)?.date).toBe('2024-05-17');
});
it('leaves the date empty when no mtime is available', () => {
expect(parseArchiveFilename('user - ABC.jpg', 'highlight')?.date).toBe('');
});
it('does not apply the loose highlight pattern outside highlight directories', () => {
// Would otherwise swallow ordinary "a - b.jpg" filenames.
expect(parseArchiveFilename('user - ABC.jpg', 'posts')).toBeNull();
});
});
describe('parseArchiveFilename — non-matching files', () => {
it.each(['4utumn07.jpg', 'profile_pic.jpg', 'README.md', 'no-separator.png'])(
'returns null for %s',
name => expect(parseArchiveFilename(name)).toBeNull(),
);
});
describe('scopedPostId', () => {
it('leaves base-profile ids untouched so permalinks keep working', () => {
expect(scopedPostId('Cq8LrxSJAJE', 'posts')).toBe('Cq8LrxSJAJE');
});
it('namespaces sidecar ids by directory', () => {
expect(scopedPostId('C5dQ', 'highlight', 'story highlights - u - Sunstory'))
.toBe('story highlights - u - Sunstory/C5dQ');
});
it('keeps the same shortcode distinct across sources', () => {
const inPosts = scopedPostId('ABC', 'posts');
const inHighlight = scopedPostId('ABC', 'highlight', 'story highlights - u - H');
expect(inPosts).not.toBe(inHighlight);
});
});
+99
View File
@@ -0,0 +1,99 @@
import { SourceKind } from '../types';
/**
* Filename parsing rules for the archive formats the viewer understands.
*
* Kept as pure functions so the riskiest part of the scanner deriving post
* identity, date and carousel order from a filename can be tested directly.
*/
/** Instagram export: `2023-04-12_user - Cq8LrxSJAJE - 2.jpg` */
export const EXPORT_RE = /^(\d{4}-\d{2}-\d{2})_(.+?) - (.+?)(?: - (\d+))?(?: - (story))?\.(.+)$/;
/** Instaloader: `2024-01-01_12-00-00_UTC_2.jpg` */
export const INSTALOADER_RE = /^(\d{4}-\d{2}-\d{2}_\d{2}-\d{2}-\d{2}_UTC)(?:_(\d+))?(?:_(story))?\.(.+)$/;
/**
* Story highlight: `user - C5dQPEYpd9W.mp4` no date prefix.
*
* Loose enough to match ordinary filenames, so it is only applied to files the
* server has already tagged as coming from a highlight directory.
*/
export const HIGHLIGHT_RE = /^(.+?) - ([A-Za-z0-9_-]+)\.(\w+)$/;
export interface ParsedFilename {
postId: string;
/** ISO date (YYYY-MM-DD), or '' when the filename carries none. */
date: string;
username: string;
/** 1-based carousel position. */
index: number;
ext: string;
isStory: boolean;
}
/**
* Parse an archive filename into post identity.
*
* `kind` selects which patterns apply; `mtime` supplies a date for formats that
* have none (highlights), so those items still sort and render sensibly.
* Returns null when no pattern matches e.g. a profile picture.
*/
export const parseArchiveFilename = (
fileName: string,
kind: SourceKind = 'posts',
mtime?: number,
): ParsedFilename | null => {
const exp = EXPORT_RE.exec(fileName);
if (exp) {
const [, date, username, postId, indexStr, story, ext] = exp;
return {
postId,
date,
username,
index: indexStr ? parseInt(indexStr, 10) : 1,
ext,
isStory: Boolean(story),
};
}
const ins = INSTALOADER_RE.exec(fileName);
if (ins) {
const [, postId, indexStr, story, ext] = ins;
return {
postId,
date: postId.split('_')[0],
username: '',
index: indexStr ? parseInt(indexStr, 10) : 1,
ext,
isStory: Boolean(story),
};
}
if (kind === 'highlight') {
const hl = HIGHLIGHT_RE.exec(fileName);
if (hl) {
const [, username, shortcode, ext] = hl;
return {
postId: shortcode,
date: mtime ? new Date(mtime).toISOString().split('T')[0] : '',
username,
index: 1,
ext,
isStory: false,
};
}
}
return null;
};
/**
* Namespace a post ID by its source directory.
*
* Base-profile IDs are left untouched so existing permalinks keep working;
* sidecar IDs are prefixed so a shortcode appearing in both the profile and a
* highlight stays two distinct posts.
*/
export const scopedPostId = (postId: string, kind: SourceKind, dir?: string): string =>
kind === 'posts' ? postId : `${dir ?? kind}/${postId}`;
+86
View File
@@ -0,0 +1,86 @@
import { LocalArchiveFile } from './archive-files';
/**
* File System Access API helpers.
*
* A `blob:` URL dies with the document, so a cached local archive whose media
* URLs are blob: URLs is worthless after a reload. A FileSystemDirectoryHandle,
* by contrast, is structured-cloneable and survives in IndexedDB so we can
* re-open the same folder on a return visit and mint fresh URLs from it.
*
* Only Chromium implements showDirectoryPicker today; callers must handle the
* unsupported case by falling back to the <input webkitdirectory> flow.
*/
// Minimal typings — TS's lib.dom does not ship these in the configured version.
type PermissionState = 'granted' | 'denied' | 'prompt';
interface FileSystemHandlePermissionDescriptor { mode?: 'read' | 'readwrite' }
export interface DirectoryHandle {
name: string;
kind: 'directory';
values(): AsyncIterableIterator<DirectoryHandle | FileHandle>;
queryPermission?(d?: FileSystemHandlePermissionDescriptor): Promise<PermissionState>;
requestPermission?(d?: FileSystemHandlePermissionDescriptor): Promise<PermissionState>;
}
interface FileHandle {
name: string;
kind: 'file';
getFile(): Promise<File>;
}
export const isDirectoryPickerSupported = () =>
typeof window !== 'undefined' && 'showDirectoryPicker' in window;
export const pickDirectory = async (): Promise<DirectoryHandle | null> => {
if (!isDirectoryPickerSupported()) return null;
try {
return await (window as any).showDirectoryPicker({ mode: 'read' });
} catch (err) {
// AbortError simply means the user dismissed the picker.
return null;
}
};
/**
* Confirm we may still read this handle. Returns false when the user declines
* or the grant has lapsed, in which case the caller should re-prompt.
*
* `requestPermission` must be called from a user gesture, so only call this
* while handling a click.
*/
export const ensureReadPermission = async (handle: DirectoryHandle): Promise<boolean> => {
try {
if (!handle.queryPermission) return true;
if ((await handle.queryPermission({ mode: 'read' })) === 'granted') return true;
if (!handle.requestPermission) return false;
return (await handle.requestPermission({ mode: 'read' })) === 'granted';
} catch {
return false;
}
};
/**
* Recursively collect every file in the directory.
*
* Paths are prefixed with the root directory's name so they line up with the
* `webkitRelativePath` values produced by <input webkitdirectory>, keeping
* cached media paths valid regardless of which picker created them.
*/
export const filesFromDirectory = async (handle: DirectoryHandle): Promise<LocalArchiveFile[]> => {
const out: LocalArchiveFile[] = [];
const walk = async (dir: DirectoryHandle, prefix: string) => {
for await (const entry of dir.values()) {
const entryPath = `${prefix}/${entry.name}`;
if (entry.kind === 'directory') {
await walk(entry as DirectoryHandle, entryPath);
} else {
const file = await (entry as FileHandle).getFile();
out.push(new LocalArchiveFile(file, entryPath));
}
}
};
await walk(handle, handle.name);
return out;
};
+20
View File
@@ -1,6 +1,26 @@
import { clsx, type ClassValue } from 'clsx'; import { clsx, type ClassValue } from 'clsx';
import { twMerge } from 'tailwind-merge'; import { twMerge } from 'tailwind-merge';
import { format, parseISO } from 'date-fns';
export function cn(...inputs: ClassValue[]) { export function cn(...inputs: ClassValue[]) {
return twMerge(clsx(inputs)); return twMerge(clsx(inputs));
} }
/**
* Format an archive date, tolerating junk.
*
* Dates are derived from filenames and arbitrary archive JSON, and date-fns
* `format` throws a RangeError on an invalid date which would take down the
* whole modal for one malformed name. Story highlights in particular carry no
* date at all when file mtimes are unavailable.
*/
export function formatDateSafe(date: string | undefined, pattern: string): string {
if (!date) return '';
try {
const parsed = parseISO(date);
if (Number.isNaN(parsed.getTime())) return '';
return format(parsed, pattern);
} catch {
return '';
}
}
+4 -1
View File
@@ -1,6 +1,7 @@
import {StrictMode} from 'react'; import {StrictMode} from 'react';
import {createRoot} from 'react-dom/client'; import {createRoot} from 'react-dom/client';
import App from './App.tsx'; import App from './App.tsx';
import { ErrorBoundary } from './components/ErrorBoundary';
import './index.css'; import './index.css';
import { registerSW } from 'virtual:pwa-register'; import { registerSW } from 'virtual:pwa-register';
@@ -26,6 +27,8 @@ const updateSW = registerSW({
createRoot(document.getElementById('root')!).render( createRoot(document.getElementById('root')!).render(
<StrictMode> <StrictMode>
<App /> <ErrorBoundary>
<App />
</ErrorBoundary>
</StrictMode>, </StrictMode>,
); );
+87
View File
@@ -1,11 +1,32 @@
export interface MediaFile { export interface MediaFile {
name: string; name: string;
/**
* Path relative to the archive root (matching webkitRelativePath for local
* folders). Unlike `url`, this survives a page reload, so it is what the
* cache persists and what URLs are rehydrated from.
*/
path: string;
url: string; url: string;
type: 'image' | 'video'; type: 'image' | 'video';
index: number; index: number;
size?: number; size?: number;
} }
/**
* Which sidecar directory a post came from. Archives store reels, stories and
* each story highlight in directories alongside the base profile; the viewer
* folds them into one profile and routes them by kind.
*/
export type SourceKind = 'posts' | 'reels' | 'stories' | 'highlight';
export interface ArchiveSource {
kind: SourceKind;
/** Directory name relative to the archives root. */
dir: string;
/** Highlight title, for kind === 'highlight'. */
title?: string;
}
export interface Post { export interface Post {
id: string; id: string;
date: string; date: string;
@@ -14,6 +35,10 @@ export interface Post {
media: MediaFile[]; media: MediaFile[];
thumbnail: string; thumbnail: string;
isStory?: boolean; isStory?: boolean;
/** Defaults to 'posts' for archives without sidecar directories. */
source?: SourceKind;
/** Highlight this post belongs to, for source === 'highlight'. */
highlightTitle?: string;
} }
/** /**
@@ -27,11 +52,73 @@ export interface ArchiveFile {
arrayBuffer(): Promise<ArrayBuffer>; arrayBuffer(): Promise<ArrayBuffer>;
stream(): ReadableStream<Uint8Array>; stream(): ReadableStream<Uint8Array>;
url?: string; url?: string;
/**
* A URL pointing at this file's contents. Local files mint a disk-backed
* blob: URL (no data is read into memory); remote files return their HTTP URL.
*/
createObjectUrl(mimeHint?: string): string;
/** True when createObjectUrl() returns a blob: URL that must be revoked. */
readonly revocable: boolean;
/** Which sidecar directory this file came from, when known. */
source?: ArchiveSource;
/** Last-modified time (ms). Used to date items whose filename has no date. */
mtime?: number;
} }
export interface ServerArchive { export interface ServerArchive {
name: string; name: string;
thumbnail: string; thumbnail: string;
path: string; path: string;
/** Null until the server has indexed this profile. */
fileCount: number | null;
/**
* Directory-mtime signature. Cheap for the server to compute and sufficient
* to detect changes, unlike a file count that would require a full walk.
*/
signature?: string;
/** Base profile plus any sidecar directories folded into it. */
sources?: ArchiveSource[];
}
/** One entry from GET /api/archives/:name/files. */
export interface ServerArchiveFile {
path: string;
size: number;
mtime: number;
kind: SourceKind;
title?: string;
}
export interface ProfileMetadata {
username: string;
fullName: string;
bio: string;
followerCount: number;
followingCount: number;
externalUrl: string;
profilePic: string | null;
allProfilePics: string[];
}
/** Shape of an archive entry persisted in IndexedDB. */
export interface CacheData {
name: string;
isLocal: boolean;
fileCount: number; fileCount: number;
/** Server archives: the signature this entry was built from. */
signature?: string;
posts: Post[];
stories: Post[];
/** Story-highlight items, grouped by `highlightTitle`. */
highlights?: Post[];
profileMetadata: ProfileMetadata;
timestamp: number;
/**
* Local archives only: whether a FileSystemDirectoryHandle was stored
* alongside this entry, meaning media URLs can be rehydrated without
* re-prompting for the folder.
*/
hasDirectoryHandle?: boolean;
/** Path of the profile picture, for rehydration (local archives). */
profilePicPath?: string;
} }
+12
View File
@@ -0,0 +1,12 @@
import 'react';
declare module 'react' {
interface InputHTMLAttributes<T> {
/**
* Non-standard attribute that makes a file input select a whole directory.
* Supported in Chromium and WebKit; used as the fallback picker where the
* File System Access API is unavailable.
*/
webkitdirectory?: string;
}
}
+7 -25
View File
@@ -1,11 +1,10 @@
import tailwindcss from '@tailwindcss/vite'; import tailwindcss from '@tailwindcss/vite';
import react from '@vitejs/plugin-react'; import react from '@vitejs/plugin-react';
import path from 'path'; import path from 'path';
import {defineConfig, loadEnv} from 'vite'; import {defineConfig} from 'vite';
import { VitePWA } from 'vite-plugin-pwa'; import { VitePWA } from 'vite-plugin-pwa';
export default defineConfig(({mode}) => { export default defineConfig(() => {
const env = loadEnv(mode, '.', '');
return { return {
plugins: [ plugins: [
react(), react(),
@@ -21,13 +20,13 @@ export default defineConfig(({mode}) => {
display: 'standalone', display: 'standalone',
icons: [ icons: [
{ {
src: 'https://cdn-icons-png.flaticon.com/512/174/174855.png', src: '/icon-512.png',
sizes: '512x512', sizes: '512x512',
type: 'image/png', type: 'image/png',
purpose: 'any maskable' purpose: 'any maskable'
}, },
{ {
src: 'https://cdn-icons-png.flaticon.com/192/174/174855.png', src: '/icon-192.png',
sizes: '192x192', sizes: '192x192',
type: 'image/png' type: 'image/png'
} }
@@ -35,29 +34,12 @@ export default defineConfig(({mode}) => {
}, },
workbox: { workbox: {
navigateFallbackDenylist: [/^\/api/, /^\/archives/], navigateFallbackDenylist: [/^\/api/, /^\/archives/],
globPatterns: ['**/*.{js,css,html,ico,png,svg,woff,woff2}'], // Fonts and icons are bundled locally, so everything the shell needs
runtimeCaching: [ // is precached and no runtime third-party caching rule is required.
{ globPatterns: ['**/*.{js,css,html,ico,png,svg,woff,woff2}']
urlPattern: /^https:\/\/fonts\.googleapis\.com\/.*/i,
handler: 'CacheFirst',
options: {
cacheName: 'google-fonts-cache',
expiration: {
maxEntries: 10,
maxAgeSeconds: 60 * 60 * 24 * 365
},
cacheableResponse: {
statuses: [0, 200]
}
}
}
]
} }
}) })
], ],
define: {
'process.env.GEMINI_API_KEY': JSON.stringify(env.GEMINI_API_KEY),
},
resolve: { resolve: {
alias: { alias: {
'@': path.resolve(__dirname, '.'), '@': path.resolve(__dirname, '.'),