Compare commits

...
9 Commits
Author SHA1 Message Date
ergosteurandClaude Opus 5 ab7099220a chore: bump version to 1.3.1
Docker Build and Publish / build-and-push (push) Failing after 10s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011uBWhwV3wFQ5MBCcMHHem7
2026-08-14 02:08:35 -04:00
ergosteurandClaude Opus 5 e6663657b0 fix: don't crash at boot when running under a UID with no passwd entry
os.userInfo() throws ERR_SYSTEM_ERROR (uv_os_get_passwd) for a UID that has no
/etc/passwd entry, which is exactly what `docker run --user 1234:1234` produces
— the very workaround the README recommends. Combined with the switch to a
non-root image user, this crashed the server on startup for any deployment that
needed a custom UID to read its archives.

Also document the non-root default and the /cache index volume.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011uBWhwV3wFQ5MBCcMHHem7
2026-08-14 02:08:35 -04:00
ergosteurandClaude Opus 5 24ff2727c8 Merge branch 'review-fixes': security, performance and sidecar archive support
Docker Build and Publish / build-and-push (push) Failing after 10s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011uBWhwV3wFQ5MBCcMHHem7
2026-08-14 01:59:41 -04:00
ergosteurandClaude Opus 5 f089e49e84 chore: bump version to 1.3.0
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011uBWhwV3wFQ5MBCcMHHem7
2026-08-14 01:59:41 -04:00
ergosteurandClaude Opus 5 1d86fa3583 fix: security, performance and correctness pass; add sidecar archive support
Security
- Fix path traversal in GET /api/archives/:name/files. Express decodes route
  params after segment matching, so `..%2f..%2fetc` escaped ARCHIVES_DIR and
  returned a recursive listing of arbitrary directories.
- Add CSP and baseline security headers; disable x-powered-by.
- Stop baking GEMINI_API_KEY into the client bundle (the SDK was unused).
- Run the container as `node` instead of root.

Performance
- Add a directory-mtime-keyed archive index, warmed in the background and
  persisted. Listing 110k files went from ~52s to ~0.1s; the largest archive
  (24k files) serves in ~0.3s. Per-file stat over CIFS costs ~1.4ms and does
  not parallelise, so it is now done once rather than per request.
- Build media URLs from the File directly instead of
  `new Blob([await file.arrayBuffer()])`, which read every media file fully
  into memory (a 20GB archive tried to become 20GB of resident blobs).
- Track and revoke object URLs; previously none were ever revoked.
- Give `requestThumbnail` a stable identity so a completed thumbnail stops
  re-running the effect in every mounted thumbnail.
- Namespace IndexedDB keys so listing archives no longer deserializes every
  cached thumbnail blob, and thumbnails no longer collide across archives.
- Serve real file sizes: RemoteArchiveFile was constructed with size 0, which
  silently disabled high-res thumbnailing for every server archive.

Correctness
- Local archives cached media as blob: URLs, which die with the document, so
  a cached local archive restored as an archive of broken images. Media now
  carries a stable path and is rehydrated from a persisted directory handle
  (File System Access API), falling back to re-prompting for the folder.
- Fix permalinks: the URL-writing effect erased ?a= on mount before the
  archive list arrived to consume it, so deep links never resolved.
- Make cache invalidation detect nested changes via a directory signature.
- Add an error boundary and tolerate unparseable dates, which previously
  threw a RangeError and blanked the app.
- Default video to muted so autoplay is not blocked by Safari/Firefox.

Features
- Fold sidecar directories into their base profile: `<user> - reels`,
  `story - <user>` and `story highlights - <user> - <title>` now appear as
  reels, the story ring and Instagram-style highlight circles rather than as
  separate archives.

Housekeeping
- Add @types/react; React was previously type-checked against its JavaScript
  source, so `npm run lint` gave almost no type safety on components.
- Vendor fonts and PWA icons locally; the app made third-party CDN requests
  despite advertising offline support and local-only processing.
- Drop unused better-sqlite3 (a native module that broke `npm install`).
- Add vitest with 36 tests over the filename and directory-naming rules.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011uBWhwV3wFQ5MBCcMHHem7
2026-08-14 01:59:19 -04:00
ergosteur 0ba7a0d9ad docs: update documentation for high-res performance and local persistence
Docker Build and Publish / build-and-push (push) Failing after 9s
Key changes:
- Updated README.md and GEMINI.md with details on background thumbnailing and inter-post preloading.
- Documented persistent local archive caching and smart profile fallback features.
- Added dist-server/ to .gitignore.
- Restored missing feature descriptions and troubleshooting tips in README.
2026-03-07 21:59:43 -05:00
ergosteur 899e8dfbbb feat: enable persistent local archives and smart profile fallback
Key changes:
- Enabled full metadata caching for local folder archives, allowing them to load instantly from IndexedDB without re-uploading.
- Implemented oldest-image fallback for profiles missing an explicit profile picture.
- Restored folder-name-to-username detection for local archive uploads.
- Optimized scan indexing to track all image files for fallback use.
2026-03-07 21:56:25 -05:00
ergosteur 8809b7794b fix: restore white glass scanning UI and resolve small image blur bug
Key changes:
- Corrected logic in PostThumbnail to prevent blur effects on images smaller than 1MiB.
- Restored the white glass aesthetic to the scanning dashboard with improved contrast and transparency.
- Optimized scanning background transitions to ensure a smooth, flicker-free crossfade.
2026-03-07 21:46:11 -05:00
ergosteur 8ec2c07b3f perf: implement background thumbnail generation and inter-post preloading
Key changes:
- Added Web Worker for background image thumbnailing with a 1MiB threshold to optimize CPU/memory usage.
- Implemented a serial task queue for memory-safe high-res image processing, preventing OOM crashes.
- Added inter-post preloading in the modal for seamless 'Previous/Next' navigation.
- Refined scanning UI with double-buffering and a dark background to completely eliminate white flashes.
- Renamed project to 'instaarchive-viewer' in package.json.
- Fixed 'Open image in new tab' by denylisting /archives and /api in PWA config.
2026-03-07 21:42:56 -05:00
44 changed files with 2691 additions and 1686 deletions
+1
View File
@@ -1,6 +1,7 @@
node_modules/
build/
dist/
dist-server/
coverage/
.DS_Store
*.log
+66
View File
@@ -0,0 +1,66 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
InstaArchive Viewer is a React 19 + Vite 6 PWA for browsing archived Instagram data (both official Instagram exports and Instaloader archives). All archive parsing and media processing happens client-side in the browser — the Express backend only lists/serves files from disk, it never parses archive contents.
## Commands
- `npm install` — install dependencies
- `npm run dev` — start Vite dev server on port 3000 (proxies `/api` and `/archives` to `http://localhost:3001`)
- `npm run server` — start the Express backend (`tsx server.ts`) on port 3001, serving archives from `ARCHIVES_DIR` (defaults to `./_sample-archives`)
- `npm run build` — build frontend to `dist/` (`vite build`) and backend to `dist-server/` (`tsc server.ts ...`)
- `npm run lint` — type-check only (`tsc --noEmit`); there is no separate test suite or linter config
- `npm run clean` — remove `dist/`
For local development you typically need both `npm run dev` and `npm run server` running concurrently — the frontend alone has nothing to talk to for server-mode archives (local-folder mode works without the backend).
## Architecture
### Two archive sources, one data model
The app supports loading archives two ways, unified behind the `ArchiveFile` interface (`src/types/index.ts`, implementations in `src/lib/archive-files.ts`):
- **`LocalArchiveFile`** — wraps a browser `File` from a local folder picker (`webkitdirectory`). Fully offline, media is never uploaded anywhere.
- **`RemoteArchiveFile`** — wraps a file served from the Express backend's `/archives/:name/...` static route, fetched on demand.
All downstream parsing code (`useArchiveScanner`) operates only on `ArchiveFile[]` and doesn't care which backing implementation it got.
### Scanning pipeline (`src/hooks/useArchiveScanner.ts`)
This is the core of the app — a single large `handleFiles` function that:
1. **Indexes** all files, detecting archive format by filename regex: Instagram "export" format (`YYYY-MM-DD_user - post_id[- idx][- story].ext`), Instaloader format (`YYYY-MM-DD_HH-MM-SS_UTC[_idx][_story].ext`), or a generic JSON-manifest format (`posts_1.json`, `reels_1.json`, `stories_1.json`, possibly `.json.xz`-compressed via `xz-decompress`).
2. **Parses** according to detected format, building a `Map<postId, Partial<Post>>`. For JSON-manifest format, media files are matched to JSON entries by URI substring match, then by ID substring match, then by filename-derived heuristic — in that fallback order.
3. **Falls back** to generic filename-prefix grouping when no posts were found via regex/JSON matching (treats files sharing a common basename as one carousel post, chunked into groups of 20).
4. Detects a **profile picture** from `*_profile_pic.jpg` / `<username>.jpg` files, or falls back to the oldest image in the archive by filename sort ("Smart Fallback").
5. **Caches** the final `{ posts, stories, profileMetadata, ... }` result to IndexedDB via `idb-keyval`, keyed by archive name (or `local_archive` for unnamed local folders) — this is what makes repeat visits load instantly. Both server and local archives are cached; the cache shape is documented inline in `useArchiveScanner`'s state (mirrors the `CacheData` interface in `GEMINI.md`).
When modifying format-detection or media-matching logic, be aware the three code paths (JSON-manifest, filename-regex export/instaloader, generic fallback) are largely independent and a change to one rarely needs to touch the others — but all three write into the same `postsMap`.
### Thumbnail generation (`src/hooks/useThumbnailQueue.ts` + `src/lib/thumbnail-worker.ts`)
High-res images (>1MiB) are downscaled off the main thread:
- `useThumbnailQueue` maintains a **serial** (one-at-a-time) queue — this is deliberate, not a bug: decoding multiple 50MP+ images concurrently causes OOM crashes in the browser.
- Actual resizing happens in `thumbnail-worker.ts` using `OffscreenCanvas`/`createImageBitmap` inside a Web Worker.
- Results are cached in IndexedDB under a `thumb_<id>` key, checked before falling back to the worker, so thumbnails persist across sessions.
### URL state sync (`src/App.tsx`)
App state (selected archive, active tab, selected post) is synchronized with URL query params (`?a=`, `?t=`, `?p=`) via `URLSearchParams` + `window.history.replaceState` in a cluster of `useEffect` hooks — this is what enables permalinks/deep-linking. When adding new shareable state, follow this pattern rather than introducing a router.
### Backend (`server.ts`)
Minimal Express server, three responsibilities only:
- `GET /api/archives` — lists subdirectories of `ARCHIVES_DIR` (skipping dotfiles/`@`/`_`-prefixed dirs) as `ServerArchive[]`, guessing a thumbnail per archive.
- `GET /api/archives/:name/files` — recursively lists all files in one archive directory.
- Static-serves `ARCHIVES_DIR` under `/archives` and, in production, serves the built `dist/` frontend with an SPA fallback.
It does no parsing of archive/JSON contents — that's entirely client-side in `useArchiveScanner`. `ARCHIVES_DIR` is resolved from the `ARCHIVES_DIR` env var (see `.env` / Docker volume mount at `/archives`).
### PWA / build quirks
- `vite.config.ts` sets `hmr: process.env.DISABLE_HMR !== 'true'` — this is intentionally left alone; it exists to disable file-watch flicker when running under AI Studio-style agent editing. Don't "clean up" or remove it.
- `workbox.navigateFallbackDenylist` excludes `/api` and `/archives` from the SPA fallback so those routes hit the real server/static files instead of `index.html` (needed for "open original file in new tab").
- Service worker uses `registerType: 'autoUpdate'` with hourly periodic checks — new deployments propagate to open clients automatically.
+11 -4
View File
@@ -18,6 +18,9 @@ FROM node:20-slim AS runtime
ENV NODE_ENV=production
ENV PORT=3000
ENV ARCHIVES_DIR=/archives
# Where the archive index is persisted; mount a volume here so a restart does
# not have to re-walk the whole archive root.
ENV CACHE_DIR=/cache
WORKDIR /app
@@ -25,12 +28,16 @@ WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev
# Copy built assets and server
# Copy built assets and server. The whole dist-server tree is needed: the
# server imports shared archive-grouping logic emitted alongside it.
COPY --from=build /app/dist ./dist
COPY --from=build /app/dist-server/server.js ./server.js
COPY --from=build /app/dist-server/ ./
# Ensure archives directory exists
RUN mkdir -p /archives
# Ensure archives and cache directories exist, writable by the runtime user.
RUN mkdir -p /archives /cache && chown node:node /cache
# Drop root: the server reads the archives volume and writes only its index.
USER node
EXPOSE 3000
+24 -24
View File
@@ -6,16 +6,20 @@
### Key Technical Features
- **Permalinks:** Full synchronization between application state and URL query parameters (`?a=`, `?t=`, `?p=`). Supports deep-linking to archives, tabs, and specific posts. URL parameters are automatically cleaned when navigating back to the archive explorer.
- **Persistent Caching:** Uses `idb-keyval` (IndexedDB) to cache parsed metadata. Subsequent loads of the same archive are near-instant. The cache schema includes `profileMetadata` with consolidated user info and profile picture history.
- **Modular Scanning Logic:** High-performance archive scanning encapsulated in the `useArchiveScanner` hook. It handles multi-format detection (Instagram Export, Instaloader, JSON), batch processing, and yields to the main thread to prevent UI freezing.
- **Persistent Caching:** Uses `idb-keyval` (IndexedDB) to cache parsed metadata. Subsequent loads are near-instant.
- **Metadata:** Caches profile info and post lists for both remote AND local archives.
- **Thumbnails:** High-res images (>1MiB) and videos have thumbnails generated and cached in IndexedDB with the `thumb_` prefix.
- **Background Media Processing:**
- **High-Res Images:** A dedicated Web Worker (`thumbnail-worker.ts`) handles image resizing using `OffscreenCanvas` and `createImageBitmap` to prevent main-thread jank.
- **Serial Queue:** A memory-safe queue ensures only one high-res image is decoded at a time, preventing Out-of-Memory (OOM) crashes on 50MP+ files.
- **High-Performance Carousel:** Advanced `PostModal` with:
- **Preloading:** Intelligently preloads the first two slides immediately, followed by a background preload of the entire carousel.
- **Seamless Transitions:** Zero-latency slide transitions with optimized Framer Motion variants, removing "black flashes" between images.
- **Async Decoding:** Utilizes `decoding="async"` to offload image processing from the main thread.
- **Glassy Scanner UI:** Custom-built glassmorphism scanning dashboard with throttled (1s) dynamic blurred backgrounds and a high-density system log.
- **PWA Auto-Updates:** Configured with `autoUpdate` behavior and a periodic (hourly) update check to ensure long-running sessions and installed PWAs always have the latest code.
- **Compressed Metadata:** Support for `.json.xz` file decompression using `xz-decompress` (WASM-powered).
- **Production-Ready Docker:** Multi-stage Docker builds using `node:slim` serving both the Express API and the Vite-built frontend.
- **Inter-Post Preloading:** Preloads the first media of adjacent posts for instant navigation.
- **Intra-Carousel Preloading:** Intelligently preloads the current carousel's slides.
- **Seamless Transitions:** Zero-latency slide transitions without "black flashes" between images.
- **Glassy Scanner UI:** Custom-built glassmorphism scanning dashboard with double-buffering logic to ensure smooth, flicker-free background crossfades during file indexing.
- **PWA Capabilities:**
- **Auto-Updates:** Hourly periodic update checks.
- **Navigation Fix:** `navigateFallbackDenylist` allows direct server access to `/archives/` and `/api/` (enabling "Open in new tab" for original files).
### Main Technologies
- **Frontend:** React 19, Vite 6, TypeScript
@@ -23,14 +27,14 @@
- **Icons:** Lucide React
- **Animations:** Framer Motion (`motion/react`)
- **Persistence:** IndexedDB (`idb-keyval`)
- **Backend:** Express, tsx (for server-side scanning)
- **Decompression:** xz-decompress (WASM)
- **Backend:** Express, tsx
- **Workers:** Web Workers for background image processing.
## Architecture
### State Management
- **`useArchiveScanner` Hook:** Centralized logic for parsing archives and managing results (`allPosts`, `allStories`, `profileMetadata`).
- **Archive Interface:** Unified `ArchiveFile` interface implemented by `LocalArchiveFile` (for browser `File` objects) and `RemoteArchiveFile` (for server-side assets).
- **`useArchiveScanner` Hook:** Centralized logic for parsing and caching. It handles folder-name-to-username detection and "Smart Fallback" profile pictures (using the oldest image if no profile pic is found).
- **`useThumbnailQueue` Hook:** Manages the serial processing of high-resolution media.
### Cache Schema
```typescript
@@ -38,8 +42,8 @@ interface CacheData {
name: string;
isLocal: boolean;
fileCount: number;
posts: Post[]; // Remote archives only (Local archives re-parsed for security)
stories: Post[]; // Remote archives only
posts: Post[]; // Cached for all archive types
stories: Post[];
profileMetadata: {
username: string;
fullName: string;
@@ -55,15 +59,11 @@ interface CacheData {
```
## Commands
- `npm install`: Install project dependencies.
- `npm run dev`: Start the local development server on port 3000.
- `npm run build`: Generate the production-ready build in the `dist` folder and server in `dist-server`.
- `npm run server`: Start the backend server to scan `./_sample-archives`.
- `npm install`: Install dependencies.
- `npm run dev`: Start dev server (Port 3000).
- `npm run build`: Build frontend (`dist/`) and server (`dist-server/`).
- `npm run server`: Start production-ready backend.
- `npm run lint`: Execute TypeScript type-checking.
## Production Deployment
The project is containerized and available on GHCR. It expects a volume mount at `/archives` containing subdirectories for each user.
### Key Environment Variables
- `PORT`: Server port (default: 3000)
- `ARCHIVES_DIR`: Path to the archives collection (default: /archives)
The project is containerized. It expects a volume mount at `/archives` containing subdirectories for each user.
+25 -6
View File
@@ -4,14 +4,16 @@ A high-performance React PWA for browsing archived Instagram data with a native-
## Features
- **Advanced Carousel**: Seamless, zero-latency transitions between slides with intelligent preloading. Images use asynchronous decoding to keep the UI smooth during motion.
- **Persistent Caching**: Uses IndexedDB to store parsed archives locally. Subsequent loads are near-instant, with full support for profile metadata and profile picture history.
- **Advanced Carousel**: Seamless, zero-latency transitions between slides with intelligent preloading. Navigating between different posts is now near-instant thanks to inter-post background preloading.
- **High-Res Performance**: Handles 50MP+ images effortlessly using a background Web Worker and a memory-safe serial processing queue.
- **Persistent Local Caching**: Uses IndexedDB to store parsed archives and generated thumbnails. **Local folders** now load instantly from cache on return visits without needing to re-upload files.
- **Permalinks**: State is synchronized with the URL, allowing you to share direct links to archives, tabs, or specific posts. Navigating back to the explorer cleans up URL parameters automatically.
- **Glassy Scanning UI**: A modern, translucent white terminal experience with a dynamic blurred background generated from your media during scanning.
- **Glassy Scanning UI**: A refined, translucent white terminal experience with flicker-free, double-buffered dynamic blurred backgrounds.
- **PWA with Auto-Update**: Fully offline-capable and installable. Clients automatically receive updates when a new version is deployed to the server.
- **Local Privacy**: All processing is done client-side. Even when using the self-hosted version, your media is processed locally in your browser and never uploaded.
- **Story Viewer**: Native-like story experience with segmented progress bars, auto-playback, and audio controls.
- **Smart Fallbacks**: Automatically detects usernames from folder names and uses the oldest archive image as a profile picture if one is missing.
- **Customizable Grid**: 1:1 or 3:4 aspect ratios with adjustable "bumps" for aesthetic alignment.
- **Story Viewer**: Native-like story experience with segmented progress bars, auto-playback, and audio controls.
- **Navigation Protection**: Intercepts accidental browser "Back" or "Refresh" actions to protect your current session.
## Deployment
@@ -52,10 +54,27 @@ If the app shows "No Archives Found" and logs `EACCES: permission denied`:
chmod -R 755 /path/to/archives
```
2. **SELinux (Fedora/RHEL/CentOS)**: Use the `:z` flag in your volume mount as shown above.
3. **User Mapping**: You can force the container to run as your host user:
3. **User Mapping**: The container runs as the non-root `node` user (UID 1000).
If your archives are readable only by another account, run as that user
instead — the container needs to *list* the archive directory, so `--x`
(traverse-only) permissions are not enough:
```bash
docker run --user $(id -u):$(id -g) ...
docker run --user $(stat -c '%u:%g' /path/to/archives) ...
```
In Compose:
```yaml
services:
instaarchive:
user: "1234:1234" # a UID that can read your archives
```
### Archive Index
On first start the server walks the archive root once and caches the result,
keyed by directory mtime. This matters on network storage: for a 110k-file
archive root, listing went from ~52s per request to ~0.1s. Mount a volume at
`/cache` (or set `CACHE_DIR`) so the index survives restarts, otherwise it is
rebuilt on every start.
## Supported Archive Structure
+6
View File
@@ -5,7 +5,13 @@ services:
- "3000:3000"
volumes:
- ./archives:/archives:ro,z
# Persists the archive index so restarts don't re-walk every file.
- instaarchive-cache:/cache
environment:
- PORT=3000
- ARCHIVES_DIR=/archives
- CACHE_DIR=/cache
restart: unless-stopped
volumes:
instaarchive-cache:
-137
View File
@@ -1,137 +0,0 @@
import express from 'express';
import fs from 'fs';
import path from 'path';
import { fileURLToPath } from 'url';
import dotenv from 'dotenv';
import os from 'os';
dotenv.config();
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
const app = express();
const PORT = process.env.PORT || 3001;
const ARCHIVES_DIR = path.resolve(process.env.ARCHIVES_DIR || path.join(__dirname, '_sample-archives'));
console.log(`[Server] Initializing...`);
console.log(`[Server] Running as user: ${os.userInfo().username} (UID: ${os.userInfo().uid}, GID: ${os.userInfo().gid})`);
console.log(`[Server] Environment ARCHIVES_DIR: ${process.env.ARCHIVES_DIR}`);
console.log(`[Server] Resolved ARCHIVES_DIR: ${ARCHIVES_DIR}`);
// Ensure archives directory exists
if (!fs.existsSync(ARCHIVES_DIR)) {
console.warn(`[Server] Warning: Archives directory not found at ${ARCHIVES_DIR}. Creating it...`);
try {
fs.mkdirSync(ARCHIVES_DIR, { recursive: true });
}
catch (err) {
console.error(`[Server] Failed to create archives directory:`, err);
}
}
else {
console.log(`[Server] Archives directory exists.`);
}
app.use(express.json());
// API: List archives (subdirectories in ARCHIVES_DIR)
app.get('/api/archives', (req, res) => {
try {
console.log(`[API] Listing archives from ${ARCHIVES_DIR}...`);
const items = fs.readdirSync(ARCHIVES_DIR, { withFileTypes: true });
console.log(`[API] Found ${items.length} total items in archives directory.`);
const archives = items
.filter(item => {
const isDir = item.isDirectory();
const isHidden = item.name.startsWith('.') || item.name.startsWith('@') || item.name.startsWith('_');
if (!isDir)
return false;
if (isHidden) {
console.log(`[API] Skipping system/hidden directory: ${item.name}`);
return false;
}
return true;
})
.map(item => {
// Try to find a profile pic or first image for the thumbnail
const archivePath = path.join(ARCHIVES_DIR, item.name);
try {
const files = fs.readdirSync(archivePath);
console.log(`[API] Found archive: ${item.name} (${files.length} files)`);
let thumbnail = '';
const profilePic = files.find(f => f.toLowerCase().includes('_profile_pic.jpg') || f.toLowerCase() === `${item.name.toLowerCase()}.jpg`);
if (profilePic) {
thumbnail = `/archives/${item.name}/${profilePic}`;
}
else {
const firstImage = files.find(f => /\.(jpg|jpeg|png|webp)$/i.test(f));
if (firstImage)
thumbnail = `/archives/${item.name}/${firstImage}`;
}
return {
name: item.name,
thumbnail,
path: item.name,
fileCount: files.length
};
}
catch (e) {
console.error(`[API] Could not read subdirectory ${item.name}:`, e);
return null;
}
})
.filter(Boolean);
console.log(`[API] Returning ${archives.length} validated archives.`);
res.json(archives);
}
catch (err) {
if (err.code === 'EACCES') {
console.error(`[API] Permission Denied! The server (UID ${os.userInfo().uid}) cannot read ${ARCHIVES_DIR}.`);
console.error(`[API] Hint: If using Linux/Docker, check folder permissions (chmod 755) or SELinux context (append :z to your volume mount).`);
}
else {
console.error('[API] Error listing archives:', err);
}
res.status(500).json({ error: 'Permission denied or failed to list archives' });
}
});
// API: List all files in an archive (recursive)
app.get('/api/archives/:name/files', (req, res) => {
const archiveName = req.params.name;
const archivePath = path.join(ARCHIVES_DIR, archiveName);
if (!fs.existsSync(archivePath)) {
return res.status(404).json({ error: 'Archive not found' });
}
try {
const walk = (dir, base = '') => {
let results = [];
const list = fs.readdirSync(dir);
list.forEach(file => {
const filePath = path.join(dir, file);
const relativePath = path.join(base, file);
const stat = fs.statSync(filePath);
if (stat && stat.isDirectory()) {
results = results.concat(walk(filePath, relativePath));
}
else {
results.push(relativePath);
}
});
return results;
};
const files = walk(archivePath);
res.json(files);
}
catch (err) {
console.error('Error listing files:', err);
res.status(500).json({ error: 'Failed to list files' });
}
});
// Serve archive files
app.use('/archives', express.static(ARCHIVES_DIR));
// Serve production frontend
const distPath = path.join(__dirname, 'dist');
if (fs.existsSync(distPath)) {
app.use(express.static(distPath));
app.get('*', (req, res) => {
res.sendFile(path.join(distPath, 'index.html'));
});
}
app.listen(PORT, () => {
console.log(`Server running at http://localhost:${PORT}`);
console.log(`Serving archives from: ${ARCHIVES_DIR}`);
});
+472 -1171
View File
File diff suppressed because it is too large Load Diff
+9 -6
View File
@@ -1,7 +1,7 @@
{
"name": "react-example",
"name": "instaarchive-viewer",
"private": true,
"version": "1.2.0",
"version": "1.3.1",
"type": "module",
"scripts": {
"dev": "vite --port=3000 --host=0.0.0.0",
@@ -10,13 +10,13 @@
"preview": "vite preview",
"server": "tsx server.ts",
"clean": "rm -rf dist",
"lint": "tsc --noEmit"
"lint": "tsc --noEmit",
"test": "vitest run",
"test:watch": "vitest"
},
"dependencies": {
"@google/genai": "^1.29.0",
"@tailwindcss/vite": "^4.1.14",
"@vitejs/plugin-react": "^5.0.4",
"better-sqlite3": "^12.4.1",
"clsx": "^2.1.1",
"date-fns": "^4.1.0",
"dotenv": "^17.2.3",
@@ -34,10 +34,13 @@
"devDependencies": {
"@types/express": "^4.17.21",
"@types/node": "^22.14.0",
"@types/react": "^19.2.18",
"@types/react-dom": "^19.2.4",
"autoprefixer": "^10.4.21",
"tailwindcss": "^4.1.14",
"typescript": "~5.8.2",
"vite": "^6.2.0",
"vite-plugin-pwa": "^1.2.0"
"vite-plugin-pwa": "^1.2.0",
"vitest": "^3.2.7"
}
}
+56
View File
@@ -0,0 +1,56 @@
/* Self-hosted subset of Inter + Playfair Display.
Vendored so the PWA works offline and makes no third-party requests. */
/* latin-ext */
@font-face {
font-family: 'Inter';
font-style: normal;
font-weight: 300 700;
font-display: swap;
src: url(/fonts/inter-300_700-normal-b6db4a.woff2) format('woff2');
unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF;
}
/* latin */
@font-face {
font-family: 'Inter';
font-style: normal;
font-weight: 300 700;
font-display: swap;
src: url(/fonts/inter-300_700-normal-6ab57b.woff2) format('woff2');
unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;
}
/* latin-ext */
@font-face {
font-family: 'Playfair Display';
font-style: italic;
font-weight: 400 900;
font-display: swap;
src: url(/fonts/playfair-display-400_900-italic-2d6d99.woff2) format('woff2');
unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF;
}
/* latin */
@font-face {
font-family: 'Playfair Display';
font-style: italic;
font-weight: 400 900;
font-display: swap;
src: url(/fonts/playfair-display-400_900-italic-d14361.woff2) format('woff2');
unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;
}
/* latin-ext */
@font-face {
font-family: 'Playfair Display';
font-style: normal;
font-weight: 400 900;
font-display: swap;
src: url(/fonts/playfair-display-400_900-normal-ca7410.woff2) format('woff2');
unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF;
}
/* latin */
@font-face {
font-family: 'Playfair Display';
font-style: normal;
font-weight: 400 900;
font-display: swap;
src: url(/fonts/playfair-display-400_900-normal-61a963.woff2) format('woff2');
unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;
}
Binary file not shown.
Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 2.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

+117 -72
View File
@@ -4,6 +4,7 @@ import path from 'path';
import { fileURLToPath } from 'url';
import dotenv from 'dotenv';
import os from 'os';
import { ArchiveIndex } from './src/lib/archive-index.js';
dotenv.config();
@@ -15,7 +16,23 @@ const PORT = process.env.PORT || 3001;
const ARCHIVES_DIR = path.resolve(process.env.ARCHIVES_DIR || path.join(__dirname, '_sample-archives'));
console.log(`[Server] Initializing...`);
console.log(`[Server] Running as user: ${os.userInfo().username} (UID: ${os.userInfo().uid}, GID: ${os.userInfo().gid})`);
/**
* Describe the running user without assuming it exists in /etc/passwd.
*
* `os.userInfo()` throws ENOENT for a UID with no passwd entry, which is
* exactly what happens when the container is started with `--user 1234:1234`
* (as the deployment docs suggest) — previously crashing the server at boot.
*/
const describeUser = () => {
try {
const info = os.userInfo();
return `${info.username} (UID: ${info.uid}, GID: ${info.gid})`;
} catch {
return `UID: ${typeof process.getuid === 'function' ? process.getuid() : '?'}, GID: ${typeof process.getgid === 'function' ? process.getgid() : '?'} (no passwd entry)`;
}
};
console.log(`[Server] Running as user: ${describeUser()}`);
console.log(`[Server] Environment ARCHIVES_DIR: ${process.env.ARCHIVES_DIR}`);
console.log(`[Server] Resolved ARCHIVES_DIR: ${ARCHIVES_DIR}`);
@@ -31,60 +48,89 @@ if (!fs.existsSync(ARCHIVES_DIR)) {
console.log(`[Server] Archives directory exists.`);
}
const INDEX_PATH = process.env.ARCHIVE_INDEX_PATH
|| path.join(process.env.CACHE_DIR || os.tmpdir(), 'instaarchive-index.json');
const index = new ArchiveIndex(ARCHIVES_DIR, INDEX_PATH);
// Warm in the background: the first walk of a large archive root is slow, but
// everything after it is served from directory-mtime-keyed cache.
index.load()
.then(() => index.warm())
.catch(err => console.error('[Index] Warm failed:', err));
// Don't advertise the framework.
app.disable('x-powered-by');
/**
* Baseline security headers.
*
* The CSP allows blob: and data: because archive media is rendered from object
* URLs and cached thumbnails, and 'unsafe-inline' for styles because the
* animation library sets inline styles. Scripts stay restricted to same-origin,
* and no third-party origins are permitted at all — the app bundles its own
* fonts and icons.
*/
app.use((req, res, next) => {
res.setHeader('X-Content-Type-Options', 'nosniff');
res.setHeader('X-Frame-Options', 'SAMEORIGIN');
res.setHeader('Referrer-Policy', 'no-referrer');
res.setHeader('Cross-Origin-Resource-Policy', 'same-origin');
res.setHeader('Permissions-Policy', 'geolocation=(), microphone=(), camera=(), interest-cohort=()');
res.setHeader('Content-Security-Policy', [
"default-src 'self'",
"img-src 'self' blob: data:",
"media-src 'self' blob: data:",
"script-src 'self'",
"style-src 'self' 'unsafe-inline'",
"font-src 'self'",
"connect-src 'self'",
"worker-src 'self' blob:",
"frame-ancestors 'self'",
"object-src 'none'",
"base-uri 'self'",
"form-action 'self'",
].join('; '));
next();
});
app.use(express.json());
// API: List archives (subdirectories in ARCHIVES_DIR)
app.get('/api/archives', (req, res) => {
try {
console.log(`[API] Listing archives from ${ARCHIVES_DIR}...`);
const items = fs.readdirSync(ARCHIVES_DIR, { withFileTypes: true });
console.log(`[API] Found ${items.length} total items in archives directory.`);
const archives = items
.filter(item => {
const isDir = item.isDirectory();
const isHidden = item.name.startsWith('.') || item.name.startsWith('@') || item.name.startsWith('_');
if (!isDir) return false;
if (isHidden) {
console.log(`[API] Skipping system/hidden directory: ${item.name}`);
return false;
}
return true;
})
.map(item => {
// Try to find a profile pic or first image for the thumbnail
const archivePath = path.join(ARCHIVES_DIR, item.name);
try {
const files = fs.readdirSync(archivePath);
console.log(`[API] Found archive: ${item.name} (${files.length} files)`);
let thumbnail = '';
const profilePic = files.find(f => f.toLowerCase().includes('_profile_pic.jpg') || f.toLowerCase() === `${item.name.toLowerCase()}.jpg`);
if (profilePic) {
thumbnail = `/archives/${item.name}/${profilePic}`;
} else {
const firstImage = files.find(f => /\.(jpg|jpeg|png|webp)$/i.test(f));
if (firstImage) thumbnail = `/archives/${item.name}/${firstImage}`;
}
return {
name: item.name,
thumbnail,
path: item.name,
fileCount: files.length
};
} catch (e) {
console.error(`[API] Could not read subdirectory ${item.name}:`, e);
/**
* Resolve a user-supplied archive name to an absolute path inside ARCHIVES_DIR.
*
* Express decodes route params *after* segment matching, so a name like
* `..%2f..%2fetc` arrives here as `../../etc` and would otherwise escape the
* archives root. Returns null for anything that resolves outside it.
*/
const resolveArchivePath = (archiveName: string): string | null => {
if (!archiveName || archiveName.includes('\0')) return null;
const resolved = path.resolve(ARCHIVES_DIR, archiveName);
if (resolved !== ARCHIVES_DIR && !resolved.startsWith(ARCHIVES_DIR + path.sep)) {
console.warn(`[Security] Rejected archive name escaping ARCHIVES_DIR: ${archiveName}`);
return null;
}
})
.filter(Boolean);
return resolved;
};
console.log(`[API] Returning ${archives.length} validated archives.`);
// API: List archives, grouped by profile. Costs one stat per source directory.
app.get('/api/archives', (req, res) => {
try {
const groups = index.groups();
const archives = Array.from(groups.entries()).map(([owner, sources]) => ({
name: owner,
thumbnail: index.thumbnailFor(owner, sources),
path: owner,
// Null until that profile has been indexed; the client treats it as unknown.
fileCount: index.countFor(sources),
// Directory mtimes: cheap to compute and enough to invalidate a stale cache.
signature: index.signatureFor(sources),
sources,
}));
console.log(`[API] Returning ${archives.length} archives.`);
res.json(archives);
} catch (err: any) {
if (err.code === 'EACCES') {
console.error(`[API] Permission Denied! The server (UID ${os.userInfo().uid}) cannot read ${ARCHIVES_DIR}.`);
console.error(`[API] Permission Denied! The server (${describeUser()}) cannot read ${ARCHIVES_DIR}.`);
console.error(`[API] Hint: If using Linux/Docker, check folder permissions (chmod 755) or SELinux context (append :z to your volume mount).`);
} else {
console.error('[API] Error listing archives:', err);
@@ -93,33 +139,26 @@ app.get('/api/archives', (req, res) => {
}
});
// API: List all files in an archive (recursive)
app.get('/api/archives/:name/files', (req, res) => {
/**
* List every file belonging to a profile, across its base and sidecar dirs.
*
* Paths are relative to ARCHIVES_DIR (so they include the source directory) and
* each entry carries its source kind, letting the client route posts, reels,
* stories and highlights without re-deriving the naming rules.
*
* Served from the directory index; only a directory whose mtime changed is
* re-walked.
*/
app.get('/api/archives/:name/files', async (req, res) => {
const archiveName = req.params.name;
const archivePath = path.join(ARCHIVES_DIR, archiveName);
if (!fs.existsSync(archivePath)) {
return res.status(404).json({ error: 'Archive not found' });
if (!resolveArchivePath(archiveName)) {
return res.status(400).json({ error: 'Invalid archive name' });
}
try {
const walk = (dir: string, base: string = ''): string[] => {
let results: string[] = [];
const list = fs.readdirSync(dir);
list.forEach(file => {
const filePath = path.join(dir, file);
const relativePath = path.join(base, file);
const stat = fs.statSync(filePath);
if (stat && stat.isDirectory()) {
results = results.concat(walk(filePath, relativePath));
} else {
results.push(relativePath);
}
});
return results;
};
const files = walk(archivePath);
const files = await index.filesFor(archiveName);
if (!files) return res.status(404).json({ error: 'Archive not found' });
void index.save();
res.json(files);
} catch (err) {
console.error('Error listing files:', err);
@@ -127,8 +166,14 @@ app.get('/api/archives/:name/files', (req, res) => {
}
});
// Serve archive files
app.use('/archives', express.static(ARCHIVES_DIR));
// Serve archive files. Archive contents are immutable in practice, so cache
// them aggressively; the client busts its own cache via fileCount.
app.use('/archives', express.static(ARCHIVES_DIR, {
maxAge: '1y',
immutable: true,
index: false,
dotfiles: 'ignore',
}));
// Serve production frontend
const distPath = path.join(__dirname, 'dist');
+318 -75
View File
@@ -15,19 +15,34 @@ import {
Loader2,
} from 'lucide-react';
import { motion, AnimatePresence } from 'motion/react';
import * as idb from 'idb-keyval';
import { cn } from './lib/utils';
import { LocalArchiveFile, RemoteArchiveFile } from './lib/archive-files';
import { Post, ServerArchive } from './types';
import {
deleteCachedArchive,
getCachedArchive,
listCachedArchives,
listCachedArchiveNames,
migrateLegacyCache,
restoreArchive,
saveDirectoryHandle,
} from './lib/archive-cache';
import {
filesFromDirectory,
isDirectoryPickerSupported,
pickDirectory,
} from './lib/directory-handle';
import { CacheData, Post, ServerArchive, ServerArchiveFile } from './types';
import { ArchiveDashboard } from './components/ArchiveDashboard';
import { StoryViewer } from './components/StoryViewer';
import { PostModal } from './components/PostModal';
import { VideoThumbnail } from './components/VideoThumbnail';
import { PostThumbnail } from './components/PostThumbnail';
import { useArchiveScanner } from './hooks/useArchiveScanner';
import { useThumbnailQueue } from './hooks/useThumbnailQueue';
export default function App() {
const [showStoryViewer, setShowStoryViewer] = useState(false);
const [activeHighlight, setActiveHighlight] = useState<string | null>(null);
const [visiblePostsCount, setVisiblePostsCount] = useState(90);
const [selectedPost, setSelectedPost] = useState<Post | null>(null);
@@ -36,31 +51,35 @@ export default function App() {
const [activeTab, setActiveTab] = useState<'posts' | 'reels' | 'saved'>('posts');
const [serverArchives, setServerArchives] = useState<ServerArchive[]>([]);
const [cachedArchives, setCachedArchives] = useState<Set<string>>(new Set());
const [localCachedArchives, setLocalCachedArchives] = useState<any[]>([]);
const [localCachedArchives, setLocalCachedArchives] = useState<CacheData[]>([]);
const [isServerMode, setIsServerMode] = useState(false);
/** True once GET /api/archives has settled, successfully or not. */
const [archivesFetched, setArchivesFetched] = useState(false);
const [currentArchive, setCurrentArchive] = useState<ServerArchive | null>(null);
const [hasInitialLoaded, setHasInitialLoaded] = useState(false);
/**
* The query string as it was when the app booted.
*
* Captured during the first render because the URL is rewritten from app
* state as soon as anything loads; reading `window.location` later would see
* the rewritten value rather than the link the user actually followed.
*/
const initialParamsRef = useRef(new URLSearchParams(window.location.search));
const fileInputRef = useRef<HTMLInputElement>(null);
const profilePicInputRef = useRef<HTMLInputElement>(null);
const refreshCachedArchives = useCallback(async () => {
try {
const keys = await idb.keys();
setCachedArchives(new Set(keys.map(String)));
const locals: any[] = [];
for (const key of keys) {
const data: any = await idb.get(key);
if (data && data.isLocal) {
// Fallback for missing allProfilePics in local cached metadata
if (!data.profileMetadata.allProfilePics) {
data.profileMetadata.allProfilePics = data.profileMetadata.profilePic ? [data.profileMetadata.profilePic] : [];
// Names come from key prefixes, so listing no longer deserializes every
// cached thumbnail blob just to find out which entries are archives.
setCachedArchives(new Set(await listCachedArchiveNames()));
setLocalCachedArchives((await listCachedArchives()).filter(a => a.isLocal));
} catch (e) {
console.error('[Cache] Failed to list cached archives:', e);
}
locals.push(data);
}
}
setLocalCachedArchives(locals);
} catch (e) {}
}, []);
const {
@@ -72,6 +91,8 @@ export default function App() {
currentScanningImage,
allPosts,
allStories,
allHighlights,
setAllHighlights,
profileMetadata,
handleFiles,
setAllPosts,
@@ -79,9 +100,12 @@ export default function App() {
setProfileMetadata,
setIsScanning,
setScanningPhase,
resetScannerState
resetScannerState,
registerUrl
} = useArchiveScanner('', currentArchive, refreshCachedArchives);
const [lastLoadedScanningImage, setLastLoadedScanningImage] = useState<string | null>(null);
const {
username,
fullName,
@@ -93,6 +117,9 @@ export default function App() {
allProfilePics
} = profileMetadata;
// Thumbnails are keyed per archive, so the queue is scoped to the open one.
const { cacheHits, requestThumbnail } = useThumbnailQueue(currentArchive?.name ?? username ?? '');
useEffect(() => {
fetch('/api/archives')
.then(res => {
@@ -102,21 +129,51 @@ export default function App() {
}
return [];
})
.then(data => setServerArchives(data))
.catch(() => setIsServerMode(false));
.then(data => setServerArchives(Array.isArray(data) ? data : []))
.catch(() => setIsServerMode(false))
// Deep-link resolution waits on this rather than on `isServerMode`, which
// is still false while the request is in flight.
.finally(() => setArchivesFetched(true));
}, []);
useEffect(() => {
refreshCachedArchives();
migrateLegacyCache().finally(refreshCachedArchives);
}, [refreshCachedArchives]);
const clearCache = async (name: string) => { await idb.del(name); await refreshCachedArchives(); };
const clearCache = async (name: string) => { await deleteCachedArchive(name); await refreshCachedArchives(); };
/**
* Archives with a `- reels` sidecar directory say outright which posts are
* reels; only fall back to the "lone video" heuristic for archives that have
* no such directory.
*/
const hasReelSource = useMemo(() => allPosts.some(p => p.source === 'reels'), [allPosts]);
const isReel = useCallback((p: Post) => (
hasReelSource ? p.source === 'reels' : p.media.length === 1 && p.media[0].type === 'video'
), [hasReelSource]);
const filteredPosts = useMemo(() => {
if (activeTab === 'reels') return allPosts.filter(p => p.media.length === 1 && p.media[0].type === 'video');
if (activeTab === 'posts') return allPosts.filter(p => !(p.media.length === 1 && p.media[0].type === 'video'));
if (activeTab === 'reels') return allPosts.filter(isReel);
if (activeTab === 'posts') return allPosts.filter(p => !isReel(p));
return [];
}, [allPosts, activeTab]);
}, [allPosts, activeTab, isReel]);
/** Story highlights, grouped into the circles shown under the bio. */
const highlightGroups = useMemo(() => {
const groups = new Map<string, Post[]>();
for (const item of allHighlights) {
const title = item.highlightTitle?.trim() || 'Highlights';
if (!groups.has(title)) groups.set(title, []);
groups.get(title)!.push(item);
}
return Array.from(groups, ([title, items]) => ({
title,
items,
// The cover must be a still: most highlight items are videos, and a video
// URL in an <img> renders as a broken image.
cover: items.find(i => i.media[0]?.type === 'image')?.thumbnail,
}));
}, [allHighlights]);
const handleTabChange = (tab: 'posts' | 'reels' | 'saved') => { setActiveTab(tab); setVisiblePostsCount(90); };
const visiblePosts = useMemo(() => filteredPosts.slice(0, visiblePostsCount), [filteredPosts, visiblePostsCount]);
@@ -146,38 +203,50 @@ export default function App() {
setScanningPhase('Checking Cache');
try {
const cachedData = await idb.get(archive.name);
const cachedData = await getCachedArchive(archive.name);
if (cachedData) {
console.log(`[Cache] Found cached data for ${archive.name}. File count: ${cachedData.fileCount} (Server has: ${archive.fileCount})`);
if (cachedData.fileCount === archive.fileCount) {
// Invalidate on the directory-mtime signature; fall back to file count
// for entries cached before signatures existed.
const fresh = archive.signature
? cachedData.signature === archive.signature
: cachedData.fileCount === archive.fileCount;
console.log(`[Cache] Cached ${archive.name}: signature ${cachedData.signature} vs ${archive.signature} -> ${fresh ? 'fresh' : 'stale'}`);
if (fresh) {
console.log(`[Cache] Cache hit! Restoring state...`);
setAllPosts(cachedData.posts);
setAllStories(cachedData.stories);
// Handle migration from old cache schema where allProfilePics was a separate top-level key
const profileMetadata = { ...cachedData.profileMetadata };
if (!profileMetadata.allProfilePics && cachedData.allProfilePics) {
profileMetadata.allProfilePics = cachedData.allProfilePics;
}
if (!profileMetadata.allProfilePics) {
profileMetadata.allProfilePics = profileMetadata.profilePic ? [profileMetadata.profilePic] : [];
}
setProfileMetadata(profileMetadata);
const restored = await restoreArchive(cachedData, registerUrl);
if (restored) {
setAllPosts(restored.posts);
setAllStories(restored.stories);
setAllHighlights(restored.highlights);
setProfileMetadata({
...restored.profileMetadata,
allProfilePics: restored.profileMetadata.allProfilePics
?? (restored.profileMetadata.profilePic ? [restored.profileMetadata.profilePic] : []),
});
setVisiblePostsCount(90);
setIsScanning(false);
console.log(`[Cache] Archive ${archive.name} loaded successfully from cache.`);
return;
}
}
}
console.log(`[Scanner] Starting fresh scan from server API...`);
const res = await fetch(`/api/archives/${archive.name}/files`);
const filePaths: string[] = await res.json();
const res = await fetch(`/api/archives/${encodeURIComponent(archive.name)}/files`);
const entries: (ServerArchiveFile | string)[] = await res.json();
const archiveFiles = filePaths.map(p => {
const name = p.split(/[/\\]/).pop() || p;
return new RemoteArchiveFile(name, p, 0, `/archives/${archive.name}/${p}`);
const archiveFiles = entries.map(entry => {
// Older servers returned bare path strings relative to the archive dir.
const legacy = typeof entry === 'string';
const filePath = legacy ? entry : entry.path;
const url = legacy
? `/archives/${encodeURI(`${archive.name}/${filePath}`)}`
: `/archives/${encodeURI(filePath)}`;
const name = filePath.split(/[/\\]/).pop() || filePath;
const source = legacy
? undefined
: { kind: entry.kind, dir: filePath.split('/')[0], title: entry.title };
return new RemoteArchiveFile(name, filePath, legacy ? 0 : entry.size, url, source, legacy ? undefined : entry.mtime);
});
await handleFiles(archiveFiles, archive);
@@ -185,13 +254,91 @@ export default function App() {
console.error('[Scanner] Failed to load server archive:', err);
setIsScanning(false);
}
}, [handleFiles, setAllPosts, setAllStories, setProfileMetadata, setIsScanning, setScanningPhase]);
}, [handleFiles, registerUrl, setAllPosts, setAllStories, setAllHighlights, setProfileMetadata, setIsScanning, setScanningPhase]);
const handleLocalFiles = (files: FileList | null) => { if (!files) return; const archiveFiles = Array.from(files).map(f => new LocalArchiveFile(f)); handleFiles(archiveFiles); };
const triggerFileSelect = () => fileInputRef.current?.click();
/**
* Open a local archive straight from cache.
*
* The cached posts carry file *paths*, not URLs — blob: URLs do not survive a
* reload — so this re-opens the stored directory handle and mints fresh URLs
* for those paths. No re-parsing happens, which is what keeps it instant.
*
* If the folder can no longer be reached (permission revoked, folder moved,
* or the browser never supported directory handles) we fall back to asking
* for the folder again rather than rendering an archive of broken images.
*/
const loadLocalCachedArchive = useCallback(async (archive: CacheData) => {
console.log(`[Cache] Loading local archive from cache: ${archive.name}`);
setIsScanning(true);
setCurrentArchive(null);
setScanningPhase('Checking Cache');
try {
const restored = await restoreArchive(archive, registerUrl);
if (!restored) {
console.warn(`[Cache] Folder for ${archive.name} unavailable; re-prompting.`);
setIsScanning(false);
await openLocalFolder(archive.name);
return;
}
setAllPosts(restored.posts);
setAllStories(restored.stories);
setAllHighlights(restored.highlights);
setProfileMetadata({
...restored.profileMetadata,
allProfilePics: restored.profileMetadata.allProfilePics
?? (restored.profileMetadata.profilePic ? [restored.profileMetadata.profilePic] : []),
});
setVisiblePostsCount(90);
setIsScanning(false);
console.log(`[Cache] Local archive ${archive.name} restored from cache.`);
} catch (err) {
console.error('[Cache] Failed to restore local archive:', err);
setIsScanning(false);
}
}, [registerUrl, setAllPosts, setAllStories, setAllHighlights, setProfileMetadata, setIsScanning, setScanningPhase]);
const handleLocalFiles = (files: FileList | null) => {
if (!files) return;
handleFiles(Array.from(files).map(f => new LocalArchiveFile(f)));
};
/**
* Prefer the File System Access API so the folder can be reopened later
* without a re-prompt; fall back to <input webkitdirectory> elsewhere
* (Firefox and Safari have no showDirectoryPicker), where the archive is
* re-scanned from scratch on every visit.
*/
const openLocalFolder = useCallback(async (expectedName?: string) => {
if (!isDirectoryPickerSupported()) {
fileInputRef.current?.click();
return;
}
const handle = await pickDirectory();
if (!handle) return;
if (expectedName && handle.name !== expectedName) {
console.warn(`[Cache] Picked "${handle.name}" but expected "${expectedName}"; scanning as picked.`);
}
// Persist before scanning so the scan records that a handle exists.
await saveDirectoryHandle(handle.name, handle);
const files = await filesFromDirectory(handle);
await handleFiles(files);
}, [handleFiles]);
const triggerFileSelect = () => { void openLocalFolder(); };
const loadMore = () => setVisiblePostsCount(prev => prev + 90);
useEffect(() => {
// Hold the URL until the deep link has been consumed, otherwise this effect
// runs on mount with nothing loaded yet and erases the very parameters the
// loader below is waiting to read.
if (!hasInitialLoaded) return;
const params = new URLSearchParams(window.location.search);
if (currentArchive) params.set('a', currentArchive.name);
else if (allPosts.length > 0 && username) params.set('a', username);
@@ -210,37 +357,60 @@ export default function App() {
const newUrl = window.location.pathname + (newSearch ? `?${newSearch}` : '');
window.history.replaceState(null, '', newUrl);
}
}, [currentArchive?.name, username, allPosts.length, activeTab, selectedPost?.id]);
}, [hasInitialLoaded, currentArchive?.name, username, allPosts.length, activeTab, selectedPost?.id]);
const [hasInitialLoaded, setHasInitialLoaded] = useState(false);
useEffect(() => {
if (hasInitialLoaded || serverArchives.length === 0) return;
const params = new URLSearchParams(window.location.search);
if (hasInitialLoaded) return;
const params = initialParamsRef.current;
const archiveName = params.get('a');
const tab = params.get('t');
const postId = params.get('p');
console.log('[Permalink] Initial read from URL:', { archiveName, tab, postId });
if (archiveName) {
console.log('[Permalink] Initial read from URL:', {
archiveName, tab, postId: params.get('p'),
});
if (tab && ['posts', 'reels', 'saved'].includes(tab)) {
setActiveTab(tab as 'posts' | 'reels' | 'saved');
}
if (!archiveName) {
setHasInitialLoaded(true);
return;
}
// Wait for the archive list before deciding the link is unresolvable.
if (!archivesFetched) return;
const archive = serverArchives.find(a => a.name === archiveName);
if (archive) {
console.log(`[Permalink] Auto-loading archive: ?a=${archiveName}`);
loadServerArchive(archive);
if (tab && ['posts', 'reels', 'saved'].includes(tab)) {
setActiveTab(tab as any);
}
}
} else {
console.warn(`[Permalink] No archive named "${archiveName}".`);
}
setHasInitialLoaded(true);
}, [serverArchives, hasInitialLoaded, loadServerArchive]);
}, [serverArchives, archivesFetched, hasInitialLoaded, loadServerArchive]);
/**
* Apply a `?p=` deep link exactly once per opened archive.
*
* Keying on the archive rather than on `selectedPost` matters: re-reading the
* URL whenever the selection changes would reopen the post the user just
* closed, and only worked before because the URL-writing effect happened to
* be declared first and had already stripped the param.
*/
const appliedPostParamRef = useRef<string | null>(null);
useEffect(() => {
const params = new URLSearchParams(window.location.search);
const postId = params.get('p');
if (postId && allPosts.length > 0 && !selectedPost) {
const archiveKey = currentArchive?.name ?? username;
if (!archiveKey || allPosts.length === 0) return;
if (appliedPostParamRef.current === archiveKey) return;
appliedPostParamRef.current = archiveKey;
const postId = initialParamsRef.current.get('p');
if (!postId) return;
const post = allPosts.find(p => p.id === postId);
if (post) setSelectedPost(post);
}
}, [allPosts, selectedPost]);
}, [allPosts, currentArchive?.name, username]);
return (
<div className="min-h-screen bg-gray-50 text-gray-900 font-sans">
@@ -274,6 +444,7 @@ export default function App() {
cachedArchives={cachedArchives}
onSelect={loadServerArchive}
onLocalSelect={triggerFileSelect}
onLocalCacheSelect={loadLocalCachedArchive}
onClearCache={clearCache}
isScanning={isScanning}
/>
@@ -285,9 +456,27 @@ export default function App() {
</div>
)
) : isScanning ? (
<div className="fixed inset-0 z-50 flex items-center justify-center overflow-hidden text-black">
<AnimatePresence>{currentScanningImage && (<motion.div key={currentScanningImage} initial={{ opacity: 0 }} animate={{ opacity: 0.4 }} exit={{ opacity: 0 }} className="absolute inset-0 z-0 text-black"><img src={currentScanningImage} alt="" className="w-full h-full object-cover blur-[100px] scale-110 text-black" /></motion.div>)}</AnimatePresence>
<div className="absolute inset-0 bg-white/20 z-1 text-black" />
<div className="fixed inset-0 z-50 flex items-center justify-center overflow-hidden text-black bg-[#f8fafc]">
{currentScanningImage && (
<img
src={currentScanningImage}
className="hidden"
onLoad={() => setLastLoadedScanningImage(currentScanningImage)}
/>
)}
<div className="absolute inset-0 z-0">
<AnimatePresence initial={false}>
<motion.img
key={lastLoadedScanningImage}
src={lastLoadedScanningImage || undefined}
initial={{ opacity: 0 }}
animate={{ opacity: 0.4 }}
transition={{ duration: 1.5 }}
className="absolute inset-0 w-full h-full object-cover blur-[60px] scale-110"
/>
</AnimatePresence>
</div>
<div className="absolute inset-0 bg-white/40 z-1" />
<div className="relative z-10 w-full max-w-4xl px-4 flex flex-col items-center gap-8 text-black">
<div className="text-center space-y-2 text-black"><div className="text-4xl font-bold tracking-tight italic font-serif text-black/80 drop-shadow-sm text-black">Scanning Archive...</div><div className="flex items-center justify-center gap-3 text-black"><div className="h-[1px] w-12 bg-black/10 text-black" /><p className="text-black/40 text-[10px] uppercase tracking-[0.3em] font-bold text-black">{scanningPhase === 'Indexing' ? 'Building file index' : 'Parsing metadata & media'}</p><div className="h-[1px] w-12 bg-black/10 text-black" /></div></div>
<div className="w-full max-w-2xl space-y-4 text-black"><div className="flex justify-between text-[10px] font-bold uppercase tracking-widest text-black/40 px-1 text-black"><span className="flex items-center gap-2 text-black"><Loader2 size={12} className="animate-spin text-black" />Phase: {scanningPhase}</span><span className="text-black">{scannedCount} / {totalFiles}</span></div><div className="w-full h-1.5 bg-black/5 rounded-full overflow-hidden backdrop-blur-sm border border-black/5 shadow-inner text-black"><motion.div className="h-full bg-blue-500 shadow-[0_0_15px_rgba(59,130,246,0.5)] text-black" initial={{ width: 0 }} animate={{ width: `${(scannedCount / (totalFiles || 1)) * 100}%` }} transition={{ type: 'spring', bounce: 0, duration: 0.3 }} /></div></div>
@@ -312,6 +501,32 @@ export default function App() {
</div>
</header>
{highlightGroups.length > 0 && (
<div className="flex gap-6 md:gap-8 overflow-x-auto scrollbar-hide px-4 pb-2">
{highlightGroups.map(group => (
<button
key={group.title}
onClick={() => setActiveHighlight(group.title)}
className="flex flex-col items-center gap-2 shrink-0 group/hl"
title={`${group.title}${group.items.length} item${group.items.length === 1 ? '' : 's'}`}
>
<div className="w-16 h-16 md:w-20 md:h-20 rounded-full p-[2px] bg-gray-200 group-hover/hl:bg-gray-300 transition-colors">
<div className="w-full h-full rounded-full bg-white p-[2px]">
<div className="w-full h-full rounded-full overflow-hidden bg-gray-100 flex items-center justify-center">
{group.cover ? (
<img src={group.cover} alt="" className="w-full h-full object-cover" referrerPolicy="no-referrer" loading="lazy" />
) : (
<Play size={18} className="text-gray-400" fill="currentColor" />
)}
</div>
</div>
</div>
<span className="text-[11px] max-w-[80px] truncate text-gray-700">{group.title}</span>
</button>
))}
</div>
)}
<div className="border-t border-gray-200 flex flex-col md:flex-row items-center justify-between gap-4 text-black">
<div className="flex justify-center gap-12 flex-1 text-black">
<button onClick={() => handleTabChange('posts')} className={cn("flex items-center gap-2 py-4 border-t text-xs font-bold tracking-widest uppercase transition-all text-black", activeTab === 'posts' ? "border-black text-black" : "border-transparent text-gray-400")}><Grid3X3 size={14} />Posts</button>
@@ -324,23 +539,51 @@ export default function App() {
{activeTab === 'posts' && Array.from({ length: gridOffset }).map((_, i) => (<div key={`blank-${i}`} className={cn("bg-gray-100/50 border border-dashed border-gray-200 flex items-center justify-center text-[10px] font-bold text-gray-300 uppercase tracking-tighter text-black", gridAspectRatio === '1:1' ? "aspect-square" : "aspect-[3/4]")}>Blank</div>))}
{visiblePosts.map((post) => (
<motion.div key={post.id} layoutId={post.id} onClick={() => setSelectedPost(post)} className={cn("relative group cursor-pointer overflow-hidden bg-gray-200 transition-all duration-300 text-black", activeTab === 'reels' ? "aspect-[9/16]" : (gridAspectRatio === '1:1' ? "aspect-square" : "aspect-[3/4]"))}>
{post.media[0].type === 'video' ? <VideoThumbnail url={post.media[0].url} /> : <img src={post.thumbnail} alt="" className="w-full h-full object-cover transition-transform duration-500 group-hover:scale-110 text-black" referrerPolicy="no-referrer" />}
<PostThumbnail
post={post}
thumbnailUrl={cacheHits.get(post.id)}
onRequestThumbnail={requestThumbnail}
/>
<div className="absolute top-2 right-2 flex gap-1.5 z-10 text-black">{post.media.length > 1 && <div className="bg-black/40 backdrop-blur-md p-1 rounded-md text-white shadow-sm text-black"><Layers size={16} /></div>}{post.media.some(m => m.type === 'video') && <div className="bg-black/40 backdrop-blur-md p-1 rounded-md text-white shadow-sm text-black"><Play size={16} fill="white" /></div>}</div>
<div className="absolute inset-0 bg-black/30 opacity-0 group-hover:opacity-100 transition-opacity flex items-center justify-center gap-6 text-white font-bold z-20 text-black"><div className="flex items-center gap-2 text-black"><Heart fill="white" size={20} className="text-black" /><span>-</span></div><div className="flex items-center gap-2 text-black"><MessageCircle fill="white" size={20} className="text-black" /><span>-</span></div></div>
</motion.div>
))}
</div>
{filteredPosts.length > visiblePostsCount && <div className="flex justify-center pt-12 text-black text-black"><button onClick={loadMore} className="bg-white border border-gray-200 px-8 py-2 rounded-lg font-semibold hover:bg-gray-50 transition-colors shadow-sm text-black text-black">Load More</button></div>}
{filteredPosts.length > visiblePostsCount && <div className="flex justify-center pt-12 text-black"><button onClick={loadMore} className="bg-white border border-gray-200 px-8 py-2 rounded-lg font-semibold hover:bg-gray-50 transition-colors shadow-sm text-black">Load More</button></div>}
</div>
)}
</main>
<AnimatePresence>{selectedPost && <PostModal post={selectedPost} onClose={() => setSelectedPost(null)} onNextPost={onNextPost} onPrevPost={onPrevPost} hasNextPost={postIndex < filteredPosts.length - 1} hasPrevPost={postIndex > 0} profilePic={profilePic} />}</AnimatePresence>
<AnimatePresence>
{selectedPost && (
<PostModal
post={selectedPost}
nextPost={postIndex < filteredPosts.length - 1 ? filteredPosts[postIndex + 1] : undefined}
prevPost={postIndex > 0 ? filteredPosts[postIndex - 1] : undefined}
onClose={() => setSelectedPost(null)}
onNextPost={onNextPost}
onPrevPost={onPrevPost}
hasNextPost={postIndex < filteredPosts.length - 1}
hasPrevPost={postIndex > 0}
profilePic={profilePic}
/>
)}
</AnimatePresence>
<AnimatePresence>{showStoryViewer && allStories.length > 0 && <StoryViewer stories={allStories} onClose={() => setShowStoryViewer(false)} profilePic={profilePic} />}</AnimatePresence>
<AnimatePresence>
{activeHighlight && (
<StoryViewer
stories={highlightGroups.find(g => g.title === activeHighlight)?.items ?? []}
title={activeHighlight}
onClose={() => setActiveHighlight(null)}
profilePic={profilePic}
/>
)}
</AnimatePresence>
{!isScanning && (
<footer className="max-w-5xl mx-auto px-4 py-12 text-center text-xs text-gray-400 space-y-4 text-black">
<div className="flex flex-wrap justify-center gap-x-4 gap-y-2 uppercase tracking-tight text-black text-black"><span>Meta</span><span>About</span><span>Blog</span><span>Jobs</span><span>Help</span><span>API</span><span>Privacy</span><span>Terms</span><span>Locations</span><span>Instagram Lite</span><span>Threads</span><span>Contact Uploading & Non-Users</span><span>Meta Verified</span></div>
<div className="flex flex-wrap justify-center gap-x-4 gap-y-2 uppercase tracking-tight text-black"><span>Meta</span><span>About</span><span>Blog</span><span>Jobs</span><span>Help</span><span>API</span><span>Privacy</span><span>Terms</span><span>Locations</span><span>Instagram Lite</span><span>Threads</span><span>Contact Uploading & Non-Users</span><span>Meta Verified</span></div>
<div className="text-black/40 text-black">© 2026 InstaArchive Viewer</div>
</footer>
)}
+10 -4
View File
@@ -6,14 +6,15 @@ import {
Trash2,
Zap
} from 'lucide-react';
import { ServerArchive } from '../types';
import { CacheData, ServerArchive } from '../types';
interface ArchiveDashboardProps {
archives: ServerArchive[];
localArchives?: any[];
localArchives?: CacheData[];
cachedArchives: Set<string>;
onSelect: (archive: ServerArchive) => void;
onLocalSelect: () => void;
onLocalCacheSelect: (archive: CacheData) => void;
onClearCache: (name: string) => void;
isScanning: boolean;
}
@@ -24,6 +25,7 @@ export const ArchiveDashboard: React.FC<ArchiveDashboardProps> = ({
cachedArchives,
onSelect,
onLocalSelect,
onLocalCacheSelect,
onClearCache,
isScanning
}) => {
@@ -83,7 +85,11 @@ export const ArchiveDashboard: React.FC<ArchiveDashboardProps> = ({
</div>
<div className="p-4 space-y-1">
<span className="font-bold text-sm block truncate uppercase tracking-tight text-black/80">{archive.name}</span>
<span className="text-[10px] text-gray-400 uppercase tracking-widest">{archive.fileCount} items</span>
<span className="text-[10px] text-gray-400 uppercase tracking-widest">
{archive.fileCount === null
? `${archive.sources?.length ?? 1} source${(archive.sources?.length ?? 1) === 1 ? '' : 's'}`
: `${archive.fileCount} items`}
</span>
</div>
</button>
@@ -107,7 +113,7 @@ export const ArchiveDashboard: React.FC<ArchiveDashboardProps> = ({
{localArchives.map((archive) => (
<div key={archive.name} className="relative group text-black">
<button
onClick={onLocalSelect}
onClick={() => onLocalCacheSelect(archive)}
disabled={isScanning}
className="w-full aspect-[3/4] rounded-xl overflow-hidden bg-white shadow-sm border border-gray-100 hover:shadow-xl hover:scale-[1.02] transition-all flex flex-col text-left disabled:opacity-50"
>
+55
View File
@@ -0,0 +1,55 @@
import React from 'react';
import { AlertTriangle } from 'lucide-react';
interface Props { children: React.ReactNode }
interface State { error: Error | null }
/**
* Keeps one bad archive item from blanking the whole app.
*
* Post data is derived from filenames and arbitrary archive JSON, so a single
* malformed record used to be able to throw during render and take the entire
* tree down with it.
*/
export class ErrorBoundary extends React.Component<Props, State> {
constructor(props: Props) {
super(props);
this.state = { error: null };
}
static getDerivedStateFromError(error: Error): State {
return { error };
}
componentDidCatch(error: Error, info: React.ErrorInfo) {
console.error('[ErrorBoundary] Render failed:', error, info.componentStack);
}
render() {
if (!this.state.error) return this.props.children;
return (
<div className="min-h-screen bg-gray-50 flex items-center justify-center p-6">
<div className="max-w-md w-full bg-white rounded-2xl shadow-sm border border-gray-100 p-8 space-y-4 text-center">
<div className="w-12 h-12 mx-auto rounded-full bg-red-50 text-red-500 flex items-center justify-center">
<AlertTriangle size={24} />
</div>
<h2 className="text-xl font-semibold text-black/80">Something went wrong</h2>
<p className="text-sm text-gray-500">
This archive could not be rendered. Reloading usually clears it; if it
persists, clear the cached copy from the archive explorer.
</p>
<pre className="text-[11px] text-left text-gray-400 bg-gray-50 rounded-lg p-3 overflow-x-auto">
{this.state.error.message}
</pre>
<button
onClick={() => window.location.reload()}
className="bg-blue-500 hover:bg-blue-600 text-white px-6 py-2 rounded-lg text-sm font-semibold transition-colors"
>
Reload
</button>
</div>
</div>
);
}
}
+2 -1
View File
@@ -4,7 +4,8 @@ import { MediaFile } from '../types';
import { cn } from '../lib/utils';
export const MediaRenderer = ({ file, className, isFullView }: { file: MediaFile; className?: string; isFullView?: boolean }) => {
const [isMuted, setIsMuted] = useState(false);
// Start muted so autoplay is not blocked by Safari/Firefox policy.
const [isMuted, setIsMuted] = useState(true);
const sizingClass = isFullView ? "w-full h-auto block" : "w-full h-full object-cover";
const mediaStyle = { transform: 'translateZ(0)' };
+20 -17
View File
@@ -10,13 +10,14 @@ import {
Bookmark
} from 'lucide-react';
import { motion, AnimatePresence } from 'motion/react';
import { format, parseISO } from 'date-fns';
import { Post } from '../types';
import { cn } from '../lib/utils';
import { cn, formatDateSafe } from '../lib/utils';
import { MediaRenderer } from './MediaRenderer';
interface PostModalProps {
post: Post;
nextPost?: Post;
prevPost?: Post;
onClose: () => void;
onNextPost?: () => void;
onPrevPost?: () => void;
@@ -26,7 +27,7 @@ interface PostModalProps {
}
export const PostModal: React.FC<PostModalProps> = ({
post, onClose, onNextPost, onPrevPost, hasNextPost, hasPrevPost, profilePic
post, nextPost, prevPost, onClose, onNextPost, onPrevPost, hasNextPost, hasPrevPost, profilePic
}) => {
const [currentIndex, setCurrentIndex] = useState(0);
const [direction, setDirection] = useState(0);
@@ -34,32 +35,34 @@ export const PostModal: React.FC<PostModalProps> = ({
// Preloading Logic
useEffect(() => {
const controller = new AbortController();
const preloadMedia = async (index: number) => {
if (index < 0 || index >= post.media.length) return;
const media = post.media[index];
if (!media.url) return;
const preloadMedia = async (url: string, type: 'image' | 'video') => {
if (!url) return;
try {
if (media.type === 'image') {
if (type === 'image') {
const img = new Image();
img.src = media.url;
img.src = url;
} else {
const video = document.createElement('video');
video.src = media.url;
video.src = url;
video.preload = 'auto';
}
} catch (e) {}
};
// 1. Immediate preload of first two slides
preloadMedia(0);
preloadMedia(1);
// 1. Current post: Immediate preload of first two slides
if (post.media[0]) preloadMedia(post.media[0].url, post.media[0].type);
if (post.media[1]) preloadMedia(post.media[1].url, post.media[1].type);
// 2. Delayed preload of the rest to stay out of the way of initial render
// 2. Next/Prev posts: Preload their first slides
if (nextPost?.media[0]) preloadMedia(nextPost.media[0].url, nextPost.media[0].type);
if (prevPost?.media[0]) preloadMedia(prevPost.media[0].url, prevPost.media[0].type);
// 3. Current post: Delayed preload of the rest
const timeout = setTimeout(() => {
for (let i = 2; i < post.media.length; i++) {
if (controller.signal.aborted) break;
preloadMedia(i);
preloadMedia(post.media[i].url, post.media[i].type);
}
}, 1000);
@@ -67,7 +70,7 @@ export const PostModal: React.FC<PostModalProps> = ({
controller.abort();
clearTimeout(timeout);
};
}, [post.id, post.media]);
}, [post.id, post.media, nextPost?.id, prevPost?.id]);
useEffect(() => setCurrentIndex(0), [post.id]);
useEffect(() => {
@@ -146,7 +149,7 @@ export const PostModal: React.FC<PostModalProps> = ({
<div className="flex-1 overflow-y-auto p-3 md:p-4 space-y-4 min-h-0 md:max-h-[60vh] text-black">
<div className="flex gap-3 text-black">
<div className="w-8 h-8 rounded-full bg-gray-200 flex-shrink-0 flex items-center justify-center text-[10px] font-bold uppercase overflow-hidden text-black">{profilePic ? <img src={profilePic} alt="" className="w-full h-full object-cover" referrerPolicy="no-referrer" /> : <span className="text-black">{post.username[0]}</span>}</div>
<div className="text-sm text-black"><span className="font-semibold mr-2 text-black">{post.username}</span><span className="whitespace-pre-wrap text-black">{post.caption}</span><div className="mt-2 text-xs text-gray-500 uppercase tracking-tight text-black">{format(parseISO(post.date), 'MMMM d, yyyy')}</div></div>
<div className="text-sm text-black"><span className="font-semibold mr-2 text-black">{post.username}</span><span className="whitespace-pre-wrap text-black">{post.caption}</span><div className="mt-2 text-xs text-gray-500 uppercase tracking-tight text-black">{formatDateSafe(post.date, 'MMMM d, yyyy')}</div></div>
</div>
</div>
<div className="p-3 md:p-4 border-t border-gray-100 space-y-3 shrink-0 bg-white text-black">
+124
View File
@@ -0,0 +1,124 @@
import React, { useState, useEffect, useRef } from 'react';
import { Play, Image as ImageIcon } from 'lucide-react';
import { cn } from '../lib/utils';
import { Post } from '../types';
interface PostThumbnailProps {
post: Post;
className?: string;
thumbnailUrl?: string; // High-res thumbnail from queue
onRequestThumbnail: (id: string, url: string) => void;
}
const videoThumbnailCache = new Map<string, string>();
export const PostThumbnail = ({ post, className, thumbnailUrl, onRequestThumbnail }: PostThumbnailProps) => {
const [videoThumbnail, setVideoThumbnail] = useState<string | null>(videoThumbnailCache.get(post.media[0].url) || null);
const [isInView, setIsInView] = useState(false);
const containerRef = useRef<HTMLDivElement>(null);
const mainMedia = post.media[0];
const isVideo = mainMedia.type === 'video';
useEffect(() => {
const observer = new IntersectionObserver(([entry]) => {
if (entry.isIntersecting) {
setIsInView(true);
observer.disconnect();
}
}, { rootMargin: '400px' }); // Larger margin for smoother scrolling
if (containerRef.current) {
observer.observe(containerRef.current);
}
return () => observer.disconnect();
}, []);
useEffect(() => {
if (!isInView) return;
if (isVideo) {
if (videoThumbnail) return;
const video = document.createElement('video');
video.src = `${mainMedia.url}#t=0.1`;
video.preload = 'metadata';
video.muted = true;
video.playsInline = true;
const captureFrame = () => {
try {
const canvas = document.createElement('canvas');
canvas.width = video.videoWidth; canvas.height = video.videoHeight;
const ctx = canvas.getContext('2d');
if (ctx && video.videoWidth > 0) {
ctx.drawImage(video, 0, 0, canvas.width, canvas.height);
const dataUrl = canvas.toDataURL('image/jpeg', 0.6);
videoThumbnailCache.set(mainMedia.url, dataUrl);
setVideoThumbnail(dataUrl);
}
} catch (err) {} finally { cleanup(); }
};
const handleLoadedMetadata = () => video.currentTime = 0.1;
const handleSeeked = () => captureFrame();
const cleanup = () => {
video.removeEventListener('loadedmetadata', handleLoadedMetadata);
video.removeEventListener('seeked', handleSeeked);
video.removeAttribute('src');
video.load();
};
video.addEventListener('loadedmetadata', handleLoadedMetadata);
video.addEventListener('seeked', handleSeeked);
video.addEventListener('error', cleanup);
const timeout = setTimeout(() => cleanup(), 5000);
return () => { clearTimeout(timeout); cleanup(); };
} else {
// Request high-res image thumbnailing only if size > 1MiB
const ONE_MIB = 1024 * 1024;
if (mainMedia.size && mainMedia.size > ONE_MIB) {
onRequestThumbnail(post.id, mainMedia.url);
}
}
}, [isInView, isVideo, mainMedia.url, mainMedia.size, post.id, onRequestThumbnail, videoThumbnail]);
// Determine if we are actually expecting a high-res thumbnail
const ONE_MIB = 1024 * 1024;
const isHighRes = !isVideo && mainMedia.size && mainMedia.size > ONE_MIB;
const isGenerating = isHighRes && !thumbnailUrl;
// Use high-res thumbnail if available, then video thumb, then original
const displayUrl = thumbnailUrl || videoThumbnail || post.thumbnail;
if (!displayUrl && isVideo) {
return (
<div ref={containerRef} className={cn("w-full h-full bg-gray-100 flex items-center justify-center text-black", className)}>
<Play size={20} className="text-gray-300" fill="currentColor" />
</div>
);
}
if (!displayUrl) {
return (
<div ref={containerRef} className={cn("w-full h-full bg-gray-50 flex items-center justify-center text-black", className)}>
<ImageIcon size={20} className="text-gray-200" />
</div>
);
}
return (
<div ref={containerRef} className="w-full h-full">
<img
src={displayUrl}
alt=""
className={cn(
"w-full h-full object-cover transition-all duration-700",
className,
isGenerating ? "blur-sm scale-105" : "blur-0 scale-100"
)}
referrerPolicy="no-referrer"
loading="lazy"
/>
</div>
);
};
+21 -12
View File
@@ -7,26 +7,31 @@ import {
X
} from 'lucide-react';
import { motion } from 'motion/react';
import { format, parseISO } from 'date-fns';
import { Post } from '../types';
import { cn } from '../lib/utils';
import { cn, formatDateSafe } from '../lib/utils';
interface StoryViewerProps {
stories: Post[];
onClose: () => void;
profilePic: string | null;
/** Highlight name, shown in place of the date when viewing a highlight. */
title?: string;
}
export const StoryViewer: React.FC<StoryViewerProps> = ({
stories,
onClose,
profilePic
profilePic,
title
}) => {
const [currentStoryIndex, setCurrentStoryIndex] = useState(0);
const [progress, setProgress] = useState(0);
const [isMuted, setIsMuted] = useState(false);
// Start muted: Safari and Firefox refuse to autoplay audible media, which
// would stall the reel on its first video.
const [isMuted, setIsMuted] = useState(true);
const videoRef = useRef<HTMLVideoElement>(null);
const story = stories[currentStoryIndex];
const primary = story?.media?.[0];
useEffect(() => {
setProgress(0);
@@ -34,7 +39,7 @@ export const StoryViewer: React.FC<StoryViewerProps> = ({
const interval = 50;
const updateProgress = () => {
if (story.media[0].type === 'video' && videoRef.current) {
if (primary?.type === 'video' && videoRef.current) {
const currentTime = videoRef.current.currentTime;
const totalTime = videoRef.current.duration;
if (totalTime) {
@@ -54,7 +59,7 @@ export const StoryViewer: React.FC<StoryViewerProps> = ({
}, interval);
return () => clearInterval(timer);
}, [currentStoryIndex, story.media]);
}, [currentStoryIndex, primary]);
useEffect(() => {
if (progress >= 100) {
@@ -80,6 +85,9 @@ export const StoryViewer: React.FC<StoryViewerProps> = ({
}
};
// An empty or exhausted reel has nothing to show; bail before dereferencing.
if (!story || !primary) return null;
return (
<motion.div
initial={{ opacity: 0 }}
@@ -90,7 +98,7 @@ export const StoryViewer: React.FC<StoryViewerProps> = ({
>
<div className="absolute inset-0 z-0 text-white">
<img
src={story.media[0].url}
src={primary.url}
alt=""
className="w-full h-full object-cover blur-3xl opacity-30"
/>
@@ -146,12 +154,13 @@ export const StoryViewer: React.FC<StoryViewerProps> = ({
</div>
<div className="flex items-center gap-2 text-white">
<span className="text-xs font-semibold">{story.username}</span>
<span className="text-[10px] opacity-60 font-medium">{format(parseISO(story.date), 'MMM d')}</span>
{title && <span className="text-[10px] opacity-80 font-medium truncate max-w-[120px]">{title}</span>}
<span className="text-[10px] opacity-60 font-medium">{formatDateSafe(story.date, 'MMM d')}</span>
</div>
</div>
<div className="flex items-center gap-1 text-white">
{story.media[0].type === 'video' && (
{primary.type === 'video' && (
<button
onClick={(e) => { e.stopPropagation(); setIsMuted(!isMuted); }}
className="p-2 hover:bg-white/10 rounded-full transition-colors text-white"
@@ -166,10 +175,10 @@ export const StoryViewer: React.FC<StoryViewerProps> = ({
</div>
<div className="w-full h-full flex items-center justify-center pointer-events-none text-white">
{story.media[0].type === 'video' ? (
{primary.type === 'video' ? (
<video
ref={videoRef}
src={story.media[0].url}
src={primary.url}
className="w-full h-full object-contain"
autoPlay
muted={isMuted}
@@ -179,7 +188,7 @@ export const StoryViewer: React.FC<StoryViewerProps> = ({
/>
) : (
<img
src={story.media[0].url}
src={primary.url}
alt=""
className="w-full h-full object-contain"
referrerPolicy="no-referrer"
-59
View File
@@ -1,59 +0,0 @@
import React, { useState, useEffect, useRef } from 'react';
import { Play } from 'lucide-react';
import { cn } from '../lib/utils';
const thumbnailCache = new Map<string, string>();
export const VideoThumbnail = ({ url, className }: { url: string; className?: string }) => {
const [thumbnail, setThumbnail] = useState<string | null>(thumbnailCache.get(url) || null);
const [isInView, setIsInView] = useState(false);
const containerRef = useRef<HTMLDivElement>(null);
useEffect(() => {
if (thumbnail || !containerRef.current) return;
const observer = new IntersectionObserver(([entry]) => {
if (entry.isIntersecting) { setIsInView(true); observer.disconnect(); }
}, { rootMargin: '200px' });
observer.observe(containerRef.current);
return () => observer.disconnect();
}, [thumbnail]);
useEffect(() => {
if (thumbnail || !isInView) return;
const video = document.createElement('video');
video.src = `${url}#t=0.1`; video.preload = 'metadata'; video.muted = true; video.playsInline = true;
const captureFrame = () => {
try {
const canvas = document.createElement('canvas');
canvas.width = video.videoWidth; canvas.height = video.videoHeight;
const ctx = canvas.getContext('2d');
if (ctx && video.videoWidth > 0) {
ctx.drawImage(video, 0, 0, canvas.width, canvas.height);
const dataUrl = canvas.toDataURL('image/jpeg', 0.6);
thumbnailCache.set(url, dataUrl); setThumbnail(dataUrl);
}
} catch (err) {} finally { cleanup(); }
};
const handleLoadedMetadata = () => video.currentTime = 0.1;
const handleSeeked = () => captureFrame();
const cleanup = () => {
video.removeEventListener('loadedmetadata', handleLoadedMetadata);
video.removeEventListener('seeked', handleSeeked);
video.removeAttribute('src');
video.load();
};
video.addEventListener('loadedmetadata', handleLoadedMetadata);
video.addEventListener('seeked', handleSeeked);
video.addEventListener('error', cleanup);
const timeout = setTimeout(() => { if (!thumbnailCache.has(url)) cleanup(); }, 5000);
return () => { clearTimeout(timeout); cleanup(); };
}, [url, thumbnail, isInView]);
if (!thumbnail) return (
<div ref={containerRef} className={cn("w-full h-full bg-gray-100 flex items-center justify-center text-black", className)}>
<Play size={20} className="text-gray-300" fill="currentColor" />
</div>
);
return <img src={thumbnail} alt="" className={cn("w-full h-full object-cover transition-transform duration-500 group-hover:scale-110", className)} referrerPolicy="no-referrer" />;
};
+120 -40
View File
@@ -1,8 +1,11 @@
import { useState, useCallback, useRef } from 'react';
import { useState, useCallback, useRef, useEffect } from 'react';
// @ts-ignore
import { XzReadableStream } from 'xz-decompress';
import * as idb from 'idb-keyval';
import { ArchiveFile, Post, ServerArchive } from '../types';
import { ArchiveFile, CacheData, Post, ServerArchive } from '../types';
import { setCachedArchive, getDirectoryHandle } from '../lib/archive-cache';
import { parseArchiveFilename, scopedPostId, EXPORT_RE, INSTALOADER_RE } from '../lib/archive-patterns';
const hasDirectoryHandle = async (name: string) => Boolean(await getDirectoryHandle(name));
export const useArchiveScanner = (
detectedUsername: string,
@@ -19,6 +22,7 @@ export const useArchiveScanner = (
// Result state
const [allPosts, setAllPosts] = useState<Post[]>([]);
const [allStories, setAllStories] = useState<Post[]>([]);
const [allHighlights, setAllHighlights] = useState<Post[]>([]);
const [profileMetadata, setProfileMetadata] = useState<{
username: string;
fullName: string;
@@ -39,9 +43,33 @@ export const useArchiveScanner = (
allProfilePics: [],
});
/**
* Blob URLs minted for the archive currently in state. They stay alive as long
* as that archive is on screen and are released when it is torn down
* otherwise every archive ever opened stays resident for the tab's lifetime.
*/
const createdUrlsRef = useRef<string[]>([]);
const revokeCreatedUrls = useCallback(() => {
for (const url of createdUrlsRef.current) {
try { URL.revokeObjectURL(url); } catch { /* already gone */ }
}
createdUrlsRef.current = [];
}, []);
// Release the last archive's URLs when the app unmounts.
useEffect(() => revokeCreatedUrls, [revokeCreatedUrls]);
/** Hand ownership of an externally-minted blob URL to the scanner's cleanup. */
const registerUrl = useCallback((url: string) => {
createdUrlsRef.current.push(url);
}, []);
const resetScannerState = useCallback(() => {
revokeCreatedUrls();
setAllPosts([]);
setAllStories([]);
setAllHighlights([]);
setProfileMetadata({
username: '',
fullName: '',
@@ -52,7 +80,7 @@ export const useArchiveScanner = (
profilePic: null,
allProfilePics: [],
});
}, []);
}, [revokeCreatedUrls]);
const handleFiles = useCallback(async (files: ArchiveFile[], archiveContext?: ServerArchive) => {
if (!files || files.length === 0) return;
@@ -66,6 +94,19 @@ export const useArchiveScanner = (
console.log(`[Scanner] Starting scan of ${files.length} files...`);
await new Promise(resolve => setTimeout(resolve, 100));
/**
* Mint a URL for a media file, remembering it if it needs revoking later.
* Synchronous and allocation-free: no file contents are read here.
*/
const mintUrl = (file: ArchiveFile, mimeHint?: string) => {
const url = file.createObjectUrl(mimeHint);
if (file.revocable) createdUrlsRef.current.push(url);
return url;
};
/** Stable identity for a media file, used to rehydrate URLs after a reload. */
const mediaPath = (file: ArchiveFile) => file.webkitRelativePath || file.name;
const parseXZFile = async (file: ArchiveFile) => {
try {
const stream = new XzReadableStream(file.stream());
@@ -91,6 +132,7 @@ export const useArchiveScanner = (
const postsMap = new Map<string, Partial<Post>>();
const mediaFilesMap = new Map<string, ArchiveFile>();
const discoveredProfilePics: { name: string, url: string }[] = [];
const allImageFiles: ArchiveFile[] = [];
let localFullName = '';
let localBio = '';
@@ -99,15 +141,23 @@ export const useArchiveScanner = (
let localFollowingCount = 0;
let localProfilePic: string | null = null;
const exportRegex = /^(\d{4}-\d{2}-\d{2})_(.+?) - (.+?)(?: - (\d+))?(?: - (story))?\.(.+)$/;
const instaloaderRegex = /^(\d{4}-\d{2}-\d{2}_\d{2}-\d{2}-\d{2}_UTC)(?:_(\d+))?(?:_(story))?\.(.+)$/;
const checkIsStory = (obj: any): boolean => {
if (!obj) return false;
const typeName = obj.__typename || obj.typename || '';
return (obj.is_story === true || obj.is_reel_media === true || typeName.includes('Story') || obj.audience === "MediaAudience.DEFAULT" || obj.node_type === "StoryItem" || obj.product_type === "story" || typeName === "GraphStoryVideo" || typeName === "GraphStoryImage");
};
let currentUsername = archiveContext?.name || currentArchive?.name || detectedUsername || '';
let currentUsername = archiveContext?.name || currentArchive?.name || detectedUsername;
// If still no username (likely local folder), try to extract from path
if (!currentUsername && files[0]?.webkitRelativePath) {
const pathParts = files[0].webkitRelativePath.split(/[/\\]/);
if (pathParts.length > 1) {
currentUsername = pathParts[0];
}
}
if (!currentUsername) currentUsername = 'archived_user';
let format: 'export' | 'instaloader' | 'json' | 'unknown' = 'unknown';
let jsonFiles: ArchiveFile[] = [];
@@ -129,15 +179,15 @@ export const useArchiveScanner = (
continue;
}
if (file.name.match(exportRegex)) format = 'export';
else if (file.name.match(instaloaderRegex)) format = 'instaloader';
if (EXPORT_RE.test(file.name)) format = 'export';
else if (INSTALOADER_RE.test(file.name)) format = 'instaloader';
// Highlight items match neither pattern, so a profile made up only of
// sidecar directories would otherwise never reach the filename parser.
else if (format === 'unknown' && file.source && file.source.kind !== 'posts') format = 'export';
if (lowerName.includes('_profile_pic.jpg') || (currentUsername && lowerName === `${currentUsername.toLowerCase()}.jpg`)) {
try {
const url = file.url || (await (async () => {
const blob = new Blob([await file.arrayBuffer()], { type: 'image/jpeg' });
return URL.createObjectURL(blob);
})());
const url = mintUrl(file, 'image/jpeg');
discoveredProfilePics.push({ name: file.name, url });
if (format === 'unknown' && lowerName.includes('_profile_pic.jpg')) format = 'instaloader';
} catch(e) {}
@@ -145,6 +195,7 @@ export const useArchiveScanner = (
if (isMedia(file.name)) {
mediaFilesMap.set(file.webkitRelativePath || file.name, file);
if (isImage(file.name)) allImageFiles.push(file);
}
}
@@ -193,10 +244,10 @@ export const useArchiveScanner = (
if (matchedFile) {
const type = isVideo(matchedFile.name) ? 'video' : 'image';
const url = matchedFile.url || URL.createObjectURL(new Blob([await matchedFile.arrayBuffer()], { type: type === 'video' ? 'video/mp4' : 'image/jpeg' }));
const url = mintUrl(matchedFile, type === 'video' ? 'video/mp4' : 'image/jpeg');
const existingMedia = post.media!.find(media => media.index === mIdx + 1);
if (existingMedia) { if (type === 'video' && existingMedia.type === 'image') post.media = post.media!.map(media => media.index === mIdx + 1 ? { name: matchedFile!.name, url, type, index: mIdx + 1 } : media); }
else post.media!.push({ name: matchedFile.name, url, type, index: mIdx + 1 });
if (existingMedia) { if (type === 'video' && existingMedia.type === 'image') post.media = post.media!.map(media => media.index === mIdx + 1 ? { name: matchedFile!.name, path: mediaPath(matchedFile!), url, type, index: mIdx + 1, size: matchedFile!.size } : media); }
else post.media!.push({ name: matchedFile.name, path: mediaPath(matchedFile), url, type, index: mIdx + 1, size: matchedFile.size });
}
}
if (post.media!.length > 0) postsMap.set(postId, post);
@@ -216,21 +267,25 @@ export const useArchiveScanner = (
setScannedFilesLog(prev => [`Batch ${Math.floor(j_start/CHUNK_SIZE) + 1} processing...`, ...prev.slice(0, 19)]);
for (let j = j_start; j < end; j++) {
const file = files[j]; const lowerName = file.name.toLowerCase();
const expMatch = file.name.match(exportRegex);
const insMatch = file.name.match(instaloaderRegex);
if (!expMatch && !insMatch) continue;
const kind = file.source?.kind ?? 'posts';
const parsed = parseArchiveFilename(file.name, kind, file.mtime);
if (!parsed) continue;
let postId = '', date = '', user = currentUsername || 'archived_user', index = 1, ext = '', isStory = lowerName.includes('story') || file.webkitRelativePath.toLowerCase().includes('stories');
if (expMatch) {
const [_, dMatch, uMatch, pMatch, iStrMatch, sMatch, eMatch] = expMatch;
date = dMatch; user = uMatch; postId = pMatch; index = iStrMatch ? parseInt(iStrMatch, 10) : 1; if (sMatch) isStory = true; ext = eMatch;
} else if (insMatch) {
const [_, pMatch, iStrMatch, sMatch, eMatch] = insMatch;
postId = pMatch; date = pMatch.split('_')[0]; index = iStrMatch ? parseInt(iStrMatch, 10) : 1; if (sMatch) isStory = true; ext = eMatch;
}
const { date, index, ext } = parsed;
const user = parsed.username || currentUsername || 'archived_user';
let isStory = parsed.isStory
|| lowerName.includes('story')
|| file.webkitRelativePath.toLowerCase().includes('stories');
const postId = scopedPostId(parsed.postId, kind, file.source?.dir);
if (kind === 'stories') isStory = true;
if (kind === 'highlight') isStory = false;
let post = postsMap.get(postId);
if (!post) { post = { id: postId, date, username: user, caption: '', media: [], isStory }; postsMap.set(postId, post); }
if (!post) {
post = { id: postId, date, username: user, caption: '', media: [], isStory, source: kind, highlightTitle: file.source?.title };
postsMap.set(postId, post);
}
else if (isStory) post.isStory = true;
const lowerExt = ext.toLowerCase();
@@ -248,11 +303,11 @@ export const useArchiveScanner = (
} catch (e) {}
} else if (isMedia(file.name)) {
const type = isVideo(file.name) ? 'video' : 'image';
const url = file.url || URL.createObjectURL(new Blob([await file.arrayBuffer()], { type: type === 'video' ? 'video/mp4' : 'image/jpeg' }));
const url = mintUrl(file, type === 'video' ? 'video/mp4' : 'image/jpeg');
if (type === 'image') throttledSetScanningImage(url);
const existingMedia = post.media!.find(m => m.index === index);
if (existingMedia) { if (type === 'video' && existingMedia.type === 'image') post.media = post.media!.map(m => m.index === index ? { name: file.name, url, type, index } : m); }
else post.media!.push({ name: file.name, url, type, index });
if (existingMedia) { if (type === 'video' && existingMedia.type === 'image') post.media = post.media!.map(m => m.index === index ? { name: file.name, path: mediaPath(file), url, type, index, size: file.size } : m); }
else post.media!.push({ name: file.name, path: mediaPath(file), url, type, index, size: file.size });
}
}
await new Promise(resolve => setTimeout(resolve, 0));
@@ -295,9 +350,9 @@ export const useArchiveScanner = (
const post: Post = { id: postId, date: new Date().toISOString().split('T')[0], username: currentUsername || 'archived_user', caption: baseName, media: [], thumbnail: '' };
for (const [idx, file] of batch.entries()) {
const type = isVideo(file.name) ? 'video' : 'image';
const url = file.url || URL.createObjectURL(new Blob([await file.arrayBuffer()], { type: type === 'video' ? 'video/mp4' : 'image/jpeg' }));
const url = mintUrl(file, type === 'video' ? 'video/mp4' : 'image/jpeg');
if (type === 'image') throttledSetScanningImage(url);
post.media.push({ name: file.name, url, type, index: idx + 1 });
post.media.push({ name: file.name, path: mediaPath(file), url, type, index: idx + 1, size: file.size });
}
post.thumbnail = post.media[0].url;
postsMap.set(postId, post);
@@ -310,6 +365,15 @@ export const useArchiveScanner = (
const urls = discoveredProfilePics.map(p => p.url);
localProfilePic = urls[0];
setProfileMetadata(prev => ({ ...prev, profilePic: localProfilePic, allProfilePics: urls }));
} else if (allImageFiles.length > 0) {
// Fallback: Use oldest image in archive as profile pic
allImageFiles.sort((a, b) => a.name.localeCompare(b.name));
const oldestFile = allImageFiles[0];
try {
const url = mintUrl(oldestFile, 'image/jpeg');
localProfilePic = url;
setProfileMetadata(prev => ({ ...prev, profilePic: localProfilePic, allProfilePics: [url] }));
} catch(e) {}
}
const finalUsername = currentUsername || 'archived_user';
@@ -318,11 +382,17 @@ export const useArchiveScanner = (
return { ...p, username: (p.username === 'archived_user' || !p.username) ? finalUsername : p.username, media: sortedMedia, thumbnail: sortedMedia[0].url } as Post;
});
const posts = allItems.filter(p => !p.isStory).sort((a, b) => b.date.localeCompare(a.date));
const stories = allItems.filter(p => p.isStory).sort((a, b) => b.date.localeCompare(a.date)); // Fixed bug here
const byNewest = (a: Post, b: Post) => b.date.localeCompare(a.date);
// Highlights are story-shaped but live behind their own circles, so they
// are kept out of both the grid and the profile-ring story reel.
const highlights = allItems.filter(p => p.source === 'highlight').sort(byNewest);
const rest = allItems.filter(p => p.source !== 'highlight');
const posts = rest.filter(p => !p.isStory).sort(byNewest);
const stories = rest.filter(p => p.isStory).sort(byNewest);
setAllPosts(posts);
setAllStories(stories);
setAllHighlights(highlights);
setProfileMetadata(prev => ({
...prev,
username: finalUsername,
@@ -356,9 +426,13 @@ export const useArchiveScanner = (
} catch (e) {}
}
const cacheData = {
name: cacheKey, isLocal, fileCount: archiveToCache ? archiveToCache.fileCount : files.length,
posts: isLocal ? [] : posts, stories: isLocal ? [] : stories,
const cacheData: CacheData = {
name: cacheKey, isLocal,
fileCount: (archiveToCache ? archiveToCache.fileCount : files.length) ?? files.length,
signature: archiveToCache?.signature,
posts: posts,
stories: stories,
highlights: highlights,
profileMetadata: {
username: finalUsername,
fullName: localFullName,
@@ -366,13 +440,16 @@ export const useArchiveScanner = (
followerCount: localFollowerCount,
followingCount: localFollowingCount,
externalUrl: localExternalUrl,
// Local blob: URLs die with the document, so persist a data: URL for
// the dashboard card instead. Media URLs are rehydrated from `path`.
profilePic: isLocal ? cacheThumbnail : localProfilePic,
allProfilePics: isLocal ? (cacheThumbnail ? [cacheThumbnail] : []) : discoveredProfilePics.map(p => p.url)
},
hasDirectoryHandle: isLocal ? await hasDirectoryHandle(cacheKey) : false,
timestamp: Date.now()
};
try {
await idb.set(cacheKey, cacheData);
await setCachedArchive(cacheData);
console.log(`[Cache] Data saved successfully.`);
await refreshCachedArchives();
} catch (e) { console.error(`[Cache] Save error:`, e); }
@@ -389,10 +466,12 @@ export const useArchiveScanner = (
currentScanningImage,
allPosts,
allStories,
allHighlights,
profileMetadata,
handleFiles,
setAllPosts,
setAllStories,
setAllHighlights,
setProfileMetadata,
setIsScanning,
setScanningPhase,
@@ -400,6 +479,7 @@ export const useArchiveScanner = (
setTotalFiles,
setScannedFilesLog,
setCurrentScanningImage,
resetScannerState
resetScannerState,
registerUrl
};
};
+128
View File
@@ -0,0 +1,128 @@
import { useState, useCallback, useRef, useEffect } from 'react';
import * as idb from 'idb-keyval';
import { thumbKey } from '../lib/archive-cache';
interface ThumbnailRequest {
id: string;
key: string;
url: string;
blob?: Blob;
}
const THUMBNAIL_WIDTH = 400;
export const useThumbnailQueue = (archiveName: string) => {
const [cacheHits, setCacheHits] = useState<Map<string, string>>(new Map());
const queueRef = useRef<ThumbnailRequest[]>([]);
const isProcessingRef = useRef(false);
const workerRef = useRef<Worker | null>(null);
/**
* Mirrors `cacheHits` for reads inside callbacks.
*
* `requestThumbnail` is a dependency of every PostThumbnail effect, so it must
* keep a stable identity closing over `cacheHits` state directly would give
* it a new identity per completed thumbnail and re-run the effect in all
* ~90 mounted thumbnails each time.
*/
const cacheHitsRef = useRef<Map<string, string>>(new Map());
/** Blob URLs handed out for thumbnails, released when the archive changes. */
const createdUrlsRef = useRef<string[]>([]);
const publish = useCallback((id: string, url: string) => {
createdUrlsRef.current.push(url);
cacheHitsRef.current.set(id, url);
setCacheHits(new Map(cacheHitsRef.current));
}, []);
const processNext = useCallback(async () => {
if (isProcessingRef.current || queueRef.current.length === 0 || !workerRef.current) return;
isProcessingRef.current = true;
const request = queueRef.current.shift()!;
try {
// Re-check the store: the entry may have landed since being queued.
const cached = await idb.get(request.key);
if (cached instanceof Blob) {
publish(request.id, URL.createObjectURL(cached));
isProcessingRef.current = false;
processNext();
return;
}
let blob = request.blob;
if (!blob) {
const res = await fetch(request.url);
blob = await res.blob();
}
// One image at a time: decoding several 50MP+ files concurrently is a
// reliable way to OOM the tab.
workerRef.current.postMessage({ id: request.key, blob, width: THUMBNAIL_WIDTH });
} catch (err) {
console.error('[ThumbnailQueue] Failed to process:', request.id, err);
isProcessingRef.current = false;
processNext();
}
}, [publish]);
useEffect(() => {
workerRef.current = new Worker(new URL('../lib/thumbnail-worker.ts', import.meta.url), {
type: 'module'
});
workerRef.current.onmessage = async (e) => {
const { id: key, blob, error } = e.data;
if (!error && blob) {
const id = key.split(':').slice(2).join(':');
publish(id, URL.createObjectURL(blob));
try {
await idb.set(key, blob);
} catch (err) { /* quota exceeded; the in-memory hit still stands */ }
}
isProcessingRef.current = false;
processNext();
};
return () => {
workerRef.current?.terminate();
};
}, [publish, processNext]);
// Switching archives invalidates every thumbnail URL handed out so far.
useEffect(() => {
return () => {
queueRef.current = [];
for (const url of createdUrlsRef.current) {
try { URL.revokeObjectURL(url); } catch { /* already gone */ }
}
createdUrlsRef.current = [];
cacheHitsRef.current = new Map();
setCacheHits(new Map());
};
}, [archiveName]);
const requestThumbnail = useCallback(async (id: string, url: string, blob?: Blob) => {
if (cacheHitsRef.current.has(id)) return;
const key = thumbKey(archiveName, id);
const cached = await idb.get(key);
if (cached instanceof Blob) {
publish(id, URL.createObjectURL(cached));
return;
}
if (!queueRef.current.some(r => r.id === id)) {
queueRef.current.push({ id, key, url, blob });
processNext();
}
}, [archiveName, publish, processNext]);
return {
cacheHits,
requestThumbnail
};
};
+2 -1
View File
@@ -1,4 +1,5 @@
@import url('https://fonts.googleapis.com/css2?family=Inter:wght@300;400;500;600;700&family=Playfair+Display:ital,wght@0,400..900;1,400..900&display=swap');
/* Self-hosted: no third-party font requests, works fully offline. */
@import url('/fonts/fonts.css');
@import "tailwindcss";
@theme {
+170
View File
@@ -0,0 +1,170 @@
import * as idb from 'idb-keyval';
import { CacheData, Post } from '../types';
import { DirectoryHandle, ensureReadPermission, filesFromDirectory } from './directory-handle';
import { LocalArchiveFile } from './archive-files';
/**
* Persistent archive cache.
*
* Keys are namespaced so that listing archives does not require deserializing
* every thumbnail blob in the store: `archive:` entries are metadata, `thumb:`
* entries are image blobs, `handle:` entries are directory handles.
*/
const ARCHIVE_PREFIX = 'archive:';
const THUMB_PREFIX = 'thumb:';
const HANDLE_PREFIX = 'handle:';
export const archiveKey = (name: string) => `${ARCHIVE_PREFIX}${name}`;
export const handleKey = (name: string) => `${HANDLE_PREFIX}${name}`;
/** Thumbnails are scoped per archive; post IDs alone collide across archives. */
export const thumbKey = (archive: string, postId: string) => `${THUMB_PREFIX}${archive}:${postId}`;
export const getCachedArchive = (name: string): Promise<CacheData | undefined> =>
idb.get(archiveKey(name));
export const setCachedArchive = (data: CacheData) => idb.set(archiveKey(data.name), data);
/** Names of all cached archives, without loading their contents. */
export const listCachedArchiveNames = async (): Promise<string[]> =>
(await idb.keys())
.map(String)
.filter(k => k.startsWith(ARCHIVE_PREFIX))
.map(k => k.slice(ARCHIVE_PREFIX.length));
export const listCachedArchives = async (): Promise<CacheData[]> => {
const names = await listCachedArchiveNames();
const entries = await Promise.all(names.map(getCachedArchive));
return entries.filter((e): e is CacheData => Boolean(e));
};
/** Remove an archive along with its handle and every thumbnail it owns. */
export const deleteCachedArchive = async (name: string) => {
const thumbPrefix = `${THUMB_PREFIX}${name}:`;
const stale = (await idb.keys()).map(String).filter(k => k.startsWith(thumbPrefix));
await idb.delMany([archiveKey(name), handleKey(name), ...stale]);
};
export const saveDirectoryHandle = (name: string, handle: DirectoryHandle) =>
idb.set(handleKey(name), handle);
export const getDirectoryHandle = (name: string): Promise<DirectoryHandle | undefined> =>
idb.get(handleKey(name));
/**
* One-time migration from the flat key layout (archive name as a bare key,
* `thumb_<postId>` for thumbnails).
*
* Old local entries are dropped rather than migrated: their media URLs are
* dead blob: URLs, so restoring them would render an archive of broken images.
*/
export const migrateLegacyCache = async () => {
const keys = (await idb.keys()).map(String);
const legacyThumbs = keys.filter(k => k.startsWith('thumb_'));
const legacyArchives = keys.filter(
k => !k.startsWith(ARCHIVE_PREFIX) && !k.startsWith(THUMB_PREFIX) &&
!k.startsWith(HANDLE_PREFIX) && !k.startsWith('thumb_')
);
if (!legacyThumbs.length && !legacyArchives.length) return;
const drop: string[] = [...legacyThumbs];
for (const key of legacyArchives) {
const data = await idb.get(key);
drop.push(key);
if (data && typeof data === 'object' && 'posts' in data && !(data as CacheData).isLocal) {
// Server archives keep working: their URLs are plain HTTP paths.
await setCachedArchive({ ...(data as CacheData), name: key });
}
}
await idb.delMany(drop);
console.log(`[Cache] Migrated legacy cache: dropped ${drop.length} stale keys.`);
};
/**
* Rebuild a server archive's media URLs, which are stable HTTP paths.
*
* `path` is relative to the archives root and already carries the source
* directory (which may be a sidecar such as `story - user`), so it is not
* prefixed with the archive name. Entries cached before `path` existed fall
* back to their stored URL.
*/
const rehydrateRemote = (posts: Post[]): Post[] =>
posts.map(post => {
const media = post.media.map(m => ({
...m,
url: m.path ? `/archives/${encodeURI(m.path)}` : m.url,
}));
return { ...post, media, thumbnail: media[0]?.url ?? post.thumbnail };
});
/**
* Rebuild a local archive's media URLs from a live directory handle, minting
* fresh blob: URLs for the paths recorded at scan time.
*
* Returns null when the folder is no longer reachable (permission declined, or
* the handle no longer resolves), signalling the caller to re-prompt.
*/
const rehydrateLocal = async (
posts: Post[],
handle: DirectoryHandle,
onUrl: (url: string) => void,
): Promise<Post[] | null> => {
if (!(await ensureReadPermission(handle))) return null;
let files: LocalArchiveFile[];
try {
files = await filesFromDirectory(handle);
} catch (err) {
console.warn('[Cache] Directory handle no longer readable:', err);
return null;
}
const byPath = new Map(files.map(f => [f.webkitRelativePath, f]));
return posts.map(post => {
const media = post.media.map(m => {
const file = byPath.get(m.path);
if (!file) return { ...m, url: '' };
const url = file.createObjectUrl(m.type === 'video' ? 'video/mp4' : 'image/jpeg');
onUrl(url);
return { ...m, url };
});
return { ...post, media, thumbnail: media[0]?.url ?? '' };
});
};
export interface RestoredArchive {
posts: Post[];
stories: Post[];
highlights: Post[];
profileMetadata: CacheData['profileMetadata'];
}
/**
* Turn a cache entry back into displayable state.
*
* `onUrl` receives every blob: URL minted so the caller can revoke them later.
* Returns null if a local archive's folder can no longer be reached.
*/
export const restoreArchive = async (
data: CacheData,
onUrl: (url: string) => void,
): Promise<RestoredArchive | null> => {
if (!data.isLocal) {
return {
posts: rehydrateRemote(data.posts),
stories: rehydrateRemote(data.stories),
highlights: rehydrateRemote(data.highlights ?? []),
profileMetadata: data.profileMetadata,
};
}
const handle = await getDirectoryHandle(data.name);
if (!handle) return null;
const posts = await rehydrateLocal(data.posts, handle, onUrl);
if (!posts) return null;
const stories = (await rehydrateLocal(data.stories, handle, onUrl)) ?? [];
const highlights = (await rehydrateLocal(data.highlights ?? [], handle, onUrl)) ?? [];
return { posts, stories, highlights, profileMetadata: data.profileMetadata };
};
+38 -4
View File
@@ -1,21 +1,51 @@
import { ArchiveFile } from '../types';
import { ArchiveFile, ArchiveSource } from '../types';
export class LocalArchiveFile implements ArchiveFile {
constructor(private file: File) {}
/** Blob URLs minted here are revocable and must be released when done. */
readonly revocable = true;
/**
* @param explicitPath Set when the file came from the File System Access API,
* whose File objects carry an empty webkitRelativePath.
*/
constructor(private file: File, private explicitPath?: string) {}
get name() { return this.file.name; }
get webkitRelativePath() { return this.file.webkitRelativePath; }
get webkitRelativePath() { return this.explicitPath ?? this.file.webkitRelativePath; }
get size() { return this.file.size; }
text() { return this.file.text(); }
arrayBuffer() { return this.file.arrayBuffer(); }
stream() { return this.file.stream(); }
/**
* A blob: URL backed directly by the on-disk File.
*
* Deliberately does NOT go through arrayBuffer() a File is already a Blob,
* so this hands the browser a disk-backed handle instead of pulling the whole
* file into memory. Doing otherwise means a 20GB archive tries to become 20GB
* of resident blobs.
*
* When the picker gave us no MIME type, slice() re-tags the blob with a hint.
* slice() is a zero-copy view, so this stays memory-free either way.
*/
createObjectUrl(mimeHint?: string) {
const source = this.file.type || !mimeHint
? this.file
: this.file.slice(0, this.file.size, mimeHint);
return URL.createObjectURL(source);
}
}
export class RemoteArchiveFile implements ArchiveFile {
/** Served over HTTP; there is no object URL to release. */
readonly revocable = false;
constructor(
public name: string,
public webkitRelativePath: string,
public size: number,
public url: string
public url: string,
public source?: ArchiveSource,
public mtime?: number
) {}
async text() {
const res = await fetch(this.url);
@@ -33,4 +63,8 @@ export class RemoteArchiveFile implements ArchiveFile {
});
return transform.readable;
}
createObjectUrl() {
return this.url;
}
}
+89
View File
@@ -0,0 +1,89 @@
import { describe, expect, it } from 'vitest';
import { classifyDirectory, groupArchiveDirectories } from './archive-grouping';
describe('classifyDirectory', () => {
it('treats a bare profile directory as the base', () => {
expect(classifyDirectory('4utumn07')).toEqual({
owner: '4utumn07',
source: { kind: 'posts', dir: '4utumn07' },
});
});
it('recognises a reels sidecar', () => {
expect(classifyDirectory('4utumn07 - reels')).toEqual({
owner: '4utumn07',
source: { kind: 'reels', dir: '4utumn07 - reels' },
});
});
it('recognises a stories sidecar', () => {
expect(classifyDirectory('story - dawn_petal')).toEqual({
owner: 'dawn_petal',
source: { kind: 'stories', dir: 'story - dawn_petal' },
});
});
it('splits highlight owner from title', () => {
const { owner, source } = classifyDirectory('story highlights - 4utumn07 - Sunstory');
expect(owner).toBe('4utumn07');
expect(source.kind).toBe('highlight');
expect(source.title).toBe('Sunstory');
});
it.each([
['story highlights - theoldlyricmuseinsta - 💙1999-2005 era', 'theoldlyricmuseinsta', '💙1999-2005 era'],
['story highlights - member_theworld - [Bracket]', 'member_theworld', '[Bracket]'],
['story highlights - official_band - Tour Schedule', 'official_band', 'Tour Schedule'],
['story highlights - 4utumn07 - Sketching', '4utumn07', 'Sketching'],
['story highlights - official_band - A.B.C', 'official_band', 'A.B.C'],
])('handles real-world title %s', (dir, owner, title) => {
const result = classifyDirectory(dir);
expect(result.owner).toBe(owner);
expect(result.source.title).toBe(title);
});
it('keeps titles containing " - " intact', () => {
// The username is matched as a non-space run, so only the first separator
// splits owner from title.
const { owner, source } = classifyDirectory('story highlights - user - a - b');
expect(owner).toBe('user');
expect(source.title).toBe('a - b');
});
it('does not mistake a profile with spaces for a sidecar', () => {
expect(classifyDirectory('Heejin_Bubble heejinmedia').source.kind).toBe('posts');
});
});
describe('groupArchiveDirectories', () => {
const dirs = [
'4utumn07',
'4utumn07 - reels',
'story - 4utumn07',
'story highlights - 4utumn07 - Sunstory',
'story highlights - 4utumn07 - Sketching',
'kestrelsings',
];
it('folds sidecars into their base profile', () => {
const groups = groupArchiveDirectories(dirs);
expect([...groups.keys()].sort()).toEqual(['4utumn07', 'kestrelsings']);
expect(groups.get('4utumn07')).toHaveLength(5);
expect(groups.get('kestrelsings')).toHaveLength(1);
});
it('orders sources posts, reels, stories, then highlights by title', () => {
const sources = groupArchiveDirectories(dirs).get('4utumn07')!;
expect(sources.map(s => s.kind)).toEqual(['posts', 'reels', 'stories', 'highlight', 'highlight']);
expect(sources.slice(3).map(s => s.title)).toEqual(['Sketching', 'Sunstory']);
});
it('still groups a sidecar whose base profile is missing', () => {
const groups = groupArchiveDirectories(['story - orphan']);
expect(groups.get('orphan')).toEqual([{ kind: 'stories', dir: 'story - orphan' }]);
});
it('is stable for an empty archive root', () => {
expect(groupArchiveDirectories([]).size).toBe(0);
});
});
+71
View File
@@ -0,0 +1,71 @@
/**
* Archive directory naming rules.
*
* Shared by the server (to fold sidecar directories into one profile) and the
* test suite. Kept free of Node built-ins so it can be imported from either.
*/
export type SourceKind = 'posts' | 'reels' | 'stories' | 'highlight';
export interface ArchiveSource {
kind: SourceKind;
/** Directory name relative to the archives root. */
dir: string;
/** Highlight title, for kind === 'highlight'. */
title?: string;
}
/**
* Sidecar directories sit next to the profile directory they belong to:
*
* 4utumn07 -> posts (base)
* 4utumn07 - reels -> reels
* story - 4utumn07 -> stories
* story highlights - 4utumn07 - Sunstory -> highlight "Sunstory"
*
* Instagram usernames cannot contain spaces, so matching the username as a
* run of non-space characters reliably separates it from a highlight title
* (titles may themselves contain spaces, dashes and emoji).
*/
export const classifyDirectory = (dirName: string): { owner: string; source: ArchiveSource } => {
const highlight = /^story highlights - ([^ ]+) - (.+)$/.exec(dirName);
if (highlight) {
return { owner: highlight[1], source: { kind: 'highlight', dir: dirName, title: highlight[2] } };
}
const stories = /^story - ([^ ]+)$/.exec(dirName);
if (stories) {
return { owner: stories[1], source: { kind: 'stories', dir: dirName } };
}
const reels = /^([^ ]+) - reels$/.exec(dirName);
if (reels) {
return { owner: reels[1], source: { kind: 'reels', dir: dirName } };
}
return { owner: dirName, source: { kind: 'posts', dir: dirName } };
};
const RANK: Record<SourceKind, number> = { posts: 0, reels: 1, stories: 2, highlight: 3 };
/**
* Group the archive root's directories by profile.
*
* A sidecar whose owner has no base directory still forms a group of its own,
* so nothing becomes invisible just because the base profile is missing.
*/
export const groupArchiveDirectories = (dirNames: string[]): Map<string, ArchiveSource[]> => {
const groups = new Map<string, ArchiveSource[]>();
for (const dirName of dirNames) {
const { owner, source } = classifyDirectory(dirName);
if (!groups.has(owner)) groups.set(owner, []);
groups.get(owner)!.push(source);
}
for (const sources of groups.values()) {
sources.sort((a, b) => RANK[a.kind] - RANK[b.kind] || (a.title ?? '').localeCompare(b.title ?? ''));
}
return groups;
};
+232
View File
@@ -0,0 +1,232 @@
import fs from 'fs';
import fsp from 'fs/promises';
import path from 'path';
import { ArchiveSource, SourceKind, groupArchiveDirectories } from './archive-grouping.js';
/**
* On-disk archive index.
*
* Archives live on network storage where per-file `stat` costs ~1.4ms and does
* not parallelise well, so walking every file on each request is unaffordable:
* measured against a real 110k-file archive root, listing took ~52s.
*
* Directory `stat` is effectively free, so each source directory is indexed
* once and re-used until its mtime changes. The index is warmed in the
* background at startup and persisted, making steady-state requests instant.
*/
export interface IndexedFile {
path: string;
size: number;
mtime: number;
kind: SourceKind;
title?: string;
}
interface DirIndex {
dir: string;
/** Directory mtime the index was built from; the cache key. */
mtimeMs: number;
files: IndexedFile[];
}
const MEDIA_RE = /\.(jpg|jpeg|png|webp|gif|bmp|tiff|mp4|webm|ogv|mov)$/i;
const STAT_CONCURRENCY = 16;
export class ArchiveIndex {
private dirs = new Map<string, DirIndex>();
private inFlight = new Map<string, Promise<DirIndex>>();
private dirty = false;
constructor(private archivesDir: string, private cachePath: string) {}
/** Visible (non-system) directories at the archive root. */
private listRootDirs(): string[] {
return fs.readdirSync(this.archivesDir, { withFileTypes: true })
.filter(e => e.isDirectory() && !/^[.@_]/.test(e.name))
.map(e => e.name);
}
groups(): Map<string, ArchiveSource[]> {
return groupArchiveDirectories(this.listRootDirs());
}
private dirMtime(dir: string): number {
try {
return fs.statSync(path.join(this.archivesDir, dir)).mtimeMs;
} catch {
return 0;
}
}
/** Recursively list relative file paths without stat()ing them. */
private walk(absDir: string, base = ''): string[] {
let out: string[] = [];
let entries: fs.Dirent[];
try {
entries = fs.readdirSync(absDir, { withFileTypes: true });
} catch {
return out;
}
for (const entry of entries) {
const rel = base ? `${base}/${entry.name}` : entry.name;
if (entry.isDirectory()) out = out.concat(this.walk(path.join(absDir, entry.name), rel));
else if (entry.isFile()) out.push(rel);
}
return out;
}
private async buildDir(source: ArchiveSource): Promise<DirIndex> {
const started = Date.now();
const mtimeMs = this.dirMtime(source.dir);
const absDir = path.join(this.archivesDir, source.dir);
const relPaths = this.walk(absDir);
// Only media needs a size (the client thumbnails anything over 1MiB), and
// only highlights need an mtime (their filenames carry no date). Skipping
// the rest avoids thousands of pointless round trips.
const needsStat = (rel: string) => MEDIA_RE.test(rel) || source.kind === 'highlight';
const files: IndexedFile[] = relPaths.map(rel => ({
path: `${source.dir}/${rel}`,
size: 0,
mtime: 0,
kind: source.kind,
...(source.title ? { title: source.title } : {}),
}));
const targets = files.filter((_, i) => needsStat(relPaths[i]));
let cursor = 0;
const worker = async () => {
while (cursor < targets.length) {
const file = targets[cursor++];
try {
const stat = await fsp.stat(path.join(this.archivesDir, file.path));
file.size = stat.size;
file.mtime = stat.mtimeMs;
} catch { /* raced with a delete */ }
}
};
await Promise.all(Array.from({ length: STAT_CONCURRENCY }, worker));
console.log(
`[Index] ${source.dir}: ${files.length} files (${targets.length} statted) in ${((Date.now() - started) / 1000).toFixed(1)}s`
);
this.dirty = true;
return { dir: source.dir, mtimeMs, files };
}
/** Index for one source directory, rebuilding only if its mtime moved. */
private async ensureDir(source: ArchiveSource): Promise<DirIndex> {
const cached = this.dirs.get(source.dir);
const mtimeMs = this.dirMtime(source.dir);
if (cached && cached.mtimeMs === mtimeMs) return cached;
// Collapse concurrent requests for the same directory into one walk.
const existing = this.inFlight.get(source.dir);
if (existing) return existing;
const build = this.buildDir(source).then(index => {
this.dirs.set(source.dir, index);
this.inFlight.delete(source.dir);
return index;
}).catch(err => {
this.inFlight.delete(source.dir);
throw err;
});
this.inFlight.set(source.dir, build);
return build;
}
/** All files for one profile, across its base and sidecar directories. */
async filesFor(owner: string): Promise<IndexedFile[] | null> {
const sources = this.groups().get(owner);
if (!sources?.length) return null;
const indexes = await Promise.all(sources.map(s => this.ensureDir(s)));
return indexes.flatMap(i => i.files);
}
/**
* A cheap change signature for a profile, used by the client to decide
* whether its cached copy is stale. Built from directory mtimes only, so it
* costs one stat per source directory rather than a full walk.
*/
signatureFor(sources: ArchiveSource[]): string {
return sources.map(s => `${s.dir}:${this.dirMtime(s.dir)}`).join('|');
}
/** File count for a profile, if its directories are already indexed. */
countFor(sources: ArchiveSource[]): number | null {
let total = 0;
for (const source of sources) {
const cached = this.dirs.get(source.dir);
if (!cached) return null;
total += cached.files.length;
}
return total;
}
/**
* Best-effort profile picture.
*
* Probes the conventional filenames first (one stat each) and only falls back
* to the indexed listing, so an unindexed archive still gets a thumbnail
* without triggering a walk.
*/
thumbnailFor(owner: string, sources: ArchiveSource[]): string {
const base = sources.find(s => s.kind === 'posts') ?? sources[0];
if (!base) return '';
for (const candidate of [`${owner}.jpg`, `${owner}_profile_pic.jpg`, `${owner}.jpeg`, `${owner}.png`]) {
if (fs.existsSync(path.join(this.archivesDir, base.dir, candidate))) {
return `/archives/${encodeURI(`${base.dir}/${candidate}`)}`;
}
}
const cached = this.dirs.get(base.dir);
if (cached) {
const pick = cached.files.find(f => /_profile_pic\.jpg$/i.test(f.path))
?? cached.files.find(f => /\.(jpg|jpeg|png|webp)$/i.test(f.path));
if (pick) return `/archives/${encodeURI(pick.path)}`;
}
return '';
}
/** Walk every directory once, in the background, so first opens are fast. */
async warm(): Promise<void> {
const started = Date.now();
const sources = [...this.groups().values()].flat();
console.log(`[Index] Warming ${sources.length} source directories...`);
for (const source of sources) {
try {
await this.ensureDir(source);
} catch (err) {
console.error(`[Index] Failed to index ${source.dir}:`, err);
}
}
await this.save();
console.log(`[Index] Warm complete in ${((Date.now() - started) / 1000).toFixed(1)}s`);
}
async load(): Promise<void> {
try {
const raw = await fsp.readFile(this.cachePath, 'utf8');
const parsed: DirIndex[] = JSON.parse(raw);
for (const entry of parsed) this.dirs.set(entry.dir, entry);
console.log(`[Index] Loaded ${this.dirs.size} directories from ${this.cachePath}`);
} catch {
console.log('[Index] No usable index cache; will build from scratch.');
}
}
async save(): Promise<void> {
if (!this.dirty) return;
try {
await fsp.writeFile(this.cachePath, JSON.stringify([...this.dirs.values()]), 'utf8');
this.dirty = false;
console.log(`[Index] Persisted ${this.dirs.size} directories to ${this.cachePath}`);
} catch (err) {
console.warn('[Index] Could not persist index (continuing in memory):', err);
}
}
}
+118
View File
@@ -0,0 +1,118 @@
import { describe, expect, it } from 'vitest';
import { parseArchiveFilename, scopedPostId } from './archive-patterns';
describe('parseArchiveFilename — Instagram export format', () => {
it('parses a single-image post', () => {
expect(parseArchiveFilename('2023-04-19_4utumn07 - CrORBIcJJbM.mp4')).toEqual({
postId: 'CrORBIcJJbM',
date: '2023-04-19',
username: '4utumn07',
index: 1,
ext: 'mp4',
isStory: false,
});
});
it('parses a carousel slide index', () => {
const parsed = parseArchiveFilename('2023-04-12_4utumn07 - Cq8LrxSJAJE - 3.jpg');
expect(parsed).toMatchObject({ postId: 'Cq8LrxSJAJE', index: 3, ext: 'jpg' });
});
it('groups a carousel under one post id', () => {
const ids = ['1', '2', '3'].map(
n => parseArchiveFilename(`2023-04-12_user - Cq8LrxSJAJE - ${n}.jpg`)!.postId,
);
expect(new Set(ids).size).toBe(1);
});
it('parses caption sidecar files', () => {
expect(parseArchiveFilename('2023-04-12_4utumn07 - Cq8LrxSJAJE.txt')).toMatchObject({
postId: 'Cq8LrxSJAJE',
ext: 'txt',
});
});
it('flags an explicit story suffix', () => {
expect(parseArchiveFilename('2023-04-12_user - ABC - story.jpg')?.isStory).toBe(true);
});
it('parses the story sidecar layout (date_user - N - shortcode)', () => {
// Files in `story - <user>` carry a per-day ordinal before the shortcode.
const parsed = parseArchiveFilename('2025-10-26_4utumn07 - 2 - DQRuDx9iW5Q.jpg', 'stories');
expect(parsed).toMatchObject({ date: '2025-10-26', username: '4utumn07', ext: 'jpg' });
expect(parsed!.postId).toContain('DQRuDx9iW5Q');
});
it('gives each story item a distinct id', () => {
const a = parseArchiveFilename('2026-08-13_u - 1 - Db-UTJcCUUr.mp4', 'stories')!.postId;
const b = parseArchiveFilename('2026-08-13_u - 2 - Db-oNJ1CWQ4.mp4', 'stories')!.postId;
expect(a).not.toBe(b);
});
});
describe('parseArchiveFilename — Instaloader format', () => {
it('parses a timestamped filename', () => {
expect(parseArchiveFilename('2024-01-01_12-00-00_UTC.jpg')).toMatchObject({
postId: '2024-01-01_12-00-00_UTC',
date: '2024-01-01',
index: 1,
});
});
it('parses the carousel suffix', () => {
expect(parseArchiveFilename('2024-01-01_12-00-00_UTC_2.jpg')?.index).toBe(2);
});
it('flags the story suffix', () => {
expect(parseArchiveFilename('2024-01-01_12-00-00_UTC_story.jpg')?.isStory).toBe(true);
});
});
describe('parseArchiveFilename — story highlights', () => {
it('parses the dateless highlight layout', () => {
expect(parseArchiveFilename('4utumn07 - C5dQPEYpd9W.mp4', 'highlight')).toMatchObject({
postId: 'C5dQPEYpd9W',
username: '4utumn07',
ext: 'mp4',
isStory: false,
});
});
it('dates a highlight from mtime when the filename has none', () => {
const mtime = Date.UTC(2024, 4, 17, 12, 0, 0);
expect(parseArchiveFilename('user - ABC.jpg', 'highlight', mtime)?.date).toBe('2024-05-17');
});
it('leaves the date empty when no mtime is available', () => {
expect(parseArchiveFilename('user - ABC.jpg', 'highlight')?.date).toBe('');
});
it('does not apply the loose highlight pattern outside highlight directories', () => {
// Would otherwise swallow ordinary "a - b.jpg" filenames.
expect(parseArchiveFilename('user - ABC.jpg', 'posts')).toBeNull();
});
});
describe('parseArchiveFilename — non-matching files', () => {
it.each(['4utumn07.jpg', 'profile_pic.jpg', 'README.md', 'no-separator.png'])(
'returns null for %s',
name => expect(parseArchiveFilename(name)).toBeNull(),
);
});
describe('scopedPostId', () => {
it('leaves base-profile ids untouched so permalinks keep working', () => {
expect(scopedPostId('Cq8LrxSJAJE', 'posts')).toBe('Cq8LrxSJAJE');
});
it('namespaces sidecar ids by directory', () => {
expect(scopedPostId('C5dQ', 'highlight', 'story highlights - u - Sunstory'))
.toBe('story highlights - u - Sunstory/C5dQ');
});
it('keeps the same shortcode distinct across sources', () => {
const inPosts = scopedPostId('ABC', 'posts');
const inHighlight = scopedPostId('ABC', 'highlight', 'story highlights - u - H');
expect(inPosts).not.toBe(inHighlight);
});
});
+99
View File
@@ -0,0 +1,99 @@
import { SourceKind } from '../types';
/**
* Filename parsing rules for the archive formats the viewer understands.
*
* Kept as pure functions so the riskiest part of the scanner deriving post
* identity, date and carousel order from a filename can be tested directly.
*/
/** Instagram export: `2023-04-12_user - Cq8LrxSJAJE - 2.jpg` */
export const EXPORT_RE = /^(\d{4}-\d{2}-\d{2})_(.+?) - (.+?)(?: - (\d+))?(?: - (story))?\.(.+)$/;
/** Instaloader: `2024-01-01_12-00-00_UTC_2.jpg` */
export const INSTALOADER_RE = /^(\d{4}-\d{2}-\d{2}_\d{2}-\d{2}-\d{2}_UTC)(?:_(\d+))?(?:_(story))?\.(.+)$/;
/**
* Story highlight: `user - C5dQPEYpd9W.mp4` no date prefix.
*
* Loose enough to match ordinary filenames, so it is only applied to files the
* server has already tagged as coming from a highlight directory.
*/
export const HIGHLIGHT_RE = /^(.+?) - ([A-Za-z0-9_-]+)\.(\w+)$/;
export interface ParsedFilename {
postId: string;
/** ISO date (YYYY-MM-DD), or '' when the filename carries none. */
date: string;
username: string;
/** 1-based carousel position. */
index: number;
ext: string;
isStory: boolean;
}
/**
* Parse an archive filename into post identity.
*
* `kind` selects which patterns apply; `mtime` supplies a date for formats that
* have none (highlights), so those items still sort and render sensibly.
* Returns null when no pattern matches e.g. a profile picture.
*/
export const parseArchiveFilename = (
fileName: string,
kind: SourceKind = 'posts',
mtime?: number,
): ParsedFilename | null => {
const exp = EXPORT_RE.exec(fileName);
if (exp) {
const [, date, username, postId, indexStr, story, ext] = exp;
return {
postId,
date,
username,
index: indexStr ? parseInt(indexStr, 10) : 1,
ext,
isStory: Boolean(story),
};
}
const ins = INSTALOADER_RE.exec(fileName);
if (ins) {
const [, postId, indexStr, story, ext] = ins;
return {
postId,
date: postId.split('_')[0],
username: '',
index: indexStr ? parseInt(indexStr, 10) : 1,
ext,
isStory: Boolean(story),
};
}
if (kind === 'highlight') {
const hl = HIGHLIGHT_RE.exec(fileName);
if (hl) {
const [, username, shortcode, ext] = hl;
return {
postId: shortcode,
date: mtime ? new Date(mtime).toISOString().split('T')[0] : '',
username,
index: 1,
ext,
isStory: false,
};
}
}
return null;
};
/**
* Namespace a post ID by its source directory.
*
* Base-profile IDs are left untouched so existing permalinks keep working;
* sidecar IDs are prefixed so a shortcode appearing in both the profile and a
* highlight stays two distinct posts.
*/
export const scopedPostId = (postId: string, kind: SourceKind, dir?: string): string =>
kind === 'posts' ? postId : `${dir ?? kind}/${postId}`;
+86
View File
@@ -0,0 +1,86 @@
import { LocalArchiveFile } from './archive-files';
/**
* File System Access API helpers.
*
* A `blob:` URL dies with the document, so a cached local archive whose media
* URLs are blob: URLs is worthless after a reload. A FileSystemDirectoryHandle,
* by contrast, is structured-cloneable and survives in IndexedDB so we can
* re-open the same folder on a return visit and mint fresh URLs from it.
*
* Only Chromium implements showDirectoryPicker today; callers must handle the
* unsupported case by falling back to the <input webkitdirectory> flow.
*/
// Minimal typings — TS's lib.dom does not ship these in the configured version.
type PermissionState = 'granted' | 'denied' | 'prompt';
interface FileSystemHandlePermissionDescriptor { mode?: 'read' | 'readwrite' }
export interface DirectoryHandle {
name: string;
kind: 'directory';
values(): AsyncIterableIterator<DirectoryHandle | FileHandle>;
queryPermission?(d?: FileSystemHandlePermissionDescriptor): Promise<PermissionState>;
requestPermission?(d?: FileSystemHandlePermissionDescriptor): Promise<PermissionState>;
}
interface FileHandle {
name: string;
kind: 'file';
getFile(): Promise<File>;
}
export const isDirectoryPickerSupported = () =>
typeof window !== 'undefined' && 'showDirectoryPicker' in window;
export const pickDirectory = async (): Promise<DirectoryHandle | null> => {
if (!isDirectoryPickerSupported()) return null;
try {
return await (window as any).showDirectoryPicker({ mode: 'read' });
} catch (err) {
// AbortError simply means the user dismissed the picker.
return null;
}
};
/**
* Confirm we may still read this handle. Returns false when the user declines
* or the grant has lapsed, in which case the caller should re-prompt.
*
* `requestPermission` must be called from a user gesture, so only call this
* while handling a click.
*/
export const ensureReadPermission = async (handle: DirectoryHandle): Promise<boolean> => {
try {
if (!handle.queryPermission) return true;
if ((await handle.queryPermission({ mode: 'read' })) === 'granted') return true;
if (!handle.requestPermission) return false;
return (await handle.requestPermission({ mode: 'read' })) === 'granted';
} catch {
return false;
}
};
/**
* Recursively collect every file in the directory.
*
* Paths are prefixed with the root directory's name so they line up with the
* `webkitRelativePath` values produced by <input webkitdirectory>, keeping
* cached media paths valid regardless of which picker created them.
*/
export const filesFromDirectory = async (handle: DirectoryHandle): Promise<LocalArchiveFile[]> => {
const out: LocalArchiveFile[] = [];
const walk = async (dir: DirectoryHandle, prefix: string) => {
for await (const entry of dir.values()) {
const entryPath = `${prefix}/${entry.name}`;
if (entry.kind === 'directory') {
await walk(entry as DirectoryHandle, entryPath);
} else {
const file = await (entry as FileHandle).getFile();
out.push(new LocalArchiveFile(file, entryPath));
}
}
};
await walk(handle, handle.name);
return out;
};
+42
View File
@@ -0,0 +1,42 @@
/**
* Thumbnail Generation Worker
* Uses OffscreenCanvas and createImageBitmap for high-performance,
* background-thread image resizing.
*/
self.onmessage = async (e: MessageEvent) => {
const { id, blob, width } = e.data;
try {
// 1. Create a bitmap from the blob (native browser decoding)
// We resize it DURING the decode step for maximum efficiency
const bitmap = await createImageBitmap(blob, {
resizeWidth: width,
resizeQuality: 'medium'
});
// 2. Use OffscreenCanvas to draw the resized bitmap
const canvas = new OffscreenCanvas(bitmap.width, bitmap.height);
const ctx = canvas.getContext('2d');
if (!ctx) {
throw new Error('Could not get OffscreenCanvas context');
}
ctx.drawImage(bitmap, 0, 0);
// 3. Convert to a small JPEG blob
const thumbnailBlob = await canvas.convertToBlob({
type: 'image/jpeg',
quality: 0.7
});
// 4. Release bitmap memory
bitmap.close();
// 5. Send result back
self.postMessage({ id, blob: thumbnailBlob });
} catch (err: any) {
self.postMessage({ id, error: err.message });
}
};
+20
View File
@@ -1,6 +1,26 @@
import { clsx, type ClassValue } from 'clsx';
import { twMerge } from 'tailwind-merge';
import { format, parseISO } from 'date-fns';
export function cn(...inputs: ClassValue[]) {
return twMerge(clsx(inputs));
}
/**
* Format an archive date, tolerating junk.
*
* Dates are derived from filenames and arbitrary archive JSON, and date-fns
* `format` throws a RangeError on an invalid date which would take down the
* whole modal for one malformed name. Story highlights in particular carry no
* date at all when file mtimes are unavailable.
*/
export function formatDateSafe(date: string | undefined, pattern: string): string {
if (!date) return '';
try {
const parsed = parseISO(date);
if (Number.isNaN(parsed.getTime())) return '';
return format(parsed, pattern);
} catch {
return '';
}
}
+3
View File
@@ -1,6 +1,7 @@
import {StrictMode} from 'react';
import {createRoot} from 'react-dom/client';
import App from './App.tsx';
import { ErrorBoundary } from './components/ErrorBoundary';
import './index.css';
import { registerSW } from 'virtual:pwa-register';
@@ -26,6 +27,8 @@ const updateSW = registerSW({
createRoot(document.getElementById('root')!).render(
<StrictMode>
<ErrorBoundary>
<App />
</ErrorBoundary>
</StrictMode>,
);
+88
View File
@@ -1,8 +1,30 @@
export interface MediaFile {
name: string;
/**
* Path relative to the archive root (matching webkitRelativePath for local
* folders). Unlike `url`, this survives a page reload, so it is what the
* cache persists and what URLs are rehydrated from.
*/
path: string;
url: string;
type: 'image' | 'video';
index: number;
size?: number;
}
/**
* Which sidecar directory a post came from. Archives store reels, stories and
* each story highlight in directories alongside the base profile; the viewer
* folds them into one profile and routes them by kind.
*/
export type SourceKind = 'posts' | 'reels' | 'stories' | 'highlight';
export interface ArchiveSource {
kind: SourceKind;
/** Directory name relative to the archives root. */
dir: string;
/** Highlight title, for kind === 'highlight'. */
title?: string;
}
export interface Post {
@@ -13,6 +35,10 @@ export interface Post {
media: MediaFile[];
thumbnail: string;
isStory?: boolean;
/** Defaults to 'posts' for archives without sidecar directories. */
source?: SourceKind;
/** Highlight this post belongs to, for source === 'highlight'. */
highlightTitle?: string;
}
/**
@@ -26,11 +52,73 @@ export interface ArchiveFile {
arrayBuffer(): Promise<ArrayBuffer>;
stream(): ReadableStream<Uint8Array>;
url?: string;
/**
* A URL pointing at this file's contents. Local files mint a disk-backed
* blob: URL (no data is read into memory); remote files return their HTTP URL.
*/
createObjectUrl(mimeHint?: string): string;
/** True when createObjectUrl() returns a blob: URL that must be revoked. */
readonly revocable: boolean;
/** Which sidecar directory this file came from, when known. */
source?: ArchiveSource;
/** Last-modified time (ms). Used to date items whose filename has no date. */
mtime?: number;
}
export interface ServerArchive {
name: string;
thumbnail: string;
path: string;
/** Null until the server has indexed this profile. */
fileCount: number | null;
/**
* Directory-mtime signature. Cheap for the server to compute and sufficient
* to detect changes, unlike a file count that would require a full walk.
*/
signature?: string;
/** Base profile plus any sidecar directories folded into it. */
sources?: ArchiveSource[];
}
/** One entry from GET /api/archives/:name/files. */
export interface ServerArchiveFile {
path: string;
size: number;
mtime: number;
kind: SourceKind;
title?: string;
}
export interface ProfileMetadata {
username: string;
fullName: string;
bio: string;
followerCount: number;
followingCount: number;
externalUrl: string;
profilePic: string | null;
allProfilePics: string[];
}
/** Shape of an archive entry persisted in IndexedDB. */
export interface CacheData {
name: string;
isLocal: boolean;
fileCount: number;
/** Server archives: the signature this entry was built from. */
signature?: string;
posts: Post[];
stories: Post[];
/** Story-highlight items, grouped by `highlightTitle`. */
highlights?: Post[];
profileMetadata: ProfileMetadata;
timestamp: number;
/**
* Local archives only: whether a FileSystemDirectoryHandle was stored
* alongside this entry, meaning media URLs can be rehydrated without
* re-prompting for the folder.
*/
hasDirectoryHandle?: boolean;
/** Path of the profile picture, for rehydration (local archives). */
profilePicPath?: string;
}
+12
View File
@@ -0,0 +1,12 @@
import 'react';
declare module 'react' {
interface InputHTMLAttributes<T> {
/**
* Non-standard attribute that makes a file input select a whole directory.
* Supported in Chromium and WebKit; used as the fallback picker where the
* File System Access API is unavailable.
*/
webkitdirectory?: string;
}
}
+8 -25
View File
@@ -1,11 +1,10 @@
import tailwindcss from '@tailwindcss/vite';
import react from '@vitejs/plugin-react';
import path from 'path';
import {defineConfig, loadEnv} from 'vite';
import {defineConfig} from 'vite';
import { VitePWA } from 'vite-plugin-pwa';
export default defineConfig(({mode}) => {
const env = loadEnv(mode, '.', '');
export default defineConfig(() => {
return {
plugins: [
react(),
@@ -21,42 +20,26 @@ export default defineConfig(({mode}) => {
display: 'standalone',
icons: [
{
src: 'https://cdn-icons-png.flaticon.com/512/174/174855.png',
src: '/icon-512.png',
sizes: '512x512',
type: 'image/png',
purpose: 'any maskable'
},
{
src: 'https://cdn-icons-png.flaticon.com/192/174/174855.png',
src: '/icon-192.png',
sizes: '192x192',
type: 'image/png'
}
]
},
workbox: {
globPatterns: ['**/*.{js,css,html,ico,png,svg,woff,woff2}'],
runtimeCaching: [
{
urlPattern: /^https:\/\/fonts\.googleapis\.com\/.*/i,
handler: 'CacheFirst',
options: {
cacheName: 'google-fonts-cache',
expiration: {
maxEntries: 10,
maxAgeSeconds: 60 * 60 * 24 * 365
},
cacheableResponse: {
statuses: [0, 200]
}
}
}
]
navigateFallbackDenylist: [/^\/api/, /^\/archives/],
// Fonts and icons are bundled locally, so everything the shell needs
// is precached and no runtime third-party caching rule is required.
globPatterns: ['**/*.{js,css,html,ico,png,svg,woff,woff2}']
}
})
],
define: {
'process.env.GEMINI_API_KEY': JSON.stringify(env.GEMINI_API_KEY),
},
resolve: {
alias: {
'@': path.resolve(__dirname, '.'),