Compare commits

..
31 Commits
Author SHA1 Message Date
ergosteurandClaude Opus 5 97f5d19ce4 chore: bump version to 1.3.0
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011uBWhwV3wFQ5MBCcMHHem7
2026-08-14 01:59:41 -04:00
ergosteurandClaude Opus 5 1b4aba54d6 fix: security, performance and correctness pass; add sidecar archive support
Security
- Fix path traversal in GET /api/archives/:name/files. Express decodes route
  params after segment matching, so `..%2f..%2fetc` escaped ARCHIVES_DIR and
  returned a recursive listing of arbitrary directories.
- Add CSP and baseline security headers; disable x-powered-by.
- Stop baking GEMINI_API_KEY into the client bundle (the SDK was unused).
- Run the container as `node` instead of root.

Performance
- Add a directory-mtime-keyed archive index, warmed in the background and
  persisted. Listing 110k files went from ~52s to ~0.1s; the largest archive
  (24k files) serves in ~0.3s. Per-file stat over CIFS costs ~1.4ms and does
  not parallelise, so it is now done once rather than per request.
- Build media URLs from the File directly instead of
  `new Blob([await file.arrayBuffer()])`, which read every media file fully
  into memory (a 20GB archive tried to become 20GB of resident blobs).
- Track and revoke object URLs; previously none were ever revoked.
- Give `requestThumbnail` a stable identity so a completed thumbnail stops
  re-running the effect in every mounted thumbnail.
- Namespace IndexedDB keys so listing archives no longer deserializes every
  cached thumbnail blob, and thumbnails no longer collide across archives.
- Serve real file sizes: RemoteArchiveFile was constructed with size 0, which
  silently disabled high-res thumbnailing for every server archive.

Correctness
- Local archives cached media as blob: URLs, which die with the document, so
  a cached local archive restored as an archive of broken images. Media now
  carries a stable path and is rehydrated from a persisted directory handle
  (File System Access API), falling back to re-prompting for the folder.
- Fix permalinks: the URL-writing effect erased ?a= on mount before the
  archive list arrived to consume it, so deep links never resolved.
- Make cache invalidation detect nested changes via a directory signature.
- Add an error boundary and tolerate unparseable dates, which previously
  threw a RangeError and blanked the app.
- Default video to muted so autoplay is not blocked by Safari/Firefox.

Features
- Fold sidecar directories into their base profile: `<user> - reels`,
  `story - <user>` and `story highlights - <user> - <title>` now appear as
  reels, the story ring and Instagram-style highlight circles rather than as
  separate archives.

Housekeeping
- Add @types/react; React was previously type-checked against its JavaScript
  source, so `npm run lint` gave almost no type safety on components.
- Vendor fonts and PWA icons locally; the app made third-party CDN requests
  despite advertising offline support and local-only processing.
- Drop unused better-sqlite3 (a native module that broke `npm install`).
- Add vitest with 36 tests over the filename and directory-naming rules.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011uBWhwV3wFQ5MBCcMHHem7
2026-08-14 01:59:19 -04:00
ergosteur b30285fe70 docs: update documentation for high-res performance and local persistence
Docker Build and Publish / build-and-push (push) Failing after 10s
Key changes:
- Updated README.md and GEMINI.md with details on background thumbnailing and inter-post preloading.
- Documented persistent local archive caching and smart profile fallback features.
- Added dist-server/ to .gitignore.
- Restored missing feature descriptions and troubleshooting tips in README.
2026-03-07 21:59:43 -05:00
ergosteur 20209bcad5 feat: enable persistent local archives and smart profile fallback
Key changes:
- Enabled full metadata caching for local folder archives, allowing them to load instantly from IndexedDB without re-uploading.
- Implemented oldest-image fallback for profiles missing an explicit profile picture.
- Restored folder-name-to-username detection for local archive uploads.
- Optimized scan indexing to track all image files for fallback use.
2026-03-07 21:56:25 -05:00
ergosteur 74902234b3 fix: restore white glass scanning UI and resolve small image blur bug
Key changes:
- Corrected logic in PostThumbnail to prevent blur effects on images smaller than 1MiB.
- Restored the white glass aesthetic to the scanning dashboard with improved contrast and transparency.
- Optimized scanning background transitions to ensure a smooth, flicker-free crossfade.
2026-03-07 21:46:11 -05:00
ergosteur d62bddc3aa perf: implement background thumbnail generation and inter-post preloading
Key changes:
- Added Web Worker for background image thumbnailing with a 1MiB threshold to optimize CPU/memory usage.
- Implemented a serial task queue for memory-safe high-res image processing, preventing OOM crashes.
- Added inter-post preloading in the modal for seamless 'Previous/Next' navigation.
- Refined scanning UI with double-buffering and a dark background to completely eliminate white flashes.
- Renamed project to 'instaarchive-viewer' in package.json.
- Fixed 'Open image in new tab' by denylisting /archives and /api in PWA config.
2026-03-07 21:42:56 -05:00
ergosteur 42c13ea106 chore: bump version to 1.2.0
Docker Build and Publish / build-and-push (push) Failing after 9s
2026-03-07 21:16:39 -05:00
ergosteur a4e9ce16a7 feat: modularize scanner, enhance carousel preloading, and improve PWA updates
Summary of changes:
- Extracted archive scanning logic into a modular 'useArchiveScanner' hook for better maintainability and performance.
- Refined PostModal carousel with intelligent media preloading and smoother, jitter-free transitions.
- Optimized image rendering with 'decoding=async' and removed 'black flashes' between slide changes.
- Updated PWA configuration to 'autoUpdate' with hourly periodic checks for fresh content.
- Fixed several bugs including stories sorting, permalink parameter cleanup, and profile metadata cache restoration.
- Comprehensive updates to documentation (README.md and GEMINI.md) reflecting the new architecture.
2026-03-07 21:16:28 -05:00
ergosteur 4f89a69ee3 fix: optimize scanning performance and resolve zero-post bug
Docker Build and Publish / build-and-push (push) Failing after 10s
2026-03-07 20:25:23 -05:00
ergosteur 147dcdf2f1 fix: restore missing UI handlers and finalize generic parser
Docker Build and Publish / build-and-push (push) Failing after 10s
2026-03-07 20:18:33 -05:00
ergosteur d4e20d9b98 fix: refine dockerignore and bump version to v1.1.4
Docker Build and Publish / build-and-push (push) Failing after 10s
2026-03-07 20:10:40 -05:00
ergosteur ec8c771733 feat: implement permalinks and document PWA cache troubleshooting
Docker Build and Publish / build-and-push (push) Failing after 10s
2026-03-07 05:20:09 -05:00
ergosteur 3784e8729b debug: add verbose logging to permalink synchronization 2026-03-07 05:12:14 -05:00
ergosteur 69d62eaa5c fix: improve permalink comparison and add debug logging 2026-03-07 05:10:43 -05:00
ergosteur 767f9c508b feat: implement permalinks for archives, tabs, and posts 2026-03-07 05:07:04 -05:00
ergosteur 103ce6f207 fix: exhaustive generic parser and implement local archive history 2026-03-07 05:05:08 -05:00
ergosteur 5267dab236 fix: address Docker EACCES errors with better logging and SELinux hints
Docker Build and Publish / build-and-push (push) Failing after 9s
2026-03-07 03:02:02 -05:00
ergosteur ebf2bf660a fix: improve Docker archive discovery and switch to compiled server
Docker Build and Publish / build-and-push (push) Failing after 1m6s
2026-03-07 02:57:19 -05:00
ergosteur c0f3523a9c docs: update README and GEMINI with Docker usage and new features 2026-03-07 02:50:18 -05:00
ergosteur 6f5021638c feat: add Dockerfile and GitHub Actions workflow for GHCR deployment
Docker Build and Publish / build-and-push (push) Canceled after 11s
2026-03-07 02:45:17 -05:00
ergosteur 67f7750157 feat: refine navigation protection to only warn when leaving the app 2026-03-07 02:41:16 -05:00
ergosteur 9e306eb85e feat: add explicit confirmation for back button and refresh in archives 2026-03-07 02:39:39 -05:00
ergosteur b2da08d52d feat: add navigation protection and refine cached badge visibility 2026-03-07 02:37:17 -05:00
ergosteur d7c13ecc19 fix: resolve Firefox media warnings by improving video cleanup 2026-03-07 02:30:46 -05:00
ergosteur d396b356be feat: implement persistent caching, glassy scanning UI, and UI refinements 2026-03-07 02:28:22 -05:00
ergosteur e23dfe4474 feat: implement self-hostable mode with server-side directory scanning 2026-03-07 00:59:31 -05:00
ergosteur 41e7c5e206 docs: update README and GEMINI.md, remove AI Studio boilerplate and .env.example 2026-03-07 00:36:46 -05:00
ergosteur f685eaebd7 feat: enhance story viewer and media playback experience 2026-03-07 00:31:04 -05:00
ergosteur 47e44ec5e9 feat: improve archive parsing, add .json.xz support, and fix profile pic display 2026-03-07 00:03:54 -05:00
ergosteur cd7dc5f981 feat: Initialize InstaArchive PWA project
Sets up a new React PWA project with Vite, Tailwind CSS, and basic PWA features. Includes essential files like README, .gitignore, package.json, and initial app structure.
2026-03-06 22:41:48 -05:00
ergosteurandGitHub a724e5bc87 Initial commit 2026-03-06 22:41:33 -05:00
43 changed files with 3874 additions and 2300 deletions
+1
View File
@@ -1,5 +1,6 @@
node_modules
dist
dist-server
.git
.github
_sample-archives
+1
View File
@@ -1,6 +1,7 @@
node_modules/
build/
dist/
dist-server/
coverage/
.DS_Store
*.log
+66
View File
@@ -0,0 +1,66 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
InstaArchive Viewer is a React 19 + Vite 6 PWA for browsing archived Instagram data (both official Instagram exports and Instaloader archives). All archive parsing and media processing happens client-side in the browser — the Express backend only lists/serves files from disk, it never parses archive contents.
## Commands
- `npm install` — install dependencies
- `npm run dev` — start Vite dev server on port 3000 (proxies `/api` and `/archives` to `http://localhost:3001`)
- `npm run server` — start the Express backend (`tsx server.ts`) on port 3001, serving archives from `ARCHIVES_DIR` (defaults to `./_sample-archives`)
- `npm run build` — build frontend to `dist/` (`vite build`) and backend to `dist-server/` (`tsc server.ts ...`)
- `npm run lint` — type-check only (`tsc --noEmit`); there is no separate test suite or linter config
- `npm run clean` — remove `dist/`
For local development you typically need both `npm run dev` and `npm run server` running concurrently — the frontend alone has nothing to talk to for server-mode archives (local-folder mode works without the backend).
## Architecture
### Two archive sources, one data model
The app supports loading archives two ways, unified behind the `ArchiveFile` interface (`src/types/index.ts`, implementations in `src/lib/archive-files.ts`):
- **`LocalArchiveFile`** — wraps a browser `File` from a local folder picker (`webkitdirectory`). Fully offline, media is never uploaded anywhere.
- **`RemoteArchiveFile`** — wraps a file served from the Express backend's `/archives/:name/...` static route, fetched on demand.
All downstream parsing code (`useArchiveScanner`) operates only on `ArchiveFile[]` and doesn't care which backing implementation it got.
### Scanning pipeline (`src/hooks/useArchiveScanner.ts`)
This is the core of the app — a single large `handleFiles` function that:
1. **Indexes** all files, detecting archive format by filename regex: Instagram "export" format (`YYYY-MM-DD_user - post_id[- idx][- story].ext`), Instaloader format (`YYYY-MM-DD_HH-MM-SS_UTC[_idx][_story].ext`), or a generic JSON-manifest format (`posts_1.json`, `reels_1.json`, `stories_1.json`, possibly `.json.xz`-compressed via `xz-decompress`).
2. **Parses** according to detected format, building a `Map<postId, Partial<Post>>`. For JSON-manifest format, media files are matched to JSON entries by URI substring match, then by ID substring match, then by filename-derived heuristic — in that fallback order.
3. **Falls back** to generic filename-prefix grouping when no posts were found via regex/JSON matching (treats files sharing a common basename as one carousel post, chunked into groups of 20).
4. Detects a **profile picture** from `*_profile_pic.jpg` / `<username>.jpg` files, or falls back to the oldest image in the archive by filename sort ("Smart Fallback").
5. **Caches** the final `{ posts, stories, profileMetadata, ... }` result to IndexedDB via `idb-keyval`, keyed by archive name (or `local_archive` for unnamed local folders) — this is what makes repeat visits load instantly. Both server and local archives are cached; the cache shape is documented inline in `useArchiveScanner`'s state (mirrors the `CacheData` interface in `GEMINI.md`).
When modifying format-detection or media-matching logic, be aware the three code paths (JSON-manifest, filename-regex export/instaloader, generic fallback) are largely independent and a change to one rarely needs to touch the others — but all three write into the same `postsMap`.
### Thumbnail generation (`src/hooks/useThumbnailQueue.ts` + `src/lib/thumbnail-worker.ts`)
High-res images (>1MiB) are downscaled off the main thread:
- `useThumbnailQueue` maintains a **serial** (one-at-a-time) queue — this is deliberate, not a bug: decoding multiple 50MP+ images concurrently causes OOM crashes in the browser.
- Actual resizing happens in `thumbnail-worker.ts` using `OffscreenCanvas`/`createImageBitmap` inside a Web Worker.
- Results are cached in IndexedDB under a `thumb_<id>` key, checked before falling back to the worker, so thumbnails persist across sessions.
### URL state sync (`src/App.tsx`)
App state (selected archive, active tab, selected post) is synchronized with URL query params (`?a=`, `?t=`, `?p=`) via `URLSearchParams` + `window.history.replaceState` in a cluster of `useEffect` hooks — this is what enables permalinks/deep-linking. When adding new shareable state, follow this pattern rather than introducing a router.
### Backend (`server.ts`)
Minimal Express server, three responsibilities only:
- `GET /api/archives` — lists subdirectories of `ARCHIVES_DIR` (skipping dotfiles/`@`/`_`-prefixed dirs) as `ServerArchive[]`, guessing a thumbnail per archive.
- `GET /api/archives/:name/files` — recursively lists all files in one archive directory.
- Static-serves `ARCHIVES_DIR` under `/archives` and, in production, serves the built `dist/` frontend with an SPA fallback.
It does no parsing of archive/JSON contents — that's entirely client-side in `useArchiveScanner`. `ARCHIVES_DIR` is resolved from the `ARCHIVES_DIR` env var (see `.env` / Docker volume mount at `/archives`).
### PWA / build quirks
- `vite.config.ts` sets `hmr: process.env.DISABLE_HMR !== 'true'` — this is intentionally left alone; it exists to disable file-watch flicker when running under AI Studio-style agent editing. Don't "clean up" or remove it.
- `workbox.navigateFallbackDenylist` excludes `/api` and `/archives` from the SPA fallback so those routes hit the real server/static files instead of `index.html` (needed for "open original file in new tab").
- Service worker uses `registerType: 'autoUpdate'` with hourly periodic checks — new deployments propagate to open clients automatically.
+14 -7
View File
@@ -7,7 +7,7 @@ WORKDIR /app
COPY package*.json ./
RUN npm ci
# Copy source and build frontend
# Copy source and build (frontend and server)
COPY . .
RUN npm run build
@@ -18,6 +18,9 @@ FROM node:20-slim AS runtime
ENV NODE_ENV=production
ENV PORT=3000
ENV ARCHIVES_DIR=/archives
# Where the archive index is persisted; mount a volume here so a restart does
# not have to re-walk the whole archive root.
ENV CACHE_DIR=/cache
WORKDIR /app
@@ -25,14 +28,18 @@ WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev
# Copy built assets and server
# Copy built assets and server. The whole dist-server tree is needed: the
# server imports shared archive-grouping logic emitted alongside it.
COPY --from=build /app/dist ./dist
COPY --from=build /app/server.ts ./
COPY --from=build /app/dist-server/ ./
# Ensure archives directory exists
RUN mkdir -p /archives
# Ensure archives and cache directories exist, writable by the runtime user.
RUN mkdir -p /archives /cache && chown node:node /cache
# Drop root: the server reads the archives volume and writes only its index.
USER node
EXPOSE 3000
# Start server using tsx
CMD ["npx", "tsx", "server.ts"]
# Start server
CMD ["node", "server.js"]
+55 -25
View File
@@ -2,38 +2,68 @@
## Project Overview
**InstaArchive** is a high-performance, React-based Progressive Web App (PWA) designed to browse and explore archived Instagram data with a native-feeling interface. It allows users to load local archive directories (either official Instagram exports or Instaloader format) and browse posts, reels, and stories.
**InstaArchive** is a high-performance, React-based Progressive Web App (PWA) designed to browse and explore archived Instagram data with a native-feeling interface. It supports both local directory loading and self-hosted server modes.
### Key Technical Features
- **Local Archive Loading:** Uses the `webkitdirectory` API to scan and process local files securely on the client-side.
- **Support for Multiple Formats:** Recognizes official Instagram export structures and Instaloader regex-based naming conventions.
- **Compressed Metadata**: Support for `.json.xz` file decompression using `xz-decompress` (WASM-powered).
- **Dynamic Media Grid**: Customizable grid layouts (1:1 and 3:4 aspect ratios) with adjustable offsets ("bumps") for aesthetic alignment.
- **Story Viewer**: Native-like story experience with segmented progress bars, automated playback, audio controls, and chronological sorting.
- **Auto-Unmute**: Videos default to unmuted when opened in full view for a better user experience.
- **PWA Ready**: Built with `vite-plugin-pwa` for offline capabilities and a standalone application experience.
- **Permalinks:** Full synchronization between application state and URL query parameters (`?a=`, `?t=`, `?p=`). Supports deep-linking to archives, tabs, and specific posts. URL parameters are automatically cleaned when navigating back to the archive explorer.
- **Persistent Caching:** Uses `idb-keyval` (IndexedDB) to cache parsed metadata. Subsequent loads are near-instant.
- **Metadata:** Caches profile info and post lists for both remote AND local archives.
- **Thumbnails:** High-res images (>1MiB) and videos have thumbnails generated and cached in IndexedDB with the `thumb_` prefix.
- **Background Media Processing:**
- **High-Res Images:** A dedicated Web Worker (`thumbnail-worker.ts`) handles image resizing using `OffscreenCanvas` and `createImageBitmap` to prevent main-thread jank.
- **Serial Queue:** A memory-safe queue ensures only one high-res image is decoded at a time, preventing Out-of-Memory (OOM) crashes on 50MP+ files.
- **High-Performance Carousel:** Advanced `PostModal` with:
- **Inter-Post Preloading:** Preloads the first media of adjacent posts for instant navigation.
- **Intra-Carousel Preloading:** Intelligently preloads the current carousel's slides.
- **Seamless Transitions:** Zero-latency slide transitions without "black flashes" between images.
- **Glassy Scanner UI:** Custom-built glassmorphism scanning dashboard with double-buffering logic to ensure smooth, flicker-free background crossfades during file indexing.
- **PWA Capabilities:**
- **Auto-Updates:** Hourly periodic update checks.
- **Navigation Fix:** `navigateFallbackDenylist` allows direct server access to `/archives/` and `/api/` (enabling "Open in new tab" for original files).
### Main Technologies
- **Frontend:** React 19, Vite, TypeScript
- **Frontend:** React 19, Vite 6, TypeScript
- **Styling:** Tailwind CSS (v4)
- **Icons:** Lucide React
- **Animations:** Framer Motion (`motion/react`)
- **Utility:** Date-fns, clsx, tailwind-merge
- **Decompression**: xz-decompress (WASM)
- **Persistence:** IndexedDB (`idb-keyval`)
- **Backend:** Express, tsx
- **Workers:** Web Workers for background image processing.
## Architecture
### State Management
- **`useArchiveScanner` Hook:** Centralized logic for parsing and caching. It handles folder-name-to-username detection and "Smart Fallback" profile pictures (using the oldest image if no profile pic is found).
- **`useThumbnailQueue` Hook:** Manages the serial processing of high-resolution media.
### Cache Schema
```typescript
interface CacheData {
name: string;
isLocal: boolean;
fileCount: number;
posts: Post[]; // Cached for all archive types
stories: Post[];
profileMetadata: {
username: string;
fullName: string;
bio: string;
followerCount: number;
followingCount: number;
externalUrl: string;
profilePic: string | null;
allProfilePics: string[];
};
timestamp: number;
}
```
## Commands
- `npm install`: Install project dependencies.
- `npm run dev`: Start the local development server on port 3000.
- `npm run build`: Generate the production-ready build in the `dist` folder.
- `npm run preview`: Locally preview the production build.
- `npm run lint`: Execute TypeScript type-checking (`tsc --noEmit`).
- `npm install`: Install dependencies.
- `npm run dev`: Start dev server (Port 3000).
- `npm run build`: Build frontend (`dist/`) and server (`dist-server/`).
- `npm run server`: Start production-ready backend.
- `npm run lint`: Execute TypeScript type-checking.
## Development Conventions
- **Component Architecture:** Functional components with modern hooks.
- **State Management**: React `useState`, `useMemo`, and `useCallback` for optimized performance during large archive scans.
- **File Handling:** Privacy-focused client-side scanning of archive directories.
- **Project Structure:**
- `src/App.tsx`: Main entry point containing application logic and UI components.
- `src/main.tsx`: React DOM mounting.
- `src/index.css`: Global styles and Tailwind imports.
- `vite.config.ts`: Project build and PWA configuration.
## Production Deployment
The project is containerized. It expects a volume mount at `/archives` containing subdirectories for each user.
+75 -32
View File
@@ -4,40 +4,83 @@ A high-performance React PWA for browsing archived Instagram data with a native-
## Features
- **Local Privacy**: All processing is done client-side using browser APIs. Your data never leaves your computer.
- **Multiple Formats**: Supports official Instagram JSON exports and Instaloader regex-based naming conventions.
- **Metadata Support**: Robust parsing of `.json` and `.json.xz` files for captions, timestamps, and story metadata.
- **Advanced Carousel**: Seamless, zero-latency transitions between slides with intelligent preloading. Navigating between different posts is now near-instant thanks to inter-post background preloading.
- **High-Res Performance**: Handles 50MP+ images effortlessly using a background Web Worker and a memory-safe serial processing queue.
- **Persistent Local Caching**: Uses IndexedDB to store parsed archives and generated thumbnails. **Local folders** now load instantly from cache on return visits without needing to re-upload files.
- **Permalinks**: State is synchronized with the URL, allowing you to share direct links to archives, tabs, or specific posts. Navigating back to the explorer cleans up URL parameters automatically.
- **Glassy Scanning UI**: A refined, translucent white terminal experience with flicker-free, double-buffered dynamic blurred backgrounds.
- **PWA with Auto-Update**: Fully offline-capable and installable. Clients automatically receive updates when a new version is deployed to the server.
- **Local Privacy**: All processing is done client-side. Even when using the self-hosted version, your media is processed locally in your browser and never uploaded.
- **Smart Fallbacks**: Automatically detects usernames from folder names and uses the oldest archive image as a profile picture if one is missing.
- **Customizable Grid**: 1:1 or 3:4 aspect ratios with adjustable "bumps" for aesthetic alignment.
- **Story Viewer**: Native-like story experience with segmented progress bars, auto-playback, and audio controls.
- **Media Grid**: Customizable 1:1 or 3:4 grid views with adjustable offsets for aesthetic alignment.
- **Auto-Deduplication**: Intelligently prefers video files over thumbnail images for the same post.
- **Navigation Protection**: Intercepts accidental browser "Back" or "Refresh" actions to protect your current session.
## Run Locally
## Deployment
### Docker (Recommended)
The easiest way to run InstaArchive is using Docker.
```bash
docker run -d \
-p 3000:3000 \
-v /path/to/your/archives:/archives:ro \
ghcr.io/ergosteur/instaarchive-viewer:latest
```
> **Note for Linux/SELinux users:** If you see "Permission Denied" in the logs, append `,z` to your volume mount: `-v /path/to/archives:/archives:ro,z`
### Docker Compose
Create a `compose.yml` file:
```yaml
services:
instaarchive:
image: ghcr.io/ergosteur/instaarchive-viewer:latest
ports:
- "3000:3000"
volumes:
- ./archives:/archives:ro,z # ,z handles SELinux permissions
```
### Troubleshooting Permissions
If the app shows "No Archives Found" and logs `EACCES: permission denied`:
1. **Check Directory Permissions**: Ensure the archive folder is world-readable:
```bash
chmod -R 755 /path/to/archives
```
2. **SELinux (Fedora/RHEL/CentOS)**: Use the `:z` flag in your volume mount as shown above.
3. **User Mapping**: You can force the container to run as your host user:
```bash
docker run --user $(id -u):$(id -g) ...
```
## Supported Archive Structure
Place your archive folders inside the mounted `/archives` directory. The directory name will be used as the account username.
### Example Structure:
```text
archives/
├── wanderlust_explorer/ # Instaloader format
│ ├── 2024-01-01_12-00-00_UTC.jpg
│ ├── 2024-01-01_12-00-00_UTC.json.xz
│ └── wanderlust_explorer_profile_pic.jpg
└── pixel_architect/ # Instagram Export format
├── 2023-12-25_pixel_architect - post_123.jpg
├── 2023-12-25_pixel_architect - post_123.json
└── pixel_architect.jpg
```
## Local Development
**Prerequisites:** Node.js (LTS recommended)
1. **Install dependencies:**
```bash
npm install
```
2. **Start the development server:**
```bash
npm run dev
```
3. **Open in browser:**
Navigate to `http://localhost:3000` and select your Instagram archive directory.
## Building for Production
To generate a production-ready build in the `dist` folder:
```bash
npm run build
```
To preview the build:
```bash
npm run preview
```
1. **Install dependencies:** `npm install`
2. **Start dev server:** `npm run dev` (Frontend on port 3000)
3. **Start local backend:** `npm run server` (Optional, serves `./_sample-archives` on port 3001)
4. **Build production:** `npm run build` (Generates `./dist` for frontend and `./dist-server` for the API)
+17
View File
@@ -0,0 +1,17 @@
services:
instaarchive:
image: ghcr.io/ergosteur/instaarchive-viewer:latest
ports:
- "3000:3000"
volumes:
- ./archives:/archives:ro,z
# Persists the archive index so restarts don't re-walk every file.
- instaarchive-cache:/cache
environment:
- PORT=3000
- ARCHIVES_DIR=/archives
- CACHE_DIR=/cache
restart: unless-stopped
volumes:
instaarchive-cache:
+472 -1171
View File
File diff suppressed because it is too large Load Diff
+11 -7
View File
@@ -1,21 +1,22 @@
{
"name": "react-example",
"name": "instaarchive-viewer",
"private": true,
"version": "1.1.0",
"version": "1.3.0",
"type": "module",
"scripts": {
"dev": "vite --port=3000 --host=0.0.0.0",
"build": "vite build",
"build": "vite build && npm run build:server",
"build:server": "tsc server.ts --esModuleInterop --module ESNext --target ES2022 --moduleResolution bundler --outDir dist-server",
"preview": "vite preview",
"server": "tsx server.ts",
"clean": "rm -rf dist",
"lint": "tsc --noEmit"
"lint": "tsc --noEmit",
"test": "vitest run",
"test:watch": "vitest"
},
"dependencies": {
"@google/genai": "^1.29.0",
"@tailwindcss/vite": "^4.1.14",
"@vitejs/plugin-react": "^5.0.4",
"better-sqlite3": "^12.4.1",
"clsx": "^2.1.1",
"date-fns": "^4.1.0",
"dotenv": "^17.2.3",
@@ -33,10 +34,13 @@
"devDependencies": {
"@types/express": "^4.17.21",
"@types/node": "^22.14.0",
"@types/react": "^19.2.18",
"@types/react-dom": "^19.2.4",
"autoprefixer": "^10.4.21",
"tailwindcss": "^4.1.14",
"typescript": "~5.8.2",
"vite": "^6.2.0",
"vite-plugin-pwa": "^1.2.0"
"vite-plugin-pwa": "^1.2.0",
"vitest": "^3.2.7"
}
}
+56
View File
@@ -0,0 +1,56 @@
/* Self-hosted subset of Inter + Playfair Display.
Vendored so the PWA works offline and makes no third-party requests. */
/* latin-ext */
@font-face {
font-family: 'Inter';
font-style: normal;
font-weight: 300 700;
font-display: swap;
src: url(/fonts/inter-300_700-normal-b6db4a.woff2) format('woff2');
unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF;
}
/* latin */
@font-face {
font-family: 'Inter';
font-style: normal;
font-weight: 300 700;
font-display: swap;
src: url(/fonts/inter-300_700-normal-6ab57b.woff2) format('woff2');
unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;
}
/* latin-ext */
@font-face {
font-family: 'Playfair Display';
font-style: italic;
font-weight: 400 900;
font-display: swap;
src: url(/fonts/playfair-display-400_900-italic-2d6d99.woff2) format('woff2');
unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF;
}
/* latin */
@font-face {
font-family: 'Playfair Display';
font-style: italic;
font-weight: 400 900;
font-display: swap;
src: url(/fonts/playfair-display-400_900-italic-d14361.woff2) format('woff2');
unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;
}
/* latin-ext */
@font-face {
font-family: 'Playfair Display';
font-style: normal;
font-weight: 400 900;
font-display: swap;
src: url(/fonts/playfair-display-400_900-normal-ca7410.woff2) format('woff2');
unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF;
}
/* latin */
@font-face {
font-family: 'Playfair Display';
font-style: normal;
font-weight: 400 900;
font-display: swap;
src: url(/fonts/playfair-display-400_900-normal-61a963.woff2) format('woff2');
unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;
}
Binary file not shown.
Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 2.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

+123 -56
View File
@@ -3,6 +3,8 @@ import fs from 'fs';
import path from 'path';
import { fileURLToPath } from 'url';
import dotenv from 'dotenv';
import os from 'os';
import { ArchiveIndex } from './src/lib/archive-index.js';
dotenv.config();
@@ -11,77 +13,136 @@ const __dirname = path.dirname(__filename);
const app = express();
const PORT = process.env.PORT || 3001;
const ARCHIVES_DIR = process.env.ARCHIVES_DIR || path.join(__dirname, '_sample-archives');
const ARCHIVES_DIR = path.resolve(process.env.ARCHIVES_DIR || path.join(__dirname, '_sample-archives'));
console.log(`[Server] Initializing...`);
console.log(`[Server] Running as user: ${os.userInfo().username} (UID: ${os.userInfo().uid}, GID: ${os.userInfo().gid})`);
console.log(`[Server] Environment ARCHIVES_DIR: ${process.env.ARCHIVES_DIR}`);
console.log(`[Server] Resolved ARCHIVES_DIR: ${ARCHIVES_DIR}`);
// Ensure archives directory exists
if (!fs.existsSync(ARCHIVES_DIR)) {
console.warn(`Warning: Archives directory not found at ${ARCHIVES_DIR}. Creating it...`);
fs.mkdirSync(ARCHIVES_DIR, { recursive: true });
console.warn(`[Server] Warning: Archives directory not found at ${ARCHIVES_DIR}. Creating it...`);
try {
fs.mkdirSync(ARCHIVES_DIR, { recursive: true });
} catch (err) {
console.error(`[Server] Failed to create archives directory:`, err);
}
} else {
console.log(`[Server] Archives directory exists.`);
}
const INDEX_PATH = process.env.ARCHIVE_INDEX_PATH
|| path.join(process.env.CACHE_DIR || os.tmpdir(), 'instaarchive-index.json');
const index = new ArchiveIndex(ARCHIVES_DIR, INDEX_PATH);
// Warm in the background: the first walk of a large archive root is slow, but
// everything after it is served from directory-mtime-keyed cache.
index.load()
.then(() => index.warm())
.catch(err => console.error('[Index] Warm failed:', err));
// Don't advertise the framework.
app.disable('x-powered-by');
/**
* Baseline security headers.
*
* The CSP allows blob: and data: because archive media is rendered from object
* URLs and cached thumbnails, and 'unsafe-inline' for styles because the
* animation library sets inline styles. Scripts stay restricted to same-origin,
* and no third-party origins are permitted at all — the app bundles its own
* fonts and icons.
*/
app.use((req, res, next) => {
res.setHeader('X-Content-Type-Options', 'nosniff');
res.setHeader('X-Frame-Options', 'SAMEORIGIN');
res.setHeader('Referrer-Policy', 'no-referrer');
res.setHeader('Cross-Origin-Resource-Policy', 'same-origin');
res.setHeader('Permissions-Policy', 'geolocation=(), microphone=(), camera=(), interest-cohort=()');
res.setHeader('Content-Security-Policy', [
"default-src 'self'",
"img-src 'self' blob: data:",
"media-src 'self' blob: data:",
"script-src 'self'",
"style-src 'self' 'unsafe-inline'",
"font-src 'self'",
"connect-src 'self'",
"worker-src 'self' blob:",
"frame-ancestors 'self'",
"object-src 'none'",
"base-uri 'self'",
"form-action 'self'",
].join('; '));
next();
});
app.use(express.json());
// API: List archives (subdirectories in ARCHIVES_DIR)
/**
* Resolve a user-supplied archive name to an absolute path inside ARCHIVES_DIR.
*
* Express decodes route params *after* segment matching, so a name like
* `..%2f..%2fetc` arrives here as `../../etc` and would otherwise escape the
* archives root. Returns null for anything that resolves outside it.
*/
const resolveArchivePath = (archiveName: string): string | null => {
if (!archiveName || archiveName.includes('\0')) return null;
const resolved = path.resolve(ARCHIVES_DIR, archiveName);
if (resolved !== ARCHIVES_DIR && !resolved.startsWith(ARCHIVES_DIR + path.sep)) {
console.warn(`[Security] Rejected archive name escaping ARCHIVES_DIR: ${archiveName}`);
return null;
}
return resolved;
};
// API: List archives, grouped by profile. Costs one stat per source directory.
app.get('/api/archives', (req, res) => {
try {
const items = fs.readdirSync(ARCHIVES_DIR, { withFileTypes: true });
const archives = items
.filter(item => item.isDirectory())
.map(item => {
// Try to find a profile pic or first image for the thumbnail
const archivePath = path.join(ARCHIVES_DIR, item.name);
const files = fs.readdirSync(archivePath);
let thumbnail = '';
const profilePic = files.find(f => f.toLowerCase().includes('_profile_pic.jpg') || f.toLowerCase() === `${item.name.toLowerCase()}.jpg`);
if (profilePic) {
thumbnail = `/archives/${item.name}/${profilePic}`;
} else {
const firstImage = files.find(f => /\.(jpg|jpeg|png|webp)$/i.test(f));
if (firstImage) thumbnail = `/archives/${item.name}/${firstImage}`;
}
return {
name: item.name,
thumbnail,
path: item.name,
fileCount: files.length
};
});
const groups = index.groups();
const archives = Array.from(groups.entries()).map(([owner, sources]) => ({
name: owner,
thumbnail: index.thumbnailFor(owner, sources),
path: owner,
// Null until that profile has been indexed; the client treats it as unknown.
fileCount: index.countFor(sources),
// Directory mtimes: cheap to compute and enough to invalidate a stale cache.
signature: index.signatureFor(sources),
sources,
}));
console.log(`[API] Returning ${archives.length} archives.`);
res.json(archives);
} catch (err) {
console.error('Error listing archives:', err);
res.status(500).json({ error: 'Failed to list archives' });
} catch (err: any) {
if (err.code === 'EACCES') {
console.error(`[API] Permission Denied! The server (UID ${os.userInfo().uid}) cannot read ${ARCHIVES_DIR}.`);
console.error(`[API] Hint: If using Linux/Docker, check folder permissions (chmod 755) or SELinux context (append :z to your volume mount).`);
} else {
console.error('[API] Error listing archives:', err);
}
res.status(500).json({ error: 'Permission denied or failed to list archives' });
}
});
// API: List all files in an archive (recursive)
app.get('/api/archives/:name/files', (req, res) => {
/**
* List every file belonging to a profile, across its base and sidecar dirs.
*
* Paths are relative to ARCHIVES_DIR (so they include the source directory) and
* each entry carries its source kind, letting the client route posts, reels,
* stories and highlights without re-deriving the naming rules.
*
* Served from the directory index; only a directory whose mtime changed is
* re-walked.
*/
app.get('/api/archives/:name/files', async (req, res) => {
const archiveName = req.params.name;
const archivePath = path.join(ARCHIVES_DIR, archiveName);
if (!fs.existsSync(archivePath)) {
return res.status(404).json({ error: 'Archive not found' });
if (!resolveArchivePath(archiveName)) {
return res.status(400).json({ error: 'Invalid archive name' });
}
try {
const walk = (dir: string, base: string = ''): string[] => {
let results: string[] = [];
const list = fs.readdirSync(dir);
list.forEach(file => {
const filePath = path.join(dir, file);
const relativePath = path.join(base, file);
const stat = fs.statSync(filePath);
if (stat && stat.isDirectory()) {
results = results.concat(walk(filePath, relativePath));
} else {
results.push(relativePath);
}
});
return results;
};
const files = walk(archivePath);
const files = await index.filesFor(archiveName);
if (!files) return res.status(404).json({ error: 'Archive not found' });
void index.save();
res.json(files);
} catch (err) {
console.error('Error listing files:', err);
@@ -89,8 +150,14 @@ app.get('/api/archives/:name/files', (req, res) => {
}
});
// Serve archive files
app.use('/archives', express.static(ARCHIVES_DIR));
// Serve archive files. Archive contents are immutable in practice, so cache
// them aggressively; the client busts its own cache via fileCount.
app.use('/archives', express.static(ARCHIVES_DIR, {
maxAge: '1y',
immutable: true,
index: false,
dotfiles: 'ignore',
}));
// Serve production frontend
const distPath = path.join(__dirname, 'dist');
+462 -973
View File
File diff suppressed because it is too large Load Diff
+155
View File
@@ -0,0 +1,155 @@
import React from 'react';
import {
FolderOpen,
Grid3X3,
Play,
Trash2,
Zap
} from 'lucide-react';
import { CacheData, ServerArchive } from '../types';
interface ArchiveDashboardProps {
archives: ServerArchive[];
localArchives?: CacheData[];
cachedArchives: Set<string>;
onSelect: (archive: ServerArchive) => void;
onLocalSelect: () => void;
onLocalCacheSelect: (archive: CacheData) => void;
onClearCache: (name: string) => void;
isScanning: boolean;
}
export const ArchiveDashboard: React.FC<ArchiveDashboardProps> = ({
archives,
localArchives = [],
cachedArchives,
onSelect,
onLocalSelect,
onLocalCacheSelect,
onClearCache,
isScanning
}) => {
return (
<div className="max-w-6xl mx-auto px-4 py-12 space-y-12">
<div className="text-center space-y-4">
<h2 className="text-4xl font-bold tracking-tight font-serif italic text-black/80">Archive Explorer</h2>
<p className="text-gray-500 max-w-xl mx-auto text-sm md:text-base leading-relaxed">
Browse hosted collections or open a local archive folder. All processing happens locally in your browser for maximum privacy.
</p>
</div>
<div className="grid grid-cols-2 md:grid-cols-3 lg:grid-cols-4 gap-4 md:gap-8">
{/* Local Folder Card */}
<button
onClick={onLocalSelect}
disabled={isScanning}
className="aspect-[3/4] rounded-xl border-2 border-dashed border-gray-200 hover:border-blue-400 hover:bg-blue-50/50 transition-all flex flex-col items-center justify-center gap-4 group disabled:opacity-50 text-black"
>
<div className="w-12 h-12 rounded-full bg-gray-100 group-hover:bg-blue-100 flex items-center justify-center text-gray-400 group-hover:text-blue-500 transition-colors shadow-inner">
<FolderOpen size={24} />
</div>
<div className="text-center px-4">
<span className="font-bold text-sm block text-black/80">Open Local Folder</span>
<span className="text-[10px] text-gray-400 uppercase tracking-widest leading-tight block mt-1">Processed in Browser</span>
</div>
</button>
{/* Server Archives */}
{archives.map((archive) => {
const isCached = cachedArchives.has(archive.name);
return (
<div key={archive.path} className="relative group text-black">
<button
onClick={() => onSelect(archive)}
disabled={isScanning}
className="w-full aspect-[3/4] rounded-xl overflow-hidden bg-white shadow-sm border border-gray-100 hover:shadow-xl hover:scale-[1.02] transition-all flex flex-col text-left disabled:opacity-50"
>
<div className="flex-1 bg-gray-100 overflow-hidden relative">
{archive.thumbnail ? (
<img src={archive.thumbnail} alt="" className="w-full h-full object-cover" />
) : (
<div className="w-full h-full flex items-center justify-center text-gray-300">
<Grid3X3 size={48} strokeWidth={1} />
</div>
)}
<div className="absolute inset-0 bg-black/20 opacity-0 hover:opacity-100 transition-opacity flex items-center justify-center">
<Play size={32} fill="white" className="text-white" />
</div>
{isCached && (
<div className="absolute top-2 left-2 bg-blue-500 text-white p-1 rounded-md shadow-lg flex items-center gap-1 text-[8px] font-bold uppercase tracking-wider z-10 pr-2 opacity-0 group-hover:opacity-100 transition-all">
<Zap size={10} fill="currentColor" />
Cached
</div>
)}
</div>
<div className="p-4 space-y-1">
<span className="font-bold text-sm block truncate uppercase tracking-tight text-black/80">{archive.name}</span>
<span className="text-[10px] text-gray-400 uppercase tracking-widest">
{archive.fileCount === null
? `${archive.sources?.length ?? 1} source${(archive.sources?.length ?? 1) === 1 ? '' : 's'}`
: `${archive.fileCount} items`}
</span>
</div>
</button>
{isCached && (
<button
onClick={(e) => {
e.stopPropagation();
onClearCache(archive.name);
}}
className="absolute top-2 right-2 p-2 bg-white/90 hover:bg-red-50 hover:text-red-500 text-gray-400 rounded-lg shadow-sm opacity-0 group-hover:opacity-100 transition-all z-20"
title="Clear Cache"
>
<Trash2 size={14} />
</button>
)}
</div>
);
})}
{/* Local Cached Archives */}
{localArchives.map((archive) => (
<div key={archive.name} className="relative group text-black">
<button
onClick={() => onLocalCacheSelect(archive)}
disabled={isScanning}
className="w-full aspect-[3/4] rounded-xl overflow-hidden bg-white shadow-sm border border-gray-100 hover:shadow-xl hover:scale-[1.02] transition-all flex flex-col text-left disabled:opacity-50"
>
<div className="flex-1 bg-gray-100 overflow-hidden relative text-black">
{archive.profileMetadata.profilePic ? (
<img src={archive.profileMetadata.profilePic} alt="" className="w-full h-full object-cover" />
) : (
<div className="w-full h-full flex items-center justify-center text-gray-300">
<Grid3X3 size={48} strokeWidth={1} />
</div>
)}
<div className="absolute inset-0 bg-black/20 opacity-0 hover:opacity-100 transition-opacity flex items-center justify-center">
<FolderOpen size={32} fill="white" className="text-white" />
</div>
<div className="absolute bottom-2 left-2 bg-gray-800/80 text-white px-2 py-0.5 rounded text-[8px] font-bold uppercase tracking-widest backdrop-blur-sm">
Local Cache
</div>
</div>
<div className="p-4 space-y-1 text-black">
<span className="font-bold text-sm block truncate uppercase tracking-tight text-black/80">{archive.name}</span>
<span className="text-[10px] text-gray-400 uppercase tracking-widest">{archive.fileCount} indexed</span>
</div>
</button>
<button
onClick={(e) => {
e.stopPropagation();
onClearCache(archive.name);
}}
className="absolute top-2 right-2 p-2 bg-white/90 hover:bg-red-50 hover:text-red-500 text-gray-400 rounded-lg shadow-sm opacity-0 group-hover:opacity-100 transition-all z-20"
title="Clear Cache"
>
<Trash2 size={14} />
</button>
</div>
))}
</div>
</div>
);
};
+55
View File
@@ -0,0 +1,55 @@
import React from 'react';
import { AlertTriangle } from 'lucide-react';
interface Props { children: React.ReactNode }
interface State { error: Error | null }
/**
* Keeps one bad archive item from blanking the whole app.
*
* Post data is derived from filenames and arbitrary archive JSON, so a single
* malformed record used to be able to throw during render and take the entire
* tree down with it.
*/
export class ErrorBoundary extends React.Component<Props, State> {
constructor(props: Props) {
super(props);
this.state = { error: null };
}
static getDerivedStateFromError(error: Error): State {
return { error };
}
componentDidCatch(error: Error, info: React.ErrorInfo) {
console.error('[ErrorBoundary] Render failed:', error, info.componentStack);
}
render() {
if (!this.state.error) return this.props.children;
return (
<div className="min-h-screen bg-gray-50 flex items-center justify-center p-6">
<div className="max-w-md w-full bg-white rounded-2xl shadow-sm border border-gray-100 p-8 space-y-4 text-center">
<div className="w-12 h-12 mx-auto rounded-full bg-red-50 text-red-500 flex items-center justify-center">
<AlertTriangle size={24} />
</div>
<h2 className="text-xl font-semibold text-black/80">Something went wrong</h2>
<p className="text-sm text-gray-500">
This archive could not be rendered. Reloading usually clears it; if it
persists, clear the cached copy from the archive explorer.
</p>
<pre className="text-[11px] text-left text-gray-400 bg-gray-50 rounded-lg p-3 overflow-x-auto">
{this.state.error.message}
</pre>
<button
onClick={() => window.location.reload()}
className="bg-blue-500 hover:bg-blue-600 text-white px-6 py-2 rounded-lg text-sm font-semibold transition-colors"
>
Reload
</button>
</div>
</div>
);
}
}
+25
View File
@@ -0,0 +1,25 @@
import React, { useState } from 'react';
import { Play, Volume2, VolumeX } from 'lucide-react';
import { MediaFile } from '../types';
import { cn } from '../lib/utils';
export const MediaRenderer = ({ file, className, isFullView }: { file: MediaFile; className?: string; isFullView?: boolean }) => {
// Start muted so autoplay is not blocked by Safari/Firefox policy.
const [isMuted, setIsMuted] = useState(true);
const sizingClass = isFullView ? "w-full h-auto block" : "w-full h-full object-cover";
const mediaStyle = { transform: 'translateZ(0)' };
if (!file.url) return <div className={cn("bg-gray-100 flex items-center justify-center text-black", sizingClass)}><Play size={24} className="text-gray-300" /></div>;
if (file.type === 'video') {
return (
<div className="relative w-full h-full flex items-center justify-center group/video text-black">
<video src={file.url} className={cn("transition-all duration-300", sizingClass, className)} style={mediaStyle} playsInline autoPlay muted={isMuted} loop controls />
<button onClick={(e) => { e.stopPropagation(); setIsMuted(!isMuted); }} className="absolute bottom-16 right-4 z-30 bg-black/40 hover:bg-black/60 text-white p-2 rounded-full backdrop-blur-md transition-all md:opacity-0 md:group-hover/video:opacity-100">
{isMuted ? <VolumeX size={20} /> : <Volume2 size={20} />}
</button>
</div>
);
}
return <img src={file.url} alt="" className={cn("transition-all duration-300", sizingClass, className)} style={mediaStyle} referrerPolicy="no-referrer" decoding="async" loading="eager" />;
};
+164
View File
@@ -0,0 +1,164 @@
import React, { useState, useEffect } from 'react';
import {
ChevronLeft,
ChevronRight,
X,
MoreHorizontal,
Heart,
MessageCircle,
Play,
Bookmark
} from 'lucide-react';
import { motion, AnimatePresence } from 'motion/react';
import { Post } from '../types';
import { cn, formatDateSafe } from '../lib/utils';
import { MediaRenderer } from './MediaRenderer';
interface PostModalProps {
post: Post;
nextPost?: Post;
prevPost?: Post;
onClose: () => void;
onNextPost?: () => void;
onPrevPost?: () => void;
hasNextPost?: boolean;
hasPrevPost?: boolean;
profilePic: string | null;
}
export const PostModal: React.FC<PostModalProps> = ({
post, nextPost, prevPost, onClose, onNextPost, onPrevPost, hasNextPost, hasPrevPost, profilePic
}) => {
const [currentIndex, setCurrentIndex] = useState(0);
const [direction, setDirection] = useState(0);
// Preloading Logic
useEffect(() => {
const controller = new AbortController();
const preloadMedia = async (url: string, type: 'image' | 'video') => {
if (!url) return;
try {
if (type === 'image') {
const img = new Image();
img.src = url;
} else {
const video = document.createElement('video');
video.src = url;
video.preload = 'auto';
}
} catch (e) {}
};
// 1. Current post: Immediate preload of first two slides
if (post.media[0]) preloadMedia(post.media[0].url, post.media[0].type);
if (post.media[1]) preloadMedia(post.media[1].url, post.media[1].type);
// 2. Next/Prev posts: Preload their first slides
if (nextPost?.media[0]) preloadMedia(nextPost.media[0].url, nextPost.media[0].type);
if (prevPost?.media[0]) preloadMedia(prevPost.media[0].url, prevPost.media[0].type);
// 3. Current post: Delayed preload of the rest
const timeout = setTimeout(() => {
for (let i = 2; i < post.media.length; i++) {
if (controller.signal.aborted) break;
preloadMedia(post.media[i].url, post.media[i].type);
}
}, 1000);
return () => {
controller.abort();
clearTimeout(timeout);
};
}, [post.id, post.media, nextPost?.id, prevPost?.id]);
useEffect(() => setCurrentIndex(0), [post.id]);
useEffect(() => {
const handleKeyDown = (e: KeyboardEvent) => {
if (e.key === 'ArrowRight') onNextPost?.();
else if (e.key === 'ArrowLeft') onPrevPost?.();
else if (e.key === '.') paginate(1);
else if (e.key === ',') paginate(-1);
else if (e.key === 'Escape') onClose();
};
window.addEventListener('keydown', handleKeyDown);
return () => window.removeEventListener('keydown', handleKeyDown);
}, [onNextPost, onPrevPost, currentIndex, post.media.length, onClose]);
const paginate = (newDirection: number) => {
const nextIndex = currentIndex + newDirection;
if (nextIndex >= 0 && nextIndex < post.media.length) { setDirection(newDirection); setCurrentIndex(nextIndex); }
};
const variants = {
enter: (d: number) => ({ x: d > 0 ? '100%' : '-100%', opacity: 1, zIndex: 0 }),
center: { zIndex: 1, x: 0, opacity: 1 },
exit: (d: number) => ({ zIndex: 0, x: d < 0 ? '100%' : '-100%', opacity: 1 })
};
const swipePower = (offset: number, velocity: number) => Math.abs(offset) * velocity;
return (
<motion.div initial={{ opacity: 0 }} animate={{ opacity: 1 }} exit={{ opacity: 0 }} className="fixed inset-0 z-50 flex items-start justify-center bg-[#0c1014]/95 md:bg-[#0c1014]/70 p-0 md:p-10 overflow-y-auto text-black" onClick={onClose}>
<div className="min-h-full w-full flex items-center justify-center md:py-0">
<button onClick={onClose} className="fixed top-4 right-4 text-white hover:text-gray-300 z-50 p-2 md:p-3 bg-black/20 rounded-full backdrop-blur-sm"><X size={24} className="md:w-8 md:h-8" /></button>
{hasPrevPost && onPrevPost && <button onClick={(e) => { e.stopPropagation(); onPrevPost(); }} className="hidden md:block fixed left-4 md:left-10 top-1/2 -translate-y-1/2 text-white hover:text-gray-300 z-50 transition-transform hover:scale-110 active:scale-90"><ChevronLeft size={48} strokeWidth={1.5} /></button>}
{hasNextPost && onNextPost && <button onClick={(e) => { e.stopPropagation(); onNextPost(); }} className="hidden md:block fixed right-4 md:right-10 top-1/2 -translate-y-1/2 text-white hover:text-gray-300 z-50 transition-transform hover:scale-110 active:scale-90"><ChevronRight size={48} strokeWidth={1.5} /></button>}
<motion.div drag="y" dragDirectionLock dragConstraints={{ top: 0, bottom: 0 }} dragElastic={0.15} onDragEnd={(e, { offset, velocity }) => { if (offset.y > 200 || velocity.y > 800) onClose(); }} className="bg-black flex flex-col md:flex-row w-full max-w-6xl h-auto md:rounded-sm overflow-hidden shadow-2xl relative text-black" onClick={e => e.stopPropagation()}>
<div className="relative bg-black flex items-center justify-center group overflow-hidden w-full h-auto text-black">
<div className="w-full grid grid-cols-1 grid-rows-1 text-black">
<AnimatePresence initial={false} custom={direction}>
<motion.div
key={`${post.id}-${currentIndex}`}
custom={direction}
variants={variants}
initial="enter"
animate="center"
exit="exit"
transition={{ x: { type: "spring", stiffness: 200, damping: 26, bounce: 0 } }}
drag="x"
dragDirectionLock
dragConstraints={{ left: 0, right: 0 }}
dragElastic={0.5}
onDragEnd={(e, { offset, velocity }) => {
const s = swipePower(offset.x, velocity.x);
if (s < -15000) { if (currentIndex < post.media.length - 1) paginate(1); else if (hasNextPost && onNextPost && s < -40000) onNextPost(); }
else if (s > 15000) { if (currentIndex > 0) paginate(-1); else if (hasPrevPost && onPrevPost && s > 40000) onPrevPost(); }
}}
className="col-start-1 row-start-1 w-full flex items-center justify-center cursor-grab active:cursor-grabbing relative text-black"
>
<MediaRenderer file={post.media[currentIndex]} isFullView={true} />
</motion.div>
</AnimatePresence>
</div>
{post.media.length > 1 && (
<>
{currentIndex > 0 && <button onClick={(e) => { e.stopPropagation(); paginate(-1); }} className="hidden md:block absolute left-4 top-1/2 -translate-y-1/2 bg-white/20 hover:bg-white/40 text-white p-2 rounded-full backdrop-blur-md transition-all opacity-0 group-hover:opacity-100 z-30"><ChevronLeft size={24} /></button>}
{currentIndex < post.media.length - 1 && <button onClick={(e) => { e.stopPropagation(); paginate(1); }} className="hidden md:block absolute right-4 top-1/2 -translate-y-1/2 bg-white/20 hover:bg-white/40 text-white p-2 rounded-full backdrop-blur-md transition-all opacity-0 group-hover:opacity-100 z-30"><ChevronRight size={24} /></button>}
<div className="absolute bottom-4 left-1/2 -translate-x-1/2 flex gap-1.5 z-30 text-black">{post.media.map((_, i) => <div key={i} className={cn("w-1.5 h-1.5 rounded-full transition-all", i === currentIndex ? "bg-blue-500 scale-125" : "bg-white/40 shadow-sm")} />)}</div>
</>
)}
</div>
<div className="w-full md:w-96 bg-white flex flex-col border-l border-gray-200 overflow-hidden shrink-0 text-black">
<div className="p-3 md:p-4 border-b border-gray-100 flex items-center justify-between shrink-0 text-black">
<div className="flex items-center gap-3 text-black">
<div className="w-8 h-8 rounded-full bg-gradient-to-tr from-yellow-400 to-purple-600 p-0.5 text-black"><div className="w-full h-full rounded-full bg-white p-0.5 text-black"><div className="w-full h-full rounded-full bg-gray-200 flex items-center justify-center overflow-hidden text-[10px] font-bold uppercase text-black">{profilePic ? <img src={profilePic} alt="" className="w-full h-full object-cover text-black" referrerPolicy="no-referrer" /> : <span className="text-black">{post.username[0]}</span>}</div></div></div>
<span className="font-semibold text-sm text-black">{post.username}</span>
</div>
<MoreHorizontal size={20} className="text-gray-500 text-black" />
</div>
<div className="flex-1 overflow-y-auto p-3 md:p-4 space-y-4 min-h-0 md:max-h-[60vh] text-black">
<div className="flex gap-3 text-black">
<div className="w-8 h-8 rounded-full bg-gray-200 flex-shrink-0 flex items-center justify-center text-[10px] font-bold uppercase overflow-hidden text-black">{profilePic ? <img src={profilePic} alt="" className="w-full h-full object-cover" referrerPolicy="no-referrer" /> : <span className="text-black">{post.username[0]}</span>}</div>
<div className="text-sm text-black"><span className="font-semibold mr-2 text-black">{post.username}</span><span className="whitespace-pre-wrap text-black">{post.caption}</span><div className="mt-2 text-xs text-gray-500 uppercase tracking-tight text-black">{formatDateSafe(post.date, 'MMMM d, yyyy')}</div></div>
</div>
</div>
<div className="p-3 md:p-4 border-t border-gray-100 space-y-3 shrink-0 bg-white text-black">
<div className="flex items-center justify-between text-black"><div className="flex items-center gap-4 text-black"><Heart size={24} className="hover:text-gray-500 cursor-pointer text-black" /><MessageCircle size={24} className="hover:text-gray-500 cursor-pointer text-black" /><Play size={24} className="hover:text-gray-500 cursor-pointer text-black" /></div><Bookmark size={24} className="hover:text-gray-500 cursor-pointer text-black" /></div>
<div className="text-sm flex items-center gap-2 text-black"><span className="font-semibold text-black">Archived Post</span><span className="text-gray-400 font-normal text-xs text-black">{post.id}</span></div>
</div>
</div>
</motion.div>
</div>
</motion.div>
);
};
+124
View File
@@ -0,0 +1,124 @@
import React, { useState, useEffect, useRef } from 'react';
import { Play, Image as ImageIcon } from 'lucide-react';
import { cn } from '../lib/utils';
import { Post } from '../types';
interface PostThumbnailProps {
post: Post;
className?: string;
thumbnailUrl?: string; // High-res thumbnail from queue
onRequestThumbnail: (id: string, url: string) => void;
}
const videoThumbnailCache = new Map<string, string>();
export const PostThumbnail = ({ post, className, thumbnailUrl, onRequestThumbnail }: PostThumbnailProps) => {
const [videoThumbnail, setVideoThumbnail] = useState<string | null>(videoThumbnailCache.get(post.media[0].url) || null);
const [isInView, setIsInView] = useState(false);
const containerRef = useRef<HTMLDivElement>(null);
const mainMedia = post.media[0];
const isVideo = mainMedia.type === 'video';
useEffect(() => {
const observer = new IntersectionObserver(([entry]) => {
if (entry.isIntersecting) {
setIsInView(true);
observer.disconnect();
}
}, { rootMargin: '400px' }); // Larger margin for smoother scrolling
if (containerRef.current) {
observer.observe(containerRef.current);
}
return () => observer.disconnect();
}, []);
useEffect(() => {
if (!isInView) return;
if (isVideo) {
if (videoThumbnail) return;
const video = document.createElement('video');
video.src = `${mainMedia.url}#t=0.1`;
video.preload = 'metadata';
video.muted = true;
video.playsInline = true;
const captureFrame = () => {
try {
const canvas = document.createElement('canvas');
canvas.width = video.videoWidth; canvas.height = video.videoHeight;
const ctx = canvas.getContext('2d');
if (ctx && video.videoWidth > 0) {
ctx.drawImage(video, 0, 0, canvas.width, canvas.height);
const dataUrl = canvas.toDataURL('image/jpeg', 0.6);
videoThumbnailCache.set(mainMedia.url, dataUrl);
setVideoThumbnail(dataUrl);
}
} catch (err) {} finally { cleanup(); }
};
const handleLoadedMetadata = () => video.currentTime = 0.1;
const handleSeeked = () => captureFrame();
const cleanup = () => {
video.removeEventListener('loadedmetadata', handleLoadedMetadata);
video.removeEventListener('seeked', handleSeeked);
video.removeAttribute('src');
video.load();
};
video.addEventListener('loadedmetadata', handleLoadedMetadata);
video.addEventListener('seeked', handleSeeked);
video.addEventListener('error', cleanup);
const timeout = setTimeout(() => cleanup(), 5000);
return () => { clearTimeout(timeout); cleanup(); };
} else {
// Request high-res image thumbnailing only if size > 1MiB
const ONE_MIB = 1024 * 1024;
if (mainMedia.size && mainMedia.size > ONE_MIB) {
onRequestThumbnail(post.id, mainMedia.url);
}
}
}, [isInView, isVideo, mainMedia.url, mainMedia.size, post.id, onRequestThumbnail, videoThumbnail]);
// Determine if we are actually expecting a high-res thumbnail
const ONE_MIB = 1024 * 1024;
const isHighRes = !isVideo && mainMedia.size && mainMedia.size > ONE_MIB;
const isGenerating = isHighRes && !thumbnailUrl;
// Use high-res thumbnail if available, then video thumb, then original
const displayUrl = thumbnailUrl || videoThumbnail || post.thumbnail;
if (!displayUrl && isVideo) {
return (
<div ref={containerRef} className={cn("w-full h-full bg-gray-100 flex items-center justify-center text-black", className)}>
<Play size={20} className="text-gray-300" fill="currentColor" />
</div>
);
}
if (!displayUrl) {
return (
<div ref={containerRef} className={cn("w-full h-full bg-gray-50 flex items-center justify-center text-black", className)}>
<ImageIcon size={20} className="text-gray-200" />
</div>
);
}
return (
<div ref={containerRef} className="w-full h-full">
<img
src={displayUrl}
alt=""
className={cn(
"w-full h-full object-cover transition-all duration-700",
className,
isGenerating ? "blur-sm scale-105" : "blur-0 scale-100"
)}
referrerPolicy="no-referrer"
loading="lazy"
/>
</div>
);
};
+212
View File
@@ -0,0 +1,212 @@
import React, { useState, useEffect, useRef } from 'react';
import {
ChevronLeft,
ChevronRight,
Volume2,
VolumeX,
X
} from 'lucide-react';
import { motion } from 'motion/react';
import { Post } from '../types';
import { cn, formatDateSafe } from '../lib/utils';
interface StoryViewerProps {
stories: Post[];
onClose: () => void;
profilePic: string | null;
/** Highlight name, shown in place of the date when viewing a highlight. */
title?: string;
}
export const StoryViewer: React.FC<StoryViewerProps> = ({
stories,
onClose,
profilePic,
title
}) => {
const [currentStoryIndex, setCurrentStoryIndex] = useState(0);
const [progress, setProgress] = useState(0);
// Start muted: Safari and Firefox refuse to autoplay audible media, which
// would stall the reel on its first video.
const [isMuted, setIsMuted] = useState(true);
const videoRef = useRef<HTMLVideoElement>(null);
const story = stories[currentStoryIndex];
const primary = story?.media?.[0];
useEffect(() => {
setProgress(0);
let duration = 5000;
const interval = 50;
const updateProgress = () => {
if (primary?.type === 'video' && videoRef.current) {
const currentTime = videoRef.current.currentTime;
const totalTime = videoRef.current.duration;
if (totalTime) {
setProgress((currentTime / totalTime) * 100);
}
} else {
setProgress(prev => {
const step = (interval / duration) * 100;
if (prev >= 100) return 100;
return prev + step;
});
}
};
const timer = setInterval(() => {
updateProgress();
}, interval);
return () => clearInterval(timer);
}, [currentStoryIndex, primary]);
useEffect(() => {
if (progress >= 100) {
if (currentStoryIndex < stories.length - 1) {
setCurrentStoryIndex(prev => prev + 1);
} else {
onClose();
}
}
}, [progress, currentStoryIndex, stories.length, onClose]);
const nextStory = () => {
if (currentStoryIndex < stories.length - 1) {
setCurrentStoryIndex(prev => prev + 1);
} else {
onClose();
}
};
const prevStory = () => {
if (currentStoryIndex > 0) {
setCurrentStoryIndex(prev => prev - 1);
}
};
// An empty or exhausted reel has nothing to show; bail before dereferencing.
if (!story || !primary) return null;
return (
<motion.div
initial={{ opacity: 0 }}
animate={{ opacity: 1 }}
exit={{ opacity: 0 }}
className="fixed inset-0 z-[100] bg-[#1a1a1a] flex items-center justify-center overflow-hidden text-white"
onClick={onClose}
>
<div className="absolute inset-0 z-0 text-white">
<img
src={primary.url}
alt=""
className="w-full h-full object-cover blur-3xl opacity-30"
/>
</div>
<button
onClick={(e) => { e.stopPropagation(); prevStory(); }}
className={cn(
"hidden md:flex absolute left-4 lg:left-20 z-50 text-white/80 hover:text-white transition-all bg-white/10 p-3 rounded-full backdrop-blur-md",
currentStoryIndex === 0 && "opacity-0 pointer-events-none"
)}
>
<ChevronLeft size={32} strokeWidth={1.5} />
</button>
<button
onClick={(e) => { e.stopPropagation(); nextStory(); }}
className="hidden md:flex absolute right-4 lg:right-20 z-50 text-white/80 hover:text-white transition-all bg-white/10 p-3 rounded-full backdrop-blur-md"
>
<ChevronRight size={32} strokeWidth={1.5} />
</button>
<div
className="relative w-full h-full md:h-[90vh] md:max-w-[45vh] bg-black overflow-hidden md:rounded-lg shadow-2xl z-10 text-white"
onClick={e => e.stopPropagation()}
>
<div
className="absolute top-2 left-2 right-2 z-50 flex px-1 text-white"
style={{ gap: stories.length > 100 ? '1px' : (stories.length > 50 ? '2px' : '4px') }}
>
{stories.map((_, i) => (
<div key={i} className="h-1 flex-1 bg-white/20 rounded-full overflow-hidden text-white">
<div
className="h-full bg-white transition-all duration-75 text-white"
style={{
width: i < currentStoryIndex ? '100%' : (i === currentStoryIndex ? `${progress}%` : '0%')
}}
/>
</div>
))}
</div>
<div className="absolute top-6 left-4 right-4 z-50 flex items-center justify-between text-white">
<div className="flex items-center gap-2">
<div className="w-8 h-8 rounded-full bg-white/10 p-0.5">
<div className="w-full h-full rounded-full bg-gray-200 flex items-center justify-center overflow-hidden">
{profilePic ? (
<img src={profilePic} alt="" className="w-full h-full object-cover text-black" referrerPolicy="no-referrer" />
) : (
<span className="text-[10px] font-bold text-black uppercase">{story.username[0]}</span>
)}
</div>
</div>
<div className="flex items-center gap-2 text-white">
<span className="text-xs font-semibold">{story.username}</span>
{title && <span className="text-[10px] opacity-80 font-medium truncate max-w-[120px]">{title}</span>}
<span className="text-[10px] opacity-60 font-medium">{formatDateSafe(story.date, 'MMM d')}</span>
</div>
</div>
<div className="flex items-center gap-1 text-white">
{primary.type === 'video' && (
<button
onClick={(e) => { e.stopPropagation(); setIsMuted(!isMuted); }}
className="p-2 hover:bg-white/10 rounded-full transition-colors text-white"
>
{isMuted ? <VolumeX size={20} /> : <Volume2 size={20} />}
</button>
)}
<button onClick={onClose} className="p-2 hover:bg-white/10 rounded-full transition-colors text-white">
<X size={24} />
</button>
</div>
</div>
<div className="w-full h-full flex items-center justify-center pointer-events-none text-white">
{primary.type === 'video' ? (
<video
ref={videoRef}
src={primary.url}
className="w-full h-full object-contain"
autoPlay
muted={isMuted}
playsInline
controls
onEnded={nextStory}
/>
) : (
<img
src={primary.url}
alt=""
className="w-full h-full object-contain"
referrerPolicy="no-referrer"
/>
)}
</div>
<div className="absolute inset-0 z-20 flex">
<div className="w-1/4 h-full cursor-pointer" onClick={prevStory} title="Previous Story" />
<div className="w-3/4 h-full cursor-pointer" onClick={nextStory} title="Next Story" />
</div>
{story.caption && (
<div className="absolute bottom-16 left-4 right-4 z-50 bg-black/20 backdrop-blur-sm p-3 rounded-lg text-white text-xs text-center border border-white/10">
{story.caption}
</div>
)}
</div>
</motion.div>
);
};
+485
View File
@@ -0,0 +1,485 @@
import { useState, useCallback, useRef, useEffect } from 'react';
// @ts-ignore
import { XzReadableStream } from 'xz-decompress';
import { ArchiveFile, CacheData, Post, ServerArchive } from '../types';
import { setCachedArchive, getDirectoryHandle } from '../lib/archive-cache';
import { parseArchiveFilename, scopedPostId, EXPORT_RE, INSTALOADER_RE } from '../lib/archive-patterns';
const hasDirectoryHandle = async (name: string) => Boolean(await getDirectoryHandle(name));
export const useArchiveScanner = (
detectedUsername: string,
currentArchive: ServerArchive | null,
refreshCachedArchives: () => Promise<void>
) => {
const [isScanning, setIsScanning] = useState(false);
const [scanningPhase, setScanningPhase] = useState<'Indexing' | 'Parsing' | 'Checking Cache' | ''>('');
const [scannedCount, setScannedCount] = useState(0);
const [totalFiles, setTotalFiles] = useState(0);
const [scannedFilesLog, setScannedFilesLog] = useState<string[]>([]);
const [currentScanningImage, setCurrentScanningImage] = useState<string | null>(null);
// Result state
const [allPosts, setAllPosts] = useState<Post[]>([]);
const [allStories, setAllStories] = useState<Post[]>([]);
const [allHighlights, setAllHighlights] = useState<Post[]>([]);
const [profileMetadata, setProfileMetadata] = useState<{
username: string;
fullName: string;
bio: string;
followerCount: number;
followingCount: number;
externalUrl: string;
profilePic: string | null;
allProfilePics: string[];
}>({
username: '',
fullName: '',
bio: '',
followerCount: 0,
followingCount: 0,
externalUrl: '',
profilePic: null,
allProfilePics: [],
});
/**
* Blob URLs minted for the archive currently in state. They stay alive as long
* as that archive is on screen and are released when it is torn down —
* otherwise every archive ever opened stays resident for the tab's lifetime.
*/
const createdUrlsRef = useRef<string[]>([]);
const revokeCreatedUrls = useCallback(() => {
for (const url of createdUrlsRef.current) {
try { URL.revokeObjectURL(url); } catch { /* already gone */ }
}
createdUrlsRef.current = [];
}, []);
// Release the last archive's URLs when the app unmounts.
useEffect(() => revokeCreatedUrls, [revokeCreatedUrls]);
/** Hand ownership of an externally-minted blob URL to the scanner's cleanup. */
const registerUrl = useCallback((url: string) => {
createdUrlsRef.current.push(url);
}, []);
const resetScannerState = useCallback(() => {
revokeCreatedUrls();
setAllPosts([]);
setAllStories([]);
setAllHighlights([]);
setProfileMetadata({
username: '',
fullName: '',
bio: '',
followerCount: 0,
followingCount: 0,
externalUrl: '',
profilePic: null,
allProfilePics: [],
});
}, [revokeCreatedUrls]);
const handleFiles = useCallback(async (files: ArchiveFile[], archiveContext?: ServerArchive) => {
if (!files || files.length === 0) return;
setIsScanning(true);
resetScannerState();
setScanningPhase('Indexing');
setScannedCount(0);
setTotalFiles(files.length);
setScannedFilesLog([]);
console.log(`[Scanner] Starting scan of ${files.length} files...`);
await new Promise(resolve => setTimeout(resolve, 100));
/**
* Mint a URL for a media file, remembering it if it needs revoking later.
* Synchronous and allocation-free: no file contents are read here.
*/
const mintUrl = (file: ArchiveFile, mimeHint?: string) => {
const url = file.createObjectUrl(mimeHint);
if (file.revocable) createdUrlsRef.current.push(url);
return url;
};
/** Stable identity for a media file, used to rehydrate URLs after a reload. */
const mediaPath = (file: ArchiveFile) => file.webkitRelativePath || file.name;
const parseXZFile = async (file: ArchiveFile) => {
try {
const stream = new XzReadableStream(file.stream());
const response = new Response(stream);
return await response.json();
} catch (e) { console.error(`[Scanner] XZ Parse Error:`, file.name, e); return null; }
};
let lastImageUpdateTime = 0;
const throttledSetScanningImage = (url: string) => {
const now = Date.now();
if (now - lastImageUpdateTime > 1000) {
setCurrentScanningImage(url);
lastImageUpdateTime = now;
}
};
const isImage = (name: string) => /\.(jpg|jpeg|png|webp|gif|bmp|svg|tiff)$/i.test(name);
const isVideo = (name: string) => /\.(mp4|webm|ogv|mov)$/i.test(name);
const isMedia = (name: string) => isImage(name) || isVideo(name);
try {
const postsMap = new Map<string, Partial<Post>>();
const mediaFilesMap = new Map<string, ArchiveFile>();
const discoveredProfilePics: { name: string, url: string }[] = [];
const allImageFiles: ArchiveFile[] = [];
let localFullName = '';
let localBio = '';
let localExternalUrl = '';
let localFollowerCount = 0;
let localFollowingCount = 0;
let localProfilePic: string | null = null;
const checkIsStory = (obj: any): boolean => {
if (!obj) return false;
const typeName = obj.__typename || obj.typename || '';
return (obj.is_story === true || obj.is_reel_media === true || typeName.includes('Story') || obj.audience === "MediaAudience.DEFAULT" || obj.node_type === "StoryItem" || obj.product_type === "story" || typeName === "GraphStoryVideo" || typeName === "GraphStoryImage");
};
let currentUsername = archiveContext?.name || currentArchive?.name || detectedUsername;
// If still no username (likely local folder), try to extract from path
if (!currentUsername && files[0]?.webkitRelativePath) {
const pathParts = files[0].webkitRelativePath.split(/[/\\]/);
if (pathParts.length > 1) {
currentUsername = pathParts[0];
}
}
if (!currentUsername) currentUsername = 'archived_user';
let format: 'export' | 'instaloader' | 'json' | 'unknown' = 'unknown';
let jsonFiles: ArchiveFile[] = [];
// Pass 1: Indexing
for (let i = 0; i < files.length; i++) {
const file = files[i];
if (i % 100 === 0 || i === files.length - 1) {
setScannedCount(i + 1);
setScannedFilesLog(prev => [`Indexed ${file.name}`, ...prev.slice(0, 19)]);
// Yield to main thread
await new Promise(resolve => setTimeout(resolve, 0));
}
const lowerName = file.name.toLowerCase();
if (lowerName.endsWith('.json') || lowerName.endsWith('.json.xz')) {
jsonFiles.push(file);
if (lowerName.includes('posts_1') || lowerName.includes('reels_1') || lowerName.includes('stories_1')) format = 'json';
continue;
}
if (EXPORT_RE.test(file.name)) format = 'export';
else if (INSTALOADER_RE.test(file.name)) format = 'instaloader';
// Highlight items match neither pattern, so a profile made up only of
// sidecar directories would otherwise never reach the filename parser.
else if (format === 'unknown' && file.source && file.source.kind !== 'posts') format = 'export';
if (lowerName.includes('_profile_pic.jpg') || (currentUsername && lowerName === `${currentUsername.toLowerCase()}.jpg`)) {
try {
const url = mintUrl(file, 'image/jpeg');
discoveredProfilePics.push({ name: file.name, url });
if (format === 'unknown' && lowerName.includes('_profile_pic.jpg')) format = 'instaloader';
} catch(e) {}
}
if (isMedia(file.name)) {
mediaFilesMap.set(file.webkitRelativePath || file.name, file);
if (isImage(file.name)) allImageFiles.push(file);
}
}
console.log(`[Scanner] Format Detection Complete. Result: ${format}. Media indexed: ${mediaFilesMap.size}`);
if (jsonFiles.length > 0 && (format === 'json' || format === 'instaloader')) {
setScanningPhase('Parsing');
for (let i = 0; i < jsonFiles.length; i++) {
const jsonFile = jsonFiles[i];
setScannedCount(i + 1);
setScannedFilesLog(prev => [`Parsing ${jsonFile.name}`, ...prev.slice(0, 19)]);
try {
const data = jsonFile.name.endsWith('.xz') ? await parseXZFile(jsonFile) : JSON.parse(await jsonFile.text());
if (!data) continue;
const items = Array.isArray(data) ? data : (data.media || [data]);
const isStoriesFile = jsonFile.name.toLowerCase().includes('stories');
if (data.node && (data.instaloader?.node_type === 'Profile' || data.node.__typename === 'User')) {
const node = data.node; const iphone = node.iphone_struct || {};
localFullName = node.full_name || ''; localBio = node.biography || iphone.biography || '';
localExternalUrl = node.external_url || '';
localFollowerCount = node.edge_followed_by?.count || iphone.follower_count || 0;
localFollowingCount = node.edge_follow?.count || iphone.following_count || 0;
if (!Array.isArray(data)) continue;
}
for (const [idx, item] of items.entries()) {
const mediaList = item.media || [item];
const postId = item.node?.id || item.id || item.title || `post_${idx}_${Date.now()}`;
const date = item.creation_timestamp ? new Date(item.creation_timestamp * 1000).toISOString().split('T')[0] : (item.node?.taken_at_timestamp ? new Date(item.node.taken_at_timestamp * 1000).toISOString().split('T')[0] : new Date().toISOString().split('T')[0]);
const isStory = isStoriesFile || checkIsStory(item) || checkIsStory(item.node) || checkIsStory(data.instaloader) || checkIsStory(item.node?.iphone_struct) || checkIsStory(item.iphone_struct) || (item.media && Array.isArray(item.media) && item.media.some((m: any) => checkIsStory(m)));
const post: Partial<Post> = { id: postId, date, username: currentUsername || 'archived_user', caption: item.title || item.node?.edge_media_to_caption?.edges?.[0]?.node?.text || item.node?.caption?.text || '', media: [], isStory };
for (const [mIdx, m] of mediaList.entries()) {
const uri = m.uri; let matchedFile: ArchiveFile | undefined;
if (uri) { for (const [path, f] of mediaFilesMap.entries()) { if (path.endsWith(uri) || uri.endsWith(path)) { matchedFile = f; break; } } }
if (!matchedFile) { const id = item.node?.id || item.id; if (id) { for (const [path, f] of mediaFilesMap.entries()) { if (f.name.includes(id)) { matchedFile = f; break; } } } }
if (!matchedFile) {
const jsonBase = jsonFile.name.substring(0, jsonFile.name.lastIndexOf('.'));
for (const ext of ['mp4', 'webm', 'jpg', 'jpeg', 'png', 'webp', 'gif']) {
const possibleName = `${jsonBase}.${ext}`;
for (const [path, f] of mediaFilesMap.entries()) { if (f.name.toLowerCase() === possibleName.toLowerCase()) { matchedFile = f; break; } }
if (matchedFile) break;
}
}
if (matchedFile) {
const type = isVideo(matchedFile.name) ? 'video' : 'image';
const url = mintUrl(matchedFile, type === 'video' ? 'video/mp4' : 'image/jpeg');
const existingMedia = post.media!.find(media => media.index === mIdx + 1);
if (existingMedia) { if (type === 'video' && existingMedia.type === 'image') post.media = post.media!.map(media => media.index === mIdx + 1 ? { name: matchedFile!.name, path: mediaPath(matchedFile!), url, type, index: mIdx + 1, size: matchedFile!.size } : media); }
else post.media!.push({ name: matchedFile.name, path: mediaPath(matchedFile), url, type, index: mIdx + 1, size: matchedFile.size });
}
}
if (post.media!.length > 0) postsMap.set(postId, post);
}
} catch (e) { console.error(`[Scanner] Error parsing JSON ${jsonFile.name}:`, e); }
// Yield to main thread
if (i % 10 === 0) await new Promise(resolve => setTimeout(resolve, 0));
}
}
if (format === 'export' || format === 'instaloader') {
setScanningPhase('Parsing');
const CHUNK_SIZE = 100;
for (let j_start = 0; j_start < files.length; j_start += CHUNK_SIZE) {
const end = Math.min(j_start + CHUNK_SIZE, files.length);
setScannedCount(j_start);
setScannedFilesLog(prev => [`Batch ${Math.floor(j_start/CHUNK_SIZE) + 1} processing...`, ...prev.slice(0, 19)]);
for (let j = j_start; j < end; j++) {
const file = files[j]; const lowerName = file.name.toLowerCase();
const kind = file.source?.kind ?? 'posts';
const parsed = parseArchiveFilename(file.name, kind, file.mtime);
if (!parsed) continue;
const { date, index, ext } = parsed;
const user = parsed.username || currentUsername || 'archived_user';
let isStory = parsed.isStory
|| lowerName.includes('story')
|| file.webkitRelativePath.toLowerCase().includes('stories');
const postId = scopedPostId(parsed.postId, kind, file.source?.dir);
if (kind === 'stories') isStory = true;
if (kind === 'highlight') isStory = false;
let post = postsMap.get(postId);
if (!post) {
post = { id: postId, date, username: user, caption: '', media: [], isStory, source: kind, highlightTitle: file.source?.title };
postsMap.set(postId, post);
}
else if (isStory) post.isStory = true;
const lowerExt = ext.toLowerCase();
if (lowerExt === 'txt') {
try { post.caption = await file.text(); } catch(e) {}
} else if (lowerExt === 'json' || lowerName.endsWith('.json.xz')) {
try {
const data = lowerName.endsWith('.xz') ? await parseXZFile(file) : JSON.parse(await file.text());
if (data) {
const node = data.node || data; const iphone = node.iphone_struct || {};
const captionText = node.edge_media_to_caption?.edges?.[0]?.node?.text || node.caption?.text || iphone.caption?.text || '';
if (captionText) post.caption = captionText;
if (checkIsStory(data) || checkIsStory(node) || checkIsStory(data.instaloader) || checkIsStory(iphone)) post.isStory = true;
}
} catch (e) {}
} else if (isMedia(file.name)) {
const type = isVideo(file.name) ? 'video' : 'image';
const url = mintUrl(file, type === 'video' ? 'video/mp4' : 'image/jpeg');
if (type === 'image') throttledSetScanningImage(url);
const existingMedia = post.media!.find(m => m.index === index);
if (existingMedia) { if (type === 'video' && existingMedia.type === 'image') post.media = post.media!.map(m => m.index === index ? { name: file.name, path: mediaPath(file), url, type, index, size: file.size } : m); }
else post.media!.push({ name: file.name, path: mediaPath(file), url, type, index, size: file.size });
}
}
await new Promise(resolve => setTimeout(resolve, 0));
}
}
if (postsMap.size === 0) {
console.log(`[Scanner] No posts found with standard patterns. Using mediaFilesMap: ${mediaFilesMap.size}`);
setScanningPhase('Parsing');
const genericGroupingMap = new Map<string, ArchiveFile[]>();
for (const [key, file] of mediaFilesMap.entries()) {
const match = file.name.match(/^(.*?)(?:(_|-|\s)+(\d+))?\.(.+)$/);
let baseName = file.name;
if (match && match[3]) { baseName = match[1].trim(); }
else { baseName = file.name.substring(0, file.name.lastIndexOf('.')); }
if (!genericGroupingMap.has(baseName)) genericGroupingMap.set(baseName, []);
genericGroupingMap.get(baseName)!.push(file);
}
console.log(`[Scanner] Generic grouping found ${genericGroupingMap.size} base groups.`);
let processedGroups = 0;
const groupEntries = Array.from(genericGroupingMap.entries());
for (const [baseName, groupFiles] of groupEntries) {
processedGroups++;
if (processedGroups % 10 === 0 || processedGroups === genericGroupingMap.size) {
setScannedCount(Math.floor((processedGroups / (genericGroupingMap.size || 1)) * (files.length || 1)));
setScannedFilesLog(prev => [`Grouping: ${baseName}`, ...prev.slice(0, 19)]);
await new Promise(resolve => setTimeout(resolve, 0));
}
groupFiles.sort((a, b) => {
const na = a.name.match(/[_-](\d+)\.\w+$/)?.[1];
const nb = b.name.match(/[_-](\d+)\.\w+$/)?.[1];
if (na && nb) return parseInt(na, 10) - parseInt(nb, 10);
return a.name.localeCompare(b.name);
});
const CAROUSEL_MAX = 20;
for (let j = 0; j < groupFiles.length; j += CAROUSEL_MAX) {
const batch = groupFiles.slice(j, j + CAROUSEL_MAX);
const partSuffix = groupFiles.length > CAROUSEL_MAX ? `_part${Math.floor(j/CAROUSEL_MAX) + 1}` : '';
const postId = `${baseName}${partSuffix}`;
const post: Post = { id: postId, date: new Date().toISOString().split('T')[0], username: currentUsername || 'archived_user', caption: baseName, media: [], thumbnail: '' };
for (const [idx, file] of batch.entries()) {
const type = isVideo(file.name) ? 'video' : 'image';
const url = mintUrl(file, type === 'video' ? 'video/mp4' : 'image/jpeg');
if (type === 'image') throttledSetScanningImage(url);
post.media.push({ name: file.name, path: mediaPath(file), url, type, index: idx + 1, size: file.size });
}
post.thumbnail = post.media[0].url;
postsMap.set(postId, post);
}
}
}
if (discoveredProfilePics.length > 0) {
discoveredProfilePics.sort((a, b) => b.name.localeCompare(a.name));
const urls = discoveredProfilePics.map(p => p.url);
localProfilePic = urls[0];
setProfileMetadata(prev => ({ ...prev, profilePic: localProfilePic, allProfilePics: urls }));
} else if (allImageFiles.length > 0) {
// Fallback: Use oldest image in archive as profile pic
allImageFiles.sort((a, b) => a.name.localeCompare(b.name));
const oldestFile = allImageFiles[0];
try {
const url = mintUrl(oldestFile, 'image/jpeg');
localProfilePic = url;
setProfileMetadata(prev => ({ ...prev, profilePic: localProfilePic, allProfilePics: [url] }));
} catch(e) {}
}
const finalUsername = currentUsername || 'archived_user';
const allItems = Array.from(postsMap.values()).filter(p => p.media && p.media.length > 0).map(p => {
const sortedMedia = p.media!.sort((a, b) => a.index - b.index);
return { ...p, username: (p.username === 'archived_user' || !p.username) ? finalUsername : p.username, media: sortedMedia, thumbnail: sortedMedia[0].url } as Post;
});
const byNewest = (a: Post, b: Post) => b.date.localeCompare(a.date);
// Highlights are story-shaped but live behind their own circles, so they
// are kept out of both the grid and the profile-ring story reel.
const highlights = allItems.filter(p => p.source === 'highlight').sort(byNewest);
const rest = allItems.filter(p => p.source !== 'highlight');
const posts = rest.filter(p => !p.isStory).sort(byNewest);
const stories = rest.filter(p => p.isStory).sort(byNewest);
setAllPosts(posts);
setAllStories(stories);
setAllHighlights(highlights);
setProfileMetadata(prev => ({
...prev,
username: finalUsername,
fullName: localFullName,
bio: localBio,
followerCount: localFollowerCount,
followingCount: localFollowingCount,
externalUrl: localExternalUrl,
profilePic: localProfilePic || prev.profilePic,
}));
console.log(`[Scanner] Finalized ${posts.length} posts and ${stories.length} stories.`);
const archiveToCache = archiveContext || currentArchive;
const isLocal = !archiveToCache;
const cacheKey = archiveToCache ? archiveToCache.name : (finalUsername || 'local_archive');
if (cacheKey && (posts.length > 0 || stories.length > 0)) {
console.log(`[Cache] Saving data for ${cacheKey} to persistent storage...`);
let cacheThumbnail = localProfilePic;
if (isLocal && posts.length > 0 && posts[0].media[0].type === 'image') {
try {
const img = new Image(); img.src = posts[0].media[0].url;
await new Promise((res) => { img.onload = res; img.onerror = res; });
if (img.complete && img.width > 0) {
const canvas = document.createElement('canvas'); const size = 200;
canvas.width = size; canvas.height = size;
const ctx = canvas.getContext('2d');
if (ctx) { ctx.drawImage(img, 0, 0, img.width, img.height, 0, 0, size, size); cacheThumbnail = canvas.toDataURL('image/jpeg', 0.7); }
}
} catch (e) {}
}
const cacheData: CacheData = {
name: cacheKey, isLocal,
fileCount: (archiveToCache ? archiveToCache.fileCount : files.length) ?? files.length,
signature: archiveToCache?.signature,
posts: posts,
stories: stories,
highlights: highlights,
profileMetadata: {
username: finalUsername,
fullName: localFullName,
bio: localBio,
followerCount: localFollowerCount,
followingCount: localFollowingCount,
externalUrl: localExternalUrl,
// Local blob: URLs die with the document, so persist a data: URL for
// the dashboard card instead. Media URLs are rehydrated from `path`.
profilePic: isLocal ? cacheThumbnail : localProfilePic,
allProfilePics: isLocal ? (cacheThumbnail ? [cacheThumbnail] : []) : discoveredProfilePics.map(p => p.url)
},
hasDirectoryHandle: isLocal ? await hasDirectoryHandle(cacheKey) : false,
timestamp: Date.now()
};
try {
await setCachedArchive(cacheData);
console.log(`[Cache] Data saved successfully.`);
await refreshCachedArchives();
} catch (e) { console.error(`[Cache] Save error:`, e); }
}
} catch (err) { console.error(`[Scanner] Critical error during scan:`, err); } finally { setIsScanning(false); }
}, [currentArchive, detectedUsername, resetScannerState, refreshCachedArchives]);
return {
isScanning,
scanningPhase,
scannedCount,
totalFiles,
scannedFilesLog,
currentScanningImage,
allPosts,
allStories,
allHighlights,
profileMetadata,
handleFiles,
setAllPosts,
setAllStories,
setAllHighlights,
setProfileMetadata,
setIsScanning,
setScanningPhase,
setScannedCount,
setTotalFiles,
setScannedFilesLog,
setCurrentScanningImage,
resetScannerState,
registerUrl
};
};
+128
View File
@@ -0,0 +1,128 @@
import { useState, useCallback, useRef, useEffect } from 'react';
import * as idb from 'idb-keyval';
import { thumbKey } from '../lib/archive-cache';
interface ThumbnailRequest {
id: string;
key: string;
url: string;
blob?: Blob;
}
const THUMBNAIL_WIDTH = 400;
export const useThumbnailQueue = (archiveName: string) => {
const [cacheHits, setCacheHits] = useState<Map<string, string>>(new Map());
const queueRef = useRef<ThumbnailRequest[]>([]);
const isProcessingRef = useRef(false);
const workerRef = useRef<Worker | null>(null);
/**
* Mirrors `cacheHits` for reads inside callbacks.
*
* `requestThumbnail` is a dependency of every PostThumbnail effect, so it must
* keep a stable identity — closing over `cacheHits` state directly would give
* it a new identity per completed thumbnail and re-run the effect in all
* ~90 mounted thumbnails each time.
*/
const cacheHitsRef = useRef<Map<string, string>>(new Map());
/** Blob URLs handed out for thumbnails, released when the archive changes. */
const createdUrlsRef = useRef<string[]>([]);
const publish = useCallback((id: string, url: string) => {
createdUrlsRef.current.push(url);
cacheHitsRef.current.set(id, url);
setCacheHits(new Map(cacheHitsRef.current));
}, []);
const processNext = useCallback(async () => {
if (isProcessingRef.current || queueRef.current.length === 0 || !workerRef.current) return;
isProcessingRef.current = true;
const request = queueRef.current.shift()!;
try {
// Re-check the store: the entry may have landed since being queued.
const cached = await idb.get(request.key);
if (cached instanceof Blob) {
publish(request.id, URL.createObjectURL(cached));
isProcessingRef.current = false;
processNext();
return;
}
let blob = request.blob;
if (!blob) {
const res = await fetch(request.url);
blob = await res.blob();
}
// One image at a time: decoding several 50MP+ files concurrently is a
// reliable way to OOM the tab.
workerRef.current.postMessage({ id: request.key, blob, width: THUMBNAIL_WIDTH });
} catch (err) {
console.error('[ThumbnailQueue] Failed to process:', request.id, err);
isProcessingRef.current = false;
processNext();
}
}, [publish]);
useEffect(() => {
workerRef.current = new Worker(new URL('../lib/thumbnail-worker.ts', import.meta.url), {
type: 'module'
});
workerRef.current.onmessage = async (e) => {
const { id: key, blob, error } = e.data;
if (!error && blob) {
const id = key.split(':').slice(2).join(':');
publish(id, URL.createObjectURL(blob));
try {
await idb.set(key, blob);
} catch (err) { /* quota exceeded; the in-memory hit still stands */ }
}
isProcessingRef.current = false;
processNext();
};
return () => {
workerRef.current?.terminate();
};
}, [publish, processNext]);
// Switching archives invalidates every thumbnail URL handed out so far.
useEffect(() => {
return () => {
queueRef.current = [];
for (const url of createdUrlsRef.current) {
try { URL.revokeObjectURL(url); } catch { /* already gone */ }
}
createdUrlsRef.current = [];
cacheHitsRef.current = new Map();
setCacheHits(new Map());
};
}, [archiveName]);
const requestThumbnail = useCallback(async (id: string, url: string, blob?: Blob) => {
if (cacheHitsRef.current.has(id)) return;
const key = thumbKey(archiveName, id);
const cached = await idb.get(key);
if (cached instanceof Blob) {
publish(id, URL.createObjectURL(cached));
return;
}
if (!queueRef.current.some(r => r.id === id)) {
queueRef.current.push({ id, key, url, blob });
processNext();
}
}, [archiveName, publish, processNext]);
return {
cacheHits,
requestThumbnail
};
};
+2 -1
View File
@@ -1,4 +1,5 @@
@import url('https://fonts.googleapis.com/css2?family=Inter:wght@300;400;500;600;700&family=Playfair+Display:ital,wght@0,400..900;1,400..900&display=swap');
/* Self-hosted: no third-party font requests, works fully offline. */
@import url('/fonts/fonts.css');
@import "tailwindcss";
@theme {
+170
View File
@@ -0,0 +1,170 @@
import * as idb from 'idb-keyval';
import { CacheData, Post } from '../types';
import { DirectoryHandle, ensureReadPermission, filesFromDirectory } from './directory-handle';
import { LocalArchiveFile } from './archive-files';
/**
* Persistent archive cache.
*
* Keys are namespaced so that listing archives does not require deserializing
* every thumbnail blob in the store: `archive:` entries are metadata, `thumb:`
* entries are image blobs, `handle:` entries are directory handles.
*/
const ARCHIVE_PREFIX = 'archive:';
const THUMB_PREFIX = 'thumb:';
const HANDLE_PREFIX = 'handle:';
export const archiveKey = (name: string) => `${ARCHIVE_PREFIX}${name}`;
export const handleKey = (name: string) => `${HANDLE_PREFIX}${name}`;
/** Thumbnails are scoped per archive; post IDs alone collide across archives. */
export const thumbKey = (archive: string, postId: string) => `${THUMB_PREFIX}${archive}:${postId}`;
export const getCachedArchive = (name: string): Promise<CacheData | undefined> =>
idb.get(archiveKey(name));
export const setCachedArchive = (data: CacheData) => idb.set(archiveKey(data.name), data);
/** Names of all cached archives, without loading their contents. */
export const listCachedArchiveNames = async (): Promise<string[]> =>
(await idb.keys())
.map(String)
.filter(k => k.startsWith(ARCHIVE_PREFIX))
.map(k => k.slice(ARCHIVE_PREFIX.length));
export const listCachedArchives = async (): Promise<CacheData[]> => {
const names = await listCachedArchiveNames();
const entries = await Promise.all(names.map(getCachedArchive));
return entries.filter((e): e is CacheData => Boolean(e));
};
/** Remove an archive along with its handle and every thumbnail it owns. */
export const deleteCachedArchive = async (name: string) => {
const thumbPrefix = `${THUMB_PREFIX}${name}:`;
const stale = (await idb.keys()).map(String).filter(k => k.startsWith(thumbPrefix));
await idb.delMany([archiveKey(name), handleKey(name), ...stale]);
};
export const saveDirectoryHandle = (name: string, handle: DirectoryHandle) =>
idb.set(handleKey(name), handle);
export const getDirectoryHandle = (name: string): Promise<DirectoryHandle | undefined> =>
idb.get(handleKey(name));
/**
* One-time migration from the flat key layout (archive name as a bare key,
* `thumb_<postId>` for thumbnails).
*
* Old local entries are dropped rather than migrated: their media URLs are
* dead blob: URLs, so restoring them would render an archive of broken images.
*/
export const migrateLegacyCache = async () => {
const keys = (await idb.keys()).map(String);
const legacyThumbs = keys.filter(k => k.startsWith('thumb_'));
const legacyArchives = keys.filter(
k => !k.startsWith(ARCHIVE_PREFIX) && !k.startsWith(THUMB_PREFIX) &&
!k.startsWith(HANDLE_PREFIX) && !k.startsWith('thumb_')
);
if (!legacyThumbs.length && !legacyArchives.length) return;
const drop: string[] = [...legacyThumbs];
for (const key of legacyArchives) {
const data = await idb.get(key);
drop.push(key);
if (data && typeof data === 'object' && 'posts' in data && !(data as CacheData).isLocal) {
// Server archives keep working: their URLs are plain HTTP paths.
await setCachedArchive({ ...(data as CacheData), name: key });
}
}
await idb.delMany(drop);
console.log(`[Cache] Migrated legacy cache: dropped ${drop.length} stale keys.`);
};
/**
* Rebuild a server archive's media URLs, which are stable HTTP paths.
*
* `path` is relative to the archives root and already carries the source
* directory (which may be a sidecar such as `story - user`), so it is not
* prefixed with the archive name. Entries cached before `path` existed fall
* back to their stored URL.
*/
const rehydrateRemote = (posts: Post[]): Post[] =>
posts.map(post => {
const media = post.media.map(m => ({
...m,
url: m.path ? `/archives/${encodeURI(m.path)}` : m.url,
}));
return { ...post, media, thumbnail: media[0]?.url ?? post.thumbnail };
});
/**
* Rebuild a local archive's media URLs from a live directory handle, minting
* fresh blob: URLs for the paths recorded at scan time.
*
* Returns null when the folder is no longer reachable (permission declined, or
* the handle no longer resolves), signalling the caller to re-prompt.
*/
const rehydrateLocal = async (
posts: Post[],
handle: DirectoryHandle,
onUrl: (url: string) => void,
): Promise<Post[] | null> => {
if (!(await ensureReadPermission(handle))) return null;
let files: LocalArchiveFile[];
try {
files = await filesFromDirectory(handle);
} catch (err) {
console.warn('[Cache] Directory handle no longer readable:', err);
return null;
}
const byPath = new Map(files.map(f => [f.webkitRelativePath, f]));
return posts.map(post => {
const media = post.media.map(m => {
const file = byPath.get(m.path);
if (!file) return { ...m, url: '' };
const url = file.createObjectUrl(m.type === 'video' ? 'video/mp4' : 'image/jpeg');
onUrl(url);
return { ...m, url };
});
return { ...post, media, thumbnail: media[0]?.url ?? '' };
});
};
export interface RestoredArchive {
posts: Post[];
stories: Post[];
highlights: Post[];
profileMetadata: CacheData['profileMetadata'];
}
/**
* Turn a cache entry back into displayable state.
*
* `onUrl` receives every blob: URL minted so the caller can revoke them later.
* Returns null if a local archive's folder can no longer be reached.
*/
export const restoreArchive = async (
data: CacheData,
onUrl: (url: string) => void,
): Promise<RestoredArchive | null> => {
if (!data.isLocal) {
return {
posts: rehydrateRemote(data.posts),
stories: rehydrateRemote(data.stories),
highlights: rehydrateRemote(data.highlights ?? []),
profileMetadata: data.profileMetadata,
};
}
const handle = await getDirectoryHandle(data.name);
if (!handle) return null;
const posts = await rehydrateLocal(data.posts, handle, onUrl);
if (!posts) return null;
const stories = (await rehydrateLocal(data.stories, handle, onUrl)) ?? [];
const highlights = (await rehydrateLocal(data.highlights ?? [], handle, onUrl)) ?? [];
return { posts, stories, highlights, profileMetadata: data.profileMetadata };
};
+70
View File
@@ -0,0 +1,70 @@
import { ArchiveFile, ArchiveSource } from '../types';
export class LocalArchiveFile implements ArchiveFile {
/** Blob URLs minted here are revocable and must be released when done. */
readonly revocable = true;
/**
* @param explicitPath Set when the file came from the File System Access API,
* whose File objects carry an empty webkitRelativePath.
*/
constructor(private file: File, private explicitPath?: string) {}
get name() { return this.file.name; }
get webkitRelativePath() { return this.explicitPath ?? this.file.webkitRelativePath; }
get size() { return this.file.size; }
text() { return this.file.text(); }
arrayBuffer() { return this.file.arrayBuffer(); }
stream() { return this.file.stream(); }
/**
* A blob: URL backed directly by the on-disk File.
*
* Deliberately does NOT go through arrayBuffer() — a File is already a Blob,
* so this hands the browser a disk-backed handle instead of pulling the whole
* file into memory. Doing otherwise means a 20GB archive tries to become 20GB
* of resident blobs.
*
* When the picker gave us no MIME type, slice() re-tags the blob with a hint.
* slice() is a zero-copy view, so this stays memory-free either way.
*/
createObjectUrl(mimeHint?: string) {
const source = this.file.type || !mimeHint
? this.file
: this.file.slice(0, this.file.size, mimeHint);
return URL.createObjectURL(source);
}
}
export class RemoteArchiveFile implements ArchiveFile {
/** Served over HTTP; there is no object URL to release. */
readonly revocable = false;
constructor(
public name: string,
public webkitRelativePath: string,
public size: number,
public url: string,
public source?: ArchiveSource,
public mtime?: number
) {}
async text() {
const res = await fetch(this.url);
return res.text();
}
async arrayBuffer() {
const res = await fetch(this.url);
return res.arrayBuffer();
}
stream() {
const transform = new TransformStream();
fetch(this.url).then(res => {
if (res.body) res.body.pipeTo(transform.writable);
else transform.writable.getWriter().close();
});
return transform.readable;
}
createObjectUrl() {
return this.url;
}
}
+89
View File
@@ -0,0 +1,89 @@
import { describe, expect, it } from 'vitest';
import { classifyDirectory, groupArchiveDirectories } from './archive-grouping';
describe('classifyDirectory', () => {
it('treats a bare profile directory as the base', () => {
expect(classifyDirectory('0ct0ber19')).toEqual({
owner: '0ct0ber19',
source: { kind: 'posts', dir: '0ct0ber19' },
});
});
it('recognises a reels sidecar', () => {
expect(classifyDirectory('0ct0ber19 - reels')).toEqual({
owner: '0ct0ber19',
source: { kind: 'reels', dir: '0ct0ber19 - reels' },
});
});
it('recognises a stories sidecar', () => {
expect(classifyDirectory('story - cher_ryppo')).toEqual({
owner: 'cher_ryppo',
source: { kind: 'stories', dir: 'story - cher_ryppo' },
});
});
it('splits highlight owner from title', () => {
const { owner, source } = classifyDirectory('story highlights - 0ct0ber19 - Heestory');
expect(owner).toBe('0ct0ber19');
expect(source.kind).toBe('highlight');
expect(source.title).toBe('Heestory');
});
it.each([
['story highlights - theoldtaylorswiftinsta - 💙2014-1989 era', 'theoldtaylorswiftinsta', '💙2014-1989 era'],
['story highlights - heejin_theworld - [Dall]', 'heejin_theworld', '[Dall]'],
['story highlights - official_artms - Cosmo Schedule', 'official_artms', 'Cosmo Schedule'],
['story highlights - 0ct0ber19 - Drawheeing', '0ct0ber19', 'Drawheeing'],
['story highlights - official_artms - G.C.I', 'official_artms', 'G.C.I'],
])('handles real-world title %s', (dir, owner, title) => {
const result = classifyDirectory(dir);
expect(result.owner).toBe(owner);
expect(result.source.title).toBe(title);
});
it('keeps titles containing " - " intact', () => {
// The username is matched as a non-space run, so only the first separator
// splits owner from title.
const { owner, source } = classifyDirectory('story highlights - user - a - b');
expect(owner).toBe('user');
expect(source.title).toBe('a - b');
});
it('does not mistake a profile with spaces for a sidecar', () => {
expect(classifyDirectory('Heejin_Bubble heejinmedia').source.kind).toBe('posts');
});
});
describe('groupArchiveDirectories', () => {
const dirs = [
'0ct0ber19',
'0ct0ber19 - reels',
'story - 0ct0ber19',
'story highlights - 0ct0ber19 - Heestory',
'story highlights - 0ct0ber19 - Drawheeing',
'carlyraejepsen',
];
it('folds sidecars into their base profile', () => {
const groups = groupArchiveDirectories(dirs);
expect([...groups.keys()].sort()).toEqual(['0ct0ber19', 'carlyraejepsen']);
expect(groups.get('0ct0ber19')).toHaveLength(5);
expect(groups.get('carlyraejepsen')).toHaveLength(1);
});
it('orders sources posts, reels, stories, then highlights by title', () => {
const sources = groupArchiveDirectories(dirs).get('0ct0ber19')!;
expect(sources.map(s => s.kind)).toEqual(['posts', 'reels', 'stories', 'highlight', 'highlight']);
expect(sources.slice(3).map(s => s.title)).toEqual(['Drawheeing', 'Heestory']);
});
it('still groups a sidecar whose base profile is missing', () => {
const groups = groupArchiveDirectories(['story - orphan']);
expect(groups.get('orphan')).toEqual([{ kind: 'stories', dir: 'story - orphan' }]);
});
it('is stable for an empty archive root', () => {
expect(groupArchiveDirectories([]).size).toBe(0);
});
});
+71
View File
@@ -0,0 +1,71 @@
/**
* Archive directory naming rules.
*
* Shared by the server (to fold sidecar directories into one profile) and the
* test suite. Kept free of Node built-ins so it can be imported from either.
*/
export type SourceKind = 'posts' | 'reels' | 'stories' | 'highlight';
export interface ArchiveSource {
kind: SourceKind;
/** Directory name relative to the archives root. */
dir: string;
/** Highlight title, for kind === 'highlight'. */
title?: string;
}
/**
* Sidecar directories sit next to the profile directory they belong to:
*
* 0ct0ber19 -> posts (base)
* 0ct0ber19 - reels -> reels
* story - 0ct0ber19 -> stories
* story highlights - 0ct0ber19 - Heestory -> highlight "Heestory"
*
* Instagram usernames cannot contain spaces, so matching the username as a
* run of non-space characters reliably separates it from a highlight title
* (titles may themselves contain spaces, dashes and emoji).
*/
export const classifyDirectory = (dirName: string): { owner: string; source: ArchiveSource } => {
const highlight = /^story highlights - ([^ ]+) - (.+)$/.exec(dirName);
if (highlight) {
return { owner: highlight[1], source: { kind: 'highlight', dir: dirName, title: highlight[2] } };
}
const stories = /^story - ([^ ]+)$/.exec(dirName);
if (stories) {
return { owner: stories[1], source: { kind: 'stories', dir: dirName } };
}
const reels = /^([^ ]+) - reels$/.exec(dirName);
if (reels) {
return { owner: reels[1], source: { kind: 'reels', dir: dirName } };
}
return { owner: dirName, source: { kind: 'posts', dir: dirName } };
};
const RANK: Record<SourceKind, number> = { posts: 0, reels: 1, stories: 2, highlight: 3 };
/**
* Group the archive root's directories by profile.
*
* A sidecar whose owner has no base directory still forms a group of its own,
* so nothing becomes invisible just because the base profile is missing.
*/
export const groupArchiveDirectories = (dirNames: string[]): Map<string, ArchiveSource[]> => {
const groups = new Map<string, ArchiveSource[]>();
for (const dirName of dirNames) {
const { owner, source } = classifyDirectory(dirName);
if (!groups.has(owner)) groups.set(owner, []);
groups.get(owner)!.push(source);
}
for (const sources of groups.values()) {
sources.sort((a, b) => RANK[a.kind] - RANK[b.kind] || (a.title ?? '').localeCompare(b.title ?? ''));
}
return groups;
};
+232
View File
@@ -0,0 +1,232 @@
import fs from 'fs';
import fsp from 'fs/promises';
import path from 'path';
import { ArchiveSource, SourceKind, groupArchiveDirectories } from './archive-grouping.js';
/**
* On-disk archive index.
*
* Archives live on network storage where per-file `stat` costs ~1.4ms and does
* not parallelise well, so walking every file on each request is unaffordable:
* measured against a real 110k-file archive root, listing took ~52s.
*
* Directory `stat` is effectively free, so each source directory is indexed
* once and re-used until its mtime changes. The index is warmed in the
* background at startup and persisted, making steady-state requests instant.
*/
export interface IndexedFile {
path: string;
size: number;
mtime: number;
kind: SourceKind;
title?: string;
}
interface DirIndex {
dir: string;
/** Directory mtime the index was built from; the cache key. */
mtimeMs: number;
files: IndexedFile[];
}
const MEDIA_RE = /\.(jpg|jpeg|png|webp|gif|bmp|tiff|mp4|webm|ogv|mov)$/i;
const STAT_CONCURRENCY = 16;
export class ArchiveIndex {
private dirs = new Map<string, DirIndex>();
private inFlight = new Map<string, Promise<DirIndex>>();
private dirty = false;
constructor(private archivesDir: string, private cachePath: string) {}
/** Visible (non-system) directories at the archive root. */
private listRootDirs(): string[] {
return fs.readdirSync(this.archivesDir, { withFileTypes: true })
.filter(e => e.isDirectory() && !/^[.@_]/.test(e.name))
.map(e => e.name);
}
groups(): Map<string, ArchiveSource[]> {
return groupArchiveDirectories(this.listRootDirs());
}
private dirMtime(dir: string): number {
try {
return fs.statSync(path.join(this.archivesDir, dir)).mtimeMs;
} catch {
return 0;
}
}
/** Recursively list relative file paths without stat()ing them. */
private walk(absDir: string, base = ''): string[] {
let out: string[] = [];
let entries: fs.Dirent[];
try {
entries = fs.readdirSync(absDir, { withFileTypes: true });
} catch {
return out;
}
for (const entry of entries) {
const rel = base ? `${base}/${entry.name}` : entry.name;
if (entry.isDirectory()) out = out.concat(this.walk(path.join(absDir, entry.name), rel));
else if (entry.isFile()) out.push(rel);
}
return out;
}
private async buildDir(source: ArchiveSource): Promise<DirIndex> {
const started = Date.now();
const mtimeMs = this.dirMtime(source.dir);
const absDir = path.join(this.archivesDir, source.dir);
const relPaths = this.walk(absDir);
// Only media needs a size (the client thumbnails anything over 1MiB), and
// only highlights need an mtime (their filenames carry no date). Skipping
// the rest avoids thousands of pointless round trips.
const needsStat = (rel: string) => MEDIA_RE.test(rel) || source.kind === 'highlight';
const files: IndexedFile[] = relPaths.map(rel => ({
path: `${source.dir}/${rel}`,
size: 0,
mtime: 0,
kind: source.kind,
...(source.title ? { title: source.title } : {}),
}));
const targets = files.filter((_, i) => needsStat(relPaths[i]));
let cursor = 0;
const worker = async () => {
while (cursor < targets.length) {
const file = targets[cursor++];
try {
const stat = await fsp.stat(path.join(this.archivesDir, file.path));
file.size = stat.size;
file.mtime = stat.mtimeMs;
} catch { /* raced with a delete */ }
}
};
await Promise.all(Array.from({ length: STAT_CONCURRENCY }, worker));
console.log(
`[Index] ${source.dir}: ${files.length} files (${targets.length} statted) in ${((Date.now() - started) / 1000).toFixed(1)}s`
);
this.dirty = true;
return { dir: source.dir, mtimeMs, files };
}
/** Index for one source directory, rebuilding only if its mtime moved. */
private async ensureDir(source: ArchiveSource): Promise<DirIndex> {
const cached = this.dirs.get(source.dir);
const mtimeMs = this.dirMtime(source.dir);
if (cached && cached.mtimeMs === mtimeMs) return cached;
// Collapse concurrent requests for the same directory into one walk.
const existing = this.inFlight.get(source.dir);
if (existing) return existing;
const build = this.buildDir(source).then(index => {
this.dirs.set(source.dir, index);
this.inFlight.delete(source.dir);
return index;
}).catch(err => {
this.inFlight.delete(source.dir);
throw err;
});
this.inFlight.set(source.dir, build);
return build;
}
/** All files for one profile, across its base and sidecar directories. */
async filesFor(owner: string): Promise<IndexedFile[] | null> {
const sources = this.groups().get(owner);
if (!sources?.length) return null;
const indexes = await Promise.all(sources.map(s => this.ensureDir(s)));
return indexes.flatMap(i => i.files);
}
/**
* A cheap change signature for a profile, used by the client to decide
* whether its cached copy is stale. Built from directory mtimes only, so it
* costs one stat per source directory rather than a full walk.
*/
signatureFor(sources: ArchiveSource[]): string {
return sources.map(s => `${s.dir}:${this.dirMtime(s.dir)}`).join('|');
}
/** File count for a profile, if its directories are already indexed. */
countFor(sources: ArchiveSource[]): number | null {
let total = 0;
for (const source of sources) {
const cached = this.dirs.get(source.dir);
if (!cached) return null;
total += cached.files.length;
}
return total;
}
/**
* Best-effort profile picture.
*
* Probes the conventional filenames first (one stat each) and only falls back
* to the indexed listing, so an unindexed archive still gets a thumbnail
* without triggering a walk.
*/
thumbnailFor(owner: string, sources: ArchiveSource[]): string {
const base = sources.find(s => s.kind === 'posts') ?? sources[0];
if (!base) return '';
for (const candidate of [`${owner}.jpg`, `${owner}_profile_pic.jpg`, `${owner}.jpeg`, `${owner}.png`]) {
if (fs.existsSync(path.join(this.archivesDir, base.dir, candidate))) {
return `/archives/${encodeURI(`${base.dir}/${candidate}`)}`;
}
}
const cached = this.dirs.get(base.dir);
if (cached) {
const pick = cached.files.find(f => /_profile_pic\.jpg$/i.test(f.path))
?? cached.files.find(f => /\.(jpg|jpeg|png|webp)$/i.test(f.path));
if (pick) return `/archives/${encodeURI(pick.path)}`;
}
return '';
}
/** Walk every directory once, in the background, so first opens are fast. */
async warm(): Promise<void> {
const started = Date.now();
const sources = [...this.groups().values()].flat();
console.log(`[Index] Warming ${sources.length} source directories...`);
for (const source of sources) {
try {
await this.ensureDir(source);
} catch (err) {
console.error(`[Index] Failed to index ${source.dir}:`, err);
}
}
await this.save();
console.log(`[Index] Warm complete in ${((Date.now() - started) / 1000).toFixed(1)}s`);
}
async load(): Promise<void> {
try {
const raw = await fsp.readFile(this.cachePath, 'utf8');
const parsed: DirIndex[] = JSON.parse(raw);
for (const entry of parsed) this.dirs.set(entry.dir, entry);
console.log(`[Index] Loaded ${this.dirs.size} directories from ${this.cachePath}`);
} catch {
console.log('[Index] No usable index cache; will build from scratch.');
}
}
async save(): Promise<void> {
if (!this.dirty) return;
try {
await fsp.writeFile(this.cachePath, JSON.stringify([...this.dirs.values()]), 'utf8');
this.dirty = false;
console.log(`[Index] Persisted ${this.dirs.size} directories to ${this.cachePath}`);
} catch (err) {
console.warn('[Index] Could not persist index (continuing in memory):', err);
}
}
}
+118
View File
@@ -0,0 +1,118 @@
import { describe, expect, it } from 'vitest';
import { parseArchiveFilename, scopedPostId } from './archive-patterns';
describe('parseArchiveFilename — Instagram export format', () => {
it('parses a single-image post', () => {
expect(parseArchiveFilename('2023-04-19_0ct0ber19 - CrORBIcJJbM.mp4')).toEqual({
postId: 'CrORBIcJJbM',
date: '2023-04-19',
username: '0ct0ber19',
index: 1,
ext: 'mp4',
isStory: false,
});
});
it('parses a carousel slide index', () => {
const parsed = parseArchiveFilename('2023-04-12_0ct0ber19 - Cq8LrxSJAJE - 3.jpg');
expect(parsed).toMatchObject({ postId: 'Cq8LrxSJAJE', index: 3, ext: 'jpg' });
});
it('groups a carousel under one post id', () => {
const ids = ['1', '2', '3'].map(
n => parseArchiveFilename(`2023-04-12_user - Cq8LrxSJAJE - ${n}.jpg`)!.postId,
);
expect(new Set(ids).size).toBe(1);
});
it('parses caption sidecar files', () => {
expect(parseArchiveFilename('2023-04-12_0ct0ber19 - Cq8LrxSJAJE.txt')).toMatchObject({
postId: 'Cq8LrxSJAJE',
ext: 'txt',
});
});
it('flags an explicit story suffix', () => {
expect(parseArchiveFilename('2023-04-12_user - ABC - story.jpg')?.isStory).toBe(true);
});
it('parses the story sidecar layout (date_user - N - shortcode)', () => {
// Files in `story - <user>` carry a per-day ordinal before the shortcode.
const parsed = parseArchiveFilename('2025-10-26_0ct0ber19 - 2 - DQRuDx9iW5Q.jpg', 'stories');
expect(parsed).toMatchObject({ date: '2025-10-26', username: '0ct0ber19', ext: 'jpg' });
expect(parsed!.postId).toContain('DQRuDx9iW5Q');
});
it('gives each story item a distinct id', () => {
const a = parseArchiveFilename('2026-08-13_u - 1 - Db-UTJcCUUr.mp4', 'stories')!.postId;
const b = parseArchiveFilename('2026-08-13_u - 2 - Db-oNJ1CWQ4.mp4', 'stories')!.postId;
expect(a).not.toBe(b);
});
});
describe('parseArchiveFilename — Instaloader format', () => {
it('parses a timestamped filename', () => {
expect(parseArchiveFilename('2024-01-01_12-00-00_UTC.jpg')).toMatchObject({
postId: '2024-01-01_12-00-00_UTC',
date: '2024-01-01',
index: 1,
});
});
it('parses the carousel suffix', () => {
expect(parseArchiveFilename('2024-01-01_12-00-00_UTC_2.jpg')?.index).toBe(2);
});
it('flags the story suffix', () => {
expect(parseArchiveFilename('2024-01-01_12-00-00_UTC_story.jpg')?.isStory).toBe(true);
});
});
describe('parseArchiveFilename — story highlights', () => {
it('parses the dateless highlight layout', () => {
expect(parseArchiveFilename('0ct0ber19 - C5dQPEYpd9W.mp4', 'highlight')).toMatchObject({
postId: 'C5dQPEYpd9W',
username: '0ct0ber19',
ext: 'mp4',
isStory: false,
});
});
it('dates a highlight from mtime when the filename has none', () => {
const mtime = Date.UTC(2024, 4, 17, 12, 0, 0);
expect(parseArchiveFilename('user - ABC.jpg', 'highlight', mtime)?.date).toBe('2024-05-17');
});
it('leaves the date empty when no mtime is available', () => {
expect(parseArchiveFilename('user - ABC.jpg', 'highlight')?.date).toBe('');
});
it('does not apply the loose highlight pattern outside highlight directories', () => {
// Would otherwise swallow ordinary "a - b.jpg" filenames.
expect(parseArchiveFilename('user - ABC.jpg', 'posts')).toBeNull();
});
});
describe('parseArchiveFilename — non-matching files', () => {
it.each(['0ct0ber19.jpg', 'profile_pic.jpg', 'README.md', 'no-separator.png'])(
'returns null for %s',
name => expect(parseArchiveFilename(name)).toBeNull(),
);
});
describe('scopedPostId', () => {
it('leaves base-profile ids untouched so permalinks keep working', () => {
expect(scopedPostId('Cq8LrxSJAJE', 'posts')).toBe('Cq8LrxSJAJE');
});
it('namespaces sidecar ids by directory', () => {
expect(scopedPostId('C5dQ', 'highlight', 'story highlights - u - Heestory'))
.toBe('story highlights - u - Heestory/C5dQ');
});
it('keeps the same shortcode distinct across sources', () => {
const inPosts = scopedPostId('ABC', 'posts');
const inHighlight = scopedPostId('ABC', 'highlight', 'story highlights - u - H');
expect(inPosts).not.toBe(inHighlight);
});
});
+99
View File
@@ -0,0 +1,99 @@
import { SourceKind } from '../types';
/**
* Filename parsing rules for the archive formats the viewer understands.
*
* Kept as pure functions so the riskiest part of the scanner — deriving post
* identity, date and carousel order from a filename — can be tested directly.
*/
/** Instagram export: `2023-04-12_user - Cq8LrxSJAJE - 2.jpg` */
export const EXPORT_RE = /^(\d{4}-\d{2}-\d{2})_(.+?) - (.+?)(?: - (\d+))?(?: - (story))?\.(.+)$/;
/** Instaloader: `2024-01-01_12-00-00_UTC_2.jpg` */
export const INSTALOADER_RE = /^(\d{4}-\d{2}-\d{2}_\d{2}-\d{2}-\d{2}_UTC)(?:_(\d+))?(?:_(story))?\.(.+)$/;
/**
* Story highlight: `user - C5dQPEYpd9W.mp4` — no date prefix.
*
* Loose enough to match ordinary filenames, so it is only applied to files the
* server has already tagged as coming from a highlight directory.
*/
export const HIGHLIGHT_RE = /^(.+?) - ([A-Za-z0-9_-]+)\.(\w+)$/;
export interface ParsedFilename {
postId: string;
/** ISO date (YYYY-MM-DD), or '' when the filename carries none. */
date: string;
username: string;
/** 1-based carousel position. */
index: number;
ext: string;
isStory: boolean;
}
/**
* Parse an archive filename into post identity.
*
* `kind` selects which patterns apply; `mtime` supplies a date for formats that
* have none (highlights), so those items still sort and render sensibly.
* Returns null when no pattern matches — e.g. a profile picture.
*/
export const parseArchiveFilename = (
fileName: string,
kind: SourceKind = 'posts',
mtime?: number,
): ParsedFilename | null => {
const exp = EXPORT_RE.exec(fileName);
if (exp) {
const [, date, username, postId, indexStr, story, ext] = exp;
return {
postId,
date,
username,
index: indexStr ? parseInt(indexStr, 10) : 1,
ext,
isStory: Boolean(story),
};
}
const ins = INSTALOADER_RE.exec(fileName);
if (ins) {
const [, postId, indexStr, story, ext] = ins;
return {
postId,
date: postId.split('_')[0],
username: '',
index: indexStr ? parseInt(indexStr, 10) : 1,
ext,
isStory: Boolean(story),
};
}
if (kind === 'highlight') {
const hl = HIGHLIGHT_RE.exec(fileName);
if (hl) {
const [, username, shortcode, ext] = hl;
return {
postId: shortcode,
date: mtime ? new Date(mtime).toISOString().split('T')[0] : '',
username,
index: 1,
ext,
isStory: false,
};
}
}
return null;
};
/**
* Namespace a post ID by its source directory.
*
* Base-profile IDs are left untouched so existing permalinks keep working;
* sidecar IDs are prefixed so a shortcode appearing in both the profile and a
* highlight stays two distinct posts.
*/
export const scopedPostId = (postId: string, kind: SourceKind, dir?: string): string =>
kind === 'posts' ? postId : `${dir ?? kind}/${postId}`;
+86
View File
@@ -0,0 +1,86 @@
import { LocalArchiveFile } from './archive-files';
/**
* File System Access API helpers.
*
* A `blob:` URL dies with the document, so a cached local archive whose media
* URLs are blob: URLs is worthless after a reload. A FileSystemDirectoryHandle,
* by contrast, is structured-cloneable and survives in IndexedDB — so we can
* re-open the same folder on a return visit and mint fresh URLs from it.
*
* Only Chromium implements showDirectoryPicker today; callers must handle the
* unsupported case by falling back to the <input webkitdirectory> flow.
*/
// Minimal typings — TS's lib.dom does not ship these in the configured version.
type PermissionState = 'granted' | 'denied' | 'prompt';
interface FileSystemHandlePermissionDescriptor { mode?: 'read' | 'readwrite' }
export interface DirectoryHandle {
name: string;
kind: 'directory';
values(): AsyncIterableIterator<DirectoryHandle | FileHandle>;
queryPermission?(d?: FileSystemHandlePermissionDescriptor): Promise<PermissionState>;
requestPermission?(d?: FileSystemHandlePermissionDescriptor): Promise<PermissionState>;
}
interface FileHandle {
name: string;
kind: 'file';
getFile(): Promise<File>;
}
export const isDirectoryPickerSupported = () =>
typeof window !== 'undefined' && 'showDirectoryPicker' in window;
export const pickDirectory = async (): Promise<DirectoryHandle | null> => {
if (!isDirectoryPickerSupported()) return null;
try {
return await (window as any).showDirectoryPicker({ mode: 'read' });
} catch (err) {
// AbortError simply means the user dismissed the picker.
return null;
}
};
/**
* Confirm we may still read this handle. Returns false when the user declines
* or the grant has lapsed, in which case the caller should re-prompt.
*
* `requestPermission` must be called from a user gesture, so only call this
* while handling a click.
*/
export const ensureReadPermission = async (handle: DirectoryHandle): Promise<boolean> => {
try {
if (!handle.queryPermission) return true;
if ((await handle.queryPermission({ mode: 'read' })) === 'granted') return true;
if (!handle.requestPermission) return false;
return (await handle.requestPermission({ mode: 'read' })) === 'granted';
} catch {
return false;
}
};
/**
* Recursively collect every file in the directory.
*
* Paths are prefixed with the root directory's name so they line up with the
* `webkitRelativePath` values produced by <input webkitdirectory>, keeping
* cached media paths valid regardless of which picker created them.
*/
export const filesFromDirectory = async (handle: DirectoryHandle): Promise<LocalArchiveFile[]> => {
const out: LocalArchiveFile[] = [];
const walk = async (dir: DirectoryHandle, prefix: string) => {
for await (const entry of dir.values()) {
const entryPath = `${prefix}/${entry.name}`;
if (entry.kind === 'directory') {
await walk(entry as DirectoryHandle, entryPath);
} else {
const file = await (entry as FileHandle).getFile();
out.push(new LocalArchiveFile(file, entryPath));
}
}
};
await walk(handle, handle.name);
return out;
};
+42
View File
@@ -0,0 +1,42 @@
/**
* Thumbnail Generation Worker
* Uses OffscreenCanvas and createImageBitmap for high-performance,
* background-thread image resizing.
*/
self.onmessage = async (e: MessageEvent) => {
const { id, blob, width } = e.data;
try {
// 1. Create a bitmap from the blob (native browser decoding)
// We resize it DURING the decode step for maximum efficiency
const bitmap = await createImageBitmap(blob, {
resizeWidth: width,
resizeQuality: 'medium'
});
// 2. Use OffscreenCanvas to draw the resized bitmap
const canvas = new OffscreenCanvas(bitmap.width, bitmap.height);
const ctx = canvas.getContext('2d');
if (!ctx) {
throw new Error('Could not get OffscreenCanvas context');
}
ctx.drawImage(bitmap, 0, 0);
// 3. Convert to a small JPEG blob
const thumbnailBlob = await canvas.convertToBlob({
type: 'image/jpeg',
quality: 0.7
});
// 4. Release bitmap memory
bitmap.close();
// 5. Send result back
self.postMessage({ id, blob: thumbnailBlob });
} catch (err: any) {
self.postMessage({ id, error: err.message });
}
};
+26
View File
@@ -0,0 +1,26 @@
import { clsx, type ClassValue } from 'clsx';
import { twMerge } from 'tailwind-merge';
import { format, parseISO } from 'date-fns';
export function cn(...inputs: ClassValue[]) {
return twMerge(clsx(inputs));
}
/**
* Format an archive date, tolerating junk.
*
* Dates are derived from filenames and arbitrary archive JSON, and date-fns
* `format` throws a RangeError on an invalid date — which would take down the
* whole modal for one malformed name. Story highlights in particular carry no
* date at all when file mtimes are unavailable.
*/
export function formatDateSafe(date: string | undefined, pattern: string): string {
if (!date) return '';
try {
const parsed = parseISO(date);
if (Number.isNaN(parsed.getTime())) return '';
return format(parsed, pattern);
} catch {
return '';
}
}
+23 -2
View File
@@ -1,13 +1,34 @@
import {StrictMode} from 'react';
import {createRoot} from 'react-dom/client';
import App from './App.tsx';
import { ErrorBoundary } from './components/ErrorBoundary';
import './index.css';
import { registerSW } from 'virtual:pwa-register';
registerSW();
// Register service worker with automatic updates
// and a periodic check every hour to ensure long-running sessions stay fresh.
const updateSW = registerSW({
onRegistered(r) {
if (r) {
// Check for updates every hour
setInterval(() => {
r.update();
}, 60 * 60 * 1000);
console.log('[PWA] Service Worker registered and update interval set.');
}
},
onNeedRefresh() {
console.log('[PWA] New content available, reloading...');
},
onOfflineReady() {
console.log('[PWA] App is ready for offline use.');
}
});
createRoot(document.getElementById('root')!).render(
<StrictMode>
<App />
<ErrorBoundary>
<App />
</ErrorBoundary>
</StrictMode>,
);
+124
View File
@@ -0,0 +1,124 @@
export interface MediaFile {
name: string;
/**
* Path relative to the archive root (matching webkitRelativePath for local
* folders). Unlike `url`, this survives a page reload, so it is what the
* cache persists and what URLs are rehydrated from.
*/
path: string;
url: string;
type: 'image' | 'video';
index: number;
size?: number;
}
/**
* Which sidecar directory a post came from. Archives store reels, stories and
* each story highlight in directories alongside the base profile; the viewer
* folds them into one profile and routes them by kind.
*/
export type SourceKind = 'posts' | 'reels' | 'stories' | 'highlight';
export interface ArchiveSource {
kind: SourceKind;
/** Directory name relative to the archives root. */
dir: string;
/** Highlight title, for kind === 'highlight'. */
title?: string;
}
export interface Post {
id: string;
date: string;
username: string;
caption: string;
media: MediaFile[];
thumbnail: string;
isStory?: boolean;
/** Defaults to 'posts' for archives without sidecar directories. */
source?: SourceKind;
/** Highlight this post belongs to, for source === 'highlight'. */
highlightTitle?: string;
}
/**
* Common interface for both local File objects and remote server-side files.
*/
export interface ArchiveFile {
name: string;
webkitRelativePath: string;
size: number;
text(): Promise<string>;
arrayBuffer(): Promise<ArrayBuffer>;
stream(): ReadableStream<Uint8Array>;
url?: string;
/**
* A URL pointing at this file's contents. Local files mint a disk-backed
* blob: URL (no data is read into memory); remote files return their HTTP URL.
*/
createObjectUrl(mimeHint?: string): string;
/** True when createObjectUrl() returns a blob: URL that must be revoked. */
readonly revocable: boolean;
/** Which sidecar directory this file came from, when known. */
source?: ArchiveSource;
/** Last-modified time (ms). Used to date items whose filename has no date. */
mtime?: number;
}
export interface ServerArchive {
name: string;
thumbnail: string;
path: string;
/** Null until the server has indexed this profile. */
fileCount: number | null;
/**
* Directory-mtime signature. Cheap for the server to compute and sufficient
* to detect changes, unlike a file count that would require a full walk.
*/
signature?: string;
/** Base profile plus any sidecar directories folded into it. */
sources?: ArchiveSource[];
}
/** One entry from GET /api/archives/:name/files. */
export interface ServerArchiveFile {
path: string;
size: number;
mtime: number;
kind: SourceKind;
title?: string;
}
export interface ProfileMetadata {
username: string;
fullName: string;
bio: string;
followerCount: number;
followingCount: number;
externalUrl: string;
profilePic: string | null;
allProfilePics: string[];
}
/** Shape of an archive entry persisted in IndexedDB. */
export interface CacheData {
name: string;
isLocal: boolean;
fileCount: number;
/** Server archives: the signature this entry was built from. */
signature?: string;
posts: Post[];
stories: Post[];
/** Story-highlight items, grouped by `highlightTitle`. */
highlights?: Post[];
profileMetadata: ProfileMetadata;
timestamp: number;
/**
* Local archives only: whether a FileSystemDirectoryHandle was stored
* alongside this entry, meaning media URLs can be rehydrated without
* re-prompting for the folder.
*/
hasDirectoryHandle?: boolean;
/** Path of the profile picture, for rehydration (local archives). */
profilePicPath?: string;
}
+12
View File
@@ -0,0 +1,12 @@
import 'react';
declare module 'react' {
interface InputHTMLAttributes<T> {
/**
* Non-standard attribute that makes a file input select a whole directory.
* Supported in Chromium and WebKit; used as the fallback picker where the
* File System Access API is unavailable.
*/
webkitdirectory?: string;
}
}
+9 -26
View File
@@ -1,17 +1,16 @@
import tailwindcss from '@tailwindcss/vite';
import react from '@vitejs/plugin-react';
import path from 'path';
import {defineConfig, loadEnv} from 'vite';
import {defineConfig} from 'vite';
import { VitePWA } from 'vite-plugin-pwa';
export default defineConfig(({mode}) => {
const env = loadEnv(mode, '.', '');
export default defineConfig(() => {
return {
plugins: [
react(),
tailwindcss(),
VitePWA({
registerType: 'prompt',
registerType: 'autoUpdate',
manifest: {
name: 'InstaArchive',
short_name: 'InstaArchive',
@@ -21,42 +20,26 @@ export default defineConfig(({mode}) => {
display: 'standalone',
icons: [
{
src: 'https://cdn-icons-png.flaticon.com/512/174/174855.png',
src: '/icon-512.png',
sizes: '512x512',
type: 'image/png',
purpose: 'any maskable'
},
{
src: 'https://cdn-icons-png.flaticon.com/192/174/174855.png',
src: '/icon-192.png',
sizes: '192x192',
type: 'image/png'
}
]
},
workbox: {
globPatterns: ['**/*.{js,css,html,ico,png,svg,woff,woff2}'],
runtimeCaching: [
{
urlPattern: /^https:\/\/fonts\.googleapis\.com\/.*/i,
handler: 'CacheFirst',
options: {
cacheName: 'google-fonts-cache',
expiration: {
maxEntries: 10,
maxAgeSeconds: 60 * 60 * 24 * 365
},
cacheableResponse: {
statuses: [0, 200]
}
}
}
]
navigateFallbackDenylist: [/^\/api/, /^\/archives/],
// Fonts and icons are bundled locally, so everything the shell needs
// is precached and no runtime third-party caching rule is required.
globPatterns: ['**/*.{js,css,html,ico,png,svg,woff,woff2}']
}
})
],
define: {
'process.env.GEMINI_API_KEY': JSON.stringify(env.GEMINI_API_KEY),
},
resolve: {
alias: {
'@': path.resolve(__dirname, '.'),