4 Commits

Author SHA1 Message Date
ergosteur
b30285fe70 docs: update documentation for high-res performance and local persistence
Key changes:
- Updated README.md and GEMINI.md with details on background thumbnailing and inter-post preloading.
- Documented persistent local archive caching and smart profile fallback features.
- Added dist-server/ to .gitignore.
- Restored missing feature descriptions and troubleshooting tips in README.
2026-03-07 21:59:43 -05:00
ergosteur
20209bcad5 feat: enable persistent local archives and smart profile fallback
Key changes:
- Enabled full metadata caching for local folder archives, allowing them to load instantly from IndexedDB without re-uploading.
- Implemented oldest-image fallback for profiles missing an explicit profile picture.
- Restored folder-name-to-username detection for local archive uploads.
- Optimized scan indexing to track all image files for fallback use.
2026-03-07 21:56:25 -05:00
ergosteur
74902234b3 fix: restore white glass scanning UI and resolve small image blur bug
Key changes:
- Corrected logic in PostThumbnail to prevent blur effects on images smaller than 1MiB.
- Restored the white glass aesthetic to the scanning dashboard with improved contrast and transparency.
- Optimized scanning background transitions to ensure a smooth, flicker-free crossfade.
2026-03-07 21:46:11 -05:00
ergosteur
d62bddc3aa perf: implement background thumbnail generation and inter-post preloading
Key changes:
- Added Web Worker for background image thumbnailing with a 1MiB threshold to optimize CPU/memory usage.
- Implemented a serial task queue for memory-safe high-res image processing, preventing OOM crashes.
- Added inter-post preloading in the modal for seamless 'Previous/Next' navigation.
- Refined scanning UI with double-buffering and a dark background to completely eliminate white flashes.
- Renamed project to 'instaarchive-viewer' in package.json.
- Fixed 'Open image in new tab' by denylisting /archives and /api in PWA config.
2026-03-07 21:42:56 -05:00
14 changed files with 436 additions and 117 deletions

1
.gitignore vendored
View File

@@ -1,6 +1,7 @@
node_modules/ node_modules/
build/ build/
dist/ dist/
dist-server/
coverage/ coverage/
.DS_Store .DS_Store
*.log *.log

View File

@@ -6,16 +6,20 @@
### Key Technical Features ### Key Technical Features
- **Permalinks:** Full synchronization between application state and URL query parameters (`?a=`, `?t=`, `?p=`). Supports deep-linking to archives, tabs, and specific posts. URL parameters are automatically cleaned when navigating back to the archive explorer. - **Permalinks:** Full synchronization between application state and URL query parameters (`?a=`, `?t=`, `?p=`). Supports deep-linking to archives, tabs, and specific posts. URL parameters are automatically cleaned when navigating back to the archive explorer.
- **Persistent Caching:** Uses `idb-keyval` (IndexedDB) to cache parsed metadata. Subsequent loads of the same archive are near-instant. The cache schema includes `profileMetadata` with consolidated user info and profile picture history. - **Persistent Caching:** Uses `idb-keyval` (IndexedDB) to cache parsed metadata. Subsequent loads are near-instant.
- **Modular Scanning Logic:** High-performance archive scanning encapsulated in the `useArchiveScanner` hook. It handles multi-format detection (Instagram Export, Instaloader, JSON), batch processing, and yields to the main thread to prevent UI freezing. - **Metadata:** Caches profile info and post lists for both remote AND local archives.
- **Thumbnails:** High-res images (>1MiB) and videos have thumbnails generated and cached in IndexedDB with the `thumb_` prefix.
- **Background Media Processing:**
- **High-Res Images:** A dedicated Web Worker (`thumbnail-worker.ts`) handles image resizing using `OffscreenCanvas` and `createImageBitmap` to prevent main-thread jank.
- **Serial Queue:** A memory-safe queue ensures only one high-res image is decoded at a time, preventing Out-of-Memory (OOM) crashes on 50MP+ files.
- **High-Performance Carousel:** Advanced `PostModal` with: - **High-Performance Carousel:** Advanced `PostModal` with:
- **Preloading:** Intelligently preloads the first two slides immediately, followed by a background preload of the entire carousel. - **Inter-Post Preloading:** Preloads the first media of adjacent posts for instant navigation.
- **Seamless Transitions:** Zero-latency slide transitions with optimized Framer Motion variants, removing "black flashes" between images. - **Intra-Carousel Preloading:** Intelligently preloads the current carousel's slides.
- **Async Decoding:** Utilizes `decoding="async"` to offload image processing from the main thread. - **Seamless Transitions:** Zero-latency slide transitions without "black flashes" between images.
- **Glassy Scanner UI:** Custom-built glassmorphism scanning dashboard with throttled (1s) dynamic blurred backgrounds and a high-density system log. - **Glassy Scanner UI:** Custom-built glassmorphism scanning dashboard with double-buffering logic to ensure smooth, flicker-free background crossfades during file indexing.
- **PWA Auto-Updates:** Configured with `autoUpdate` behavior and a periodic (hourly) update check to ensure long-running sessions and installed PWAs always have the latest code. - **PWA Capabilities:**
- **Compressed Metadata:** Support for `.json.xz` file decompression using `xz-decompress` (WASM-powered). - **Auto-Updates:** Hourly periodic update checks.
- **Production-Ready Docker:** Multi-stage Docker builds using `node:slim` serving both the Express API and the Vite-built frontend. - **Navigation Fix:** `navigateFallbackDenylist` allows direct server access to `/archives/` and `/api/` (enabling "Open in new tab" for original files).
### Main Technologies ### Main Technologies
- **Frontend:** React 19, Vite 6, TypeScript - **Frontend:** React 19, Vite 6, TypeScript
@@ -23,14 +27,14 @@
- **Icons:** Lucide React - **Icons:** Lucide React
- **Animations:** Framer Motion (`motion/react`) - **Animations:** Framer Motion (`motion/react`)
- **Persistence:** IndexedDB (`idb-keyval`) - **Persistence:** IndexedDB (`idb-keyval`)
- **Backend:** Express, tsx (for server-side scanning) - **Backend:** Express, tsx
- **Decompression:** xz-decompress (WASM) - **Workers:** Web Workers for background image processing.
## Architecture ## Architecture
### State Management ### State Management
- **`useArchiveScanner` Hook:** Centralized logic for parsing archives and managing results (`allPosts`, `allStories`, `profileMetadata`). - **`useArchiveScanner` Hook:** Centralized logic for parsing and caching. It handles folder-name-to-username detection and "Smart Fallback" profile pictures (using the oldest image if no profile pic is found).
- **Archive Interface:** Unified `ArchiveFile` interface implemented by `LocalArchiveFile` (for browser `File` objects) and `RemoteArchiveFile` (for server-side assets). - **`useThumbnailQueue` Hook:** Manages the serial processing of high-resolution media.
### Cache Schema ### Cache Schema
```typescript ```typescript
@@ -38,8 +42,8 @@ interface CacheData {
name: string; name: string;
isLocal: boolean; isLocal: boolean;
fileCount: number; fileCount: number;
posts: Post[]; // Remote archives only (Local archives re-parsed for security) posts: Post[]; // Cached for all archive types
stories: Post[]; // Remote archives only stories: Post[];
profileMetadata: { profileMetadata: {
username: string; username: string;
fullName: string; fullName: string;
@@ -55,15 +59,11 @@ interface CacheData {
``` ```
## Commands ## Commands
- `npm install`: Install project dependencies. - `npm install`: Install dependencies.
- `npm run dev`: Start the local development server on port 3000. - `npm run dev`: Start dev server (Port 3000).
- `npm run build`: Generate the production-ready build in the `dist` folder and server in `dist-server`. - `npm run build`: Build frontend (`dist/`) and server (`dist-server/`).
- `npm run server`: Start the backend server to scan `./_sample-archives`. - `npm run server`: Start production-ready backend.
- `npm run lint`: Execute TypeScript type-checking. - `npm run lint`: Execute TypeScript type-checking.
## Production Deployment ## Production Deployment
The project is containerized and available on GHCR. It expects a volume mount at `/archives` containing subdirectories for each user. The project is containerized. It expects a volume mount at `/archives` containing subdirectories for each user.
### Key Environment Variables
- `PORT`: Server port (default: 3000)
- `ARCHIVES_DIR`: Path to the archives collection (default: /archives)

View File

@@ -4,14 +4,16 @@ A high-performance React PWA for browsing archived Instagram data with a native-
## Features ## Features
- **Advanced Carousel**: Seamless, zero-latency transitions between slides with intelligent preloading. Images use asynchronous decoding to keep the UI smooth during motion. - **Advanced Carousel**: Seamless, zero-latency transitions between slides with intelligent preloading. Navigating between different posts is now near-instant thanks to inter-post background preloading.
- **Persistent Caching**: Uses IndexedDB to store parsed archives locally. Subsequent loads are near-instant, with full support for profile metadata and profile picture history. - **High-Res Performance**: Handles 50MP+ images effortlessly using a background Web Worker and a memory-safe serial processing queue.
- **Persistent Local Caching**: Uses IndexedDB to store parsed archives and generated thumbnails. **Local folders** now load instantly from cache on return visits without needing to re-upload files.
- **Permalinks**: State is synchronized with the URL, allowing you to share direct links to archives, tabs, or specific posts. Navigating back to the explorer cleans up URL parameters automatically. - **Permalinks**: State is synchronized with the URL, allowing you to share direct links to archives, tabs, or specific posts. Navigating back to the explorer cleans up URL parameters automatically.
- **Glassy Scanning UI**: A modern, translucent white terminal experience with a dynamic blurred background generated from your media during scanning. - **Glassy Scanning UI**: A refined, translucent white terminal experience with flicker-free, double-buffered dynamic blurred backgrounds.
- **PWA with Auto-Update**: Fully offline-capable and installable. Clients automatically receive updates when a new version is deployed to the server. - **PWA with Auto-Update**: Fully offline-capable and installable. Clients automatically receive updates when a new version is deployed to the server.
- **Local Privacy**: All processing is done client-side. Even when using the self-hosted version, your media is processed locally in your browser and never uploaded. - **Local Privacy**: All processing is done client-side. Even when using the self-hosted version, your media is processed locally in your browser and never uploaded.
- **Story Viewer**: Native-like story experience with segmented progress bars, auto-playback, and audio controls. - **Smart Fallbacks**: Automatically detects usernames from folder names and uses the oldest archive image as a profile picture if one is missing.
- **Customizable Grid**: 1:1 or 3:4 aspect ratios with adjustable "bumps" for aesthetic alignment. - **Customizable Grid**: 1:1 or 3:4 aspect ratios with adjustable "bumps" for aesthetic alignment.
- **Story Viewer**: Native-like story experience with segmented progress bars, auto-playback, and audio controls.
- **Navigation Protection**: Intercepts accidental browser "Back" or "Refresh" actions to protect your current session. - **Navigation Protection**: Intercepts accidental browser "Back" or "Refresh" actions to protect your current session.
## Deployment ## Deployment

View File

@@ -1,5 +1,5 @@
{ {
"name": "react-example", "name": "instaarchive-viewer",
"private": true, "private": true,
"version": "1.2.0", "version": "1.2.0",
"type": "module", "type": "module",

View File

@@ -23,8 +23,9 @@ import { Post, ServerArchive } from './types';
import { ArchiveDashboard } from './components/ArchiveDashboard'; import { ArchiveDashboard } from './components/ArchiveDashboard';
import { StoryViewer } from './components/StoryViewer'; import { StoryViewer } from './components/StoryViewer';
import { PostModal } from './components/PostModal'; import { PostModal } from './components/PostModal';
import { VideoThumbnail } from './components/VideoThumbnail'; import { PostThumbnail } from './components/PostThumbnail';
import { useArchiveScanner } from './hooks/useArchiveScanner'; import { useArchiveScanner } from './hooks/useArchiveScanner';
import { useThumbnailQueue } from './hooks/useThumbnailQueue';
export default function App() { export default function App() {
const [showStoryViewer, setShowStoryViewer] = useState(false); const [showStoryViewer, setShowStoryViewer] = useState(false);
@@ -43,6 +44,8 @@ export default function App() {
const fileInputRef = useRef<HTMLInputElement>(null); const fileInputRef = useRef<HTMLInputElement>(null);
const profilePicInputRef = useRef<HTMLInputElement>(null); const profilePicInputRef = useRef<HTMLInputElement>(null);
const { cacheHits, requestThumbnail } = useThumbnailQueue();
const refreshCachedArchives = useCallback(async () => { const refreshCachedArchives = useCallback(async () => {
try { try {
const keys = await idb.keys(); const keys = await idb.keys();
@@ -82,6 +85,8 @@ export default function App() {
resetScannerState resetScannerState
} = useArchiveScanner('', currentArchive, refreshCachedArchives); } = useArchiveScanner('', currentArchive, refreshCachedArchives);
const [lastLoadedScanningImage, setLastLoadedScanningImage] = useState<string | null>(null);
const { const {
username, username,
fullName, fullName,
@@ -187,6 +192,37 @@ export default function App() {
} }
}, [handleFiles, setAllPosts, setAllStories, setProfileMetadata, setIsScanning, setScanningPhase]); }, [handleFiles, setAllPosts, setAllStories, setProfileMetadata, setIsScanning, setScanningPhase]);
const loadLocalCachedArchive = useCallback(async (archive: any) => {
console.log(`[Cache] Loading local archive from cache: ${archive.name}`);
setIsScanning(true);
setCurrentArchive(null);
setScanningPhase('Checking Cache');
try {
// Small delay for UI transition
await new Promise(resolve => setTimeout(resolve, 300));
setAllPosts(archive.posts || []);
setAllStories(archive.stories || []);
const profileMetadata = { ...archive.profileMetadata };
if (!profileMetadata.allProfilePics && archive.allProfilePics) {
profileMetadata.allProfilePics = archive.allProfilePics;
}
if (!profileMetadata.allProfilePics) {
profileMetadata.allProfilePics = profileMetadata.profilePic ? [profileMetadata.profilePic] : [];
}
setProfileMetadata(profileMetadata);
setVisiblePostsCount(90);
setIsScanning(false);
console.log(`[Cache] Local archive ${archive.name} restored from cache.`);
} catch (err) {
console.error('[Cache] Failed to restore local archive:', err);
setIsScanning(false);
}
}, [setAllPosts, setAllStories, setProfileMetadata, setIsScanning, setScanningPhase]);
const handleLocalFiles = (files: FileList | null) => { if (!files) return; const archiveFiles = Array.from(files).map(f => new LocalArchiveFile(f)); handleFiles(archiveFiles); }; const handleLocalFiles = (files: FileList | null) => { if (!files) return; const archiveFiles = Array.from(files).map(f => new LocalArchiveFile(f)); handleFiles(archiveFiles); };
const triggerFileSelect = () => fileInputRef.current?.click(); const triggerFileSelect = () => fileInputRef.current?.click();
const loadMore = () => setVisiblePostsCount(prev => prev + 90); const loadMore = () => setVisiblePostsCount(prev => prev + 90);
@@ -274,6 +310,7 @@ export default function App() {
cachedArchives={cachedArchives} cachedArchives={cachedArchives}
onSelect={loadServerArchive} onSelect={loadServerArchive}
onLocalSelect={triggerFileSelect} onLocalSelect={triggerFileSelect}
onLocalCacheSelect={loadLocalCachedArchive}
onClearCache={clearCache} onClearCache={clearCache}
isScanning={isScanning} isScanning={isScanning}
/> />
@@ -285,9 +322,27 @@ export default function App() {
</div> </div>
) )
) : isScanning ? ( ) : isScanning ? (
<div className="fixed inset-0 z-50 flex items-center justify-center overflow-hidden text-black"> <div className="fixed inset-0 z-50 flex items-center justify-center overflow-hidden text-black bg-[#f8fafc]">
<AnimatePresence>{currentScanningImage && (<motion.div key={currentScanningImage} initial={{ opacity: 0 }} animate={{ opacity: 0.4 }} exit={{ opacity: 0 }} className="absolute inset-0 z-0 text-black"><img src={currentScanningImage} alt="" className="w-full h-full object-cover blur-[100px] scale-110 text-black" /></motion.div>)}</AnimatePresence> {currentScanningImage && (
<div className="absolute inset-0 bg-white/20 z-1 text-black" /> <img
src={currentScanningImage}
className="hidden"
onLoad={() => setLastLoadedScanningImage(currentScanningImage)}
/>
)}
<div className="absolute inset-0 z-0">
<AnimatePresence initial={false}>
<motion.img
key={lastLoadedScanningImage}
src={lastLoadedScanningImage || undefined}
initial={{ opacity: 0 }}
animate={{ opacity: 0.4 }}
transition={{ duration: 1.5 }}
className="absolute inset-0 w-full h-full object-cover blur-[60px] scale-110"
/>
</AnimatePresence>
</div>
<div className="absolute inset-0 bg-white/40 z-1" />
<div className="relative z-10 w-full max-w-4xl px-4 flex flex-col items-center gap-8 text-black"> <div className="relative z-10 w-full max-w-4xl px-4 flex flex-col items-center gap-8 text-black">
<div className="text-center space-y-2 text-black"><div className="text-4xl font-bold tracking-tight italic font-serif text-black/80 drop-shadow-sm text-black">Scanning Archive...</div><div className="flex items-center justify-center gap-3 text-black"><div className="h-[1px] w-12 bg-black/10 text-black" /><p className="text-black/40 text-[10px] uppercase tracking-[0.3em] font-bold text-black">{scanningPhase === 'Indexing' ? 'Building file index' : 'Parsing metadata & media'}</p><div className="h-[1px] w-12 bg-black/10 text-black" /></div></div> <div className="text-center space-y-2 text-black"><div className="text-4xl font-bold tracking-tight italic font-serif text-black/80 drop-shadow-sm text-black">Scanning Archive...</div><div className="flex items-center justify-center gap-3 text-black"><div className="h-[1px] w-12 bg-black/10 text-black" /><p className="text-black/40 text-[10px] uppercase tracking-[0.3em] font-bold text-black">{scanningPhase === 'Indexing' ? 'Building file index' : 'Parsing metadata & media'}</p><div className="h-[1px] w-12 bg-black/10 text-black" /></div></div>
<div className="w-full max-w-2xl space-y-4 text-black"><div className="flex justify-between text-[10px] font-bold uppercase tracking-widest text-black/40 px-1 text-black"><span className="flex items-center gap-2 text-black"><Loader2 size={12} className="animate-spin text-black" />Phase: {scanningPhase}</span><span className="text-black">{scannedCount} / {totalFiles}</span></div><div className="w-full h-1.5 bg-black/5 rounded-full overflow-hidden backdrop-blur-sm border border-black/5 shadow-inner text-black"><motion.div className="h-full bg-blue-500 shadow-[0_0_15px_rgba(59,130,246,0.5)] text-black" initial={{ width: 0 }} animate={{ width: `${(scannedCount / (totalFiles || 1)) * 100}%` }} transition={{ type: 'spring', bounce: 0, duration: 0.3 }} /></div></div> <div className="w-full max-w-2xl space-y-4 text-black"><div className="flex justify-between text-[10px] font-bold uppercase tracking-widest text-black/40 px-1 text-black"><span className="flex items-center gap-2 text-black"><Loader2 size={12} className="animate-spin text-black" />Phase: {scanningPhase}</span><span className="text-black">{scannedCount} / {totalFiles}</span></div><div className="w-full h-1.5 bg-black/5 rounded-full overflow-hidden backdrop-blur-sm border border-black/5 shadow-inner text-black"><motion.div className="h-full bg-blue-500 shadow-[0_0_15px_rgba(59,130,246,0.5)] text-black" initial={{ width: 0 }} animate={{ width: `${(scannedCount / (totalFiles || 1)) * 100}%` }} transition={{ type: 'spring', bounce: 0, duration: 0.3 }} /></div></div>
@@ -324,7 +379,11 @@ export default function App() {
{activeTab === 'posts' && Array.from({ length: gridOffset }).map((_, i) => (<div key={`blank-${i}`} className={cn("bg-gray-100/50 border border-dashed border-gray-200 flex items-center justify-center text-[10px] font-bold text-gray-300 uppercase tracking-tighter text-black", gridAspectRatio === '1:1' ? "aspect-square" : "aspect-[3/4]")}>Blank</div>))} {activeTab === 'posts' && Array.from({ length: gridOffset }).map((_, i) => (<div key={`blank-${i}`} className={cn("bg-gray-100/50 border border-dashed border-gray-200 flex items-center justify-center text-[10px] font-bold text-gray-300 uppercase tracking-tighter text-black", gridAspectRatio === '1:1' ? "aspect-square" : "aspect-[3/4]")}>Blank</div>))}
{visiblePosts.map((post) => ( {visiblePosts.map((post) => (
<motion.div key={post.id} layoutId={post.id} onClick={() => setSelectedPost(post)} className={cn("relative group cursor-pointer overflow-hidden bg-gray-200 transition-all duration-300 text-black", activeTab === 'reels' ? "aspect-[9/16]" : (gridAspectRatio === '1:1' ? "aspect-square" : "aspect-[3/4]"))}> <motion.div key={post.id} layoutId={post.id} onClick={() => setSelectedPost(post)} className={cn("relative group cursor-pointer overflow-hidden bg-gray-200 transition-all duration-300 text-black", activeTab === 'reels' ? "aspect-[9/16]" : (gridAspectRatio === '1:1' ? "aspect-square" : "aspect-[3/4]"))}>
{post.media[0].type === 'video' ? <VideoThumbnail url={post.media[0].url} /> : <img src={post.thumbnail} alt="" className="w-full h-full object-cover transition-transform duration-500 group-hover:scale-110 text-black" referrerPolicy="no-referrer" />} <PostThumbnail
post={post}
thumbnailUrl={cacheHits.get(post.id)}
onRequestThumbnail={requestThumbnail}
/>
<div className="absolute top-2 right-2 flex gap-1.5 z-10 text-black">{post.media.length > 1 && <div className="bg-black/40 backdrop-blur-md p-1 rounded-md text-white shadow-sm text-black"><Layers size={16} /></div>}{post.media.some(m => m.type === 'video') && <div className="bg-black/40 backdrop-blur-md p-1 rounded-md text-white shadow-sm text-black"><Play size={16} fill="white" /></div>}</div> <div className="absolute top-2 right-2 flex gap-1.5 z-10 text-black">{post.media.length > 1 && <div className="bg-black/40 backdrop-blur-md p-1 rounded-md text-white shadow-sm text-black"><Layers size={16} /></div>}{post.media.some(m => m.type === 'video') && <div className="bg-black/40 backdrop-blur-md p-1 rounded-md text-white shadow-sm text-black"><Play size={16} fill="white" /></div>}</div>
<div className="absolute inset-0 bg-black/30 opacity-0 group-hover:opacity-100 transition-opacity flex items-center justify-center gap-6 text-white font-bold z-20 text-black"><div className="flex items-center gap-2 text-black"><Heart fill="white" size={20} className="text-black" /><span>-</span></div><div className="flex items-center gap-2 text-black"><MessageCircle fill="white" size={20} className="text-black" /><span>-</span></div></div> <div className="absolute inset-0 bg-black/30 opacity-0 group-hover:opacity-100 transition-opacity flex items-center justify-center gap-6 text-white font-bold z-20 text-black"><div className="flex items-center gap-2 text-black"><Heart fill="white" size={20} className="text-black" /><span>-</span></div><div className="flex items-center gap-2 text-black"><MessageCircle fill="white" size={20} className="text-black" /><span>-</span></div></div>
</motion.div> </motion.div>
@@ -335,7 +394,21 @@ export default function App() {
)} )}
</main> </main>
<AnimatePresence>{selectedPost && <PostModal post={selectedPost} onClose={() => setSelectedPost(null)} onNextPost={onNextPost} onPrevPost={onPrevPost} hasNextPost={postIndex < filteredPosts.length - 1} hasPrevPost={postIndex > 0} profilePic={profilePic} />}</AnimatePresence> <AnimatePresence>
{selectedPost && (
<PostModal
post={selectedPost}
nextPost={postIndex < filteredPosts.length - 1 ? filteredPosts[postIndex + 1] : undefined}
prevPost={postIndex > 0 ? filteredPosts[postIndex - 1] : undefined}
onClose={() => setSelectedPost(null)}
onNextPost={onNextPost}
onPrevPost={onPrevPost}
hasNextPost={postIndex < filteredPosts.length - 1}
hasPrevPost={postIndex > 0}
profilePic={profilePic}
/>
)}
</AnimatePresence>
<AnimatePresence>{showStoryViewer && allStories.length > 0 && <StoryViewer stories={allStories} onClose={() => setShowStoryViewer(false)} profilePic={profilePic} />}</AnimatePresence> <AnimatePresence>{showStoryViewer && allStories.length > 0 && <StoryViewer stories={allStories} onClose={() => setShowStoryViewer(false)} profilePic={profilePic} />}</AnimatePresence>
{!isScanning && ( {!isScanning && (

View File

@@ -14,6 +14,7 @@ interface ArchiveDashboardProps {
cachedArchives: Set<string>; cachedArchives: Set<string>;
onSelect: (archive: ServerArchive) => void; onSelect: (archive: ServerArchive) => void;
onLocalSelect: () => void; onLocalSelect: () => void;
onLocalCacheSelect: (archive: any) => void;
onClearCache: (name: string) => void; onClearCache: (name: string) => void;
isScanning: boolean; isScanning: boolean;
} }
@@ -24,6 +25,7 @@ export const ArchiveDashboard: React.FC<ArchiveDashboardProps> = ({
cachedArchives, cachedArchives,
onSelect, onSelect,
onLocalSelect, onLocalSelect,
onLocalCacheSelect,
onClearCache, onClearCache,
isScanning isScanning
}) => { }) => {
@@ -107,7 +109,7 @@ export const ArchiveDashboard: React.FC<ArchiveDashboardProps> = ({
{localArchives.map((archive) => ( {localArchives.map((archive) => (
<div key={archive.name} className="relative group text-black"> <div key={archive.name} className="relative group text-black">
<button <button
onClick={onLocalSelect} onClick={() => onLocalCacheSelect(archive)}
disabled={isScanning} disabled={isScanning}
className="w-full aspect-[3/4] rounded-xl overflow-hidden bg-white shadow-sm border border-gray-100 hover:shadow-xl hover:scale-[1.02] transition-all flex flex-col text-left disabled:opacity-50" className="w-full aspect-[3/4] rounded-xl overflow-hidden bg-white shadow-sm border border-gray-100 hover:shadow-xl hover:scale-[1.02] transition-all flex flex-col text-left disabled:opacity-50"
> >

View File

@@ -17,6 +17,8 @@ import { MediaRenderer } from './MediaRenderer';
interface PostModalProps { interface PostModalProps {
post: Post; post: Post;
nextPost?: Post;
prevPost?: Post;
onClose: () => void; onClose: () => void;
onNextPost?: () => void; onNextPost?: () => void;
onPrevPost?: () => void; onPrevPost?: () => void;
@@ -26,7 +28,7 @@ interface PostModalProps {
} }
export const PostModal: React.FC<PostModalProps> = ({ export const PostModal: React.FC<PostModalProps> = ({
post, onClose, onNextPost, onPrevPost, hasNextPost, hasPrevPost, profilePic post, nextPost, prevPost, onClose, onNextPost, onPrevPost, hasNextPost, hasPrevPost, profilePic
}) => { }) => {
const [currentIndex, setCurrentIndex] = useState(0); const [currentIndex, setCurrentIndex] = useState(0);
const [direction, setDirection] = useState(0); const [direction, setDirection] = useState(0);
@@ -34,32 +36,34 @@ export const PostModal: React.FC<PostModalProps> = ({
// Preloading Logic // Preloading Logic
useEffect(() => { useEffect(() => {
const controller = new AbortController(); const controller = new AbortController();
const preloadMedia = async (index: number) => {
if (index < 0 || index >= post.media.length) return; const preloadMedia = async (url: string, type: 'image' | 'video') => {
const media = post.media[index]; if (!url) return;
if (!media.url) return;
try { try {
if (media.type === 'image') { if (type === 'image') {
const img = new Image(); const img = new Image();
img.src = media.url; img.src = url;
} else { } else {
const video = document.createElement('video'); const video = document.createElement('video');
video.src = media.url; video.src = url;
video.preload = 'auto'; video.preload = 'auto';
} }
} catch (e) {} } catch (e) {}
}; };
// 1. Immediate preload of first two slides // 1. Current post: Immediate preload of first two slides
preloadMedia(0); if (post.media[0]) preloadMedia(post.media[0].url, post.media[0].type);
preloadMedia(1); if (post.media[1]) preloadMedia(post.media[1].url, post.media[1].type);
// 2. Delayed preload of the rest to stay out of the way of initial render // 2. Next/Prev posts: Preload their first slides
if (nextPost?.media[0]) preloadMedia(nextPost.media[0].url, nextPost.media[0].type);
if (prevPost?.media[0]) preloadMedia(prevPost.media[0].url, prevPost.media[0].type);
// 3. Current post: Delayed preload of the rest
const timeout = setTimeout(() => { const timeout = setTimeout(() => {
for (let i = 2; i < post.media.length; i++) { for (let i = 2; i < post.media.length; i++) {
if (controller.signal.aborted) break; if (controller.signal.aborted) break;
preloadMedia(i); preloadMedia(post.media[i].url, post.media[i].type);
} }
}, 1000); }, 1000);
@@ -67,7 +71,7 @@ export const PostModal: React.FC<PostModalProps> = ({
controller.abort(); controller.abort();
clearTimeout(timeout); clearTimeout(timeout);
}; };
}, [post.id, post.media]); }, [post.id, post.media, nextPost?.id, prevPost?.id]);
useEffect(() => setCurrentIndex(0), [post.id]); useEffect(() => setCurrentIndex(0), [post.id]);
useEffect(() => { useEffect(() => {

View File

@@ -0,0 +1,124 @@
import React, { useState, useEffect, useRef } from 'react';
import { Play, Image as ImageIcon } from 'lucide-react';
import { cn } from '../lib/utils';
import { Post } from '../types';
interface PostThumbnailProps {
post: Post;
className?: string;
thumbnailUrl?: string; // High-res thumbnail from queue
onRequestThumbnail: (id: string, url: string) => void;
}
const videoThumbnailCache = new Map<string, string>();
export const PostThumbnail = ({ post, className, thumbnailUrl, onRequestThumbnail }: PostThumbnailProps) => {
const [videoThumbnail, setVideoThumbnail] = useState<string | null>(videoThumbnailCache.get(post.media[0].url) || null);
const [isInView, setIsInView] = useState(false);
const containerRef = useRef<HTMLDivElement>(null);
const mainMedia = post.media[0];
const isVideo = mainMedia.type === 'video';
useEffect(() => {
const observer = new IntersectionObserver(([entry]) => {
if (entry.isIntersecting) {
setIsInView(true);
observer.disconnect();
}
}, { rootMargin: '400px' }); // Larger margin for smoother scrolling
if (containerRef.current) {
observer.observe(containerRef.current);
}
return () => observer.disconnect();
}, []);
useEffect(() => {
if (!isInView) return;
if (isVideo) {
if (videoThumbnail) return;
const video = document.createElement('video');
video.src = `${mainMedia.url}#t=0.1`;
video.preload = 'metadata';
video.muted = true;
video.playsInline = true;
const captureFrame = () => {
try {
const canvas = document.createElement('canvas');
canvas.width = video.videoWidth; canvas.height = video.videoHeight;
const ctx = canvas.getContext('2d');
if (ctx && video.videoWidth > 0) {
ctx.drawImage(video, 0, 0, canvas.width, canvas.height);
const dataUrl = canvas.toDataURL('image/jpeg', 0.6);
videoThumbnailCache.set(mainMedia.url, dataUrl);
setVideoThumbnail(dataUrl);
}
} catch (err) {} finally { cleanup(); }
};
const handleLoadedMetadata = () => video.currentTime = 0.1;
const handleSeeked = () => captureFrame();
const cleanup = () => {
video.removeEventListener('loadedmetadata', handleLoadedMetadata);
video.removeEventListener('seeked', handleSeeked);
video.removeAttribute('src');
video.load();
};
video.addEventListener('loadedmetadata', handleLoadedMetadata);
video.addEventListener('seeked', handleSeeked);
video.addEventListener('error', cleanup);
const timeout = setTimeout(() => cleanup(), 5000);
return () => { clearTimeout(timeout); cleanup(); };
} else {
// Request high-res image thumbnailing only if size > 1MiB
const ONE_MIB = 1024 * 1024;
if (mainMedia.size && mainMedia.size > ONE_MIB) {
onRequestThumbnail(post.id, mainMedia.url);
}
}
}, [isInView, isVideo, mainMedia.url, mainMedia.size, post.id, onRequestThumbnail, videoThumbnail]);
// Determine if we are actually expecting a high-res thumbnail
const ONE_MIB = 1024 * 1024;
const isHighRes = !isVideo && mainMedia.size && mainMedia.size > ONE_MIB;
const isGenerating = isHighRes && !thumbnailUrl;
// Use high-res thumbnail if available, then video thumb, then original
const displayUrl = thumbnailUrl || videoThumbnail || post.thumbnail;
if (!displayUrl && isVideo) {
return (
<div ref={containerRef} className={cn("w-full h-full bg-gray-100 flex items-center justify-center text-black", className)}>
<Play size={20} className="text-gray-300" fill="currentColor" />
</div>
);
}
if (!displayUrl) {
return (
<div ref={containerRef} className={cn("w-full h-full bg-gray-50 flex items-center justify-center text-black", className)}>
<ImageIcon size={20} className="text-gray-200" />
</div>
);
}
return (
<div ref={containerRef} className="w-full h-full">
<img
src={displayUrl}
alt=""
className={cn(
"w-full h-full object-cover transition-all duration-700",
className,
isGenerating ? "blur-sm scale-105" : "blur-0 scale-100"
)}
referrerPolicy="no-referrer"
loading="lazy"
/>
</div>
);
};

View File

@@ -1,59 +0,0 @@
import React, { useState, useEffect, useRef } from 'react';
import { Play } from 'lucide-react';
import { cn } from '../lib/utils';
const thumbnailCache = new Map<string, string>();
export const VideoThumbnail = ({ url, className }: { url: string; className?: string }) => {
const [thumbnail, setThumbnail] = useState<string | null>(thumbnailCache.get(url) || null);
const [isInView, setIsInView] = useState(false);
const containerRef = useRef<HTMLDivElement>(null);
useEffect(() => {
if (thumbnail || !containerRef.current) return;
const observer = new IntersectionObserver(([entry]) => {
if (entry.isIntersecting) { setIsInView(true); observer.disconnect(); }
}, { rootMargin: '200px' });
observer.observe(containerRef.current);
return () => observer.disconnect();
}, [thumbnail]);
useEffect(() => {
if (thumbnail || !isInView) return;
const video = document.createElement('video');
video.src = `${url}#t=0.1`; video.preload = 'metadata'; video.muted = true; video.playsInline = true;
const captureFrame = () => {
try {
const canvas = document.createElement('canvas');
canvas.width = video.videoWidth; canvas.height = video.videoHeight;
const ctx = canvas.getContext('2d');
if (ctx && video.videoWidth > 0) {
ctx.drawImage(video, 0, 0, canvas.width, canvas.height);
const dataUrl = canvas.toDataURL('image/jpeg', 0.6);
thumbnailCache.set(url, dataUrl); setThumbnail(dataUrl);
}
} catch (err) {} finally { cleanup(); }
};
const handleLoadedMetadata = () => video.currentTime = 0.1;
const handleSeeked = () => captureFrame();
const cleanup = () => {
video.removeEventListener('loadedmetadata', handleLoadedMetadata);
video.removeEventListener('seeked', handleSeeked);
video.removeAttribute('src');
video.load();
};
video.addEventListener('loadedmetadata', handleLoadedMetadata);
video.addEventListener('seeked', handleSeeked);
video.addEventListener('error', cleanup);
const timeout = setTimeout(() => { if (!thumbnailCache.has(url)) cleanup(); }, 5000);
return () => { clearTimeout(timeout); cleanup(); };
}, [url, thumbnail, isInView]);
if (!thumbnail) return (
<div ref={containerRef} className={cn("w-full h-full bg-gray-100 flex items-center justify-center text-black", className)}>
<Play size={20} className="text-gray-300" fill="currentColor" />
</div>
);
return <img src={thumbnail} alt="" className={cn("w-full h-full object-cover transition-transform duration-500 group-hover:scale-110", className)} referrerPolicy="no-referrer" />;
};

View File

@@ -91,6 +91,7 @@ export const useArchiveScanner = (
const postsMap = new Map<string, Partial<Post>>(); const postsMap = new Map<string, Partial<Post>>();
const mediaFilesMap = new Map<string, ArchiveFile>(); const mediaFilesMap = new Map<string, ArchiveFile>();
const discoveredProfilePics: { name: string, url: string }[] = []; const discoveredProfilePics: { name: string, url: string }[] = [];
const allImageFiles: ArchiveFile[] = [];
let localFullName = ''; let localFullName = '';
let localBio = ''; let localBio = '';
@@ -107,7 +108,17 @@ export const useArchiveScanner = (
return (obj.is_story === true || obj.is_reel_media === true || typeName.includes('Story') || obj.audience === "MediaAudience.DEFAULT" || obj.node_type === "StoryItem" || obj.product_type === "story" || typeName === "GraphStoryVideo" || typeName === "GraphStoryImage"); return (obj.is_story === true || obj.is_reel_media === true || typeName.includes('Story') || obj.audience === "MediaAudience.DEFAULT" || obj.node_type === "StoryItem" || obj.product_type === "story" || typeName === "GraphStoryVideo" || typeName === "GraphStoryImage");
}; };
let currentUsername = archiveContext?.name || currentArchive?.name || detectedUsername || ''; let currentUsername = archiveContext?.name || currentArchive?.name || detectedUsername;
// If still no username (likely local folder), try to extract from path
if (!currentUsername && files[0]?.webkitRelativePath) {
const pathParts = files[0].webkitRelativePath.split(/[/\\]/);
if (pathParts.length > 1) {
currentUsername = pathParts[0];
}
}
if (!currentUsername) currentUsername = 'archived_user';
let format: 'export' | 'instaloader' | 'json' | 'unknown' = 'unknown'; let format: 'export' | 'instaloader' | 'json' | 'unknown' = 'unknown';
let jsonFiles: ArchiveFile[] = []; let jsonFiles: ArchiveFile[] = [];
@@ -145,6 +156,7 @@ export const useArchiveScanner = (
if (isMedia(file.name)) { if (isMedia(file.name)) {
mediaFilesMap.set(file.webkitRelativePath || file.name, file); mediaFilesMap.set(file.webkitRelativePath || file.name, file);
if (isImage(file.name)) allImageFiles.push(file);
} }
} }
@@ -195,8 +207,8 @@ export const useArchiveScanner = (
const type = isVideo(matchedFile.name) ? 'video' : 'image'; const type = isVideo(matchedFile.name) ? 'video' : 'image';
const url = matchedFile.url || URL.createObjectURL(new Blob([await matchedFile.arrayBuffer()], { type: type === 'video' ? 'video/mp4' : 'image/jpeg' })); const url = matchedFile.url || URL.createObjectURL(new Blob([await matchedFile.arrayBuffer()], { type: type === 'video' ? 'video/mp4' : 'image/jpeg' }));
const existingMedia = post.media!.find(media => media.index === mIdx + 1); const existingMedia = post.media!.find(media => media.index === mIdx + 1);
if (existingMedia) { if (type === 'video' && existingMedia.type === 'image') post.media = post.media!.map(media => media.index === mIdx + 1 ? { name: matchedFile!.name, url, type, index: mIdx + 1 } : media); } if (existingMedia) { if (type === 'video' && existingMedia.type === 'image') post.media = post.media!.map(media => media.index === mIdx + 1 ? { name: matchedFile!.name, url, type, index: mIdx + 1, size: matchedFile!.size } : media); }
else post.media!.push({ name: matchedFile.name, url, type, index: mIdx + 1 }); else post.media!.push({ name: matchedFile.name, url, type, index: mIdx + 1, size: matchedFile.size });
} }
} }
if (post.media!.length > 0) postsMap.set(postId, post); if (post.media!.length > 0) postsMap.set(postId, post);
@@ -251,8 +263,8 @@ export const useArchiveScanner = (
const url = file.url || URL.createObjectURL(new Blob([await file.arrayBuffer()], { type: type === 'video' ? 'video/mp4' : 'image/jpeg' })); const url = file.url || URL.createObjectURL(new Blob([await file.arrayBuffer()], { type: type === 'video' ? 'video/mp4' : 'image/jpeg' }));
if (type === 'image') throttledSetScanningImage(url); if (type === 'image') throttledSetScanningImage(url);
const existingMedia = post.media!.find(m => m.index === index); const existingMedia = post.media!.find(m => m.index === index);
if (existingMedia) { if (type === 'video' && existingMedia.type === 'image') post.media = post.media!.map(m => m.index === index ? { name: file.name, url, type, index } : m); } if (existingMedia) { if (type === 'video' && existingMedia.type === 'image') post.media = post.media!.map(m => m.index === index ? { name: file.name, url, type, index, size: file.size } : m); }
else post.media!.push({ name: file.name, url, type, index }); else post.media!.push({ name: file.name, url, type, index, size: file.size });
} }
} }
await new Promise(resolve => setTimeout(resolve, 0)); await new Promise(resolve => setTimeout(resolve, 0));
@@ -297,7 +309,7 @@ export const useArchiveScanner = (
const type = isVideo(file.name) ? 'video' : 'image'; const type = isVideo(file.name) ? 'video' : 'image';
const url = file.url || URL.createObjectURL(new Blob([await file.arrayBuffer()], { type: type === 'video' ? 'video/mp4' : 'image/jpeg' })); const url = file.url || URL.createObjectURL(new Blob([await file.arrayBuffer()], { type: type === 'video' ? 'video/mp4' : 'image/jpeg' }));
if (type === 'image') throttledSetScanningImage(url); if (type === 'image') throttledSetScanningImage(url);
post.media.push({ name: file.name, url, type, index: idx + 1 }); post.media.push({ name: file.name, url, type, index: idx + 1, size: file.size });
} }
post.thumbnail = post.media[0].url; post.thumbnail = post.media[0].url;
postsMap.set(postId, post); postsMap.set(postId, post);
@@ -310,6 +322,15 @@ export const useArchiveScanner = (
const urls = discoveredProfilePics.map(p => p.url); const urls = discoveredProfilePics.map(p => p.url);
localProfilePic = urls[0]; localProfilePic = urls[0];
setProfileMetadata(prev => ({ ...prev, profilePic: localProfilePic, allProfilePics: urls })); setProfileMetadata(prev => ({ ...prev, profilePic: localProfilePic, allProfilePics: urls }));
} else if (allImageFiles.length > 0) {
// Fallback: Use oldest image in archive as profile pic
allImageFiles.sort((a, b) => a.name.localeCompare(b.name));
const oldestFile = allImageFiles[0];
try {
const url = oldestFile.url || URL.createObjectURL(new Blob([await oldestFile.arrayBuffer()], { type: 'image/jpeg' }));
localProfilePic = url;
setProfileMetadata(prev => ({ ...prev, profilePic: localProfilePic, allProfilePics: [url] }));
} catch(e) {}
} }
const finalUsername = currentUsername || 'archived_user'; const finalUsername = currentUsername || 'archived_user';
@@ -358,7 +379,8 @@ export const useArchiveScanner = (
const cacheData = { const cacheData = {
name: cacheKey, isLocal, fileCount: archiveToCache ? archiveToCache.fileCount : files.length, name: cacheKey, isLocal, fileCount: archiveToCache ? archiveToCache.fileCount : files.length,
posts: isLocal ? [] : posts, stories: isLocal ? [] : stories, posts: posts, // Enable caching for local archives
stories: stories,
profileMetadata: { profileMetadata: {
username: finalUsername, username: finalUsername,
fullName: localFullName, fullName: localFullName,

View File

@@ -0,0 +1,106 @@
import { useState, useCallback, useRef, useEffect } from 'react';
import * as idb from 'idb-keyval';
interface ThumbnailRequest {
id: string;
url: string;
blob?: Blob;
}
const THUMBNAIL_WIDTH = 400;
export const useThumbnailQueue = () => {
const [cacheHits, setCacheHits] = useState<Map<string, string>>(new Map());
const queueRef = useRef<ThumbnailRequest[]>([]);
const isProcessingRef = useRef(false);
const workerRef = useRef<Worker | null>(null);
useEffect(() => {
// Initialize worker with relative URL (vite will handle this)
workerRef.current = new Worker(new URL('../lib/thumbnail-worker.ts', import.meta.url), {
type: 'module'
});
workerRef.current.onmessage = async (e) => {
const { id, blob, error } = e.data;
if (!error && blob) {
const url = URL.createObjectURL(blob);
setCacheHits(prev => new Map(prev).set(id, url));
// Persist to IndexedDB (as dataURL for simple retrieval or keep as Blob)
try {
await idb.set(`thumb_${id}`, blob);
} catch (err) {}
}
// Process next in queue
isProcessingRef.current = false;
processNext();
};
return () => {
workerRef.current?.terminate();
};
}, []);
const processNext = useCallback(async () => {
if (isProcessingRef.current || queueRef.current.length === 0 || !workerRef.current) return;
isProcessingRef.current = true;
const request = queueRef.current.shift()!;
try {
// 1. Double check cache before expensive work
const cached = await idb.get(`thumb_${request.id}`);
if (cached instanceof Blob) {
const url = URL.createObjectURL(cached);
setCacheHits(prev => new Map(prev).set(request.id, url));
isProcessingRef.current = false;
processNext();
return;
}
// 2. Fetch original if no blob provided
let blob = request.blob;
if (!blob) {
const res = await fetch(request.url);
blob = await res.blob();
}
// 3. Send to worker
workerRef.current.postMessage({
id: request.id,
blob,
width: THUMBNAIL_WIDTH
});
} catch (err) {
console.error('[ThumbnailQueue] Failed to process:', request.id, err);
isProcessingRef.current = false;
processNext();
}
}, []);
const requestThumbnail = useCallback(async (id: string, url: string, blob?: Blob) => {
// 1. Sync check state
if (cacheHits.has(id)) return;
// 2. Async check idb
const cached = await idb.get(`thumb_${id}`);
if (cached instanceof Blob) {
setCacheHits(prev => new Map(prev).set(id, URL.createObjectURL(cached)));
return;
}
// 3. Add to queue if not already there
if (!queueRef.current.some(r => r.id === id)) {
queueRef.current.push({ id, url, blob });
processNext();
}
}, [cacheHits, processNext]);
return {
cacheHits,
requestThumbnail
};
};

View File

@@ -0,0 +1,42 @@
/**
* Thumbnail Generation Worker
* Uses OffscreenCanvas and createImageBitmap for high-performance,
* background-thread image resizing.
*/
self.onmessage = async (e: MessageEvent) => {
const { id, blob, width } = e.data;
try {
// 1. Create a bitmap from the blob (native browser decoding)
// We resize it DURING the decode step for maximum efficiency
const bitmap = await createImageBitmap(blob, {
resizeWidth: width,
resizeQuality: 'medium'
});
// 2. Use OffscreenCanvas to draw the resized bitmap
const canvas = new OffscreenCanvas(bitmap.width, bitmap.height);
const ctx = canvas.getContext('2d');
if (!ctx) {
throw new Error('Could not get OffscreenCanvas context');
}
ctx.drawImage(bitmap, 0, 0);
// 3. Convert to a small JPEG blob
const thumbnailBlob = await canvas.convertToBlob({
type: 'image/jpeg',
quality: 0.7
});
// 4. Release bitmap memory
bitmap.close();
// 5. Send result back
self.postMessage({ id, blob: thumbnailBlob });
} catch (err: any) {
self.postMessage({ id, error: err.message });
}
};

View File

@@ -3,6 +3,7 @@ export interface MediaFile {
url: string; url: string;
type: 'image' | 'video'; type: 'image' | 'video';
index: number; index: number;
size?: number;
} }
export interface Post { export interface Post {

View File

@@ -34,6 +34,7 @@ export default defineConfig(({mode}) => {
] ]
}, },
workbox: { workbox: {
navigateFallbackDenylist: [/^\/api/, /^\/archives/],
globPatterns: ['**/*.{js,css,html,ico,png,svg,woff,woff2}'], globPatterns: ['**/*.{js,css,html,ico,png,svg,woff,woff2}'],
runtimeCaching: [ runtimeCaching: [
{ {