Compare commits

..
31 Commits
Author SHA1 Message Date
ergosteurandClaude Opus 5 97f5d19ce4 chore: bump version to 1.3.0
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011uBWhwV3wFQ5MBCcMHHem7
2026-08-14 01:59:41 -04:00
ergosteurandClaude Opus 5 1b4aba54d6 fix: security, performance and correctness pass; add sidecar archive support
Security
- Fix path traversal in GET /api/archives/:name/files. Express decodes route
  params after segment matching, so `..%2f..%2fetc` escaped ARCHIVES_DIR and
  returned a recursive listing of arbitrary directories.
- Add CSP and baseline security headers; disable x-powered-by.
- Stop baking GEMINI_API_KEY into the client bundle (the SDK was unused).
- Run the container as `node` instead of root.

Performance
- Add a directory-mtime-keyed archive index, warmed in the background and
  persisted. Listing 110k files went from ~52s to ~0.1s; the largest archive
  (24k files) serves in ~0.3s. Per-file stat over CIFS costs ~1.4ms and does
  not parallelise, so it is now done once rather than per request.
- Build media URLs from the File directly instead of
  `new Blob([await file.arrayBuffer()])`, which read every media file fully
  into memory (a 20GB archive tried to become 20GB of resident blobs).
- Track and revoke object URLs; previously none were ever revoked.
- Give `requestThumbnail` a stable identity so a completed thumbnail stops
  re-running the effect in every mounted thumbnail.
- Namespace IndexedDB keys so listing archives no longer deserializes every
  cached thumbnail blob, and thumbnails no longer collide across archives.
- Serve real file sizes: RemoteArchiveFile was constructed with size 0, which
  silently disabled high-res thumbnailing for every server archive.

Correctness
- Local archives cached media as blob: URLs, which die with the document, so
  a cached local archive restored as an archive of broken images. Media now
  carries a stable path and is rehydrated from a persisted directory handle
  (File System Access API), falling back to re-prompting for the folder.
- Fix permalinks: the URL-writing effect erased ?a= on mount before the
  archive list arrived to consume it, so deep links never resolved.
- Make cache invalidation detect nested changes via a directory signature.
- Add an error boundary and tolerate unparseable dates, which previously
  threw a RangeError and blanked the app.
- Default video to muted so autoplay is not blocked by Safari/Firefox.

Features
- Fold sidecar directories into their base profile: `<user> - reels`,
  `story - <user>` and `story highlights - <user> - <title>` now appear as
  reels, the story ring and Instagram-style highlight circles rather than as
  separate archives.

Housekeeping
- Add @types/react; React was previously type-checked against its JavaScript
  source, so `npm run lint` gave almost no type safety on components.
- Vendor fonts and PWA icons locally; the app made third-party CDN requests
  despite advertising offline support and local-only processing.
- Drop unused better-sqlite3 (a native module that broke `npm install`).
- Add vitest with 36 tests over the filename and directory-naming rules.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011uBWhwV3wFQ5MBCcMHHem7
2026-08-14 01:59:19 -04:00
ergosteur b30285fe70 docs: update documentation for high-res performance and local persistence
Docker Build and Publish / build-and-push (push) Failing after 10s
Key changes:
- Updated README.md and GEMINI.md with details on background thumbnailing and inter-post preloading.
- Documented persistent local archive caching and smart profile fallback features.
- Added dist-server/ to .gitignore.
- Restored missing feature descriptions and troubleshooting tips in README.
2026-03-07 21:59:43 -05:00
ergosteur 20209bcad5 feat: enable persistent local archives and smart profile fallback
Key changes:
- Enabled full metadata caching for local folder archives, allowing them to load instantly from IndexedDB without re-uploading.
- Implemented oldest-image fallback for profiles missing an explicit profile picture.
- Restored folder-name-to-username detection for local archive uploads.
- Optimized scan indexing to track all image files for fallback use.
2026-03-07 21:56:25 -05:00
ergosteur 74902234b3 fix: restore white glass scanning UI and resolve small image blur bug
Key changes:
- Corrected logic in PostThumbnail to prevent blur effects on images smaller than 1MiB.
- Restored the white glass aesthetic to the scanning dashboard with improved contrast and transparency.
- Optimized scanning background transitions to ensure a smooth, flicker-free crossfade.
2026-03-07 21:46:11 -05:00
ergosteur d62bddc3aa perf: implement background thumbnail generation and inter-post preloading
Key changes:
- Added Web Worker for background image thumbnailing with a 1MiB threshold to optimize CPU/memory usage.
- Implemented a serial task queue for memory-safe high-res image processing, preventing OOM crashes.
- Added inter-post preloading in the modal for seamless 'Previous/Next' navigation.
- Refined scanning UI with double-buffering and a dark background to completely eliminate white flashes.
- Renamed project to 'instaarchive-viewer' in package.json.
- Fixed 'Open image in new tab' by denylisting /archives and /api in PWA config.
2026-03-07 21:42:56 -05:00
ergosteur 42c13ea106 chore: bump version to 1.2.0
Docker Build and Publish / build-and-push (push) Failing after 9s
2026-03-07 21:16:39 -05:00
ergosteur a4e9ce16a7 feat: modularize scanner, enhance carousel preloading, and improve PWA updates
Summary of changes:
- Extracted archive scanning logic into a modular 'useArchiveScanner' hook for better maintainability and performance.
- Refined PostModal carousel with intelligent media preloading and smoother, jitter-free transitions.
- Optimized image rendering with 'decoding=async' and removed 'black flashes' between slide changes.
- Updated PWA configuration to 'autoUpdate' with hourly periodic checks for fresh content.
- Fixed several bugs including stories sorting, permalink parameter cleanup, and profile metadata cache restoration.
- Comprehensive updates to documentation (README.md and GEMINI.md) reflecting the new architecture.
2026-03-07 21:16:28 -05:00
ergosteur 4f89a69ee3 fix: optimize scanning performance and resolve zero-post bug
Docker Build and Publish / build-and-push (push) Failing after 10s
2026-03-07 20:25:23 -05:00
ergosteur 147dcdf2f1 fix: restore missing UI handlers and finalize generic parser
Docker Build and Publish / build-and-push (push) Failing after 10s
2026-03-07 20:18:33 -05:00
ergosteur d4e20d9b98 fix: refine dockerignore and bump version to v1.1.4
Docker Build and Publish / build-and-push (push) Failing after 10s
2026-03-07 20:10:40 -05:00
ergosteur ec8c771733 feat: implement permalinks and document PWA cache troubleshooting
Docker Build and Publish / build-and-push (push) Failing after 10s
2026-03-07 05:20:09 -05:00
ergosteur 3784e8729b debug: add verbose logging to permalink synchronization 2026-03-07 05:12:14 -05:00
ergosteur 69d62eaa5c fix: improve permalink comparison and add debug logging 2026-03-07 05:10:43 -05:00
ergosteur 767f9c508b feat: implement permalinks for archives, tabs, and posts 2026-03-07 05:07:04 -05:00
ergosteur 103ce6f207 fix: exhaustive generic parser and implement local archive history 2026-03-07 05:05:08 -05:00
ergosteur 5267dab236 fix: address Docker EACCES errors with better logging and SELinux hints
Docker Build and Publish / build-and-push (push) Failing after 9s
2026-03-07 03:02:02 -05:00
ergosteur ebf2bf660a fix: improve Docker archive discovery and switch to compiled server
Docker Build and Publish / build-and-push (push) Failing after 1m6s
2026-03-07 02:57:19 -05:00
ergosteur c0f3523a9c docs: update README and GEMINI with Docker usage and new features 2026-03-07 02:50:18 -05:00
ergosteur 6f5021638c feat: add Dockerfile and GitHub Actions workflow for GHCR deployment
Docker Build and Publish / build-and-push (push) Canceled after 11s
2026-03-07 02:45:17 -05:00
ergosteur 67f7750157 feat: refine navigation protection to only warn when leaving the app 2026-03-07 02:41:16 -05:00
ergosteur 9e306eb85e feat: add explicit confirmation for back button and refresh in archives 2026-03-07 02:39:39 -05:00
ergosteur b2da08d52d feat: add navigation protection and refine cached badge visibility 2026-03-07 02:37:17 -05:00
ergosteur d7c13ecc19 fix: resolve Firefox media warnings by improving video cleanup 2026-03-07 02:30:46 -05:00
ergosteur d396b356be feat: implement persistent caching, glassy scanning UI, and UI refinements 2026-03-07 02:28:22 -05:00
ergosteur e23dfe4474 feat: implement self-hostable mode with server-side directory scanning 2026-03-07 00:59:31 -05:00
ergosteur 41e7c5e206 docs: update README and GEMINI.md, remove AI Studio boilerplate and .env.example 2026-03-07 00:36:46 -05:00
ergosteur f685eaebd7 feat: enhance story viewer and media playback experience 2026-03-07 00:31:04 -05:00
ergosteur 47e44ec5e9 feat: improve archive parsing, add .json.xz support, and fix profile pic display 2026-03-07 00:03:54 -05:00
ergosteur cd7dc5f981 feat: Initialize InstaArchive PWA project
Sets up a new React PWA project with Vite, Tailwind CSS, and basic PWA features. Includes essential files like README, .gitignore, package.json, and initial app structure.
2026-03-06 22:41:48 -05:00
ergosteurandGitHub a724e5bc87 Initial commit 2026-03-06 22:41:33 -05:00
67 changed files with 220 additions and 7692 deletions
-2
View File
@@ -9,5 +9,3 @@ coverage/
!.env.example
_sample-archives
_gemini-plans
__pycache__/
*.pyc
+33 -131
View File
@@ -4,161 +4,63 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
## Project Overview
InstaArchive Viewer is a React 19 + Vite 6 PWA for browsing archived Instagram data (official Instagram exports and Instaloader archives). All archive *parsing* happens client-side in the browser; the Express backend only indexes and serves files from disk, and never parses archive contents.
InstaArchive Viewer is a React 19 + Vite 6 PWA for browsing archived Instagram data (both official Instagram exports and Instaloader archives). All archive parsing and media processing happens client-side in the browser — the Express backend only lists/serves files from disk, it never parses archive contents.
## Commands
- `npm install` — install dependencies
- `npm run dev` — Vite dev server on port 3000 (proxies `/api` and `/archives` to `http://localhost:3001`)
- `npm run server` — Express backend (`tsx server.ts`) on port 3001, serving `ARCHIVES_DIR` (defaults to `./_sample-archives`)
- `npm run build` — frontend to `dist/`, backend to `dist-server/`
- `npm run lint` — type-check only (`tsc --noEmit`)
- `npm test` / `npm run test:watch` — vitest
- `npx vitest run src/lib/archive-patterns.test.ts` — a single test file
- `npm run dev` — start Vite dev server on port 3000 (proxies `/api` and `/archives` to `http://localhost:3001`)
- `npm run server` — start the Express backend (`tsx server.ts`) on port 3001, serving archives from `ARCHIVES_DIR` (defaults to `./_sample-archives`)
- `npm run build` — build frontend to `dist/` (`vite build`) and backend to `dist-server/` (`tsc server.ts ...`)
- `npm run lint` — type-check only (`tsc --noEmit`); there is no separate test suite or linter config
- `npm run clean` — remove `dist/`
Local development usually needs both `npm run dev` and `npm run server`. Local-folder mode works without the backend; server-mode archives do not.
For local development you typically need both `npm run dev` and `npm run server` running concurrently — the frontend alone has nothing to talk to for server-mode archives (local-folder mode works without the backend).
## Architecture
### Two archive sources, one data model
Loading is unified behind the `ArchiveFile` interface (`src/types/index.ts`, implementations in `src/lib/archive-files.ts`):
The app supports loading archives two ways, unified behind the `ArchiveFile` interface (`src/types/index.ts`, implementations in `src/lib/archive-files.ts`):
- **`LocalArchiveFile`** — wraps a browser `File`. `createObjectUrl()` mints a **disk-backed** blob URL directly from the File; never route media through `arrayBuffer()`, which pulls whole files into memory.
- **`RemoteArchiveFile`** — wraps a file served from `/archives/...`, fetched on demand.
- **`LocalArchiveFile`** — wraps a browser `File` from a local folder picker (`webkitdirectory`). Fully offline, media is never uploaded anywhere.
- **`RemoteArchiveFile`** — wraps a file served from the Express backend's `/archives/:name/...` static route, fetched on demand.
`revocable` tells callers whether the returned URL must be revoked. The scanner tracks every minted URL and releases them on archive teardown.
### Sidecar directories
An archive root holds one directory per profile plus *sidecars* that belong to it:
```
4utumn07 -> posts (base)
4utumn07 - reels -> reels
story - 4utumn07 -> stories
story highlights - 4utumn07 - Sunstory -> highlight "Sunstory"
```
`src/lib/archive-grouping.ts` (shared by server and tests) folds these into a single profile with a `sources` list. Sidecars never appear as standalone archives. Each file the server returns carries its `kind`, so the client routes posts / reels / story ring / highlight circles without re-deriving naming rules.
### Server-side archive index (`src/lib/archive-index.ts`)
**Do not reintroduce per-request filesystem walks.** Archives typically live on network storage where per-file `stat` costs ~1.4ms and does not parallelise; a naive walk of a 110k-file root took ~52s per listing. Instead:
- Each source directory is indexed once and cached, keyed by its **directory mtime** (a directory `stat` is effectively free).
- The index is warmed in the background at startup and persisted to `CACHE_DIR` (mount a volume at `/cache`).
- `GET /api/archives` does no file walking at all — it returns directory-mtime `signature`s, which the client uses for cache invalidation instead of a file count.
- Only media files are stat'd (for `size`, which gates thumbnailing) and only highlights need `mtime` (their filenames carry no date).
All downstream parsing code (`useArchiveScanner`) operates only on `ArchiveFile[]` and doesn't care which backing implementation it got.
### Scanning pipeline (`src/hooks/useArchiveScanner.ts`)
`handleFiles` indexes files, detects format, then parses via one of three largely independent paths that all write into a shared `postsMap`:
This is the core of the app — a single large `handleFiles` function that:
1. **Indexes** all files, detecting archive format by filename regex: Instagram "export" format (`YYYY-MM-DD_user - post_id[- idx][- story].ext`), Instaloader format (`YYYY-MM-DD_HH-MM-SS_UTC[_idx][_story].ext`), or a generic JSON-manifest format (`posts_1.json`, `reels_1.json`, `stories_1.json`, possibly `.json.xz`-compressed via `xz-decompress`).
2. **Parses** according to detected format, building a `Map<postId, Partial<Post>>`. For JSON-manifest format, media files are matched to JSON entries by URI substring match, then by ID substring match, then by filename-derived heuristic — in that fallback order.
3. **Falls back** to generic filename-prefix grouping when no posts were found via regex/JSON matching (treats files sharing a common basename as one carousel post, chunked into groups of 20).
4. Detects a **profile picture** from `*_profile_pic.jpg` / `<username>.jpg` files, or falls back to the oldest image in the archive by filename sort ("Smart Fallback").
5. **Caches** the final `{ posts, stories, profileMetadata, ... }` result to IndexedDB via `idb-keyval`, keyed by archive name (or `local_archive` for unnamed local folders) — this is what makes repeat visits load instantly. Both server and local archives are cached; the cache shape is documented inline in `useArchiveScanner`'s state (mirrors the `CacheData` interface in `GEMINI.md`).
1. **JSON manifest** (`posts_1.json`, possibly `.json.xz` via `xz-decompress`) — media matched to entries by URI, then ID, then filename heuristic, in that fallback order.
2. **Filename patterns** — see `src/lib/archive-patterns.ts` for the export / Instaloader / highlight regexes, extracted as pure functions and covered by tests. Prefer changing them there.
3. **Generic grouping fallback** — when nothing else matched, files sharing a basename become one carousel.
When modifying format-detection or media-matching logic, be aware the three code paths (JSON-manifest, filename-regex export/instaloader, generic fallback) are largely independent and a change to one rarely needs to touch the others — but all three write into the same `postsMap`.
Results are cached to IndexedDB. Media records store a stable `path`; **`url` is not persistable** for local archives because blob URLs die with the document.
### Thumbnail generation (`src/hooks/useThumbnailQueue.ts` + `src/lib/thumbnail-worker.ts`)
Three different JSON shapes turn up as `.json`, so they are told apart structurally, not by filename (`src/lib/gallery-dl-sidecar.ts`):
High-res images (>1MiB) are downscaled off the main thread:
- `useThumbnailQueue` maintains a **serial** (one-at-a-time) queue — this is deliberate, not a bug: decoding multiple 50MP+ images concurrently causes OOM crashes in the browser.
- Actual resizing happens in `thumbnail-worker.ts` using `OffscreenCanvas`/`createImageBitmap` inside a Web Worker.
- Results are cached in IndexedDB under a `thumb_<id>` key, checked before falling back to the worker, so thumbnails persist across sessions.
| shape | marker |
|---|---|
| Instagram export manifest | top-level `media` array |
| Instaloader `.json.xz` | GraphQL node under `node` / `__typename` |
| gallery-dl sidecar | flat, `post_shortcode` + `type`, none of the above |
### URL state sync (`src/App.tsx`)
The gallery-dl sidecar is the only source that states what a post *is*: its `type` (`post` / `reel` / `story` / `highlight`) is Instagram's own classification, so `post.isReel` set from it beats every fallback in `post-tabs.ts`. This matters — of the 781 items in `official_band - reels`, the sidecars say only **360 are reels**; the other 421 are ordinary feed videos the clips endpoint returns via `include_feed_video`. Directory-based classification counted all 781.
**Dates are ranked, not last-write-wins** (`src/lib/post-dates.ts`): sidecar (what Instagram reported) beats filename (what the fetcher wrote) beats mtime (when the file hit disk, and unrelated to when it was posted). Ties keep the incumbent. Several files describe one post and they are scanned in directory order, not in order of trustworthiness, so without the ranking the date was decided by whichever file came first. Only JDownloader highlights fall to mtime at all — `parseArchiveFilename` flags those via `dateFromMtime`.
### Cache and local-archive persistence (`src/lib/archive-cache.ts`)
IndexedDB keys are namespaced (`archive:`, `thumb:`, `handle:`) so listing archives does not deserialize every cached thumbnail blob, and thumbnails are scoped per archive to avoid cross-archive collisions.
Restoring an archive **rehydrates URLs from `path`**: server archives rebuild HTTP URLs; local archives re-open a persisted `FileSystemDirectoryHandle` and mint fresh blob URLs. If the folder is unreachable (permission lapsed, or the browser lacks `showDirectoryPicker` — Firefox/Safari), the app re-prompts rather than rendering broken images.
### Thumbnails (`src/hooks/useThumbnailQueue.ts` + `src/lib/thumbnail-worker.ts`)
Images over 1MiB are downscaled in a Web Worker via `OffscreenCanvas`. The queue is **serial on purpose** — decoding several 50MP+ images at once OOMs the tab. `requestThumbnail` must keep a stable identity (it reads cache state through a ref), or every completed thumbnail re-runs the effect in all mounted thumbnails.
### Profile tabs (`src/lib/post-tabs.ts`)
The grid holds **everything**, reels included, and the Reels tab is a *filtered view* of that same set. Only the Reels tab filters. The tabs were mutually exclusive until v1.7.0, which hid a lot: 1100 of `groupfandom`'s 3813 posts and 533 of `for.member`'s 1225 never appeared in the grid at all.
This *approximates* Instagram rather than matching it. Instagram's grid includes a reel only if the creator shared it to feed — a per-post choice, measured live on 2026-08-16: `official_band` had 21 reels in its first 34 grid tiles, `4utumn07` just 1 in 214. That flag appears nowhere in an archive (JD2 stores no metadata, and Instaloader's `product_type` says what a post *is*, not whether it was shared to feed), so showing everything is the closest reachable behaviour. Instagram's "N posts" counter equals its grid, which is why the header counts `postsForTab(allPosts, 'posts')` and not `allPosts` — the raw list still holds both copies of a double-fetched post.
When checking the live site, note that grid reels link to `/reel/<code>/` while ordinary posts link to `/<user>/p/<code>/`. Matching only `/p/` silently drops every reel, which once produced a confident and completely wrong conclusion that Instagram never shows reels in the grid.
Deciding *what is a reel* has no good answer for most archives. Instagram's own marker is `product_type` on the post's GraphQL node (`clips` = reel, `feed` = ordinary feed video, `igtv`, `story`) — `__typename` is `GraphVideo` for all three, and aspect ratio does not separate them either. But:
- Only Instaloader archives carry that metadata, and only newer captures. A survey of `hazelofficial` found `product_type` on 1101 of 5919 sidecars, and just **2** posts marked `clips`.
- JDownloader archives carry none at all — media plus a `.txt` holding the bare caption.
So the viewer believes a `- reels` sidecar directory when one exists, and otherwise falls back to treating a lone video as a reel. **The fallback is a guess**: it cannot tell a reel from a feed video or an old IGTV upload, and it misses videos inside carousels.
`dedupePostCopies` exists because the JDownloader flow crawls the profile URL and the `/reels/` URL separately (the profile page misses some reels), so the two overlap and a reel can land on disk twice. Those become two posts with distinct directory-scoped ids, which the grid would otherwise render side by side. It dedupes by shortcode, preferring the reels-source copy. It is only safe over `allPosts` — stories and highlights are excluded there, and a shortcode may legitimately appear in both a profile and a highlight.
### URL state (`src/App.tsx`, `src/lib/routing.ts`)
Paths mirror Instagram: `/<archive>/`, `/<archive>/reels/`, `/<archive>/p/<shortcode>/`. The old `?a=&t=&p=` form is still parsed for existing links but never written. Reserved prefixes (`api`, `archives`, `assets`…) can't be mistaken for a profile name.
A post URL carries no tab, as on Instagram — the tab is re-derived from the post's `source`, so a reel link lands on the Reels tab and pages through reels. Sidecar posts keep directory-scoped ids internally but expose only the shortcode.
Three rules, all learned from real bugs:
- The initial route is captured into a ref on first render; the URL is rewritten from state as soon as anything loads, so reading `window.location` later sees the rewrite, not the user's link.
- URL writing is gated on `hasInitialLoaded`, otherwise it erases the deep link before the loader consumes it.
- Deep-link resolution waits on the archive fetch having *settled* (`archivesFetched`), not on `isServerMode`, which is still false while the request is in flight.
### Mobile feed (`src/components/PostFeed.tsx`)
Below `md`, opening a post renders a scrolling feed page rather than the modal (`useIsMobile` decides). Only a window of posts is mounted; it grows both ways, and prepending corrects `scrollTop` in a `useLayoutEffect` so content doesn't jump. Only the post crossing the viewport centre plays its video and drives the URL. Desktop keeps `PostModal`; both share `MediaCarousel`.
App state (selected archive, active tab, selected post) is synchronized with URL query params (`?a=`, `?t=`, `?p=`) via `URLSearchParams` + `window.history.replaceState` in a cluster of `useEffect` hooks — this is what enables permalinks/deep-linking. When adding new shareable state, follow this pattern rather than introducing a router.
### Backend (`server.ts`)
Serves `/api/archives`, `/api/archives/:name/files`, static `/archives`, and the built SPA. Notes:
Minimal Express server, three responsibilities only:
- `GET /api/archives` — lists subdirectories of `ARCHIVES_DIR` (skipping dotfiles/`@`/`_`-prefixed dirs) as `ServerArchive[]`, guessing a thumbnail per archive.
- `GET /api/archives/:name/files` — recursively lists all files in one archive directory.
- Static-serves `ARCHIVES_DIR` under `/archives` and, in production, serves the built `dist/` frontend with an SPA fallback.
- Express decodes route params **after** segment matching, so `..%2f` reaches the handler as `../`. All user-supplied archive names go through `resolveArchivePath`.
- `os.userInfo()` throws for a UID with no `/etc/passwd` entry, which is what `--user 1234:1234` produces — use `describeUser()`.
### CSP: do not tighten `script-src` or `connect-src` without testing xz
The xz decompressor for Instaloader `.json.xz` sidecars is **WebAssembly**, embedded as a `data:` URL the library fetches at startup. The policy must keep:
```
script-src 'self' 'wasm-unsafe-eval' // compile wasm, without allowing eval() of JS
connect-src 'self' data: // fetch the embedded module
```
Removing either breaks decoding with a bare `TypeError: Failed to fetch` **and no stack** — it surfaces through `new Response(stream).json()`, so it reads like a network fault rather than a policy block. The visible symptom is not an error page: archives silently lose captions, story flags and all profile metadata (follower counts, bio, name). This shipped broken for several releases.
To check quickly, run in the page console:
```js
await fetch('data:application/wasm;base64,AGFzbQEAAAA=') // connect-src
await WebAssembly.instantiate(Uint8Array.of(0,97,115,109,1,0,0,0)) // script-src
```
**The Vite dev server does not send these headers**, so anything CSP-related is invisible in `npm run dev`. Verify security-header and PWA behaviour by building and serving `dist/` through `server.js`, not against the dev server.
### PWA: server-only changes do not reach installed clients
The service worker precaches `index.html` **together with its response headers**. A change that touches only the server (a CSP fix, a new header) leaves the client build byte-identical, so the precache manifest and `sw.js` are unchanged, the worker never updates, and installed clients keep replaying the old shell with the old headers — indefinitely.
`vite.config.ts` therefore compiles the package version into the client via `define: { __APP_VERSION__ }`, and `App.tsx` renders it in the footer. That is **load-bearing**: it makes every release change the bundle hash → `index.html` → its precache revision → `sw.js`, which is what browsers byte-compare to decide whether to update. Don't remove it as dead weight.
To recover a client stuck on an old shell: unregister the service worker, delete its caches, reload.
### Deployment
Live archives live in `<share>/Instagram-archive/archives/` — one directory per profile plus sidecars. Directories that are not Instagram profiles (tool output, exports from other services) sit *outside* that folder so they never reach the viewer.
Container runs as non-root. The image defaults to `node`, but the archive share must be *listable* by that UID — a mode-711 share owned by another account needs `user: "<uid>:<gid>"` in compose. Mount a volume at `/cache` so the index survives restarts.
It does no parsing of archive/JSON contents — that's entirely client-side in `useArchiveScanner`. `ARCHIVES_DIR` is resolved from the `ARCHIVES_DIR` env var (see `.env` / Docker volume mount at `/archives`).
### PWA / build quirks
- `vite.config.ts` sets `hmr: process.env.DISABLE_HMR !== 'true'` — intentional, leave it.
- `workbox.navigateFallbackDenylist` excludes `/api` and `/archives` so those hit the real server.
- Fonts and icons are vendored in `public/` — do not reintroduce CDN references; the app advertises offline support and local-only processing.
- `vite.config.ts` sets `hmr: process.env.DISABLE_HMR !== 'true'` — this is intentionally left alone; it exists to disable file-watch flicker when running under AI Studio-style agent editing. Don't "clean up" or remove it.
- `workbox.navigateFallbackDenylist` excludes `/api` and `/archives` from the SPA fallback so those routes hit the real server/static files instead of `index.html` (needed for "open original file in new tab").
- Service worker uses `registerType: 'autoUpdate'` with hourly periodic checks — new deployments propagate to open clients automatically.
-109
View File
@@ -1,109 +0,0 @@
# Manual run — scrape, archive, push to the NAS
**`tooling` branch only. Never push this to GitHub.** Like `TOOLING.md`, it names the fetch host,
the NAS hosts and their paths.
How to do by hand everything the timers normally do, plus the second NAS hop that feeds
`cosmo_web`. `TOOLING.md` has the details behind each step.
## Where things go
1. **mattellite → ana-nas.** Every scrape script ends by rsyncing its new files to
`ana-nas:/volume1/rslsync/sync/Instagram-archive/archives/` (10.20.28.200).
<https://instaarchive.ergosteur.com> reads straight from there. **There is no separate step
for this hop.**
2. **ana-nas → DH2300.** The `archiver-instagram` container on `DH2300-D7CA` pulls that archive,
makes thumbnails, remuxes video, normalizes into `cosmo_web`'s data and translates captions.
It runs weekly (Wednesday 04:00). Step 6 below runs it by hand.
## 1. Pre-flight (mattellite)
```sh
ssh matt@mattellite
~/gdl/check-session.py # want: "session ok ... expires <date>"
curl -s http://localhost:9222/json/version | head -2 # Chrome DevTools port is up
~/gdl/gdl-upgrade.sh # gallery-dl + yt-dlp to latest
```
If the session check fails, log in to Instagram in the VNC Chrome (display `:1`), **reload the
tab** (a dead session can still look logged in) and re-check.
## 2. Stories — first; they expire in 24 h
```sh
~/gdl/run-stories.sh
```
About 2 minutes. `rsync ... (code 23)` / `failed to set permissions` is expected: the script
says "files landed" when that is the only problem. `No results` for a profile just means it has
no active story.
## 3. Posts and reels
**While gallery-dl's listing is broken** ([gh#9829](https://github.com/mikf/gallery-dl/issues/9829),
from 2026-10-22), use the browser scraper, one profile at a time:
```sh
for p in heejin kimxxlip; do ~/gdl/posts-sync.sh $p; done
for p in withaseul cher_ryppo zindoriyam; do ~/gdl/posts-sync.sh $p; done
~/gdl/posts-sync.sh official_artms
```
- `posts-sync.sh` scrolls `/<user>/` in the signed-in Chrome, stops once it reaches posts the
archive already holds, and fetches only the new ones as single posts. Each profile takes a few
minutes, mostly the 12–20 s gap between fetches.
- It prints `no new posts for <user>` when a profile is caught up.
- Logs: `~/gdl/logs/posts-<user>-*.log`.
- **Then sweep the Reels tab too:** `~/gdl/run-reels-all.sh` (~15 min). Most reels never
appear in the main grid. On 2026-10-23 only 3 of heejin's 114 reels did, so `posts-sync.sh`
alone misses them. `reels-sync.sh <user>` does a single profile.
**Or the whole thing in one command**, the same run the weekly timer does: gallery-dl for
stories and highlights, then the browser for every profile's posts grid and Reels tab, each
stopping once it reaches archived content:
```sh
~/gdl/run-profiles.sh
```
Once a gallery-dl release fixes the listing, `GDL_LISTING=api ~/gdl/run-profiles.sh` does posts
and reels through gallery-dl again (~5 min).
For long runs, detach:
`setsid nohup ~/gdl/run-profiles.sh >/dev/null 2>&1 &`, then
`tail -f ~/gdl/logs/$(ls -1t ~/gdl/logs | head -1)`.
**Stop at once** on a `429` from `scontent-*.cdninstagram.com` or a `400 Bad Request` on
`/api/v1/feed/reels_media/`. See "When a run fails" in `TOOLING.md`.
## 4. Highlights
These only come through `run-profiles.sh`. Expect them to fail with the same
`Requested user could not be found` until gh#9829 is fixed in a release, so skip them until then.
## 5. Check it landed (any machine)
```sh
for u in heejin kimxxlip withaseul cher_ryppo zindoriyam official_artms; do
n=$(curl -s "https://instaarchive.ergosteur.com/api/archives/$u/files" | python3 -c 'import json,sys; d=json.load(sys.stdin); print(len(d if isinstance(d,list) else d["files"]))')
printf '%-16s %s\n' "$u" "$n"
done
```
Or browse <https://instaarchive.ergosteur.com>. New files appear without a redeploy.
## 6. Push to `cosmo_web` (the DH2300 hop)
This runs the same command the container's cron line runs (from `docker/archiver-instagram-entrypoint.sh`
in Cosmo-Live-Downloads):
```sh
ssh matt@DH2300-D7CA
docker exec cosmo-web-archiver-instagram-1 sh -c \
'cd /work && . /app/instagram-archiver.env && python3 scripts/instagram_archive_run.py'
docker logs --tail 50 cosmo-web-archiver-instagram-1
```
It pings that container's Healthchecks check, same as a scheduled run. The pipeline itself
(mirror, thumbnail, remux, normalize, collaborators, translate) is described in the docstring of
`scripts/instagram_archive_run.py` in Cosmo-Live-Downloads.
+2 -19
View File
@@ -54,27 +54,10 @@ If the app shows "No Archives Found" and logs `EACCES: permission denied`:
chmod -R 755 /path/to/archives
```
2. **SELinux (Fedora/RHEL/CentOS)**: Use the `:z` flag in your volume mount as shown above.
3. **User Mapping**: The container runs as the non-root `node` user (UID 1000).
If your archives are readable only by another account, run as that user
instead — the container needs to *list* the archive directory, so `--x`
(traverse-only) permissions are not enough:
3. **User Mapping**: You can force the container to run as your host user:
```bash
docker run --user $(stat -c '%u:%g' /path/to/archives) ...
docker run --user $(id -u):$(id -g) ...
```
In Compose:
```yaml
services:
instaarchive:
user: "1234:1234" # a UID that can read your archives
```
### Archive Index
On first start the server walks the archive root once and caches the result,
keyed by directory mtime. This matters on network storage: for a 110k-file
archive root, listing went from ~52s per request to ~0.1s. Mount a volume at
`/cache` (or set `CACHE_DIR`) so the index survives restarts, otherwise it is
rebuilt on every start.
## Supported Archive Structure
-1036
View File
File diff suppressed because it is too large Load Diff
-6
View File
@@ -1,6 +0,0 @@
https://www.instagram.com/heejin/
https://www.instagram.com/kimxxlip/
https://www.instagram.com/withaseul/
https://www.instagram.com/cher_ryppo/
https://www.instagram.com/zindoriyam/
https://www.instagram.com/official_artms/
-221
View File
@@ -1,221 +0,0 @@
[
[
2,
{
"category": "instagram",
"coauthors": [
{
"full_name": "\ucd5c\ub9ac \ucd5c\uc608\ub9bc Choerry",
"id": "57668363710",
"username": "cher_ryppo"
},
{
"full_name": "\uc9c4\uc194 \ud835\udc09\ud835\udc08\ud835\udc0d\ud835\udc12\ud835\udc0e\ud835\udc14\ud835\udc0b",
"id": "55465048543",
"username": "zindoriyam"
},
{
"full_name": "Official ARTMS",
"id": "58524253183",
"username": "official_artms"
},
{
"full_name": "HEEJIN",
"id": "57723176039",
"username": "0ct0ber19"
},
{
"full_name": "\uae40\ub9bd KIMLIP",
"id": "57506288936",
"username": "kimxxlip"
}
],
"count": 1,
"date": "2026-08-18 07:18:54",
"description": "#\uc81c\uc791\uc9c0\uc6d0 ARTMS(\uc544\ub974\ud14c\ubbf8\uc2a4) \u2018Blue Blood Night\u2019 \ub9ac\uc2a4\ub2dd \ud30c\ud2f0 \ud83c\udf19\n\n\uc131\uc218\ub3d9\uc5d0\uc11c \ud3bc\uccd0\uc9c4 ARTMS\uc640 300\uba85 \ud32c\ub4e4\uc758 \uc5ed\ub300\uae09 \ubc24!\n\nARTMS\uac00 \uc57d 300\uba85\uc758 \ud32c\ub4e4\uacfc \ud568\uaed8 \ud074\ub7fd \ud615\ud0dc\uc758\n\ub3c5\ud2b9\ud558\uace0 \ubabd\ud658\uc801\uc778 \uacf5\uac04\uc5d0\uc11c \uc624\ud504\ub77c\uc778 \ud32c \uc774\ubca4\ud2b8\n\u2018Blue Blood Night\u2019 \ub9ac\uc2a4\ub2dd \ud30c\ud2f0\ub97c \uac1c\ucd5c\ud588\uc2b5\ub2c8\ub2e4 \u2728\n\n\ud558\uc774 \ub370\ud328\ub274\ub3c4 \uc678\uce58\uace0 ARTMS \uc0ac\ub791\ud574\uc694\ub3c4 \uc678\ucce4\ub358\n\ub728\uac70\uc6e0\ub358 \ud604\uc7a5\uc5d0 \ub370\ud328\ub274\uac00 \ub2e4\ub140\uc654\uc2b5\ub2c8\ub2e4 \ud83d\ude0e\n\n\ud83d\udccc\uc544\ub974\ud14c\ubbf8\uc2a4 (ARTMS) \n@official_artms \n\n\u25aa\ufe0f2024\ub144\uc5d0 \ub370\ubdd4\ud55c \ubaa8\ub4dc\ud558\uc6b0\uc2a4(MODHAUS) \uc18c\uc18d\uc758 5\uc778\uc870 \uac78\uadf8\ub8f9 \n\u25aa\ufe0f\uba64\ubc84: \ud76c\uc9c4, \ud558\uc2ac, \uae40\ub9bd, \uc9c4\uc194, \ucd5c\ub9ac \n\u25aa\ufe0f\uc774\ub2ec\uc758 \uc18c\ub140 \ucd9c\uc2e0 \uba64\ubc84\ub4e4\uc774 \ub73b\uc744 \ubaa8\uc544 \uacb0\uc131\ud55c \ud504\ub85c\uc81d\ud2b8\uc774\uc790 \uc815\uc2dd \uac78\uadf8\ub8f9 \n\u25aa\ufe0f\uccab \uc815\uaddc \uc568\ubc94\uacfc \ud0c0\uc774\ud2c0\uace1 \u2018Virtual Angel\u2019 \ub4f1\uc744 \uc120\ubcf4\uc784\n\n#ARTMS #\uc544\ub974\ud14c\ubbf8\uc2a4 #BlueBloodNight\n\n\ud83c\udfa5 @dailyfashion_news",
"fullname": "\u270d\ud83c\udffb\ud83e\udd13\ud328\uc158 \ub274\uc2a4 \uc694\uc57d \u2b50\ufe0f \ub370\ud328\ub274\ud83d\ude0e\ud83d\udc85\ud83c\udffb",
"liked": false,
"likes": 7219,
"owner_id": "38980901318",
"pinned": [],
"post_date": "2026-08-18 07:18:54",
"post_id": "3966279735525149864",
"post_shortcode": "DcLDme7zoyo",
"post_url": "https://www.instagram.com/reel/DcLDme7zoyo/",
"subcategory": "reel",
"tags": [
"#ARTMS",
"#BlueBloodNight",
"#\uc544\ub974\ud14c\ubbf8\uc2a4",
"#\uc81c\uc791\uc9c0\uc6d0"
],
"type": "reel",
"username": "dailyfashion_news"
}
],
[
3,
"ytdl:https://www.instagram.com/reel/DcLDme7zoyo/1.mp4",
{
"category": "instagram",
"coauthors": [
{
"full_name": "\ucd5c\ub9ac \ucd5c\uc608\ub9bc Choerry",
"id": "57668363710",
"username": "cher_ryppo"
},
{
"full_name": "\uc9c4\uc194 \ud835\udc09\ud835\udc08\ud835\udc0d\ud835\udc12\ud835\udc0e\ud835\udc14\ud835\udc0b",
"id": "55465048543",
"username": "zindoriyam"
},
{
"full_name": "Official ARTMS",
"id": "58524253183",
"username": "official_artms"
},
{
"full_name": "HEEJIN",
"id": "57723176039",
"username": "0ct0ber19"
},
{
"full_name": "\uae40\ub9bd KIMLIP",
"id": "57506288936",
"username": "kimxxlip"
}
],
"count": 1,
"date": "2026-08-18 07:18:54",
"description": "#\uc81c\uc791\uc9c0\uc6d0 ARTMS(\uc544\ub974\ud14c\ubbf8\uc2a4) \u2018Blue Blood Night\u2019 \ub9ac\uc2a4\ub2dd \ud30c\ud2f0 \ud83c\udf19\n\n\uc131\uc218\ub3d9\uc5d0\uc11c \ud3bc\uccd0\uc9c4 ARTMS\uc640 300\uba85 \ud32c\ub4e4\uc758 \uc5ed\ub300\uae09 \ubc24!\n\nARTMS\uac00 \uc57d 300\uba85\uc758 \ud32c\ub4e4\uacfc \ud568\uaed8 \ud074\ub7fd \ud615\ud0dc\uc758\n\ub3c5\ud2b9\ud558\uace0 \ubabd\ud658\uc801\uc778 \uacf5\uac04\uc5d0\uc11c \uc624\ud504\ub77c\uc778 \ud32c \uc774\ubca4\ud2b8\n\u2018Blue Blood Night\u2019 \ub9ac\uc2a4\ub2dd \ud30c\ud2f0\ub97c \uac1c\ucd5c\ud588\uc2b5\ub2c8\ub2e4 \u2728\n\n\ud558\uc774 \ub370\ud328\ub274\ub3c4 \uc678\uce58\uace0 ARTMS \uc0ac\ub791\ud574\uc694\ub3c4 \uc678\ucce4\ub358\n\ub728\uac70\uc6e0\ub358 \ud604\uc7a5\uc5d0 \ub370\ud328\ub274\uac00 \ub2e4\ub140\uc654\uc2b5\ub2c8\ub2e4 \ud83d\ude0e\n\n\ud83d\udccc\uc544\ub974\ud14c\ubbf8\uc2a4 (ARTMS) \n@official_artms \n\n\u25aa\ufe0f2024\ub144\uc5d0 \ub370\ubdd4\ud55c \ubaa8\ub4dc\ud558\uc6b0\uc2a4(MODHAUS) \uc18c\uc18d\uc758 5\uc778\uc870 \uac78\uadf8\ub8f9 \n\u25aa\ufe0f\uba64\ubc84: \ud76c\uc9c4, \ud558\uc2ac, \uae40\ub9bd, \uc9c4\uc194, \ucd5c\ub9ac \n\u25aa\ufe0f\uc774\ub2ec\uc758 \uc18c\ub140 \ucd9c\uc2e0 \uba64\ubc84\ub4e4\uc774 \ub73b\uc744 \ubaa8\uc544 \uacb0\uc131\ud55c \ud504\ub85c\uc81d\ud2b8\uc774\uc790 \uc815\uc2dd \uac78\uadf8\ub8f9 \n\u25aa\ufe0f\uccab \uc815\uaddc \uc568\ubc94\uacfc \ud0c0\uc774\ud2c0\uace1 \u2018Virtual Angel\u2019 \ub4f1\uc744 \uc120\ubcf4\uc784\n\n#ARTMS #\uc544\ub974\ud14c\ubbf8\uc2a4 #BlueBloodNight\n\n\ud83c\udfa5 @dailyfashion_news",
"display_url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.82787-15/777982919_18124593284301319_6754878398807305029_n.jpg?stp=dst-jpg_e15_tt6&_nc_cat=107&ig_cache_key=Mzk2NjI3OTczNTUyNTE0OTg2NDE4MTI0NTkzMjgxMzAxMzE5.3-ccb7-5&ccb=7-5&_nc_sid=58cdad&efg=eyJ2ZW5jb2RlX3RhZyI6IkNMSVBTLnhwaWRzLjEzMjAuc2RyLnZpZGVvX2RlZmF1bHRfY292ZXJfZnJhbWUuQzMifQ%3D%3D&_nc_ohc=VTiIkvHCoBMQ7kNvwFnD08V&_nc_oc=Adp2K1e-pF18avq4GpMbXRgiZv_T6ySFEqTFdXPRtemyWnFeG8p6cgvxaXxZtyDDw-U&_nc_ad=z-m&_nc_cid=0&_nc_zt=23&_nc_ht=instagram.fyto3-1.fna&_nc_gid=-Kd8mdFfght8nFvwkJmRPA&_nc_ss=7a22e&oh=00_AQIKaJEGeU5iyrLIjnfXozkpbabHpemHWlDoSEEx11SMFQ&oe=6A9D5761",
"extension": "mp4",
"filename": "AQNbHAasCyJwuQWdYi7vWw3w6iCiMdBAXZQ58cnOKhanFZgw_IAhVC_f380GVo23sNeeCD559rVhmUSr0bmruolOjAgXMVwrmoRu88A",
"fullname": "\u270d\ud83c\udffb\ud83e\udd13\ud328\uc158 \ub274\uc2a4 \uc694\uc57d \u2b50\ufe0f \ub370\ud328\ub274\ud83d\ude0e\ud83d\udc85\ud83c\udffb",
"height": 1920,
"height_original": 1920,
"liked": false,
"likes": 7219,
"media_id": "3966279735525149864",
"num": 1,
"owner": {
"account_badges": [],
"account_type": 3,
"can_see_quiet_post_attribution": true,
"eligible_for_text_app_activation_badge": false,
"fan_club_info": {
"autosave_to_exclusive_highlight": null,
"connected_member_count": null,
"fan_club_id": null,
"fan_club_name": null,
"fan_consideration_page_revamp_eligiblity": null,
"has_created_ssc": null,
"has_enough_subscribers_for_ssc": null,
"is_fan_club_gifting_eligible": null,
"is_fan_club_referral_eligible": null,
"is_free_trial_eligible": null,
"largest_public_bc_id": null,
"should_show_playlists_in_profile_tab": null,
"subscriber_count": null
},
"fbid_v2": "17841439039552612",
"feed_post_reshare_disabled": false,
"friendship_status": {
"followed_by": false,
"following": false,
"is_bestie": false,
"is_feed_favorite": false,
"is_muting_reel": false,
"is_private": false,
"is_restricted": false
},
"full_name": "\u270d\ud83c\udffb\ud83e\udd13\ud328\uc158 \ub274\uc2a4 \uc694\uc57d \u2b50\ufe0f \ub370\ud328\ub274\ud83d\ude0e\ud83d\udc85\ud83c\udffb",
"has_anonymous_profile_picture": false,
"hd_profile_pic_url_info": {
"height": 1080,
"url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.2885-19/283399158_1201757910664534_1998021056617019217_n.jpg?efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gE8DUM4skVKFnTxJ_6S8QN3RWaHsgKT9sigiWMRE8sqvl9r488ADsJFU6ErFHln2eg&_nc_ohc=LjOAkQaNPtMQ7kNvwHhtSrv&_nc_gid=-Kd8mdFfght8nFvwkJmRPA&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQLdX6t5krhp725dzvNOvQbGOFpVdvwgZWqDkFvv1AeQPA&oe=6A9D6555&_nc_sid=fc8dfb",
"width": 1080
},
"hd_profile_pic_versions": [
{
"height": 320,
"url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.2885-19/283399158_1201757910664534_1998021056617019217_n.jpg?stp=dst-jpg_s320x320_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gE8DUM4skVKFnTxJ_6S8QN3RWaHsgKT9sigiWMRE8sqvl9r488ADsJFU6ErFHln2eg&_nc_ohc=LjOAkQaNPtMQ7kNvwHhtSrv&_nc_gid=-Kd8mdFfght8nFvwkJmRPA&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQKux7g7JpZsP_Q7z6rX3bFlcdVCUCwwDAQfmRRWbB6ePQ&oe=6A9D6555&_nc_sid=fc8dfb",
"width": 320
},
{
"height": 640,
"url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.2885-19/283399158_1201757910664534_1998021056617019217_n.jpg?stp=dst-jpg_s640x640_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gE8DUM4skVKFnTxJ_6S8QN3RWaHsgKT9sigiWMRE8sqvl9r488ADsJFU6ErFHln2eg&_nc_ohc=LjOAkQaNPtMQ7kNvwHhtSrv&_nc_gid=-Kd8mdFfght8nFvwkJmRPA&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQJJz0fXTbLsfMqkiG7KHQtZbaHwHAU8nS1eTzB0sQqRSA&oe=6A9D6555&_nc_sid=fc8dfb",
"width": 640
}
],
"id": "38980901318",
"is_active_on_text_post_app": true,
"is_embeds_disabled": false,
"is_favorite": false,
"is_private": false,
"is_ring_creator": false,
"is_unpublished": false,
"is_verified": true,
"latest_reel_media": 1788306389,
"pk": "38980901318",
"pk_id": "38980901318",
"profile_pic_id": "2844231369895585868_38980901318",
"profile_pic_url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.2885-19/283399158_1201757910664534_1998021056617019217_n.jpg?stp=dst-jpg_s150x150_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gE8DUM4skVKFnTxJ_6S8QN3RWaHsgKT9sigiWMRE8sqvl9r488ADsJFU6ErFHln2eg&_nc_ohc=LjOAkQaNPtMQ7kNvwHhtSrv&_nc_gid=-Kd8mdFfght8nFvwkJmRPA&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQI28hrlvg9XAWGFkTjCyv96HLuMEZb5q6AlNbUUqw2WVQ&oe=6A9D6555&_nc_sid=fc8dfb",
"show_account_transparency_details": true,
"show_ring_award": false,
"strong_id__": "38980901318",
"text_post_app_is_private": false,
"third_party_downloads_enabled": 2,
"transparency_product_enabled": false,
"user_activation_info": {},
"username": "dailyfashion_news"
},
"owner_id": "38980901318",
"pinned": [],
"post_date": "2026-08-18 07:18:54",
"post_id": "3966279735525149864",
"post_shortcode": "DcLDme7zoyo",
"post_url": "https://www.instagram.com/reel/DcLDme7zoyo/",
"shortcode": "DcLDme7zoyo",
"subcategory": "reel",
"tagged_users": [
{
"full_name": "\uc9c4\uc194 \ud835\udc09\ud835\udc08\ud835\udc0d\ud835\udc12\ud835\udc0e\ud835\udc14\ud835\udc0b",
"id": "55465048543",
"username": "zindoriyam"
},
{
"full_name": "\uae40\ub9bd KIMLIP",
"id": "57506288936",
"username": "kimxxlip"
},
{
"full_name": "\ucd5c\ub9ac \ucd5c\uc608\ub9bc Choerry",
"id": "57668363710",
"username": "cher_ryppo"
},
{
"full_name": "HEEJIN",
"id": "57723176039",
"username": "0ct0ber19"
},
{
"full_name": "Official ARTMS",
"id": "58524253183",
"username": "official_artms"
}
],
"tags": [
"#ARTMS",
"#BlueBloodNight",
"#\uc544\ub974\ud14c\ubbf8\uc2a4",
"#\uc81c\uc791\uc9c0\uc6d0"
],
"type": "reel",
"username": "dailyfashion_news",
"video_url": "https://instagram.fyto3-1.fna.fbcdn.net/o1/v/t2/f2/m86/AQNbHAasCyJwuQWdYi7vWw3w6iCiMdBAXZQ58cnOKhanFZgw_IAhVC_f380GVo23sNeeCD559rVhmUSr0bmruolOjAgXMVwrmoRu88A.mp4?_nc_cat=105&_nc_oc=AdomTMuitkVGoe5U3qTaa9WVmrQNaUmC-hpxQMgtcKNDrT2Htj3cu6iPiBDfh9M0Hyk&_nc_sid=5e9851&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_ohc=O3edipmWdLQQ7kNvwFP9r4s&efg=eyJ2ZW5jb2RlX3RhZyI6Inhwdl9wcm9ncmVzc2l2ZS5JTlNUQUdSQU0uQ0xJUFMuQzMuNzIwLmRhc2hfYmFzZWxpbmVfMV92MSIsInhwdl9hc3NldF9pZCI6MTgxMTkyNTgxMzEyMzA0OSwiYXNzZXRfYWdlX2RheXMiOjE0LCJ2aV91c2VjYXNlX2lkIjoxMDA5OSwiZHVyYXRpb25fcyI6NTQsInVybGdlbl9zb3VyY2UiOiJ3d3cifQ%3D%3D&ccb=17-1&vs=69de96aad46e550e&_nc_vs=HBksFQIYUmlnX3hwdl9yZWVsc19wZXJtYW5lbnRfc3JfcHJvZC83RTRFMjc3MDhFMjVCMzUxNjA5MURDQUM2QTQ3MTlBQV92aWRlb19kYXNoaW5pdC5tcDQVAALIARIAFQIYUWlnX3hwdl9wbGFjZW1lbnRfcGVybWFuZW50X3YyL0I3NDdFMUM4NTRCODk2NDRENDM2NTIwQzVBODBCOTk5X2F1ZGlvX2Rhc2hpbml0Lm1wNBUCAsgBEgAoABgAGwKIB3VzZV9vaWwBMRJwcm9ncmVzc2l2ZV9yZWNpcGUBMRUAACbS35qn-_u3BhUCKAJDMywXQEtRBiTdLxsYEmRhc2hfYmFzZWxpbmVfMV92MREAdf4HZeadAQA&_nc_gid=-Kd8mdFfght8nFvwkJmRPA&_nc_zt=28&_nc_ss=7a22e&oh=00_AQLoXUFyQixJHvIcp3UE-6sEsjKuOlmhiPWmQphL7U7o5g&oe=6A996A42",
"width": 1080,
"width_original": 1080
}
]
]
-181
View File
@@ -1,181 +0,0 @@
[
[
2,
{
"audio_artist": null,
"audio_duration": 15.0,
"audio_timestamps": null,
"audio_title": "Original audio",
"audio_user": {
"full_name": "\uc2e0\ubaa8\ucc0c \ud835\udde0\ud835\uddfc\ud835\uddf0\ud835\uddf5\ud835\uddf6 \ud83c\udf58",
"id": "61396137942",
"is_private": false,
"is_verified": false,
"pk": "61396137942",
"pk_id": "61396137942",
"profile_pic_id": "3506506412954824214_61396137942",
"profile_pic_url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.2885-19/468037362_1017459960091671_5806837525855946557_n.jpg?stp=dst-jpg_s150x150_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby42MDAuYzIifQ&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=103&_nc_oc=Q6cZ2gEBkafhMKkLe-YbSz6FVQiMLEw4juIKWvM8PNmZ2nTcMpNyjdl0kV_-fWna0tytdLA&_nc_ohc=sPpx_bESe4cQ7kNvwFvI70J&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQIS_52Qhu7n6Fo7XNtQaIKXGEhkwhgz-C9vpWo4yb6dnw&oe=6A9D5107&_nc_sid=fc8dfb",
"strong_id__": "61396137942",
"username": "mochi.053"
},
"category": "instagram",
"coauthors": [
{
"full_name": "HEEJIN",
"id": "57723176039",
"username": "0ct0ber19"
}
],
"count": 2,
"date": "2026-08-22 02:48:05",
"description": "\u3060\u3044\u3058\u3087\u270c\ufe0f\u3067\u3057\u3087",
"fullname": "\uce04",
"liked": true,
"likes": 178797,
"owner_id": "54301371254",
"pinned": [],
"post_date": "2026-08-22 02:48:05",
"post_id": "3969041070976751617",
"post_shortcode": "DcU3dM-gVgB",
"post_url": "https://www.instagram.com/p/DcU3dM-gVgB/",
"subcategory": "post",
"type": "post",
"username": "chuuo3o"
}
],
[
3,
"ytdl:https://www.instagram.com/p/DcU3dM-gVgB/1.mp4",
{
"audio_artist": null,
"audio_duration": 15.0,
"audio_timestamps": null,
"audio_title": "Original audio",
"audio_user": {
"full_name": "\uc2e0\ubaa8\ucc0c \ud835\udde0\ud835\uddfc\ud835\uddf0\ud835\uddf5\ud835\uddf6 \ud83c\udf58",
"id": "61396137942",
"is_private": false,
"is_verified": false,
"pk": "61396137942",
"pk_id": "61396137942",
"profile_pic_id": "3506506412954824214_61396137942",
"profile_pic_url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.2885-19/468037362_1017459960091671_5806837525855946557_n.jpg?stp=dst-jpg_s150x150_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby42MDAuYzIifQ&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=103&_nc_oc=Q6cZ2gEBkafhMKkLe-YbSz6FVQiMLEw4juIKWvM8PNmZ2nTcMpNyjdl0kV_-fWna0tytdLA&_nc_ohc=sPpx_bESe4cQ7kNvwFvI70J&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQIS_52Qhu7n6Fo7XNtQaIKXGEhkwhgz-C9vpWo4yb6dnw&oe=6A9D5107&_nc_sid=fc8dfb",
"strong_id__": "61396137942",
"username": "mochi.053"
},
"category": "instagram",
"coauthors": [
{
"full_name": "HEEJIN",
"id": "57723176039",
"username": "0ct0ber19"
}
],
"count": 2,
"date": "2026-08-22 02:48:05",
"description": "\u3060\u3044\u3058\u3087\u270c\ufe0f\u3067\u3057\u3087",
"display_url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.82787-15/783856831_18043412798811255_1724949997749570795_n.jpg?stp=dst-jpg_e15_tt6&_nc_cat=108&ig_cache_key=Mzk2OTA0MTA3MDk3Njc1MTYxNzE4MDQzNDEyNzkyODExMjU1.3-ccb7-5&ccb=7-5&_nc_sid=58cdad&efg=eyJ2ZW5jb2RlX3RhZyI6IkNMSVBTLnhwaWRzLjcyMC5zZHIudmlkZW9fZGVmYXVsdF9jb3Zlcl9mcmFtZS5DMyJ9&_nc_ohc=JWzEND3gUxAQ7kNvwGDGCLf&_nc_oc=AdpwpFkh8Wj84hyF4zWcsaNB5zUnYJh2bJijNe9AxrOR1WKa08Uy35khb5BpFnG9XhM&_nc_ad=z-m&_nc_cid=0&_nc_zt=23&_nc_ht=instagram.fyto3-1.fna&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&_nc_ss=7a22e&oh=00_AQLQlu6EItD2G6QrlOmQSas2IHDOpucLIhxVWcCPFE1BOw&oe=6A9D4B4F",
"extension": "mp4",
"filename": "AQN8nhc_L3hdDSQVVRF2OXm2gKwDpsdh_jw81ZnA9Uh4nqOwukKke8QXrGCe1jrMtOXzPT5Bkuipu8RShT651VV7q6EVbjvxC-fxgUE",
"fullname": "\uce04",
"height": 1920,
"height_original": 1920,
"liked": true,
"likes": 178797,
"media_id": "3969041070976751617",
"num": 1,
"owner": {
"account_badges": [],
"account_type": 3,
"can_see_quiet_post_attribution": true,
"eligible_for_text_app_activation_badge": false,
"fan_club_info": {
"autosave_to_exclusive_highlight": null,
"connected_member_count": null,
"fan_club_id": null,
"fan_club_name": null,
"fan_consideration_page_revamp_eligiblity": null,
"has_created_ssc": null,
"has_enough_subscribers_for_ssc": null,
"is_fan_club_gifting_eligible": null,
"is_fan_club_referral_eligible": null,
"is_free_trial_eligible": null,
"largest_public_bc_id": null,
"should_show_playlists_in_profile_tab": null,
"subscriber_count": null
},
"fbid_v2": "17841454337627671",
"feed_post_reshare_disabled": false,
"friendship_status": {
"followed_by": false,
"following": true,
"is_bestie": false,
"is_feed_favorite": false,
"is_muting_reel": false,
"is_private": false,
"is_restricted": false
},
"full_name": "\uce04",
"has_anonymous_profile_picture": false,
"hd_profile_pic_url_info": {
"height": 1080,
"url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.82787-19/714930835_18032224961811255_6816644959905468206_n.jpg?efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gEBkafhMKkLe-YbSz6FVQiMLEw4juIKWvM8PNmZ2nTcMpNyjdl0kV_-fWna0tytdLA&_nc_ohc=hpoiFipmAqsQ7kNvwE3y5Ow&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQJiPho8ymPf1bC2B5dAlKCbdXW8d4KK-9qqsVrcOytVGQ&oe=6A9D4CB4&_nc_sid=fc8dfb",
"width": 1080
},
"hd_profile_pic_versions": [
{
"height": 320,
"url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.82787-19/714930835_18032224961811255_6816644959905468206_n.jpg?stp=dst-jpg_s320x320_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gEBkafhMKkLe-YbSz6FVQiMLEw4juIKWvM8PNmZ2nTcMpNyjdl0kV_-fWna0tytdLA&_nc_ohc=hpoiFipmAqsQ7kNvwE3y5Ow&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQIn43F6SprbhopBuJxmswE2wd4C39mDZoL9Fbqs8Ih-cw&oe=6A9D4CB4&_nc_sid=fc8dfb",
"width": 320
},
{
"height": 640,
"url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.82787-19/714930835_18032224961811255_6816644959905468206_n.jpg?stp=dst-jpg_s640x640_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gEBkafhMKkLe-YbSz6FVQiMLEw4juIKWvM8PNmZ2nTcMpNyjdl0kV_-fWna0tytdLA&_nc_ohc=hpoiFipmAqsQ7kNvwE3y5Ow&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQJF1vYqhGCDkV-v6WbrTb_PoVC4GA3QCbEGJU0y0F96BA&oe=6A9D4CB4&_nc_sid=fc8dfb",
"width": 640
}
],
"id": "54301371254",
"is_active_on_text_post_app": false,
"is_embeds_disabled": false,
"is_favorite": false,
"is_private": false,
"is_ring_creator": false,
"is_unpublished": false,
"is_verified": true,
"latest_reel_media": 0,
"pk": "54301371254",
"pk_id": "54301371254",
"profile_pic_id": "3912796961274695044_54301371254",
"profile_pic_url": "https://instagram.fyto3-1.fna.fbcdn.net/v/t51.82787-19/714930835_18032224961811255_6816644959905468206_n.jpg?stp=dst-jpg_s150x150_tt6&efg=eyJ2ZW5jb2RlX3RhZyI6InByb2ZpbGVfcGljLmRqYW5nby4xMDgwLmMyIn0&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_cat=1&_nc_oc=Q6cZ2gEBkafhMKkLe-YbSz6FVQiMLEw4juIKWvM8PNmZ2nTcMpNyjdl0kV_-fWna0tytdLA&_nc_ohc=hpoiFipmAqsQ7kNvwE3y5Ow&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&edm=ALQROFkBAAAA&ccb=7-5&oh=00_AQJu85hrHiCGK3enXnfDQExy_ZbgzCV_AGYJ4yOy7mi_QQ&oe=6A9D4CB4&_nc_sid=fc8dfb",
"show_account_transparency_details": true,
"show_ring_award": false,
"strong_id__": "54301371254",
"text_post_app_is_private": true,
"third_party_downloads_enabled": 1,
"transparency_product_enabled": false,
"user_activation_info": {},
"username": "chuuo3o"
},
"owner_id": "54301371254",
"pinned": [],
"post_date": "2026-08-22 02:48:05",
"post_id": "3969041070976751617",
"post_shortcode": "DcU3dM-gVgB",
"post_url": "https://www.instagram.com/p/DcU3dM-gVgB/",
"shortcode": "DcU3dM-gVgB",
"subcategory": "post",
"tagged_users": [
{
"full_name": "HEEJIN",
"id": "57723176039",
"username": "0ct0ber19"
}
],
"type": "post",
"username": "chuuo3o",
"video_url": "https://instagram.fyto3-1.fna.fbcdn.net/o1/v/t2/f2/m86/AQN8nhc_L3hdDSQVVRF2OXm2gKwDpsdh_jw81ZnA9Uh4nqOwukKke8QXrGCe1jrMtOXzPT5Bkuipu8RShT651VV7q6EVbjvxC-fxgUE.mp4?_nc_cat=102&_nc_oc=AdqR865Bo8jW90mx5U9WR94Q0Eh1YI3ycT_ZhhW71XQNFHxUR3aBwhQEgcQ9Z2Hnrzs&_nc_sid=5e9851&_nc_ht=instagram.fyto3-1.fna.fbcdn.net&_nc_ohc=2F0lG8gquXwQ7kNvwGIIkUT&efg=eyJ2ZW5jb2RlX3RhZyI6Inhwdl9wcm9ncmVzc2l2ZS5JTlNUQUdSQU0uQ0xJUFMuQzMuNzIwLmRhc2hfYmFzZWxpbmVfMV92MSIsInhwdl9hc3NldF9pZCI6MTYxMDAzNjE0Mzk3NzkxMywiYXNzZXRfYWdlX2RheXMiOjExLCJ2aV91c2VjYXNlX2lkIjoxMDA5OSwiZHVyYXRpb25fcyI6MTUsInVybGdlbl9zb3VyY2UiOiJ3d3cifQ%3D%3D&ccb=17-1&vs=93986893873e15f7&_nc_vs=HBksFQIYUmlnX3hwdl9yZWVsc19wZXJtYW5lbnRfc3JfcHJvZC80ODQ5NDdDRjM0NzcyNkI2RkE4RDZDRkFGQjQzQTlCQl92aWRlb19kYXNoaW5pdC5tcDQVAALIARIAFQIYUWlnX3hwdl9wbGFjZW1lbnRfcGVybWFuZW50X3YyLzkwNDc3OEIwRTY2MDYxNEY5RjA4MUMxQzNDMDVBMkFDX2F1ZGlvX2Rhc2hpbml0Lm1wNBUCAsgBEgAoABgAGwKIB3VzZV9vaWwBMRJwcm9ncmVzc2l2ZV9yZWNpcGUBMRUAACby1v-GuJTcBRUCKAJDMywXQC6qfvnbItEYEmRhc2hfYmFzZWxpbmVfMV92MREAdf4HZeadAQA&_nc_gid=RGZW24yVvqRHKLuQvnRkKQ&_nc_zt=28&_nc_ss=7a22e&oh=00_AQIrUM_ZU8b3LTxlgefSs0x6vjvCX9BV4piQfWrs74dXIg&oe=6A99835F",
"width": 1080,
"width_original": 1080
}
]
]
-732
View File
@@ -1,732 +0,0 @@
# gallery-dl — a CLI replacement for JDownloader2
Status: **in production.** All six ARTMS profiles are synced with
`scripts/gdl-sync.py`; JD2 is no longer used for them.
Everything below was measured against the live site and the real archive on
2026-08-16 and 2026-08-20, not inferred from documentation.
## Why gallery-dl and not a hand-rolled script
The hard parts of fetching Instagram are pagination, cookie handling, CDN URL
expiry and resumption. gallery-dl already has all of them, plus extractors that
map 1:1 onto our sidecar directory layout (`posts`, `reels`, `stories`,
`highlights`). Rolling our own would mean reimplementing the ban-sensitive part
by hand.
## The account was suspended on 2026-08-17 — read this first
> [!NOTE]
> **The scraping account was replaced on 2026-10-14.** It is now **`erg05teur`**,
> which is also signed in on the owner's phone (a separate session; the cookie
> gdl uses lives only in the dedicated Chrome on `mattellite`). Everything below
> about the suspension is the **previous** account's history and the reasons for
> every rule; none of it has stopped applying, and the new account has no track
> record, so treat it as a replacement account (see "For a replacement account"
> below). The new session cookie was created 2026-10-14 ~10:07 EDT.
>
> What this account has been asked since then, honestly counted, because the
> warm-up advice below was not followed to the letter: ~13 single-post / format
> probes (resolution check, 2026-10-14), plus ~80 listing requests from an
> **unplanned** reels catch-up run (installing the weekly profiles timer
> triggered it via `Persistent=true`; stopped partway, `heejin / reels`
> completed, `kimxxlip / reels` was abandoned ~7 min in). Scheduled runs were
> deliberately left on their normal timers by the owner.
The account used for all of the below was suspended the same day this tooling
was built, for "activity that doesn't follow our Community Standards on spam".
The fetching was not the expensive part. **Verification was.**
**It was restored, and synced normally again on 2026-08-20** — a full run
across all six profiles with 0 failures and 0 CDN 429s. That is not evidence
the limits were imagined; it is one data point on a restored account that has
been treated carefully since. Everything below still applies, and the budget is
still per session rather than per command.
What was actually spent against `instagram.com` in a few hours, from one
session and one IP:
| activity | rough requests | downloaded |
|---|---:|---|
| enumerating a profile grid by scrolling it in an automated browser | ~18 pages | nothing |
| the same profile again, after a bug in the scraping selector | ~18 pages | nothing |
| a Reels tab enumerated the same way | ~9 pages | nothing |
| full `-j` metadata dumps of one profile, twice | ~16 pages | nothing |
| `--simulate` runs over the same profile, three times | ~24 pages | nothing |
| single-post `/p/<code>/` fetches while testing filename formats | ~8 | a handful |
| an aborted sync that re-ran every listing pass before dying | ~40 pages | ~270 MB |
| the real sync, 24 sources across 6 profiles | ~150 pages | 2.2 GB |
The two rows that actually mattered to the archive are the last one and part of
the second-to-last. **Everything above them produced no files at all**, and
together they were a comparable number of requests.
The warnings arrived in this order and were each rationalised:
1. `429 Too Many Requests` from `scontent-*.cdninstagram.com`, losing two
videos. Treated as a pacing problem — pacing was lowered and the run
continued.
2. `400 Bad Request` from `/api/v1/highlights/<id>/highlights_tray/`, on an
endpoint that had worked hours earlier. Correctly read as a possible block;
requests stopped.
3. Suspension.
**Treat the first CDN 429 as a stop signal for the session, not a tuning
parameter.** It is the tolerant surface complaining; if that surface is
complaining, the rate-limited one has been unhappy for a while.
### Rules that follow from this
- **Count verification requests against the same budget as fetching.** A
`--simulate`, a `-j` dump and a browser scroll all hit `instagram.com` and
download nothing. Being read-only does not make them free; it makes them
invisible, which is worse.
- **Never enumerate the live site with an automated browser.** Scrolling a
214-post grid is ~18 paginated GraphQL loads at machine speed with no dwell
time between them. It is the most obviously non-human thing in this whole
document, and it was done here twice on one profile.
- **Verify against the archive, not against Instagram.** Every naming, dating
and classification question answered in this file could have been answered
from files already on disk plus a single listing pass.
- **`probe_live` is not cached, so every restart re-enumerates everything.**
The aborted run cost a full duplicate set of listing passes for five
profiles. Cache probe output to disk before running anything twice.
- **Budget per session, not per command.** Nothing in the tooling knows what
the last command spent.
### For a replacement account
(Applies to `erg05teur`, 2026-10-14 onward — see the note at the top of this
section for what it has already been asked.)
- Let it exist and be used normally for a while before pointing any tool at it.
- Keep the cookie on one machine and one public IP, as before.
- Start with a single small profile and stop for the day afterwards.
- Prefer Instagram's own "Download a copy" export where possible: it is
first-party, costs no scraping requests, and carries the metadata this whole
document works around not having.
## The safety model — read this before changing any option
The ban vector is **requests to `instagram.com`**, not bandwidth. See
`docs/jdownloader.md` for the history; Instaloader got this account banned by
asking `instagram.com` a question *per post*.
gallery-dl has two API backends and the difference is exactly that vector:
```python
if self.config("api") == "graphql":
self.api = InstagramGraphqlAPI(self) # per-post api.media() for every
else: # video and every carousel
self.api = InstagramRestAPI(self) # <- default, listing-only
```
The REST backend paginates at `count: 30` (feed) / `page_size: 50` (clips), and
those responses already carry `carousel_media`, `image_versions2`,
`video_versions` and `product_type`. **No per-post request.** A 300-post
profile costs roughly 10 requests to `instagram.com`.
Rules, in order of importance:
1. **`"api": "rest"` always.** Never `graphql`. This is the whole ballgame.
2. **Never enable `metadata`-style options that trigger extra calls.** If a
field is not already in the listing response, it is not worth a request.
3. **Pace it.** `"sleep-request": [4.0, 7.0]` — a randomised gap, not a fixed
one. Also `"sleep": [1.0, 3.0]` between downloads.
4. **Cap the download rate** (`downloader.http.rate`) so the CDN side looks like
a person, not a mirror.
5. **Run from the same public IP as the browser the cookie came from.** At time
of writing that is `mattellite` (`66.23.52.196`); the dev workstation is a
*different* public IP and using the cookie from there is precisely what
session-hijack detection looks for.
6. **No programmatic login, ever.** gallery-dl's username/password path is
disabled upstream anyway; use `--cookies-from-browser`.
Do not add proxy rotation, fingerprint spoofing or account rotation. Throttling
and request-avoidance are welcome; evasion is not.
### Cookies
The logged-in Chrome on `mattellite` runs with a non-default profile:
```
--user-data-dir=/home/matt/.config/google-chrome-devtools
```
so the cookie flag is:
```
--cookies-from-browser "chrome:/home/matt/.config/google-chrome-devtools"
```
Plain `--cookies-from-browser chrome` fails with "Unable to find chrome cookies
database" because it looks in `~/.config/google-chrome/`.
Anonymous access is **not** a viable fallback: it serves lower-resolution media,
caps profile pagination at 12 posts, and returns `AuthRequired` for stories and
highlights.
(Observed 2026-10-07, after the session died: with no `sessionid` gallery-dl did
not fall back to anonymous downloads at all — most sources failed with
`HTTP redirect to login page` / `HTTP redirect to home page`, and stories came
back as "No results". The lower-resolution claim above was **not** re-measured
then; what *was* measured is that archived image sizes match a logged-in fetch.)
**The session can die silently, and nothing about the browser window shows it** —
an open Instagram tab keeps rendering its logged-in UI until its next failed
request. The only reliable signal is the cookie DB: `scripts/check-session.py`
looks for an unexpired `sessionid` in this same profile, and `gdl-cron.sh` runs
it before every scheduled run (exit 76 = refuse to scrape). See `TOOLING.md`,
"2026-10-14".
## Output format
The viewer's parser is the contract, not JD2's exact bytes. `EXPORT_RE` in
`src/lib/archive-patterns.ts` accepts all of these, and normalises the index
with `parseInt`, so **JD2 and gallery-dl naming interoperate**:
```
"… - CrORBIcJJbM.mp4" -> postId=CrORBIcJJbM index=1
"… - CrORBIcJJbM - 1.mp4" -> postId=CrORBIcJJbM index=1
"… - C53YPQzp7Wj - 09.jpg" -> postId=C53YPQzp7Wj index=9
```
That means zero-padding and the presence/absence of ` - N` on single-media posts
are cosmetic. Don't spend effort forcing them.
### Directory layout
| kind | directory | note |
|---|---|---|
| posts | `<user>` | |
| reels | `<user> - reels` | |
| stories | `story - <user>` | |
| highlights | `story highlights - <user> - <title>` | |
**Force the directory with `-D`; never use `{username}` for it.** A profile's
reels tab returns *collab reels owned by other accounts* — `/0ct0ber19/reels/`
served 6 reels owned by `official_artms` and 1 by `chuuo3o`. With
`{username}` those would scatter into `official_artms - reels/`. JD2 got this
right and the archive proves it: `chuuo3o` and `official_artms` filenames sit
inside `0ct0ber19 - reels/`.
So: **owner in the filename, crawl scope in the directory.**
### Filenames
```jsonc
"filename": {
"sidecar_shortcode and count >= 10":
"{date:Olocal/%Y-%m-%d}_{username} - {post_shortcode} - {num:02}.{extension}",
"sidecar_shortcode":
"{date:Olocal/%Y-%m-%d}_{username} - {post_shortcode} - {num}.{extension}",
"":
"{date:Olocal/%Y-%m-%d}_{username} - {post_shortcode}.{extension}"
}
```
`sidecar_shortcode` is set only when the post is a carousel, so it is the
carousel discriminator. Conditions are evaluated in order, first match wins
(`path.py:265`).
Stories and highlights use the per-item `{shortcode}`, not `{post_shortcode}`
(which is the *reel's* id, shared by every item in it):
```
"{date:Olocal/%Y-%m-%d}_{username} - {shortcode}.{extension}"
```
`{date}` on a story/highlight file is the **per-item** `taken_at`
(`instagram.py:337` prefers `item["taken_at"]`), verified on a 154-item
highlight whose items carried distinct times while `post_date` stayed pinned to
the reel. Highlights therefore gain real dates — today they fall back to
directory mtime.
### The timezone is not UTC
JD2 stamped filenames in **desktop local time (US Eastern)**. Measured across
212 comparable posts:
| model | mismatches |
|---|---:|
| UTC | 19 |
| UTC−5 (EST) | 10 |
| UTC−4 (EDT) | **0** |
| America/New_York (DST-aware) | **0** |
`{date:Olocal/%Y-%m-%d}` uses the machine's local zone with per-timestamp DST
awareness, which reproduces it — `mattellite` is `America/Toronto`, the same
offsets. Note the **trailing `/` must be omitted**: `Olocal/%Y-%m-%d/` puts the
separator into the strftime format and it sanitises to an underscore, giving
`2026-08-15__0ct0ber19`.
If the sync ever moves to a host in another timezone, set an explicit
`{date:O-4/…}` or the dates will silently shift for ~9% of posts.
### Caption sidecars
JD2 writes one `.txt` per post, named without the index, containing the caption
with **no trailing newline**, and writes nothing when the caption is empty
(measured: 197 of 217 posts, 86 of 86 reels, 0 of 10 stories, 0 of 16
highlights). gallery-dl reproduces this exactly with the default
`"empty": false`:
```jsonc
{ "name": "metadata", "event": "post", "mode": "custom",
"content-format": "{description}", "extension": "txt",
"filename": "{date:Olocal/%Y-%m-%d}_{username} - {post_shortcode}.txt" }
```
`"event": "post"` is what makes it one file per post rather than per media file.
### Metadata sidecar (new — JD2 had no equivalent)
```jsonc
{ "name": "metadata", "event": "post", "mode": "json",
"filename": "{date:Olocal/%Y-%m-%d}_{username} - {post_shortcode}.json",
"include": ["post_shortcode","post_id","type","date","post_date","username",
"fullname","owner_id","description","count","likes","post_url",
"sidecar_shortcode"] }
```
Use **`include`**, not `fields` — `fields` is for `mode: custom` and silently
does nothing here, leaving `audio_user` blobs (including another user's profile
picture URL) in the output.
The payoff is `type`, which is Instagram's own classification:
```json
{ "post_shortcode": "DbdG9L9jU4m", "type": "post", "count": 2 } // feed video
{ "post_shortcode": "Db-lNCoib9m", "type": "reel", "count": 1 } // real reel
```
This is the `product_type: "clips"` signal, delivered free in the listing
response. It is the authoritative answer to "is this a reel", and would let the
viewer retire the lone-video heuristic in `src/lib/post-tabs.ts` — see
"Scanner work" below.
**`type` is only populated by listing extractors.** Extracting a single
`/p/<shortcode>/` URL leaves it `null`. Sync always uses listing URLs, so this
only matters when testing by hand.
### Collab posts: the JSON and the media disagree about whose post it is
Verified directly against two saved raw API responses (`docs/example-api-
response-DcU3dM-gVgB.json`, a 2-way collab, and `docs/example-api-response-
DcLDme7zoyo.json`, a 5-way collab owned by an external account), 2026-09-01.
For an Instagram Collab, `username`/`fullname`/`owner_id` in the listing
response are always the **original poster's**, never the scraped account's —
even when the scraped account is one of the collaborators, not the owner. The
metadata `.json` sidecar (built from `include`, above) is filed under that
same original-poster identity, since its filename template uses
`{username}`. But the **media file** for that same post is written into
whichever profile's own crawl directory triggered the download, and its
filename's `{username}` slot took the *scraped* account's name, not the true
owner's. Net effect: one physical post produces a JSON named for the real
owner and a media file (or files, for a carousel) named for whoever we were
crawling — two different identities for one post, in the same directory.
Downstream (`cosmo_normalize_instagram.py` in `Cosmo-Live-Downloads`) had to
stop matching media to its JSON by username and match on shortcode/`{num}`
only, and separately built a cross-profile `collab_with` pass so every
participant's page shows the post, not just the one whose directory JD2/
gallery-dl happened to land the JSON in. See that repo's `NOTES.md`,
2026-09-01 entries, for the full fix.
**`coauthors` is a native, richer signal for this, and is now captured.**
The raw API response carries a `coauthors` array (`{"full_name", "id",
"username"}` per collaborator) that **excludes the post's own owner** —
confirmed on both saved examples, including one where the owner
(`dailyfashion_news`) is an external account with no ARTMS members in her
own name, present only via `coauthors`. Added to the post-level metadata
JSON's `include` list on 2026-09-01: a direct field instead of inferring a
collab from filename/directory identity mismatches, and it never needs the
"is this the owner" branch `coauthors` already excludes for us.
**Per-carousel-item fields got their own sidecar, added the same day.**
`width`, `height`, `width_original`, `height_original` and `tagged_users`
live on the per-FILE kwdict, not the per-post one the metadata JSON above
reads — a carousel's items can each have different dimensions and tags,
which one post-level JSON can't represent. `gdl-sync.py`'s `media_pp` is a
second `metadata` postprocessor, `event: "file"` (gallery-dl's default when
omitted), so it runs once per downloaded file and writes `<filename>.json`
alongside it — e.g. `... - 01.jpg.json` next to `... - 01.jpg`, never
colliding with the post-level `....json`, which has no per-item number.
`owner` — a full user object (profile pic URLs, privacy flags) for whoever
posted that specific item — is deliberately left out, the same reasoning as
`audio_user` above. Verified against the same two saved examples: correct
per-item `width`/`height` and `tagged_users` came back on a live re-fetch of
an already-archived carousel, with zero media re-downloaded (the existing
skip-archive still applies; only the new sidecars are new files).
### The shared archive-db dedups media across profiles too, not just within one
The skip-archive DB (see "Incremental sync" above) keys purely on
`instagram_<media_id>`, with no per-profile scoping. When the *same* media_id
is reachable from more than one profile's listing — a Collab post, or a
repost — whichever profile's crawl reaches it **first** downloads the file;
every other profile that later lists the same media_id sees it as
already-in-the-archive-db and skips the download, even though that file has
never actually landed in *that profile's own* directory tree. The metadata
`.json`/`.txt` sidecars are written regardless (they aren't gated by the
download-archive), so the symptom is a JSON with zero matching media files
in its own directory — 510 such posts were found across the real archive on
2026-09-01, entirely from this mechanism, not from anything actually missing
from Instagram.
This isn't fixable on the gallery-dl side without per-profile archive DBs
(which would defeat the point of skip-archive — re-downloading anything a
sibling profile already fetched). The fix lives downstream instead: `cosmo_
normalize_instagram.py` builds one archive-wide `shortcode → path` index once
and falls back to it when a post's own directory has no matching media. Worth
knowing before assuming a JSON-with-no-media post reflects a real scrape gap.
## Cadence, and the budget that enforces it
**Monthly for everything, daily for stories only.** Stories expire in 24h and
cannot be backfilled, so they are the one surface where missing a day means
losing the content permanently. Everything else can wait — the skip-archive
means an infrequent full sync costs barely more than a frequent one, because it
only fetches what is new.
```
# weekly, everything
gdl-sync.py --index <viewer-url> --staging ~/gdl/staging \
--publish <user>@<nas>:<archives> --archive-db ~/gdl/artms.db \
--urls-file artms_account_links.txt --execute
# daily, stories only -- one request per profile
gdl-sync.py ... --only stories --execute
```
A stories-only run is one source per profile and **never seeds**, because a
story cannot be in the archive before it is fetched; probing would double the
cost of the cheapest surface for no benefit. Six profiles is a handful of
requests.
When scheduling it, **randomise the minute and avoid the hour boundary**. A job
that fires at exactly 09:00 every day is a machine; one that fires somewhere in
a window looks like someone opening the app.
The tool now refuses to repeat itself:
| flag | default | what it prevents |
|---|---|---|
| `--min-interval` | 20h | re-fetching a source touched recently — the aborted-restart case that re-enumerated five profiles |
| `--probe-ttl` | 24h | paying for a listing pass twice within a run cycle |
| `--max-sources` | off | a runaway list touching more than intended |
| `--force` | off | (escape hatch: ignores both guards) |
State lives beside the archive DB as `<db>.state.json`, recording per source
when it was seeded and last fetched. **Seeding is a one-time bootstrap**: after
the first successful sync the archive DB records everything gallery-dl has
seen, so the source is never probed again. That is the single biggest saving
here — a second full sync costs roughly half what the first did.
## Incremental sync — why the fetch host needs no copy of the archive
gallery-dl can skip already-held media two ways, and the difference decides
whether the fetcher needs the archive mounted:
- **By file existence** (default). Needs the destination to already contain the
files, so it only works if the archive is mounted where gallery-dl writes.
- **By skip-archive** (`--download-archive`). A sqlite DB of ids. Needs nothing
on disk.
We use the second, so the fetch host can write to **local disk and rsync
afterwards**. That avoids writing tens of thousands of small files over CIFS,
and keeps a mid-sync failure from leaving partial files on the live Resilio
share.
The key is `archive_prefix + archive_fmt`, which for this extractor is the
literal `instagram` plus the per-media numeric pk (`instagram.py:25`,
`job.py:713-719`). Verified: a 3-image carousel produced
```
instagram3079387627521318672
instagram3079387627521429433
instagram3079387627529716672
```
and a second run skipped every media file, rewriting only the idempotent
`.txt`/`.json` sidecars.
**Seeding.** `media_id` is not in our filenames, so the DB cannot be built from
names alone — but one listing pass (the pass we make anyway) maps every live
item to its `media_id`, and the archive's *file listing* says which we already
hold. No extra Instagram requests, and no archive content — a listing is
enough, which `GET /api/archives/:name/files` already serves.
Measured on `0ct0ber19`: 2275 live media items, 2248 seeded from the existing
listing, **27 left to download** — precisely the media of the two posts added
since the last crawl.
The one trap, which silently seeds almost nothing if you get it backwards:
| surface | filed under | why |
|---|---|---|
| posts, reels | `post_shortcode` | carousel children each have their own `shortcode`, which never appears in a filename |
| stories, highlights | `shortcode` (per item) | `post_shortcode` is the containing reel's id, shared by every item |
`live_key()` encodes this. Matching on the wrong field seeded 5 of 2275.
### The skip-archive saves the CDN, not `instagram.com`
Worth being exact about, because the two costs land on different surfaces and
only one of them bans accounts:
| what | which surface | scales with |
|---|---|---|
| downloading media | `scontent-*.cdninstagram.com` | how much is **new** |
| enumerating the profile to find it | `instagram.com` | how **big** the profile is |
The skip-archive suppresses the first. It does nothing about the second, so a
2275-post profile costs ~76 pages of pagination every run, forever, whether it
has three new posts or none. Seeding (above) saved a *second* full pass, not
the first.
Measured on the 2026-08-20 run, from sidecar write times in staging — free,
since the run was paying for the listing anyway:
```
1787248852 2026-08-19 … DcOeoVxkthi new, +0s
1787248944 2026-08-18 … DcLpfoJCZtp new, +92s
1787249058 2026-08-17 … DcIlGbxCUk0 new, +114s
1787249162 2026-07-24 … DbKr1TxlPSX ┐ all one second: nothing
1787249162 2026-08-15 … DcD-FdBCYGm ┘ downloaded, sidecars only
```
Three posts took ~100s each; the remaining 2272 were enumeration with nothing
to show for it.
**Pinned posts do not break early abort.** Test case 16 previously claimed
`0ct0ber19` returns its 3 pinned posts out of date order — that is true of the
*web grid*, but the REST `/posts/` listing came back strictly
reverse-chronological, newest first, no hoisting. That matters because
front-loaded old posts are the one thing that would make `skip: abort:N`
dangerous: it would trip on them and abort before reaching anything new.
So `skip: abort:N` is viable, and cuts ~420 requests per run to ~40-60:
| surface | live items | pages | with `abort:50` |
|---|---:|---:|---:|
| posts, 6 profiles | 11,248 | ~377 | ~12 |
| reels, 6 profiles | 1,080 | ~24 | ~8 |
| stories + highlights | — | ~20 | ~20 |
N counts consecutive skipped **files**, not posts, so it must clear the largest
already-held carousel — `DcD-FdBCYGm` alone is 22 media. 50 is comfortable; 5
would not be.
**The tradeoff is edited carousels.** Test case 15 is a post that gained items
after we archived it, and only a full enumeration finds those. Suggested
policy: `abort:50` for routine runs, a full sweep occasionally.
Measured the same day, resuming a stopped run with `--abort 50`:
| source | live items | enumerated |
|---|---:|---:|
| `cher_ryppo` posts | 2,151 | **7** |
| `cher_ryppo` reels | 92 | 53 |
One page instead of 72, and every new post was still caught. The 7 is roughly
3 new posts plus 4 already-held carousels making up the 50 skipped files.
Reels need 53 because they are single-media, so 50 consecutive skips really is
50 reels — another reminder that N counts files, and that the same N behaves
very differently on a carousel-heavy surface than on a reels tab.
## Publishing
The fetch host stages to local disk and rsyncs afterwards. `rsync
--ignore-existing` is not an optimisation but the safety property: the archive
deliberately outlives Instagram, so publishing must only ever **add**. No
`--delete`, and nothing already present is overwritten — including sidecars,
which are rewritten every run and would otherwise churn the synced share.
Publishing happens once at the end of a run, so a profile that fails midway
never reaches the archive half-written.
## Status
In use for all six ARTMS profiles.
`withaseul` first — 322 files added (74 media, 241 `.json`, 7 `.txt`), nothing
overwritten or deleted. Of the 74 new media, **zero** duplicated media already
held under a different name, which is the check that says JD2 and gallery-dl
naming really do converge.
**2026-08-20**, the first full incremental sync, four days after the previous
one. 184 new media, 299 files published, 0 failures and **0 CDN 429s**:
| profile | posts | reels | stories | files added |
|---|---:|---:|---:|---:|
| 0ct0ber19 | 58 | 2 | 4 | +77 |
| official_artms | 12 | — | 2 | +85 |
| cher_ryppo | 41 | 1 | 8 | +63 |
| zindoriyam | 23 | — | 4 | +35 |
| kimxxlip | 16 | — | 2 | +23 |
| withaseul | 10 | — | — | +16 |
The 20 story items are the part that could not have been recovered later.
Two things made it cheap, and both are worth keeping:
- The archive DB was already seeded from the previous run, so `--min-interval`
and the recorded `seeded` state meant **no probe passes at all**. A state
file has to exist for this; if one is missing after a manual run, write it
rather than letting the tool re-seed 24 sources.
- `--abort 50` (see above) cut the remaining listing cost by roughly 85%.
The run was deliberately **stopped and resumed** halfway to pick up `--abort`.
That is safe precisely because of the state file: the 12 finished sources were
already marked `fetched`, so the 20h floor skipped them and only the remaining
12 re-ran. Stopping a run is cheap now; it was not before.
Published files land owned by the SSH user rather than `rslsync`. The viewer
reads them fine (world-readable), but Resilio does not own what it syncs; worth
a `chown` if that ever matters. This also makes **`rsync` exit 23**
("some files/attrs were not transferred") the *normal* outcome of a publish —
it is the failed `chown`, not lost data. Confirm by re-running the same rsync
with `--dry-run`: an empty file list means everything arrived.
The profiles to fetch live in `artms_account_links.txt` at the archive root,
passed with `--urls-file`.
## Verified run
`withaseul`, all four surfaces, staged locally and published to a scratch
directory before the live publish above:
```
==> withaseul / posts seeded 915 of 984 live items
==> withaseul / reels seeded 28 of 34 live items
==> withaseul / stories no results (none active)
==> withaseul / highlights no results
```
Output landed correctly, including the collab-reel case — `withaseul - reels`
contains 53 files owned by `withaseul`, 10 by `cher_ryppo`, 3 by `0ct0ber19`
and 2 by `official_artms`, all with the owner in the filename and the crawl
scope as the directory.
### The CDN rate-limits, and the first run tripped it
At `rate: 3M` with `sleep: [1.0, 3.0]`, `scontent-*.cdninstagram.com` returned
**`429 Too Many Requests`** and two videos were lost (gallery-dl retried, then
gave up with exit 4). This is the *tolerant* surface complaining, which is a
clear signal the pacing was too aggressive.
Defaults are now:
| option | value |
|---|---|
| `--rate` | `1M` |
| `--sleep-request` | 6–10 s |
| `--sleep` | 3–6 s |
| `sleep-429` | 120 s |
| `retries` (extractor and downloader) | 8 |
Re-running with those recovered both videos and produced **0 failures and 0
429s**. Do not raise them for speed; an archive sync has no deadline.
### yt-dlp is worth installing
Without it, gallery-dl logs `Cannot import yt-dlp or youtube-dl` and falls back
to a progressive URL for DASH videos. The fallback mostly works but is what the
429s hit hardest.
**`pipx install yt-dlp` does not work** — it was the advice here until
2026-08-20, and it is wrong. It gives yt-dlp its own venv, so the binary lands
on `PATH` while gallery-dl, in a *different* venv, still cannot `import yt_dlp`.
The symptom is that everything looks installed and the log keeps saying
`Cannot import yt-dlp`. gallery-dl needs it importable, not runnable:
```sh
pipx inject gallery-dl yt-dlp
```
Verify by asking gallery-dl's own interpreter, not the shell:
```sh
/home/matt/.local/share/pipx/venvs/gallery-dl/bin/python -c 'import yt_dlp'
```
## Known quirks
- **`count` is not the emitted file count.** For 135 of 214 posts it was exactly
one higher than the number of files written. This makes the `count >= 10`
padding condition mis-pad a handful of 9-item posts (10 of 214 measured). Since
the parser normalises the index, this is cosmetic — but it means a re-fetch
over an existing JD2 tree writes `- 01.jpg` beside an existing `- 1.jpg`.
- **Carousels get edited.** Two posts had a different media count live than on
disk. Padding width follows the count *at download time*, so a grown carousel
produces mixed widths — the archive already contains one such post from JD2.
- **Highlights already have two naming styles on disk**, and every undated file
has a dated twin. The scanner dedupes by index so they render once; it is
wasted disk, not a display bug.
- **Some `video_versions` entries are VP9, and format selection is
codec-blind.** The extractor picks `max(video_versions, key=lambda x:
(x["width"], x["height"], x["type"]))` — resolution only, no codec check
(`instagram.py`). Instagram appears to have started serving VP9-encoded
highest-resolution variants for some posts around when the new gdl-based
workflow started (2026-08); 87 such files were found archive-wide on
2026-09-01. VP9-in-MP4 plays fine everywhere gallery-dl was tested from
except **Safari/WebKit**, which wires VP9 decode only into its WebM
demuxer, never its MP4/ISOBMFF path — confirmed via WebKit bug trackers,
not guessed. Fetching a lower-resolution non-VP9 variant instead was
considered and rejected (quality loss); the fix is downstream, a one-time
`-c:v copy -c:a libopus` remux to `.webm` (`cosmo_remux_instagram_vp9.py`
in `Cosmo-Live-Downloads`) that keeps VP9 losslessly and only re-encodes
audio (WebM disallows AAC). Confirmed live via gallery-dl/yt-dlp that
Instagram never offers a native WebM option to request instead — this has
to be done locally, there's no source-side fix.
## Scanner work — done
Shipped in `53b1f80` ("read gallery-dl sidecars for reel type and post dates").
`useArchiveScanner` tells the three `.json` shapes apart **structurally**, not
by filename, in `src/lib/gallery-dl-sidecar.ts`:
1. Instagram export manifests (`posts_1.json`) — top-level `media` array.
2. Instaloader `.json.xz` — GraphQL node under `node` / `__typename`.
3. gallery-dl `.json` — flat, `post_shortcode` + `type`, none of the above.
`post.isReel` now comes from the sidecar's `type`, which is Instagram's own
classification, and beats every fallback in `post-tabs.ts`. Post dates are
ranked rather than last-write-wins (`src/lib/post-dates.ts`): sidecar beats
filename beats mtime.
Note those files live on **`main`** — they parse the archive at display time
and are viewer code, not fetching tooling.
## Test cases
Real subjects, all present in the archive today. See
`scripts/gdl-sync.py --selftest` for the harness.
| # | case | shortcode | expected |
|---|---|---|---|
| 1 | single image | `CwcXnQhOqFG` | one `.jpg`, no index |
| 2 | single feed video | `DbdG9L9jU4m` | one `.mp4`, `type: post` |
| 3 | carousel, images only | `Cq8LrxSJAJE` | `- 1 … - 3` |
| 4 | carousel, image + video | `CtohvHxLnWO` | `- 1.jpg … - 4.mp4`, **no `.txt`** |
| 5 | carousel of exactly 9 | `Cv2Hb_brx_N` | 1-digit index |
| 6 | carousel of 10+ | `CzM8Uf6B6H_` | 2-digit index `- 01 … - 10` |
| 7 | reel shown on the posts grid | `C8FHM6EJl15` | in `<user>`, `type: reel` |
| 8 | reel on the reels tab | `Db-lNCoib9m` | in `<user> - reels`, `type: reel` |
| 9 | collab reel (other owner) | `DYcZOb0h6Sv` | dir `0ct0ber19 - reels`, filename `chuuo3o` |
| 10 | story | live only | `story - <user>`, per-item shortcode + date |
| 11 | story highlight | `C-IImhvpFuk` | `story highlights - <user> - <title>` |
| 12 | highlight, unicode title | `Drawheeing⠀` | trailing U+2800 preserved in dirname |
| 13 | empty caption | `CrdsY5CrSsO` | media written, `.txt` absent |
| 14 | deleted post | `C0TgI7sphfZ` | on disk, absent live — must not be removed |
| 15 | edited carousel | `C7zG7-jJMlq` | 18 on disk, 8 live — must not be removed |
| 16 | pinned posts | `0ct0ber19` | REST listing is strictly reverse-chronological; see below |
| 17 | profile avatar | `0ct0ber19.jpg` | base dir, undated |
Cases 14–16 are reconciliation, not naming: **a sync must never delete**, since
the archive deliberately outlives Instagram.
Not covered, decide before relying on them: the `/reposts/` tab (`0ct0ber19`
has one) and `/tagged/`. Neither is fetched today.
-184
View File
@@ -1,184 +0,0 @@
slug,username,archive_dir,shortcode,type,date,post_url,json_file
choerry,cher_ryppo,cher_ryppo,C5SBC--Syik,post,2024-04-03 01:50:25,https://www.instagram.com/p/C5SBC--Syik/,2024-04-02_cher_ryppo - C5SBC--Syik.json
choerry,cher_ryppo,cher_ryppo,C811ib_ys7D,post,2024-06-30 13:16:41,https://www.instagram.com/p/C811ib_ys7D/,2024-06-30_cher_ryppo - C811ib_ys7D.json
choerry,cher_ryppo,cher_ryppo,C9JTMgnSCHF,post,2024-07-08 02:41:25,https://www.instagram.com/p/C9JTMgnSCHF/,2024-07-07_cher_ryppo - C9JTMgnSCHF.json
choerry,cher_ryppo,cher_ryppo,C-K6kxZy6qq,post,2024-08-02 14:16:09,https://www.instagram.com/p/C-K6kxZy6qq/,2024-08-02_cher_ryppo - C-K6kxZy6qq.json
choerry,cher_ryppo,cher_ryppo,DBXI0zVs_Iv,post,2024-10-20 21:45:44,https://www.instagram.com/p/DBXI0zVs_Iv/,2024-10-20_cher_ryppo - DBXI0zVs_Iv.json
choerry,cher_ryppo,cher_ryppo,DRRDiezjdyV,post,2025-11-20 05:26:24,https://www.instagram.com/p/DRRDiezjdyV/,2025-11-20_cher_ryppo - DRRDiezjdyV.json
haseul,withaseul,withaseul,Cqps3ZWBlav,post,2023-04-05 10:44:56,https://www.instagram.com/p/Cqps3ZWBlav/,2023-04-05_withaseul - Cqps3ZWBlav.json
haseul,withaseul,withaseul,CqsjfUzSelq,post,2023-04-06 13:20:43,https://www.instagram.com/p/CqsjfUzSelq/,2023-04-06_withaseul - CqsjfUzSelq.json
haseul,withaseul,withaseul,CqxiTfjhNJ2,post,2023-04-08 11:46:34,https://www.instagram.com/p/CqxiTfjhNJ2/,2023-04-08_withaseul - CqxiTfjhNJ2.json
haseul,withaseul,withaseul,Cq4qymBhftL,post,2023-04-11 06:15:24,https://www.instagram.com/p/Cq4qymBhftL/,2023-04-11_withaseul - Cq4qymBhftL.json
haseul,withaseul,withaseul,CrnhJtrhoZi,post,2023-04-29 10:55:29,https://www.instagram.com/p/CrnhJtrhoZi/,2023-04-29_withaseul - CrnhJtrhoZi.json
haseul,withaseul,withaseul,Cr7fZZyB4Sp,post,2023-05-07 05:04:58,https://www.instagram.com/p/Cr7fZZyB4Sp/,2023-05-07_withaseul - Cr7fZZyB4Sp.json
haseul,withaseul,withaseul,Cr8QeaFhFsq,post,2023-05-07 12:13:49,https://www.instagram.com/p/Cr8QeaFhFsq/,2023-05-07_withaseul - Cr8QeaFhFsq.json
haseul,withaseul,withaseul,CsBIA4HBUOX,post,2023-05-09 09:36:05,https://www.instagram.com/p/CsBIA4HBUOX/,2023-05-09_withaseul - CsBIA4HBUOX.json
haseul,withaseul,withaseul,CsgbxFDhNmj,post,2023-05-21 13:25:08,https://www.instagram.com/p/CsgbxFDhNmj/,2023-05-21_withaseul - CsgbxFDhNmj.json
haseul,withaseul,withaseul,Csn9aO4B1r-,post,2023-05-24 11:33:48,https://www.instagram.com/p/Csn9aO4B1r-/,2023-05-24_withaseul - Csn9aO4B1r-.json
haseul,withaseul,withaseul,CsvY2jrBjMc,post,2023-05-27 08:48:17,https://www.instagram.com/p/CsvY2jrBjMc/,2023-05-27_withaseul - CsvY2jrBjMc.json
haseul,withaseul,withaseul,CtEHmRDPesc,post,2023-06-04 10:01:34,https://www.instagram.com/p/CtEHmRDPesc/,2023-06-04_withaseul - CtEHmRDPesc.json
haseul,withaseul,withaseul,CtRmUjsBUoc,post,2023-06-09 15:40:09,https://www.instagram.com/p/CtRmUjsBUoc/,2023-06-09_withaseul - CtRmUjsBUoc.json
haseul,withaseul,withaseul,Cuzar_3LqAA,post,2023-07-17 15:24:04,https://www.instagram.com/p/Cuzar_3LqAA/,2023-07-17_withaseul - Cuzar_3LqAA.json
haseul,withaseul,withaseul,CvHe1VMr7C-,post,2023-07-25 10:25:06,https://www.instagram.com/p/CvHe1VMr7C-/,2023-07-25_withaseul - CvHe1VMr7C-.json
haseul,withaseul,withaseul,CvhRFq7rnHc,post,2023-08-04 10:45:15,https://www.instagram.com/p/CvhRFq7rnHc/,2023-08-04_withaseul - CvhRFq7rnHc.json
haseul,withaseul,withaseul,CwIr3qEBPgF,post,2023-08-19 18:09:39,https://www.instagram.com/p/CwIr3qEBPgF/,2023-08-19_withaseul - CwIr3qEBPgF.json
haseul,withaseul,withaseul,CwzPlMThh5m,post,2023-09-05 06:49:48,https://www.instagram.com/p/CwzPlMThh5m/,2023-09-05_withaseul - CwzPlMThh5m.json
haseul,withaseul,withaseul,Cw9s3LNBhsq,post,2023-09-09 08:18:04,https://www.instagram.com/p/Cw9s3LNBhsq/,2023-09-09_withaseul - Cw9s3LNBhsq.json
haseul,withaseul,withaseul,CxIma4nBlaP,post,2023-09-13 13:53:26,https://www.instagram.com/p/CxIma4nBlaP/,2023-09-13_withaseul - CxIma4nBlaP.json
haseul,withaseul,withaseul,CxLcZ2jBME5,post,2023-09-14 16:23:38,https://www.instagram.com/p/CxLcZ2jBME5/,2023-09-14_withaseul - CxLcZ2jBME5.json
haseul,withaseul,withaseul,CxN3g95hVDH,post,2023-09-15 14:59:00,https://www.instagram.com/p/CxN3g95hVDH/,2023-09-15_withaseul - CxN3g95hVDH.json
haseul,withaseul,withaseul,CxTGBb_BAYw,post,2023-09-17 15:41:59,https://www.instagram.com/p/CxTGBb_BAYw/,2023-09-17_withaseul - CxTGBb_BAYw.json
haseul,withaseul,withaseul,CxnTyJIhUc0,post,2023-09-25 12:07:02,https://www.instagram.com/p/CxnTyJIhUc0/,2023-09-25_withaseul - CxnTyJIhUc0.json
haseul,withaseul,withaseul,CxsPZAur6Xv,post,2023-09-27 10:04:51,https://www.instagram.com/p/CxsPZAur6Xv/,2023-09-27_withaseul - CxsPZAur6Xv.json
haseul,withaseul,withaseul,Cx7mOFqhU7R,post,2023-10-03 09:12:57,https://www.instagram.com/p/Cx7mOFqhU7R/,2023-10-03_withaseul - Cx7mOFqhU7R.json
haseul,withaseul,withaseul,CyqV0lfruX-,post,2023-10-21 12:53:58,https://www.instagram.com/p/CyqV0lfruX-/,2023-10-21_withaseul - CyqV0lfruX-.json
haseul,withaseul,withaseul,Cy7RByYh2BH,post,2023-10-28 02:39:10,https://www.instagram.com/p/Cy7RByYh2BH/,2023-10-27_withaseul - Cy7RByYh2BH.json
haseul,withaseul,withaseul,Cz7nefxBJT0,post,2023-11-22 02:26:43,https://www.instagram.com/p/Cz7nefxBJT0/,2023-11-21_withaseul - Cz7nefxBJT0.json
haseul,withaseul,withaseul,C0YxsROhG4W,post,2023-12-03 10:13:57,https://www.instagram.com/p/C0YxsROhG4W/,2023-12-03_withaseul - C0YxsROhG4W.json
haseul,withaseul,withaseul,C0zOk2JBXTY,post,2023-12-13 16:46:36,https://www.instagram.com/p/C0zOk2JBXTY/,2023-12-13_withaseul - C0zOk2JBXTY.json
haseul,withaseul,withaseul,C1q_9TzBrdt,post,2024-01-04 08:36:20,https://www.instagram.com/p/C1q_9TzBrdt/,2024-01-04_withaseul - C1q_9TzBrdt.json
haseul,withaseul,withaseul,C1t-aeHBCef,post,2024-01-05 12:20:34,https://www.instagram.com/p/C1t-aeHBCef/,2024-01-05_withaseul - C1t-aeHBCef.json
haseul,withaseul,withaseul,C1yrgmMhsmz,post,2024-01-07 08:11:35,https://www.instagram.com/p/C1yrgmMhsmz/,2024-01-07_withaseul - C1yrgmMhsmz.json
haseul,withaseul,withaseul,C1yrxqIhGBZ,post,2024-01-07 08:13:54,https://www.instagram.com/p/C1yrxqIhGBZ/,2024-01-07_withaseul - C1yrxqIhGBZ.json
haseul,withaseul,withaseul,C10p70xh-6x,post,2024-01-08 02:36:18,https://www.instagram.com/p/C10p70xh-6x/,2024-01-07_withaseul - C10p70xh-6x.json
haseul,withaseul,withaseul,C2APujnBSF-,post,2024-01-12 14:38:11,https://www.instagram.com/p/C2APujnBSF-/,2024-01-12_withaseul - C2APujnBSF-.json
haseul,withaseul,withaseul,C3enxknhzeF,post,2024-02-18 06:16:55,https://www.instagram.com/p/C3enxknhzeF/,2024-02-18_withaseul - C3enxknhzeF.json
haseul,withaseul,withaseul,C4f8UQIBZAe,post,2024-03-14 15:07:03,https://www.instagram.com/p/C4f8UQIBZAe/,2024-03-14_withaseul - C4f8UQIBZAe.json
haseul,withaseul,withaseul,C4qIrJTh3po,post,2024-03-18 14:07:26,https://www.instagram.com/p/C4qIrJTh3po/,2024-03-18_withaseul - C4qIrJTh3po.json
haseul,withaseul,withaseul,C4rjDNhhJu8,post,2024-03-19 03:17:09,https://www.instagram.com/p/C4rjDNhhJu8/,2024-03-18_withaseul - C4rjDNhhJu8.json
haseul,withaseul,withaseul,C4xSiGGhP40,post,2024-03-21 08:48:16,https://www.instagram.com/p/C4xSiGGhP40/,2024-03-21_withaseul - C4xSiGGhP40.json
haseul,withaseul,withaseul,C5GBz4FB-kE,post,2024-03-29 10:06:12,https://www.instagram.com/p/C5GBz4FB-kE/,2024-03-29_withaseul - C5GBz4FB-kE.json
haseul,withaseul,withaseul,C5GJmlABYug,post,2024-03-29 11:14:17,https://www.instagram.com/p/C5GJmlABYug/,2024-03-29_withaseul - C5GJmlABYug.json
haseul,withaseul,withaseul,C5LgFFtB8ri,post,2024-03-31 13:06:54,https://www.instagram.com/p/C5LgFFtB8ri/,2024-03-31_withaseul - C5LgFFtB8ri.json
haseul,withaseul,withaseul,C5VZGPDB-Qo,post,2024-04-04 09:18:17,https://www.instagram.com/p/C5VZGPDB-Qo/,2024-04-04_withaseul - C5VZGPDB-Qo.json
haseul,withaseul,withaseul - reels,C5bWrT3BaGy,post,2024-04-06 16:52:50,https://www.instagram.com/p/C5bWrT3BaGy/,2024-04-06_withaseul - C5bWrT3BaGy.json
haseul,withaseul,withaseul,C6LAH8vBs_a,post,2024-04-25 04:59:04,https://www.instagram.com/p/C6LAH8vBs_a/,2024-04-25_withaseul - C6LAH8vBs_a.json
haseul,withaseul,withaseul,C6ypIzVBmiL,post,2024-05-10 14:27:49,https://www.instagram.com/p/C6ypIzVBmiL/,2024-05-10_withaseul - C6ypIzVBmiL.json
haseul,withaseul,withaseul,C68cTpzhUzQ,post,2024-05-14 09:48:07,https://www.instagram.com/p/C68cTpzhUzQ/,2024-05-14_withaseul - C68cTpzhUzQ.json
haseul,withaseul,withaseul,C68oiiahwZd,post,2024-05-14 11:35:00,https://www.instagram.com/p/C68oiiahwZd/,2024-05-14_withaseul - C68oiiahwZd.json
haseul,withaseul,withaseul,C7Oh6mYB7oF,post,2024-05-21 10:23:27,https://www.instagram.com/p/C7Oh6mYB7oF/,2024-05-21_withaseul - C7Oh6mYB7oF.json
haseul,withaseul,withaseul - reels,C7RnCv6BZT3,post,2024-05-22 15:06:48,https://www.instagram.com/p/C7RnCv6BZT3/,2024-05-22_withaseul - C7RnCv6BZT3.json
haseul,withaseul,withaseul,C7R7owoBd4n,post,2024-05-22 18:05:56,https://www.instagram.com/p/C7R7owoBd4n/,2024-05-22_withaseul - C7R7owoBd4n.json
haseul,withaseul,withaseul,C7mPzLRhCRz,post,2024-05-30 15:26:55,https://www.instagram.com/p/C7mPzLRhCRz/,2024-05-30_withaseul - C7mPzLRhCRz.json
haseul,withaseul,withaseul,C7tNIlPh7NK,post,2024-06-02 08:18:19,https://www.instagram.com/p/C7tNIlPh7NK/,2024-06-02_withaseul - C7tNIlPh7NK.json
haseul,withaseul,withaseul,C70yRTmBgng,post,2024-06-05 06:57:30,https://www.instagram.com/p/C70yRTmBgng/,2024-06-05_withaseul - C70yRTmBgng.json
haseul,withaseul,withaseul,C71aPMWBfqs,post,2024-06-05 12:46:44,https://www.instagram.com/p/C71aPMWBfqs/,2024-06-05_withaseul - C71aPMWBfqs.json
haseul,withaseul,withaseul,C8NBuqRhCJv,post,2024-06-14 16:54:21,https://www.instagram.com/p/C8NBuqRhCJv/,2024-06-14_withaseul - C8NBuqRhCJv.json
haseul,withaseul,withaseul,C8PPpUKhyLf,post,2024-06-15 13:34:26,https://www.instagram.com/p/C8PPpUKhyLf/,2024-06-15_withaseul - C8PPpUKhyLf.json
haseul,withaseul,withaseul,C8UjYG8B74s,post,2024-06-17 15:03:03,https://www.instagram.com/p/C8UjYG8B74s/,2024-06-17_withaseul - C8UjYG8B74s.json
haseul,withaseul,withaseul,C87PlKyBTU6,post,2024-07-02 15:40:27,https://www.instagram.com/p/C87PlKyBTU6/,2024-07-02_withaseul - C87PlKyBTU6.json
haseul,withaseul,withaseul,C9FqomJhky4,post,2024-07-06 16:49:16,https://www.instagram.com/p/C9FqomJhky4/,2024-07-06_withaseul - C9FqomJhky4.json
haseul,withaseul,withaseul,C9ICMZ5BUfr,post,2024-07-07 14:53:36,https://www.instagram.com/p/C9ICMZ5BUfr/,2024-07-07_withaseul - C9ICMZ5BUfr.json
haseul,withaseul,withaseul,C9V8bjJBPdk,post,2024-07-13 00:32:37,https://www.instagram.com/p/C9V8bjJBPdk/,2024-07-12_withaseul - C9V8bjJBPdk.json
haseul,withaseul,withaseul,C9xK8b1SZAY,post,2024-07-23 14:18:56,https://www.instagram.com/p/C9xK8b1SZAY/,2024-07-23_withaseul - C9xK8b1SZAY.json
haseul,withaseul,withaseul,C-DLjJ6BE0y,post,2024-07-30 14:10:33,https://www.instagram.com/p/C-DLjJ6BE0y/,2024-07-30_withaseul - C-DLjJ6BE0y.json
haseul,withaseul,withaseul,C-Ncp1MyhiD,post,2024-08-03 13:52:25,https://www.instagram.com/p/C-Ncp1MyhiD/,2024-08-03_withaseul - C-Ncp1MyhiD.json
haseul,withaseul,withaseul,C_A4QkApuLO,post,2024-08-23 13:14:54,https://www.instagram.com/p/C_A4QkApuLO/,2024-08-23_withaseul - C_A4QkApuLO.json
haseul,withaseul,withaseul,C_CcGblt8X9,post,2024-08-24 03:47:20,https://www.instagram.com/p/C_CcGblt8X9/,2024-08-23_withaseul - C_CcGblt8X9.json
haseul,withaseul,withaseul,C_ZCyXPylSu,post,2024-09-01 22:28:40,https://www.instagram.com/p/C_ZCyXPylSu/,2024-09-01_withaseul - C_ZCyXPylSu.json
haseul,withaseul,withaseul,C_gRp4gPuY3,post,2024-09-04 17:53:16,https://www.instagram.com/p/C_gRp4gPuY3/,2024-09-04_withaseul - C_gRp4gPuY3.json
haseul,withaseul,withaseul,C_7XyvJMmm7,post,2024-09-15 06:26:24,https://www.instagram.com/p/C_7XyvJMmm7/,2024-09-15_withaseul - C_7XyvJMmm7.json
haseul,withaseul,withaseul,DACAVOGyOEc,post,2024-09-17 20:16:04,https://www.instagram.com/p/DACAVOGyOEc/,2024-09-17_withaseul - DACAVOGyOEc.json
haseul,withaseul,withaseul,DAqwdZIBjY5,post,2024-10-03 16:06:14,https://www.instagram.com/p/DAqwdZIBjY5/,2024-10-03_withaseul - DAqwdZIBjY5.json
haseul,withaseul,withaseul,DAv8DxvyGbr,post,2024-10-05 16:23:48,https://www.instagram.com/p/DAv8DxvyGbr/,2024-10-05_withaseul - DAv8DxvyGbr.json
haseul,withaseul,withaseul,DBhdsL1NIvC,post,2024-10-24 22:00:28,https://www.instagram.com/p/DBhdsL1NIvC/,2024-10-24_withaseul - DBhdsL1NIvC.json
haseul,withaseul,withaseul,DBrP-pYSDEn,post,2024-10-28 17:13:03,https://www.instagram.com/p/DBrP-pYSDEn/,2024-10-28_withaseul - DBrP-pYSDEn.json
haseul,withaseul,withaseul,DBu5zCHBgrT,post,2024-10-30 03:16:12,https://www.instagram.com/p/DBu5zCHBgrT/,2024-10-29_withaseul - DBu5zCHBgrT.json
haseul,withaseul,withaseul,DByVUvOBSQy,post,2024-10-31 11:14:27,https://www.instagram.com/p/DByVUvOBSQy/,2024-10-31_withaseul - DByVUvOBSQy.json
haseul,withaseul,withaseul,DB5re5DhW8a,post,2024-11-03 07:42:45,https://www.instagram.com/p/DB5re5DhW8a/,2024-11-03_withaseul - DB5re5DhW8a.json
haseul,withaseul,withaseul,DCBsfwGhFYd,post,2024-11-06 10:25:32,https://www.instagram.com/p/DCBsfwGhFYd/,2024-11-06_withaseul - DCBsfwGhFYd.json
haseul,withaseul,withaseul,DCObB0SR64m,post,2024-11-11 09:03:02,https://www.instagram.com/p/DCObB0SR64m/,2024-11-11_withaseul - DCObB0SR64m.json
haseul,withaseul,withaseul,DCRhOPphLWU,post,2024-11-12 13:54:52,https://www.instagram.com/p/DCRhOPphLWU/,2024-11-12_withaseul - DCRhOPphLWU.json
haseul,withaseul,withaseul,DCbgyVghiT-,post,2024-11-16 11:03:28,https://www.instagram.com/p/DCbgyVghiT-/,2024-11-16_withaseul - DCbgyVghiT-.json
haseul,withaseul,withaseul,DCe4qpXhgBV,post,2024-11-17 18:29:51,https://www.instagram.com/p/DCe4qpXhgBV/,2024-11-17_withaseul - DCe4qpXhgBV.json
haseul,withaseul,withaseul,DC_4KddBmo_,post,2024-11-30 14:00:24,https://www.instagram.com/p/DC_4KddBmo_/,2024-11-30_withaseul - DC_4KddBmo_.json
haseul,withaseul,withaseul,DDRNi9Bhicn,post,2024-12-07 07:34:20,https://www.instagram.com/p/DDRNi9Bhicn/,2024-12-07_withaseul - DDRNi9Bhicn.json
haseul,withaseul,withaseul,DDW2JdFBUMs,post,2024-12-09 12:05:19,https://www.instagram.com/p/DDW2JdFBUMs/,2024-12-09_withaseul - DDW2JdFBUMs.json
haseul,withaseul,withaseul,DDjWETEBWw2,post,2024-12-14 08:35:07,https://www.instagram.com/p/DDjWETEBWw2/,2024-12-14_withaseul - DDjWETEBWw2.json
haseul,withaseul,withaseul,DDpkLTyhUdt,post,2024-12-16 18:33:51,https://www.instagram.com/p/DDpkLTyhUdt/,2024-12-16_withaseul - DDpkLTyhUdt.json
haseul,withaseul,withaseul,DE1NVFihF2g,post,2025-01-15 03:36:30,https://www.instagram.com/p/DE1NVFihF2g/,2025-01-14_withaseul - DE1NVFihF2g.json
haseul,withaseul,withaseul,DE5GO2-S5PB,post,2025-01-16 15:51:26,https://www.instagram.com/p/DE5GO2-S5PB/,2025-01-16_withaseul - DE5GO2-S5PB.json
haseul,withaseul,withaseul - reels,DE5H7LcB5aX,post,2025-01-16 16:07:31,https://www.instagram.com/p/DE5H7LcB5aX/,2025-01-16_withaseul - DE5H7LcB5aX.json
haseul,withaseul,withaseul,DFFtju9hTdv,post,2025-01-21 13:25:58,https://www.instagram.com/p/DFFtju9hTdv/,2025-01-21_withaseul - DFFtju9hTdv.json
haseul,withaseul,withaseul,DFsLxphhwqE,post,2025-02-05 12:01:09,https://www.instagram.com/p/DFsLxphhwqE/,2025-02-05_withaseul - DFsLxphhwqE.json
haseul,withaseul,withaseul,DHvlOgAxE_j,post,2025-03-28 12:44:03,https://www.instagram.com/p/DHvlOgAxE_j/,2025-03-28_withaseul - DHvlOgAxE_j.json
haseul,withaseul,withaseul,DH96B0QtxfT,post,2025-04-03 02:15:11,https://www.instagram.com/p/DH96B0QtxfT/,2025-04-02_withaseul - DH96B0QtxfT.json
haseul,withaseul,withaseul,DIL6wNXRGDC,post,2025-04-08 12:50:53,https://www.instagram.com/p/DIL6wNXRGDC/,2025-04-08_withaseul - DIL6wNXRGDC.json
haseul,withaseul,withaseul,DIP1mift6QV,post,2025-04-10 01:22:50,https://www.instagram.com/p/DIP1mift6QV/,2025-04-09_withaseul - DIP1mift6QV.json
haseul,withaseul,withaseul,DIfvDIKxwuT,post,2025-04-16 05:33:25,https://www.instagram.com/p/DIfvDIKxwuT/,2025-04-16_withaseul - DIfvDIKxwuT.json
haseul,withaseul,withaseul,DIm-MshvCJy,post,2025-04-19 01:00:03,https://www.instagram.com/p/DIm-MshvCJy/,2025-04-18_withaseul - DIm-MshvCJy.json
haseul,withaseul,withaseul,DIuQyfphWnR,post,2025-04-21 20:57:37,https://www.instagram.com/p/DIuQyfphWnR/,2025-04-21_withaseul - DIuQyfphWnR.json
haseul,withaseul,withaseul,DI3hOzrBLs-,post,2025-04-25 11:14:27,https://www.instagram.com/p/DI3hOzrBLs-/,2025-04-25_withaseul - DI3hOzrBLs-.json
haseul,withaseul,withaseul,DK0lklBhrH6,post,2025-06-13 00:57:27,https://www.instagram.com/p/DK0lklBhrH6/,2025-06-12_withaseul - DK0lklBhrH6.json
haseul,withaseul,withaseul,DK1EdUZBJtr,post,2025-06-13 05:27:20,https://www.instagram.com/p/DK1EdUZBJtr/,2025-06-13_withaseul - DK1EdUZBJtr.json
haseul,withaseul,withaseul,DLJiMzrhM83,post,2025-06-21 04:12:02,https://www.instagram.com/p/DLJiMzrhM83/,2025-06-21_withaseul - DLJiMzrhM83.json
haseul,withaseul,withaseul,DLL1iePBcwm,post,2025-06-22 01:39:30,https://www.instagram.com/p/DLL1iePBcwm/,2025-06-21_withaseul - DLL1iePBcwm.json
haseul,withaseul,withaseul,DLM-sUIhK5P,post,2025-06-22 12:18:44,https://www.instagram.com/p/DLM-sUIhK5P/,2025-06-22_withaseul - DLM-sUIhK5P.json
haseul,withaseul,withaseul,DLU66yCvWZh,post,2025-06-25 14:19:41,https://www.instagram.com/p/DLU66yCvWZh/,2025-06-25_withaseul - DLU66yCvWZh.json
haseul,withaseul,withaseul,DLxAZg5hpnW,post,2025-07-06 12:06:18,https://www.instagram.com/p/DLxAZg5hpnW/,2025-07-06_withaseul - DLxAZg5hpnW.json
haseul,withaseul,withaseul,DMFfDqABJgM,post,2025-07-14 10:59:00,https://www.instagram.com/p/DMFfDqABJgM/,2025-07-14_withaseul - DMFfDqABJgM.json
haseul,withaseul,withaseul,DM0NH9yhkzT,post,2025-08-01 14:26:37,https://www.instagram.com/p/DM0NH9yhkzT/,2025-08-01_withaseul - DM0NH9yhkzT.json
haseul,withaseul,withaseul,DM9_Ol7SzmK,post,2025-08-05 09:37:35,https://www.instagram.com/p/DM9_Ol7SzmK/,2025-08-05_withaseul - DM9_Ol7SzmK.json
haseul,withaseul,withaseul,DNnih4pBd_M,post,2025-08-21 12:54:55,https://www.instagram.com/p/DNnih4pBd_M/,2025-08-21_withaseul - DNnih4pBd_M.json
haseul,withaseul,withaseul,DOIxptrgXgv,post,2025-09-03 10:42:00,https://www.instagram.com/p/DOIxptrgXgv/,2025-09-03_withaseul - DOIxptrgXgv.json
haseul,withaseul,withaseul,DOa8a_ugc2Z,post,2025-09-10 12:02:26,https://www.instagram.com/p/DOa8a_ugc2Z/,2025-09-10_withaseul - DOa8a_ugc2Z.json
haseul,withaseul,withaseul,DPMLxsmgUJJ,post,2025-09-29 14:59:24,https://www.instagram.com/p/DPMLxsmgUJJ/,2025-09-29_withaseul - DPMLxsmgUJJ.json
haseul,withaseul,withaseul,DPRez1QgUxE,post,2025-10-01 16:21:55,https://www.instagram.com/p/DPRez1QgUxE/,2025-10-01_withaseul - DPRez1QgUxE.json
haseul,withaseul,withaseul,DQg9WOOgaKD,post,2025-11-01 13:08:45,https://www.instagram.com/p/DQg9WOOgaKD/,2025-11-01_withaseul - DQg9WOOgaKD.json
haseul,withaseul,withaseul,DQ_JrMdDciC,post,2025-11-13 06:33:42,https://www.instagram.com/p/DQ_JrMdDciC/,2025-11-13_withaseul - DQ_JrMdDciC.json
haseul,withaseul,withaseul,DROhLjLjVqY,post,2025-11-19 05:47:42,https://www.instagram.com/p/DROhLjLjVqY/,2025-11-19_withaseul - DROhLjLjVqY.json
haseul,withaseul,withaseul,DRwiTEDDRHR,post,2025-12-02 10:51:38,https://www.instagram.com/p/DRwiTEDDRHR/,2025-12-02_withaseul - DRwiTEDDRHR.json
haseul,withaseul,withaseul,DTw-4d9gRUx,post,2026-01-21 08:04:12,https://www.instagram.com/p/DTw-4d9gRUx/,2026-01-21_withaseul - DTw-4d9gRUx.json
haseul,withaseul,withaseul,DT0rHDCjPpE,post,2026-01-22 18:28:24,https://www.instagram.com/p/DT0rHDCjPpE/,2026-01-22_withaseul - DT0rHDCjPpE.json
haseul,withaseul,withaseul,DT6HqajDJfF,post,2026-01-24 21:14:05,https://www.instagram.com/p/DT6HqajDJfF/,2026-01-24_withaseul - DT6HqajDJfF.json
haseul,withaseul,withaseul,DU0oU25gWso,post,2026-02-16 14:19:54,https://www.instagram.com/p/DU0oU25gWso/,2026-02-16_withaseul - DU0oU25gWso.json
haseul,withaseul,withaseul,DWZPNAKAZcl,post,2026-03-27 16:19:14,https://www.instagram.com/p/DWZPNAKAZcl/,2026-03-27_withaseul - DWZPNAKAZcl.json
haseul,withaseul,withaseul,DW3yBm2gctK,post,2026-04-08 13:00:00,https://www.instagram.com/p/DW3yBm2gctK/,2026-04-08_withaseul - DW3yBm2gctK.json
haseul,withaseul,withaseul,DZj6uVWhV8M,post,2026-06-14 09:26:10,https://www.instagram.com/p/DZj6uVWhV8M/,2026-06-14_withaseul - DZj6uVWhV8M.json
haseul,withaseul,withaseul,DaZm_xLAZeL,post,2026-07-05 05:52:43,https://www.instagram.com/p/DaZm_xLAZeL/,2026-07-05_withaseul - DaZm_xLAZeL.json
haseul,withaseul,withaseul,Dbnq4k9gb5U,post,2026-08-04 13:27:27,https://www.instagram.com/p/Dbnq4k9gb5U/,2026-08-04_withaseul - Dbnq4k9gb5U.json
heejin,0ct0ber19,0ct0ber19,CrdsY5CrSsO,post,2023-04-25 15:21:16,https://www.instagram.com/p/CrdsY5CrSsO/,2023-04-25_0ct0ber19 - CrdsY5CrSsO.json
heejin,0ct0ber19,0ct0ber19,CtohvHxLnWO,post,2023-06-18 13:22:37,https://www.instagram.com/p/CtohvHxLnWO/,2023-06-18_0ct0ber19 - CtohvHxLnWO.json
heejin,0ct0ber19,0ct0ber19,CuEOMWppp5S,post,2023-06-29 07:30:35,https://www.instagram.com/p/CuEOMWppp5S/,2023-06-29_0ct0ber19 - CuEOMWppp5S.json
heejin,0ct0ber19,0ct0ber19,CwC0Y-prGoB,post,2023-08-17 11:28:40,https://www.instagram.com/p/CwC0Y-prGoB/,2023-08-17_0ct0ber19 - CwC0Y-prGoB.json
heejin,0ct0ber19,0ct0ber19,CxsKPWgpzU4,post,2023-09-27 09:19:51,https://www.instagram.com/p/CxsKPWgpzU4/,2023-09-27_0ct0ber19 - CxsKPWgpzU4.json
heejin,0ct0ber19,0ct0ber19,CzJf78xryub,post,2023-11-02 15:18:48,https://www.instagram.com/p/CzJf78xryub/,2023-11-02_0ct0ber19 - CzJf78xryub.json
heejin,0ct0ber19,0ct0ber19,CzOVVzFLTnD,post,2023-11-04 12:22:25,https://www.instagram.com/p/CzOVVzFLTnD/,2023-11-04_0ct0ber19 - CzOVVzFLTnD.json
heejin,0ct0ber19,0ct0ber19,C14VKVmpdtC,post,2024-01-09 12:51:44,https://www.instagram.com/p/C14VKVmpdtC/,2024-01-09_0ct0ber19 - C14VKVmpdtC.json
heejin,0ct0ber19,0ct0ber19,C2FnIelpue9,post,2024-01-14 16:38:54,https://www.instagram.com/p/C2FnIelpue9/,2024-01-14_0ct0ber19 - C2FnIelpue9.json
heejin,0ct0ber19,0ct0ber19,C5SDY_UJIn6,post,2024-04-03 02:10:53,https://www.instagram.com/p/C5SDY_UJIn6/,2024-04-02_0ct0ber19 - C5SDY_UJIn6.json
heejin,0ct0ber19,0ct0ber19,C5WB6zwp_rX,post,2024-04-04 15:15:00,https://www.instagram.com/p/C5WB6zwp_rX/,2024-04-04_0ct0ber19 - C5WB6zwp_rX.json
heejin,0ct0ber19,0ct0ber19,C5dJ0xXpJpX,post,2024-04-07 09:38:46,https://www.instagram.com/p/C5dJ0xXpJpX/,2024-04-07_0ct0ber19 - C5dJ0xXpJpX.json
heejin,0ct0ber19,0ct0ber19,C5oJdT-pQBI,post,2024-04-11 16:07:12,https://www.instagram.com/p/C5oJdT-pQBI/,2024-04-11_0ct0ber19 - C5oJdT-pQBI.json
heejin,0ct0ber19,0ct0ber19,C53YPQzp7Wj,post,2024-04-17 14:04:58,https://www.instagram.com/p/C53YPQzp7Wj/,2024-04-17_0ct0ber19 - C53YPQzp7Wj.json
heejin,0ct0ber19,0ct0ber19,C5--hdUJKZH,post,2024-04-20 12:54:11,https://www.instagram.com/p/C5--hdUJKZH/,2024-04-20_0ct0ber19 - C5--hdUJKZH.json
heejin,0ct0ber19,0ct0ber19,C71iAmbpekA,post,2024-06-05 13:54:39,https://www.instagram.com/p/C71iAmbpekA/,2024-06-05_0ct0ber19 - C71iAmbpekA.json
heejin,0ct0ber19,0ct0ber19,C8M8TxnSOKI,post,2024-06-14 16:07:00,https://www.instagram.com/p/C8M8TxnSOKI/,2024-06-14_0ct0ber19 - C8M8TxnSOKI.json
heejin,0ct0ber19,0ct0ber19,C8hxpuYyWGv,post,2024-06-22 18:17:55,https://www.instagram.com/p/C8hxpuYyWGv/,2024-06-22_0ct0ber19 - C8hxpuYyWGv.json
heejin,0ct0ber19,0ct0ber19,C-AbtWSJbHB,post,2024-07-29 12:34:02,https://www.instagram.com/p/C-AbtWSJbHB/,2024-07-29_0ct0ber19 - C-AbtWSJbHB.json
heejin,0ct0ber19,0ct0ber19,C_WxFtuvqXK,post,2024-09-01 01:15:32,https://www.instagram.com/p/C_WxFtuvqXK/,2024-08-31_0ct0ber19 - C_WxFtuvqXK.json
heejin,0ct0ber19,0ct0ber19,DcgTbz-iTAH,post,2026-08-26 13:19:15,https://www.instagram.com/p/DcgTbz-iTAH/,2026-08-26_0ct0ber19 - DcgTbz-iTAH.json
jinsoul,zindoriyam,zindoriyam,CndmboMBHeB,post,2023-01-16 04:23:38,https://www.instagram.com/p/CndmboMBHeB/,2023-01-15_zindoriyam - CndmboMBHeB.json
jinsoul,zindoriyam,zindoriyam,Ctb5AnZxCwh,post,2023-06-13 15:35:51,https://www.instagram.com/p/Ctb5AnZxCwh/,2023-06-13_zindoriyam - Ctb5AnZxCwh.json
jinsoul,zindoriyam,zindoriyam,Cu6ZRz8hj_T,post,2023-07-20 08:26:26,https://www.instagram.com/p/Cu6ZRz8hj_T/,2023-07-20_zindoriyam - Cu6ZRz8hj_T.json
jinsoul,zindoriyam,zindoriyam,Cwrydt6OXmo,post,2023-09-02 09:20:42,https://www.instagram.com/p/Cwrydt6OXmo/,2023-09-02_zindoriyam - Cwrydt6OXmo.json
jinsoul,zindoriyam,zindoriyam,Cxsj3_IhwNv,post,2023-09-27 13:03:51,https://www.instagram.com/p/Cxsj3_IhwNv/,2023-09-27_zindoriyam - Cxsj3_IhwNv.json
jinsoul,zindoriyam,zindoriyam,Cza0C4gBkb5,post,2023-11-09 08:41:36,https://www.instagram.com/p/Cza0C4gBkb5/,2023-11-09_zindoriyam - Cza0C4gBkb5.json
jinsoul,zindoriyam,zindoriyam,C5LqGJUBTIK,post,2024-03-31 14:34:25,https://www.instagram.com/p/C5LqGJUBTIK/,2024-03-31_zindoriyam - C5LqGJUBTIK.json
jinsoul,zindoriyam,zindoriyam,C5QZg9BBc12,post,2024-04-02 10:45:44,https://www.instagram.com/p/C5QZg9BBc12/,2024-04-02_zindoriyam - C5QZg9BBc12.json
jinsoul,zindoriyam,zindoriyam,C8NAliShdHm,post,2024-06-14 16:44:22,https://www.instagram.com/p/C8NAliShdHm/,2024-06-14_zindoriyam - C8NAliShdHm.json
jinsoul,zindoriyam,zindoriyam,C9-Ot__hyGY,post,2024-07-28 16:02:02,https://www.instagram.com/p/C9-Ot__hyGY/,2024-07-28_zindoriyam - C9-Ot__hyGY.json
jinsoul,zindoriyam,zindoriyam,DANQU24um-N,post,2024-09-22 05:07:29,https://www.instagram.com/p/DANQU24um-N/,2024-09-22_zindoriyam - DANQU24um-N.json
jinsoul,zindoriyam,zindoriyam,DHz88e2sS_J,post,2025-03-30 05:28:16,https://www.instagram.com/p/DHz88e2sS_J/,2025-03-30_zindoriyam - DHz88e2sS_J.json
jinsoul,zindoriyam,zindoriyam,DJXAVrANWfp,post,2025-05-07 16:42:44,https://www.instagram.com/p/DJXAVrANWfp/,2025-05-07_zindoriyam - DJXAVrANWfp.json
jinsoul,zindoriyam,zindoriyam,DLhdKkZJ2_5,post,2025-06-30 11:09:49,https://www.instagram.com/p/DLhdKkZJ2_5/,2025-06-30_zindoriyam - DLhdKkZJ2_5.json
jinsoul,zindoriyam,zindoriyam,DMiZJbaJZXv,post,2025-07-25 16:25:21,https://www.instagram.com/p/DMiZJbaJZXv/,2025-07-25_zindoriyam - DMiZJbaJZXv.json
jinsoul,zindoriyam,zindoriyam,DNJCd9jpT-J,post,2025-08-09 16:37:33,https://www.instagram.com/p/DNJCd9jpT-J/,2025-08-09_zindoriyam - DNJCd9jpT-J.json
jinsoul,zindoriyam,zindoriyam,DRl8jAxFnwE,post,2025-11-28 08:09:22,https://www.instagram.com/p/DRl8jAxFnwE/,2025-11-28_zindoriyam - DRl8jAxFnwE.json
jinsoul,zindoriyam,zindoriyam,DTQR5cDk6kY,post,2026-01-08 15:15:25,https://www.instagram.com/p/DTQR5cDk6kY/,2026-01-08_zindoriyam - DTQR5cDk6kY.json
jinsoul,zindoriyam,zindoriyam,DVv1oRSEyIy,post,2026-03-11 14:26:54,https://www.instagram.com/p/DVv1oRSEyIy/,2026-03-11_zindoriyam - DVv1oRSEyIy.json
jinsoul,zindoriyam,zindoriyam,DW_6oxSkz8u,post,2026-04-11 16:49:54,https://www.instagram.com/p/DW_6oxSkz8u/,2026-04-11_zindoriyam - DW_6oxSkz8u.json
jinsoul,zindoriyam,zindoriyam,DcRLAVJEwlG,post,2026-08-20 16:16:59,https://www.instagram.com/p/DcRLAVJEwlG/,2026-08-20_zindoriyam - DcRLAVJEwlG.json
kimlip,kimxxlip,kimxxlip,CndmBanNyrR,post,2023-01-16 04:20:04,https://www.instagram.com/p/CndmBanNyrR/,2023-01-15_kimxxlip - CndmBanNyrR.json
kimlip,kimxxlip,kimxxlip,CwXG1L3vQFH,post,2023-08-25 08:34:37,https://www.instagram.com/p/CwXG1L3vQFH/,2023-08-25_kimxxlip - CwXG1L3vQFH.json
kimlip,kimxxlip,kimxxlip,CxsL_FQvHna,post,2023-09-27 09:35:06,https://www.instagram.com/p/CxsL_FQvHna/,2023-09-27_kimxxlip - CxsL_FQvHna.json
1 slug username archive_dir shortcode type date post_url json_file
2 choerry cher_ryppo cher_ryppo C5SBC--Syik post 2024-04-03 01:50:25 https://www.instagram.com/p/C5SBC--Syik/ 2024-04-02_cher_ryppo - C5SBC--Syik.json
3 choerry cher_ryppo cher_ryppo C811ib_ys7D post 2024-06-30 13:16:41 https://www.instagram.com/p/C811ib_ys7D/ 2024-06-30_cher_ryppo - C811ib_ys7D.json
4 choerry cher_ryppo cher_ryppo C9JTMgnSCHF post 2024-07-08 02:41:25 https://www.instagram.com/p/C9JTMgnSCHF/ 2024-07-07_cher_ryppo - C9JTMgnSCHF.json
5 choerry cher_ryppo cher_ryppo C-K6kxZy6qq post 2024-08-02 14:16:09 https://www.instagram.com/p/C-K6kxZy6qq/ 2024-08-02_cher_ryppo - C-K6kxZy6qq.json
6 choerry cher_ryppo cher_ryppo DBXI0zVs_Iv post 2024-10-20 21:45:44 https://www.instagram.com/p/DBXI0zVs_Iv/ 2024-10-20_cher_ryppo - DBXI0zVs_Iv.json
7 choerry cher_ryppo cher_ryppo DRRDiezjdyV post 2025-11-20 05:26:24 https://www.instagram.com/p/DRRDiezjdyV/ 2025-11-20_cher_ryppo - DRRDiezjdyV.json
8 haseul withaseul withaseul Cqps3ZWBlav post 2023-04-05 10:44:56 https://www.instagram.com/p/Cqps3ZWBlav/ 2023-04-05_withaseul - Cqps3ZWBlav.json
9 haseul withaseul withaseul CqsjfUzSelq post 2023-04-06 13:20:43 https://www.instagram.com/p/CqsjfUzSelq/ 2023-04-06_withaseul - CqsjfUzSelq.json
10 haseul withaseul withaseul CqxiTfjhNJ2 post 2023-04-08 11:46:34 https://www.instagram.com/p/CqxiTfjhNJ2/ 2023-04-08_withaseul - CqxiTfjhNJ2.json
11 haseul withaseul withaseul Cq4qymBhftL post 2023-04-11 06:15:24 https://www.instagram.com/p/Cq4qymBhftL/ 2023-04-11_withaseul - Cq4qymBhftL.json
12 haseul withaseul withaseul CrnhJtrhoZi post 2023-04-29 10:55:29 https://www.instagram.com/p/CrnhJtrhoZi/ 2023-04-29_withaseul - CrnhJtrhoZi.json
13 haseul withaseul withaseul Cr7fZZyB4Sp post 2023-05-07 05:04:58 https://www.instagram.com/p/Cr7fZZyB4Sp/ 2023-05-07_withaseul - Cr7fZZyB4Sp.json
14 haseul withaseul withaseul Cr8QeaFhFsq post 2023-05-07 12:13:49 https://www.instagram.com/p/Cr8QeaFhFsq/ 2023-05-07_withaseul - Cr8QeaFhFsq.json
15 haseul withaseul withaseul CsBIA4HBUOX post 2023-05-09 09:36:05 https://www.instagram.com/p/CsBIA4HBUOX/ 2023-05-09_withaseul - CsBIA4HBUOX.json
16 haseul withaseul withaseul CsgbxFDhNmj post 2023-05-21 13:25:08 https://www.instagram.com/p/CsgbxFDhNmj/ 2023-05-21_withaseul - CsgbxFDhNmj.json
17 haseul withaseul withaseul Csn9aO4B1r- post 2023-05-24 11:33:48 https://www.instagram.com/p/Csn9aO4B1r-/ 2023-05-24_withaseul - Csn9aO4B1r-.json
18 haseul withaseul withaseul CsvY2jrBjMc post 2023-05-27 08:48:17 https://www.instagram.com/p/CsvY2jrBjMc/ 2023-05-27_withaseul - CsvY2jrBjMc.json
19 haseul withaseul withaseul CtEHmRDPesc post 2023-06-04 10:01:34 https://www.instagram.com/p/CtEHmRDPesc/ 2023-06-04_withaseul - CtEHmRDPesc.json
20 haseul withaseul withaseul CtRmUjsBUoc post 2023-06-09 15:40:09 https://www.instagram.com/p/CtRmUjsBUoc/ 2023-06-09_withaseul - CtRmUjsBUoc.json
21 haseul withaseul withaseul Cuzar_3LqAA post 2023-07-17 15:24:04 https://www.instagram.com/p/Cuzar_3LqAA/ 2023-07-17_withaseul - Cuzar_3LqAA.json
22 haseul withaseul withaseul CvHe1VMr7C- post 2023-07-25 10:25:06 https://www.instagram.com/p/CvHe1VMr7C-/ 2023-07-25_withaseul - CvHe1VMr7C-.json
23 haseul withaseul withaseul CvhRFq7rnHc post 2023-08-04 10:45:15 https://www.instagram.com/p/CvhRFq7rnHc/ 2023-08-04_withaseul - CvhRFq7rnHc.json
24 haseul withaseul withaseul CwIr3qEBPgF post 2023-08-19 18:09:39 https://www.instagram.com/p/CwIr3qEBPgF/ 2023-08-19_withaseul - CwIr3qEBPgF.json
25 haseul withaseul withaseul CwzPlMThh5m post 2023-09-05 06:49:48 https://www.instagram.com/p/CwzPlMThh5m/ 2023-09-05_withaseul - CwzPlMThh5m.json
26 haseul withaseul withaseul Cw9s3LNBhsq post 2023-09-09 08:18:04 https://www.instagram.com/p/Cw9s3LNBhsq/ 2023-09-09_withaseul - Cw9s3LNBhsq.json
27 haseul withaseul withaseul CxIma4nBlaP post 2023-09-13 13:53:26 https://www.instagram.com/p/CxIma4nBlaP/ 2023-09-13_withaseul - CxIma4nBlaP.json
28 haseul withaseul withaseul CxLcZ2jBME5 post 2023-09-14 16:23:38 https://www.instagram.com/p/CxLcZ2jBME5/ 2023-09-14_withaseul - CxLcZ2jBME5.json
29 haseul withaseul withaseul CxN3g95hVDH post 2023-09-15 14:59:00 https://www.instagram.com/p/CxN3g95hVDH/ 2023-09-15_withaseul - CxN3g95hVDH.json
30 haseul withaseul withaseul CxTGBb_BAYw post 2023-09-17 15:41:59 https://www.instagram.com/p/CxTGBb_BAYw/ 2023-09-17_withaseul - CxTGBb_BAYw.json
31 haseul withaseul withaseul CxnTyJIhUc0 post 2023-09-25 12:07:02 https://www.instagram.com/p/CxnTyJIhUc0/ 2023-09-25_withaseul - CxnTyJIhUc0.json
32 haseul withaseul withaseul CxsPZAur6Xv post 2023-09-27 10:04:51 https://www.instagram.com/p/CxsPZAur6Xv/ 2023-09-27_withaseul - CxsPZAur6Xv.json
33 haseul withaseul withaseul Cx7mOFqhU7R post 2023-10-03 09:12:57 https://www.instagram.com/p/Cx7mOFqhU7R/ 2023-10-03_withaseul - Cx7mOFqhU7R.json
34 haseul withaseul withaseul CyqV0lfruX- post 2023-10-21 12:53:58 https://www.instagram.com/p/CyqV0lfruX-/ 2023-10-21_withaseul - CyqV0lfruX-.json
35 haseul withaseul withaseul Cy7RByYh2BH post 2023-10-28 02:39:10 https://www.instagram.com/p/Cy7RByYh2BH/ 2023-10-27_withaseul - Cy7RByYh2BH.json
36 haseul withaseul withaseul Cz7nefxBJT0 post 2023-11-22 02:26:43 https://www.instagram.com/p/Cz7nefxBJT0/ 2023-11-21_withaseul - Cz7nefxBJT0.json
37 haseul withaseul withaseul C0YxsROhG4W post 2023-12-03 10:13:57 https://www.instagram.com/p/C0YxsROhG4W/ 2023-12-03_withaseul - C0YxsROhG4W.json
38 haseul withaseul withaseul C0zOk2JBXTY post 2023-12-13 16:46:36 https://www.instagram.com/p/C0zOk2JBXTY/ 2023-12-13_withaseul - C0zOk2JBXTY.json
39 haseul withaseul withaseul C1q_9TzBrdt post 2024-01-04 08:36:20 https://www.instagram.com/p/C1q_9TzBrdt/ 2024-01-04_withaseul - C1q_9TzBrdt.json
40 haseul withaseul withaseul C1t-aeHBCef post 2024-01-05 12:20:34 https://www.instagram.com/p/C1t-aeHBCef/ 2024-01-05_withaseul - C1t-aeHBCef.json
41 haseul withaseul withaseul C1yrgmMhsmz post 2024-01-07 08:11:35 https://www.instagram.com/p/C1yrgmMhsmz/ 2024-01-07_withaseul - C1yrgmMhsmz.json
42 haseul withaseul withaseul C1yrxqIhGBZ post 2024-01-07 08:13:54 https://www.instagram.com/p/C1yrxqIhGBZ/ 2024-01-07_withaseul - C1yrxqIhGBZ.json
43 haseul withaseul withaseul C10p70xh-6x post 2024-01-08 02:36:18 https://www.instagram.com/p/C10p70xh-6x/ 2024-01-07_withaseul - C10p70xh-6x.json
44 haseul withaseul withaseul C2APujnBSF- post 2024-01-12 14:38:11 https://www.instagram.com/p/C2APujnBSF-/ 2024-01-12_withaseul - C2APujnBSF-.json
45 haseul withaseul withaseul C3enxknhzeF post 2024-02-18 06:16:55 https://www.instagram.com/p/C3enxknhzeF/ 2024-02-18_withaseul - C3enxknhzeF.json
46 haseul withaseul withaseul C4f8UQIBZAe post 2024-03-14 15:07:03 https://www.instagram.com/p/C4f8UQIBZAe/ 2024-03-14_withaseul - C4f8UQIBZAe.json
47 haseul withaseul withaseul C4qIrJTh3po post 2024-03-18 14:07:26 https://www.instagram.com/p/C4qIrJTh3po/ 2024-03-18_withaseul - C4qIrJTh3po.json
48 haseul withaseul withaseul C4rjDNhhJu8 post 2024-03-19 03:17:09 https://www.instagram.com/p/C4rjDNhhJu8/ 2024-03-18_withaseul - C4rjDNhhJu8.json
49 haseul withaseul withaseul C4xSiGGhP40 post 2024-03-21 08:48:16 https://www.instagram.com/p/C4xSiGGhP40/ 2024-03-21_withaseul - C4xSiGGhP40.json
50 haseul withaseul withaseul C5GBz4FB-kE post 2024-03-29 10:06:12 https://www.instagram.com/p/C5GBz4FB-kE/ 2024-03-29_withaseul - C5GBz4FB-kE.json
51 haseul withaseul withaseul C5GJmlABYug post 2024-03-29 11:14:17 https://www.instagram.com/p/C5GJmlABYug/ 2024-03-29_withaseul - C5GJmlABYug.json
52 haseul withaseul withaseul C5LgFFtB8ri post 2024-03-31 13:06:54 https://www.instagram.com/p/C5LgFFtB8ri/ 2024-03-31_withaseul - C5LgFFtB8ri.json
53 haseul withaseul withaseul C5VZGPDB-Qo post 2024-04-04 09:18:17 https://www.instagram.com/p/C5VZGPDB-Qo/ 2024-04-04_withaseul - C5VZGPDB-Qo.json
54 haseul withaseul withaseul - reels C5bWrT3BaGy post 2024-04-06 16:52:50 https://www.instagram.com/p/C5bWrT3BaGy/ 2024-04-06_withaseul - C5bWrT3BaGy.json
55 haseul withaseul withaseul C6LAH8vBs_a post 2024-04-25 04:59:04 https://www.instagram.com/p/C6LAH8vBs_a/ 2024-04-25_withaseul - C6LAH8vBs_a.json
56 haseul withaseul withaseul C6ypIzVBmiL post 2024-05-10 14:27:49 https://www.instagram.com/p/C6ypIzVBmiL/ 2024-05-10_withaseul - C6ypIzVBmiL.json
57 haseul withaseul withaseul C68cTpzhUzQ post 2024-05-14 09:48:07 https://www.instagram.com/p/C68cTpzhUzQ/ 2024-05-14_withaseul - C68cTpzhUzQ.json
58 haseul withaseul withaseul C68oiiahwZd post 2024-05-14 11:35:00 https://www.instagram.com/p/C68oiiahwZd/ 2024-05-14_withaseul - C68oiiahwZd.json
59 haseul withaseul withaseul C7Oh6mYB7oF post 2024-05-21 10:23:27 https://www.instagram.com/p/C7Oh6mYB7oF/ 2024-05-21_withaseul - C7Oh6mYB7oF.json
60 haseul withaseul withaseul - reels C7RnCv6BZT3 post 2024-05-22 15:06:48 https://www.instagram.com/p/C7RnCv6BZT3/ 2024-05-22_withaseul - C7RnCv6BZT3.json
61 haseul withaseul withaseul C7R7owoBd4n post 2024-05-22 18:05:56 https://www.instagram.com/p/C7R7owoBd4n/ 2024-05-22_withaseul - C7R7owoBd4n.json
62 haseul withaseul withaseul C7mPzLRhCRz post 2024-05-30 15:26:55 https://www.instagram.com/p/C7mPzLRhCRz/ 2024-05-30_withaseul - C7mPzLRhCRz.json
63 haseul withaseul withaseul C7tNIlPh7NK post 2024-06-02 08:18:19 https://www.instagram.com/p/C7tNIlPh7NK/ 2024-06-02_withaseul - C7tNIlPh7NK.json
64 haseul withaseul withaseul C70yRTmBgng post 2024-06-05 06:57:30 https://www.instagram.com/p/C70yRTmBgng/ 2024-06-05_withaseul - C70yRTmBgng.json
65 haseul withaseul withaseul C71aPMWBfqs post 2024-06-05 12:46:44 https://www.instagram.com/p/C71aPMWBfqs/ 2024-06-05_withaseul - C71aPMWBfqs.json
66 haseul withaseul withaseul C8NBuqRhCJv post 2024-06-14 16:54:21 https://www.instagram.com/p/C8NBuqRhCJv/ 2024-06-14_withaseul - C8NBuqRhCJv.json
67 haseul withaseul withaseul C8PPpUKhyLf post 2024-06-15 13:34:26 https://www.instagram.com/p/C8PPpUKhyLf/ 2024-06-15_withaseul - C8PPpUKhyLf.json
68 haseul withaseul withaseul C8UjYG8B74s post 2024-06-17 15:03:03 https://www.instagram.com/p/C8UjYG8B74s/ 2024-06-17_withaseul - C8UjYG8B74s.json
69 haseul withaseul withaseul C87PlKyBTU6 post 2024-07-02 15:40:27 https://www.instagram.com/p/C87PlKyBTU6/ 2024-07-02_withaseul - C87PlKyBTU6.json
70 haseul withaseul withaseul C9FqomJhky4 post 2024-07-06 16:49:16 https://www.instagram.com/p/C9FqomJhky4/ 2024-07-06_withaseul - C9FqomJhky4.json
71 haseul withaseul withaseul C9ICMZ5BUfr post 2024-07-07 14:53:36 https://www.instagram.com/p/C9ICMZ5BUfr/ 2024-07-07_withaseul - C9ICMZ5BUfr.json
72 haseul withaseul withaseul C9V8bjJBPdk post 2024-07-13 00:32:37 https://www.instagram.com/p/C9V8bjJBPdk/ 2024-07-12_withaseul - C9V8bjJBPdk.json
73 haseul withaseul withaseul C9xK8b1SZAY post 2024-07-23 14:18:56 https://www.instagram.com/p/C9xK8b1SZAY/ 2024-07-23_withaseul - C9xK8b1SZAY.json
74 haseul withaseul withaseul C-DLjJ6BE0y post 2024-07-30 14:10:33 https://www.instagram.com/p/C-DLjJ6BE0y/ 2024-07-30_withaseul - C-DLjJ6BE0y.json
75 haseul withaseul withaseul C-Ncp1MyhiD post 2024-08-03 13:52:25 https://www.instagram.com/p/C-Ncp1MyhiD/ 2024-08-03_withaseul - C-Ncp1MyhiD.json
76 haseul withaseul withaseul C_A4QkApuLO post 2024-08-23 13:14:54 https://www.instagram.com/p/C_A4QkApuLO/ 2024-08-23_withaseul - C_A4QkApuLO.json
77 haseul withaseul withaseul C_CcGblt8X9 post 2024-08-24 03:47:20 https://www.instagram.com/p/C_CcGblt8X9/ 2024-08-23_withaseul - C_CcGblt8X9.json
78 haseul withaseul withaseul C_ZCyXPylSu post 2024-09-01 22:28:40 https://www.instagram.com/p/C_ZCyXPylSu/ 2024-09-01_withaseul - C_ZCyXPylSu.json
79 haseul withaseul withaseul C_gRp4gPuY3 post 2024-09-04 17:53:16 https://www.instagram.com/p/C_gRp4gPuY3/ 2024-09-04_withaseul - C_gRp4gPuY3.json
80 haseul withaseul withaseul C_7XyvJMmm7 post 2024-09-15 06:26:24 https://www.instagram.com/p/C_7XyvJMmm7/ 2024-09-15_withaseul - C_7XyvJMmm7.json
81 haseul withaseul withaseul DACAVOGyOEc post 2024-09-17 20:16:04 https://www.instagram.com/p/DACAVOGyOEc/ 2024-09-17_withaseul - DACAVOGyOEc.json
82 haseul withaseul withaseul DAqwdZIBjY5 post 2024-10-03 16:06:14 https://www.instagram.com/p/DAqwdZIBjY5/ 2024-10-03_withaseul - DAqwdZIBjY5.json
83 haseul withaseul withaseul DAv8DxvyGbr post 2024-10-05 16:23:48 https://www.instagram.com/p/DAv8DxvyGbr/ 2024-10-05_withaseul - DAv8DxvyGbr.json
84 haseul withaseul withaseul DBhdsL1NIvC post 2024-10-24 22:00:28 https://www.instagram.com/p/DBhdsL1NIvC/ 2024-10-24_withaseul - DBhdsL1NIvC.json
85 haseul withaseul withaseul DBrP-pYSDEn post 2024-10-28 17:13:03 https://www.instagram.com/p/DBrP-pYSDEn/ 2024-10-28_withaseul - DBrP-pYSDEn.json
86 haseul withaseul withaseul DBu5zCHBgrT post 2024-10-30 03:16:12 https://www.instagram.com/p/DBu5zCHBgrT/ 2024-10-29_withaseul - DBu5zCHBgrT.json
87 haseul withaseul withaseul DByVUvOBSQy post 2024-10-31 11:14:27 https://www.instagram.com/p/DByVUvOBSQy/ 2024-10-31_withaseul - DByVUvOBSQy.json
88 haseul withaseul withaseul DB5re5DhW8a post 2024-11-03 07:42:45 https://www.instagram.com/p/DB5re5DhW8a/ 2024-11-03_withaseul - DB5re5DhW8a.json
89 haseul withaseul withaseul DCBsfwGhFYd post 2024-11-06 10:25:32 https://www.instagram.com/p/DCBsfwGhFYd/ 2024-11-06_withaseul - DCBsfwGhFYd.json
90 haseul withaseul withaseul DCObB0SR64m post 2024-11-11 09:03:02 https://www.instagram.com/p/DCObB0SR64m/ 2024-11-11_withaseul - DCObB0SR64m.json
91 haseul withaseul withaseul DCRhOPphLWU post 2024-11-12 13:54:52 https://www.instagram.com/p/DCRhOPphLWU/ 2024-11-12_withaseul - DCRhOPphLWU.json
92 haseul withaseul withaseul DCbgyVghiT- post 2024-11-16 11:03:28 https://www.instagram.com/p/DCbgyVghiT-/ 2024-11-16_withaseul - DCbgyVghiT-.json
93 haseul withaseul withaseul DCe4qpXhgBV post 2024-11-17 18:29:51 https://www.instagram.com/p/DCe4qpXhgBV/ 2024-11-17_withaseul - DCe4qpXhgBV.json
94 haseul withaseul withaseul DC_4KddBmo_ post 2024-11-30 14:00:24 https://www.instagram.com/p/DC_4KddBmo_/ 2024-11-30_withaseul - DC_4KddBmo_.json
95 haseul withaseul withaseul DDRNi9Bhicn post 2024-12-07 07:34:20 https://www.instagram.com/p/DDRNi9Bhicn/ 2024-12-07_withaseul - DDRNi9Bhicn.json
96 haseul withaseul withaseul DDW2JdFBUMs post 2024-12-09 12:05:19 https://www.instagram.com/p/DDW2JdFBUMs/ 2024-12-09_withaseul - DDW2JdFBUMs.json
97 haseul withaseul withaseul DDjWETEBWw2 post 2024-12-14 08:35:07 https://www.instagram.com/p/DDjWETEBWw2/ 2024-12-14_withaseul - DDjWETEBWw2.json
98 haseul withaseul withaseul DDpkLTyhUdt post 2024-12-16 18:33:51 https://www.instagram.com/p/DDpkLTyhUdt/ 2024-12-16_withaseul - DDpkLTyhUdt.json
99 haseul withaseul withaseul DE1NVFihF2g post 2025-01-15 03:36:30 https://www.instagram.com/p/DE1NVFihF2g/ 2025-01-14_withaseul - DE1NVFihF2g.json
100 haseul withaseul withaseul DE5GO2-S5PB post 2025-01-16 15:51:26 https://www.instagram.com/p/DE5GO2-S5PB/ 2025-01-16_withaseul - DE5GO2-S5PB.json
101 haseul withaseul withaseul - reels DE5H7LcB5aX post 2025-01-16 16:07:31 https://www.instagram.com/p/DE5H7LcB5aX/ 2025-01-16_withaseul - DE5H7LcB5aX.json
102 haseul withaseul withaseul DFFtju9hTdv post 2025-01-21 13:25:58 https://www.instagram.com/p/DFFtju9hTdv/ 2025-01-21_withaseul - DFFtju9hTdv.json
103 haseul withaseul withaseul DFsLxphhwqE post 2025-02-05 12:01:09 https://www.instagram.com/p/DFsLxphhwqE/ 2025-02-05_withaseul - DFsLxphhwqE.json
104 haseul withaseul withaseul DHvlOgAxE_j post 2025-03-28 12:44:03 https://www.instagram.com/p/DHvlOgAxE_j/ 2025-03-28_withaseul - DHvlOgAxE_j.json
105 haseul withaseul withaseul DH96B0QtxfT post 2025-04-03 02:15:11 https://www.instagram.com/p/DH96B0QtxfT/ 2025-04-02_withaseul - DH96B0QtxfT.json
106 haseul withaseul withaseul DIL6wNXRGDC post 2025-04-08 12:50:53 https://www.instagram.com/p/DIL6wNXRGDC/ 2025-04-08_withaseul - DIL6wNXRGDC.json
107 haseul withaseul withaseul DIP1mift6QV post 2025-04-10 01:22:50 https://www.instagram.com/p/DIP1mift6QV/ 2025-04-09_withaseul - DIP1mift6QV.json
108 haseul withaseul withaseul DIfvDIKxwuT post 2025-04-16 05:33:25 https://www.instagram.com/p/DIfvDIKxwuT/ 2025-04-16_withaseul - DIfvDIKxwuT.json
109 haseul withaseul withaseul DIm-MshvCJy post 2025-04-19 01:00:03 https://www.instagram.com/p/DIm-MshvCJy/ 2025-04-18_withaseul - DIm-MshvCJy.json
110 haseul withaseul withaseul DIuQyfphWnR post 2025-04-21 20:57:37 https://www.instagram.com/p/DIuQyfphWnR/ 2025-04-21_withaseul - DIuQyfphWnR.json
111 haseul withaseul withaseul DI3hOzrBLs- post 2025-04-25 11:14:27 https://www.instagram.com/p/DI3hOzrBLs-/ 2025-04-25_withaseul - DI3hOzrBLs-.json
112 haseul withaseul withaseul DK0lklBhrH6 post 2025-06-13 00:57:27 https://www.instagram.com/p/DK0lklBhrH6/ 2025-06-12_withaseul - DK0lklBhrH6.json
113 haseul withaseul withaseul DK1EdUZBJtr post 2025-06-13 05:27:20 https://www.instagram.com/p/DK1EdUZBJtr/ 2025-06-13_withaseul - DK1EdUZBJtr.json
114 haseul withaseul withaseul DLJiMzrhM83 post 2025-06-21 04:12:02 https://www.instagram.com/p/DLJiMzrhM83/ 2025-06-21_withaseul - DLJiMzrhM83.json
115 haseul withaseul withaseul DLL1iePBcwm post 2025-06-22 01:39:30 https://www.instagram.com/p/DLL1iePBcwm/ 2025-06-21_withaseul - DLL1iePBcwm.json
116 haseul withaseul withaseul DLM-sUIhK5P post 2025-06-22 12:18:44 https://www.instagram.com/p/DLM-sUIhK5P/ 2025-06-22_withaseul - DLM-sUIhK5P.json
117 haseul withaseul withaseul DLU66yCvWZh post 2025-06-25 14:19:41 https://www.instagram.com/p/DLU66yCvWZh/ 2025-06-25_withaseul - DLU66yCvWZh.json
118 haseul withaseul withaseul DLxAZg5hpnW post 2025-07-06 12:06:18 https://www.instagram.com/p/DLxAZg5hpnW/ 2025-07-06_withaseul - DLxAZg5hpnW.json
119 haseul withaseul withaseul DMFfDqABJgM post 2025-07-14 10:59:00 https://www.instagram.com/p/DMFfDqABJgM/ 2025-07-14_withaseul - DMFfDqABJgM.json
120 haseul withaseul withaseul DM0NH9yhkzT post 2025-08-01 14:26:37 https://www.instagram.com/p/DM0NH9yhkzT/ 2025-08-01_withaseul - DM0NH9yhkzT.json
121 haseul withaseul withaseul DM9_Ol7SzmK post 2025-08-05 09:37:35 https://www.instagram.com/p/DM9_Ol7SzmK/ 2025-08-05_withaseul - DM9_Ol7SzmK.json
122 haseul withaseul withaseul DNnih4pBd_M post 2025-08-21 12:54:55 https://www.instagram.com/p/DNnih4pBd_M/ 2025-08-21_withaseul - DNnih4pBd_M.json
123 haseul withaseul withaseul DOIxptrgXgv post 2025-09-03 10:42:00 https://www.instagram.com/p/DOIxptrgXgv/ 2025-09-03_withaseul - DOIxptrgXgv.json
124 haseul withaseul withaseul DOa8a_ugc2Z post 2025-09-10 12:02:26 https://www.instagram.com/p/DOa8a_ugc2Z/ 2025-09-10_withaseul - DOa8a_ugc2Z.json
125 haseul withaseul withaseul DPMLxsmgUJJ post 2025-09-29 14:59:24 https://www.instagram.com/p/DPMLxsmgUJJ/ 2025-09-29_withaseul - DPMLxsmgUJJ.json
126 haseul withaseul withaseul DPRez1QgUxE post 2025-10-01 16:21:55 https://www.instagram.com/p/DPRez1QgUxE/ 2025-10-01_withaseul - DPRez1QgUxE.json
127 haseul withaseul withaseul DQg9WOOgaKD post 2025-11-01 13:08:45 https://www.instagram.com/p/DQg9WOOgaKD/ 2025-11-01_withaseul - DQg9WOOgaKD.json
128 haseul withaseul withaseul DQ_JrMdDciC post 2025-11-13 06:33:42 https://www.instagram.com/p/DQ_JrMdDciC/ 2025-11-13_withaseul - DQ_JrMdDciC.json
129 haseul withaseul withaseul DROhLjLjVqY post 2025-11-19 05:47:42 https://www.instagram.com/p/DROhLjLjVqY/ 2025-11-19_withaseul - DROhLjLjVqY.json
130 haseul withaseul withaseul DRwiTEDDRHR post 2025-12-02 10:51:38 https://www.instagram.com/p/DRwiTEDDRHR/ 2025-12-02_withaseul - DRwiTEDDRHR.json
131 haseul withaseul withaseul DTw-4d9gRUx post 2026-01-21 08:04:12 https://www.instagram.com/p/DTw-4d9gRUx/ 2026-01-21_withaseul - DTw-4d9gRUx.json
132 haseul withaseul withaseul DT0rHDCjPpE post 2026-01-22 18:28:24 https://www.instagram.com/p/DT0rHDCjPpE/ 2026-01-22_withaseul - DT0rHDCjPpE.json
133 haseul withaseul withaseul DT6HqajDJfF post 2026-01-24 21:14:05 https://www.instagram.com/p/DT6HqajDJfF/ 2026-01-24_withaseul - DT6HqajDJfF.json
134 haseul withaseul withaseul DU0oU25gWso post 2026-02-16 14:19:54 https://www.instagram.com/p/DU0oU25gWso/ 2026-02-16_withaseul - DU0oU25gWso.json
135 haseul withaseul withaseul DWZPNAKAZcl post 2026-03-27 16:19:14 https://www.instagram.com/p/DWZPNAKAZcl/ 2026-03-27_withaseul - DWZPNAKAZcl.json
136 haseul withaseul withaseul DW3yBm2gctK post 2026-04-08 13:00:00 https://www.instagram.com/p/DW3yBm2gctK/ 2026-04-08_withaseul - DW3yBm2gctK.json
137 haseul withaseul withaseul DZj6uVWhV8M post 2026-06-14 09:26:10 https://www.instagram.com/p/DZj6uVWhV8M/ 2026-06-14_withaseul - DZj6uVWhV8M.json
138 haseul withaseul withaseul DaZm_xLAZeL post 2026-07-05 05:52:43 https://www.instagram.com/p/DaZm_xLAZeL/ 2026-07-05_withaseul - DaZm_xLAZeL.json
139 haseul withaseul withaseul Dbnq4k9gb5U post 2026-08-04 13:27:27 https://www.instagram.com/p/Dbnq4k9gb5U/ 2026-08-04_withaseul - Dbnq4k9gb5U.json
140 heejin 0ct0ber19 0ct0ber19 CrdsY5CrSsO post 2023-04-25 15:21:16 https://www.instagram.com/p/CrdsY5CrSsO/ 2023-04-25_0ct0ber19 - CrdsY5CrSsO.json
141 heejin 0ct0ber19 0ct0ber19 CtohvHxLnWO post 2023-06-18 13:22:37 https://www.instagram.com/p/CtohvHxLnWO/ 2023-06-18_0ct0ber19 - CtohvHxLnWO.json
142 heejin 0ct0ber19 0ct0ber19 CuEOMWppp5S post 2023-06-29 07:30:35 https://www.instagram.com/p/CuEOMWppp5S/ 2023-06-29_0ct0ber19 - CuEOMWppp5S.json
143 heejin 0ct0ber19 0ct0ber19 CwC0Y-prGoB post 2023-08-17 11:28:40 https://www.instagram.com/p/CwC0Y-prGoB/ 2023-08-17_0ct0ber19 - CwC0Y-prGoB.json
144 heejin 0ct0ber19 0ct0ber19 CxsKPWgpzU4 post 2023-09-27 09:19:51 https://www.instagram.com/p/CxsKPWgpzU4/ 2023-09-27_0ct0ber19 - CxsKPWgpzU4.json
145 heejin 0ct0ber19 0ct0ber19 CzJf78xryub post 2023-11-02 15:18:48 https://www.instagram.com/p/CzJf78xryub/ 2023-11-02_0ct0ber19 - CzJf78xryub.json
146 heejin 0ct0ber19 0ct0ber19 CzOVVzFLTnD post 2023-11-04 12:22:25 https://www.instagram.com/p/CzOVVzFLTnD/ 2023-11-04_0ct0ber19 - CzOVVzFLTnD.json
147 heejin 0ct0ber19 0ct0ber19 C14VKVmpdtC post 2024-01-09 12:51:44 https://www.instagram.com/p/C14VKVmpdtC/ 2024-01-09_0ct0ber19 - C14VKVmpdtC.json
148 heejin 0ct0ber19 0ct0ber19 C2FnIelpue9 post 2024-01-14 16:38:54 https://www.instagram.com/p/C2FnIelpue9/ 2024-01-14_0ct0ber19 - C2FnIelpue9.json
149 heejin 0ct0ber19 0ct0ber19 C5SDY_UJIn6 post 2024-04-03 02:10:53 https://www.instagram.com/p/C5SDY_UJIn6/ 2024-04-02_0ct0ber19 - C5SDY_UJIn6.json
150 heejin 0ct0ber19 0ct0ber19 C5WB6zwp_rX post 2024-04-04 15:15:00 https://www.instagram.com/p/C5WB6zwp_rX/ 2024-04-04_0ct0ber19 - C5WB6zwp_rX.json
151 heejin 0ct0ber19 0ct0ber19 C5dJ0xXpJpX post 2024-04-07 09:38:46 https://www.instagram.com/p/C5dJ0xXpJpX/ 2024-04-07_0ct0ber19 - C5dJ0xXpJpX.json
152 heejin 0ct0ber19 0ct0ber19 C5oJdT-pQBI post 2024-04-11 16:07:12 https://www.instagram.com/p/C5oJdT-pQBI/ 2024-04-11_0ct0ber19 - C5oJdT-pQBI.json
153 heejin 0ct0ber19 0ct0ber19 C53YPQzp7Wj post 2024-04-17 14:04:58 https://www.instagram.com/p/C53YPQzp7Wj/ 2024-04-17_0ct0ber19 - C53YPQzp7Wj.json
154 heejin 0ct0ber19 0ct0ber19 C5--hdUJKZH post 2024-04-20 12:54:11 https://www.instagram.com/p/C5--hdUJKZH/ 2024-04-20_0ct0ber19 - C5--hdUJKZH.json
155 heejin 0ct0ber19 0ct0ber19 C71iAmbpekA post 2024-06-05 13:54:39 https://www.instagram.com/p/C71iAmbpekA/ 2024-06-05_0ct0ber19 - C71iAmbpekA.json
156 heejin 0ct0ber19 0ct0ber19 C8M8TxnSOKI post 2024-06-14 16:07:00 https://www.instagram.com/p/C8M8TxnSOKI/ 2024-06-14_0ct0ber19 - C8M8TxnSOKI.json
157 heejin 0ct0ber19 0ct0ber19 C8hxpuYyWGv post 2024-06-22 18:17:55 https://www.instagram.com/p/C8hxpuYyWGv/ 2024-06-22_0ct0ber19 - C8hxpuYyWGv.json
158 heejin 0ct0ber19 0ct0ber19 C-AbtWSJbHB post 2024-07-29 12:34:02 https://www.instagram.com/p/C-AbtWSJbHB/ 2024-07-29_0ct0ber19 - C-AbtWSJbHB.json
159 heejin 0ct0ber19 0ct0ber19 C_WxFtuvqXK post 2024-09-01 01:15:32 https://www.instagram.com/p/C_WxFtuvqXK/ 2024-08-31_0ct0ber19 - C_WxFtuvqXK.json
160 heejin 0ct0ber19 0ct0ber19 DcgTbz-iTAH post 2026-08-26 13:19:15 https://www.instagram.com/p/DcgTbz-iTAH/ 2026-08-26_0ct0ber19 - DcgTbz-iTAH.json
161 jinsoul zindoriyam zindoriyam CndmboMBHeB post 2023-01-16 04:23:38 https://www.instagram.com/p/CndmboMBHeB/ 2023-01-15_zindoriyam - CndmboMBHeB.json
162 jinsoul zindoriyam zindoriyam Ctb5AnZxCwh post 2023-06-13 15:35:51 https://www.instagram.com/p/Ctb5AnZxCwh/ 2023-06-13_zindoriyam - Ctb5AnZxCwh.json
163 jinsoul zindoriyam zindoriyam Cu6ZRz8hj_T post 2023-07-20 08:26:26 https://www.instagram.com/p/Cu6ZRz8hj_T/ 2023-07-20_zindoriyam - Cu6ZRz8hj_T.json
164 jinsoul zindoriyam zindoriyam Cwrydt6OXmo post 2023-09-02 09:20:42 https://www.instagram.com/p/Cwrydt6OXmo/ 2023-09-02_zindoriyam - Cwrydt6OXmo.json
165 jinsoul zindoriyam zindoriyam Cxsj3_IhwNv post 2023-09-27 13:03:51 https://www.instagram.com/p/Cxsj3_IhwNv/ 2023-09-27_zindoriyam - Cxsj3_IhwNv.json
166 jinsoul zindoriyam zindoriyam Cza0C4gBkb5 post 2023-11-09 08:41:36 https://www.instagram.com/p/Cza0C4gBkb5/ 2023-11-09_zindoriyam - Cza0C4gBkb5.json
167 jinsoul zindoriyam zindoriyam C5LqGJUBTIK post 2024-03-31 14:34:25 https://www.instagram.com/p/C5LqGJUBTIK/ 2024-03-31_zindoriyam - C5LqGJUBTIK.json
168 jinsoul zindoriyam zindoriyam C5QZg9BBc12 post 2024-04-02 10:45:44 https://www.instagram.com/p/C5QZg9BBc12/ 2024-04-02_zindoriyam - C5QZg9BBc12.json
169 jinsoul zindoriyam zindoriyam C8NAliShdHm post 2024-06-14 16:44:22 https://www.instagram.com/p/C8NAliShdHm/ 2024-06-14_zindoriyam - C8NAliShdHm.json
170 jinsoul zindoriyam zindoriyam C9-Ot__hyGY post 2024-07-28 16:02:02 https://www.instagram.com/p/C9-Ot__hyGY/ 2024-07-28_zindoriyam - C9-Ot__hyGY.json
171 jinsoul zindoriyam zindoriyam DANQU24um-N post 2024-09-22 05:07:29 https://www.instagram.com/p/DANQU24um-N/ 2024-09-22_zindoriyam - DANQU24um-N.json
172 jinsoul zindoriyam zindoriyam DHz88e2sS_J post 2025-03-30 05:28:16 https://www.instagram.com/p/DHz88e2sS_J/ 2025-03-30_zindoriyam - DHz88e2sS_J.json
173 jinsoul zindoriyam zindoriyam DJXAVrANWfp post 2025-05-07 16:42:44 https://www.instagram.com/p/DJXAVrANWfp/ 2025-05-07_zindoriyam - DJXAVrANWfp.json
174 jinsoul zindoriyam zindoriyam DLhdKkZJ2_5 post 2025-06-30 11:09:49 https://www.instagram.com/p/DLhdKkZJ2_5/ 2025-06-30_zindoriyam - DLhdKkZJ2_5.json
175 jinsoul zindoriyam zindoriyam DMiZJbaJZXv post 2025-07-25 16:25:21 https://www.instagram.com/p/DMiZJbaJZXv/ 2025-07-25_zindoriyam - DMiZJbaJZXv.json
176 jinsoul zindoriyam zindoriyam DNJCd9jpT-J post 2025-08-09 16:37:33 https://www.instagram.com/p/DNJCd9jpT-J/ 2025-08-09_zindoriyam - DNJCd9jpT-J.json
177 jinsoul zindoriyam zindoriyam DRl8jAxFnwE post 2025-11-28 08:09:22 https://www.instagram.com/p/DRl8jAxFnwE/ 2025-11-28_zindoriyam - DRl8jAxFnwE.json
178 jinsoul zindoriyam zindoriyam DTQR5cDk6kY post 2026-01-08 15:15:25 https://www.instagram.com/p/DTQR5cDk6kY/ 2026-01-08_zindoriyam - DTQR5cDk6kY.json
179 jinsoul zindoriyam zindoriyam DVv1oRSEyIy post 2026-03-11 14:26:54 https://www.instagram.com/p/DVv1oRSEyIy/ 2026-03-11_zindoriyam - DVv1oRSEyIy.json
180 jinsoul zindoriyam zindoriyam DW_6oxSkz8u post 2026-04-11 16:49:54 https://www.instagram.com/p/DW_6oxSkz8u/ 2026-04-11_zindoriyam - DW_6oxSkz8u.json
181 jinsoul zindoriyam zindoriyam DcRLAVJEwlG post 2026-08-20 16:16:59 https://www.instagram.com/p/DcRLAVJEwlG/ 2026-08-20_zindoriyam - DcRLAVJEwlG.json
182 kimlip kimxxlip kimxxlip CndmBanNyrR post 2023-01-16 04:20:04 https://www.instagram.com/p/CndmBanNyrR/ 2023-01-15_kimxxlip - CndmBanNyrR.json
183 kimlip kimxxlip kimxxlip CwXG1L3vQFH post 2023-08-25 08:34:37 https://www.instagram.com/p/CwXG1L3vQFH/ 2023-08-25_kimxxlip - CwXG1L3vQFH.json
184 kimlip kimxxlip kimxxlip CxsL_FQvHna post 2023-09-27 09:35:06 https://www.instagram.com/p/CxsL_FQvHna/ 2023-09-27_kimxxlip - CxsL_FQvHna.json
-194
View File
@@ -1,194 +0,0 @@
# JDownloader2 — archive fetching
> **This file lives only on the `tooling` branch.** `main` is published to
> GitHub and deliberately carries none of this — not the host details, not the
> IPs, and not the account names. `main`'s history was redacted on 2026-08-20;
> real names exist only here.
>
> There is no `npm run jd2` script — `package.json` and `CLAUDE.md` are kept
> byte-identical to `main` so that merging `main` into `tooling` never
> conflicts. Run the crawljob generator directly:
>
> ```sh
> npx tsx scripts/jd2-sync.ts --archives <dir> --dry-run
> ```
How content gets into this archive, and why the setup is shaped the way it is.
## Why JDownloader and not Instaloader
There are two surfaces, and they're treated very differently:
| Surface | What hits it | Risk |
|---|---|---|
| `instagram.com` | profile pages, GraphQL/API metadata | Tied to your session, heavily rate-limited. **This is where bans come from.** |
| `scontent*.cdninstagram.com` | the actual media | Signed URLs, CDN-served, tolerant. Mostly a bandwidth question. |
JDownloader does nearly all its work on the CDN. Instaloader's value — the rich
`.json.xz` metadata — comes from asking `instagram.com` a question *per post*.
Concretely, from this archive: `rivvsofficial` has 188 post-metadata files, so
backfilling it cost 188 API requests for one 605-file profile. That's the ban
vector. Downloading the 238 photos was never the problem.
Instaloader got this account banned once. JDownloader with throttling did not.
> **The account was suspended anyway, on 2026-08-17, for "spam".** Not by
> JDownloader, and not by downloading. It was suspended during a day of
> *building and verifying* the gallery-dl replacement — automated browser
> scrolling to enumerate profile grids, repeated `--simulate` and `-j` metadata
> passes, and one aborted sync that re-ran every listing pass before dying.
>
> The framing above is right about which surface is dangerous and wrong about
> what reaches it. **Every read of `instagram.com` counts, including the ones
> that download nothing** — and read-only work is easy not to count precisely
> because it leaves no files behind. See the post-mortem at the top of
> `docs/gallery-dl.md`.
>
> The rule that would have prevented it: *verify against the archive, never
> against the live site*, and treat the first CDN `429` as the end of the
> session rather than a pacing knob.
### What the metadata gap actually costs
Comparing a JDownloader profile against an Instaloader one:
| | JDownloader | Instaloader |
|---|---|---|
| Media | ✅ | ✅ |
| Captions (`.txt`) | ✅ | ✅ |
| Dates (from filenames) | ✅ | ✅ |
| Bio / full name | ❌ | ✅ |
| Follower counts | ❌ | ✅ |
| External URL | ❌ | ✅ |
Captions already work — the viewer reads the `.txt` sidecars. Everything missing
lives in a *single* profile-level record, not the per-post ones. That's why
JDownloader-sourced profiles show "0 followers" and a placeholder bio.
Not worth extra requests. If you ever want it, the zero-request option is a
hand-written `profile.json` sidecar (not implemented yet — ask).
## Settings that matter
**Chunks per download → 1.** The single most important one. JDownloader splits
each file into multiple ranged requests by default; that `Range` pattern looks
nothing like a browser or the app. One chunk = one sequential GET per file.
`jd2-sync` sets `chunks=1` per job, so no global change is needed — but set it
globally too if you ever add links by hand.
**Max simultaneous downloads → 2–3**, connections-per-host low. Concurrency is
what turns "a user" into a statistic.
**Leave reconnect / IP-change features off.** A mid-session IP change on a live
cookie is a *stronger* anomaly signal than the request rate you'd be avoiding.
## The cookie
Exported manually from a real browser session. This is the right approach — no
programmatic login anywhere, which is the thing that actually gets flagged.
- Use it from the **same public IP** as the browser it came from. A cookie used
from a different network is what session-hijack detection looks for.
- When it expires, **re-export from the browser**. Never add a login step to a tool.
- It's a full account credential. Keep it off the NAS share and out of the repo.
## Workflow
Two URLs per profile, because the profile grid misses some reels:
```
https://www.instagram.com/<user>/
https://www.instagram.com/<user>/reels/
```
They overlap slightly — a reel caught by both lands in each directory and shows
up twice in the viewer. That's correct and matches Instagram, which also shows
reels in the profile grid *and* the Reels tab.
## Generating jobs
Instead of pasting URLs and setting output folders by hand:
```bash
npm run jd2 -- --archives /volume1/rslsync/sync/Instagram-archive/archives --dry-run
```
Review, then write it into JDownloader's folder-watch directory:
```bash
npm run jd2 -- --archives /volume1/rslsync/sync/Instagram-archive/archives \
--out ~/.jd2/folderwatch
```
JDownloader runs on the desktop while the archive lives on the NAS, so tell it
the path *it* sees:
```bash
npm run jd2 -- --archives /mnt/nas/Instagram-archive/archives \
--download-base 'Z:\Instagram-archive\archives' \
--out ~/.jd2/folderwatch
```
| Flag | Purpose |
|---|---|
| `--archives <dir>` | Archive root to scan (or `$ARCHIVES_DIR`) |
| `--out <dir>` | JDownloader folder-watch directory |
| `--download-base <dir>` | Root path as JDownloader sees it (Windows paths fine) |
| `--user <name>` | Just this profile (repeatable) |
| `--skip <name>` | Never emit jobs for this directory (repeatable) |
| `--chunks <n>` | Connections per file (default 1) |
| `--auto-start` | Start immediately instead of parking in LinkGrabber |
| `--all-reels` | Emit a reels job even where no reels directory exists |
| `--dry-run` | Print instead of writing |
Defaults are deliberately conservative: `chunks=1`, and links park in the
LinkGrabber for review rather than auto-starting.
Only posts and reels are emitted. Highlight URLs need a numeric id and story
URLs expire, so those stay manual.
Directories that aren't Instagram profiles are skipped by username shape
(letters, digits, dots, underscores, ≤30 chars) — pointing a crawl at those
spends `instagram.com` requests to be told the profile doesn't exist. For names
that *look* like usernames but aren't, use `--skip` or a `.jd2ignore` file in
the archive root, one name per line.
Format reference: `src/org/jdownloader/extensions/folderwatchV2/explain.txt`.
JDownloader develops on SVN — read it via the daily mirror at
<https://github.com/mycodedoesnotcompile2/jdownloader_mirror> (`svn_trunk/`),
not one of the abandoned GitHub copies.
## Expected layout
Everything downloads into `<archives>/`, one directory per source:
```
archives/
0ct0ber19/ posts
0ct0ber19 - reels/ reels
story - 0ct0ber19/ stories
story highlights - 0ct0ber19 - Heestory/ a highlight
```
Non-archive directories (tool output, exports from elsewhere) live *outside*
`archives/` so they never reach the viewer.
The server picks up changes automatically — its index is keyed on directory
mtime, so a new file invalidates only that directory.
## If something goes wrong
**429 / rate limited** — stop for hours, not seconds. Retrying into a limit is
what converts a soft throttle into something worse.
**Cookie stops working** — re-export from the browser. Don't add a login step.
**Files land in the wrong folder** — a Packagizer rule is overriding the job.
Generated jobs set `overwritePackagizerEnabled=TRUE` to prevent this; check that
rules aren't set to run after it.
**Viewer doesn't show new posts** — check the file is in the right directory and
matches the naming pattern (`YYYY-MM-DD_<user> - <shortcode>[ - NN].<ext>`).
The index refreshes on directory mtime, so a genuinely new file is picked up on
the next request.
-85
View File
@@ -1,85 +0,0 @@
# Video upgrade pass — plan and findings (2026-10-14)
**Status: designed and partly measured, NOT built. Nothing in the archive has been changed.**
Written so this can be picked up cold. Account context: the scraping account is the
fresh `erg05teur` (see `docs/gallery-dl.md`, top of "The account was suspended…") — every
request in this plan is spent on a new account, so the budget rules there apply in full.
## Why this exists
The question was whether a logged-out fetch had given us lower-resolution media and whether a
logged-in run could upgrade it. Measured result:
- **Images: no upgrade needed.** Header-scan of 38.9k archived images, plus live metadata for 8
posts across 2023–2026 (including the oddly small ones): archived sizes match exactly what a
logged-in fetch offers (1440×1800, 3277×4096, 2976×3720; 400×400 / 742×928 items are small at
the source, sitting next to 1440 px siblings in the same carousel).
- **Logged-out gdl does not download smaller files — it fails** (login redirect). No logged-out
media ever entered the archive.
- **Videos: this is where there is something to gain.** Of ~1,300 archived videos in the six scraped
profiles, ~1,200 are 720p (min side ≤ 720) and ~100 (2026 reels) are 1080p. A 2026 reel
(`DTpycSlDKHs`, archived 720×1280) offers **1080×1920 VP9 video-only DASH + separate AAC audio**.
A 2024 post (`C7RmlCcSWNR`) tops out at 1276×720 — same as the archive — so older videos
probably gain nothing. Expected beneficiaries: **2025–26 videos, roughly 700 posts**. Not
verified how far back 1080p exists.
## Decisions already made (by the owner)
1. **Videos in a separate pass** from anything else (images need none).
2. **Keep the replaced files for review** in a `_superseded/` area — never overwrite/delete.
3. **Output `.webm`, not `.mp4`** (video stream copied, audio re-encoded AAC → Opus 96 kbps; the
audio stream is ~43 kbps HE-AAC so the loss is negligible). **Tested on real hardware:**
iPhone, iOS 18.7 Safari plays the VP9+Opus `.webm` at 1080×1920 and **fails the VP9+AAC `.mp4`**
(`MediaError` code 4, no video track) — even though `canPlayType` reports "probably" for it, so
never trust `canPlayType` here. VP9-in-MP4 (the lossless remux) is therefore out.
Re-encoding VP9 → H.264 was rejected (bigger, softer, no gain over the 720p original).
## How the sample was produced (reproducible, one post)
```sh
# portrait video: 1080 is the WIDTH — `height=1080` selects nothing and fails "format not available"
yt-dlp --cookies-from-browser chrome:/home/matt/.config/google-chrome-devtools \
-f "bv*[width>=1080]+ba" --merge-output-format mp4 -o 'X.%(ext)s' https://www.instagram.com/p/<shortcode>/
ffmpeg -i X.mp4 -c:v copy -c:a libopus -b:a 96k X.webm # ffmpeg 7.1.5 on mattellite
```
Sizes for that post: 6.46 MB `.mp4` / 6.53 MB `.webm`. For a browser test you need HTTP **Range**
support (Python's `http.server` has none; Safari requires it) — the throwaway server used is not kept.
`gallery-dl` needs `$HOME/.local/bin` on PATH over plain ssh.
## Open decisions / work before building
| item | notes |
|---|---|
| **Viewer MIME type** | `src/lib/archive-cache.ts:127` and `src/hooks/useArchiveScanner.ts:272,344,391` mint every video blob URL as `video/mp4`. Derive the type from the extension (`.webm` → `video/webm`) or Safari/iOS may refuse a WebM labelled mp4. `MEDIA_RE` / `isVideo` already recognise `.webm`. |
| **Filename / dedup** | The archive names `YYYY-MM-DD_<user> - <shortcode>[ - <n>].mp4`. Swapping to `.webm` must not leave **both** in `archives/` (the viewer would show duplicates) — the old `.mp4` has to be *moved out*, not just joined by a `.webm`. Check `parseArchiveFilename` (`src/lib/archive-patterns.ts`) for the extension handling and the JD2-vs-gdl same-stem duplicate logic. |
| **Skip-archive** | `artms.db` keys on `instagram<media_id>`, so a normal run will never re-fetch these. The upgrade tool must fetch targeted items deliberately and must not touch/poison `artms.db` semantics. |
| **`_superseded/` location** | Must be **outside** `/volume1/rslsync/sync/Instagram-archive/archives/` — that tree is scanned by the viewer **and** inside the Resilio-synced share (it would sync every replaced file to devices). Not decided. Also unchecked: whether `agentapi` can write outside that share. Use rsync `--backup --backup-dir=<dir>/<date>/` so the old copy is kept. |
| **Carousel videos** | yt-dlp on a carousel post URL fails ("No video formats found") — see `DIa5NpWRnIP`. gdl's own metadata gives per-child `ytdl:` URLs for videos in carousels; the tool needs to handle child items, not post URLs. |
| **Request budget** | One `yt-dlp -F`-style request per post to learn what 1080p exists (plus the download). ~700 candidates ⇒ suggested **≤ 40 posts/day**, resumable, behind `check-session.py`, hand-started or a slow timer — never a burst. Candidates can be pre-ranked from the archive alone (720p, 2025–26) at zero request cost. |
| **Storage** | 1080p VP9 ≈ 1.6× the 720p file (6 MB vs ~4 MB for that sample, plus the kept original). Estimate before running at scale. |
## Planned design
1. **Candidate list, zero requests:** scan the NAS archive (read-only, ssh `agentapi@<nas>` from
mattellite; the scans in this doc used header parsing in plain Python over stdin because the
NAS has no ffprobe) for videos ≤ 720 min-side from 2025-01 onward, newest first.
2. **Dry-run mode (default):** for each candidate, one request to learn the best offered stream,
record `{shortcode, archived WxH, offered WxH}` in a state file; print counts + estimated GB.
Downloads nothing.
3. **Execute mode:** only where offered width > archived width; download video+audio, `ffmpeg
-c:v copy -c:a libopus -b:a 96k`, verify with `ffprobe` (dimensions, duration within 0.5 s of
the original, both streams present), publish the `.webm`, move the old file to `_superseded/`.
Cap per run (`--max-posts`), pace with the same 12–20 s gaps, stop on any 429 / login redirect.
4. **First step when resuming:** do ONE reel end to end by hand (`DTpycSlDKHs`, ~3 requests) to prove
download → remux → verify → publish → supersede → viewer playback, *then* the dry-run, and only after
the weekly profiles timer has run cleanly on the new account (first fire ≈ 2026-10-23).
## Things measured along the way (for reference)
- Image header scan by year (six scraped profiles): ≥ 1440 px for the vast majority; ~75 images
< 1000 px, all checked samples genuinely small at the source. `official_artms` 2023–24 has a
cluster < 1440 px (~1080-class originals) — not probed.
- Video min-side classes (six profiles): 2023 ≤720: 304, 2024 ≤720: 248, 2025 ≤720: 368 (+1 at
1080), 2026 ≤720: 302 and ≤1080: 100 (+1 at 1440×2560). A handful ≤540.
- The scan tools (read-only, stdlib only, run on the NAS user over ssh stdin) were throwaway
and are not in the repo; recreate from this description if needed.
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "instaarchive-viewer",
"version": "1.8.1",
"version": "1.3.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "instaarchive-viewer",
"version": "1.8.1",
"version": "1.3.0",
"dependencies": {
"@tailwindcss/vite": "^4.1.14",
"@vitejs/plugin-react": "^5.0.4",
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "instaarchive-viewer",
"private": true,
"version": "1.8.1",
"version": "1.3.0",
"type": "module",
"scripts": {
"dev": "vite --port=3000 --host=0.0.0.0",
"build": "vite build && npm run build:server",
"build:server": "tsc server.ts --esModuleInterop --module ESNext --target ES2022 --moduleResolution bundler --removeComments --outDir dist-server",
"build:server": "tsc server.ts --esModuleInterop --module ESNext --target ES2022 --moduleResolution bundler --outDir dist-server",
"preview": "vite preview",
"server": "tsx server.ts",
"clean": "rm -rf dist",
-70
View File
@@ -1,70 +0,0 @@
#!/usr/bin/env python3
"""
Preflight for gdl-cron.sh: is there a usable Instagram login in the cookie jar gdl reads?
gdl-sync.py runs gallery-dl with `--cookies-from-browser chrome:<profile>`, so what matters
is not what the interactive Chrome window LOOKS like (an open tab keeps showing its logged-in
UI long after the session died -- found 2026-10-14, stories had been empty for a week) but
whether the profile's cookie DB holds a `sessionid`. This reads the same DB, the same way.
Never prints cookie values -- only the name, expiry, and a verdict.
Exit 0 logged in (a non-expired sessionid exists)
Exit 1 logged out / unreadable -- the caller must not scrape
"""
import datetime as dt
import os
import shutil
import sqlite3
import sys
import tempfile
import time
from pathlib import Path
PROFILE = Path(os.environ.get("GDL_COOKIE_PROFILE",
str(Path.home() / ".config/google-chrome-devtools")))
WARN_DAYS = 7
def main() -> int:
src = PROFILE / "Default" / "Cookies"
if not src.exists():
print(f"no cookie DB at {src} -- Chrome profile missing?")
return 1
with tempfile.TemporaryDirectory() as tmp:
# Copy first: Chrome holds the live DB open, and a copy keeps this read-only.
for suffix in ("", "-journal", "-wal", "-shm"):
f = Path(str(src) + suffix)
if f.exists():
shutil.copy2(f, Path(tmp) / f.name)
try:
con = sqlite3.connect(str(Path(tmp) / "Cookies"))
rows = con.execute(
"SELECT expires_utc, has_expires FROM cookies "
"WHERE host_key LIKE '%instagram.com' AND name = 'sessionid'").fetchall()
except sqlite3.Error as exc:
print(f"cannot read cookie DB: {exc}")
return 1
if not rows:
print("NOT LOGGED IN: no Instagram sessionid in the Chrome profile "
f"{PROFILE}. Log in to Instagram in that Chrome window (the open tab can "
"look logged in while stale -- reload it), then re-run.")
return 1
expires, has_expires = rows[0]
if has_expires:
exp = dt.datetime.fromtimestamp(expires / 1_000_000 - 11644473600, dt.timezone.utc)
left = (exp.timestamp() - time.time()) / 86400
if left <= 0:
print(f"NOT LOGGED IN: the Instagram sessionid expired {exp:%Y-%m-%d}.")
return 1
note = f"expires {exp:%Y-%m-%d}"
if left < WARN_DAYS:
note += f" (WARNING: only {left:.1f} days left)"
print(f"session ok: sessionid present, {note}")
else:
print("session ok: sessionid present (session cookie, no expiry)")
return 0
if __name__ == "__main__":
sys.exit(main())
-120
View File
@@ -1,120 +0,0 @@
#!/bin/bash
# The full scrape-then-fetch pipeline for one profile: scroll the real page in
# the signed-in Chrome (instagram-chrome-scrape.py), then fetch and publish
# whatever's new as single posts. See TOOLING.md ("Reels: the API is blocked,
# scrape by scrolling instead", and 2026-10-22 for the posts grid) for why this
# exists at all -- it drives the actual signed-in browser session instead of
# the listing API, and is slower by design.
#
# Usage: GDL_TAB=reels|posts ./chrome-sync.sh <username-or-profile-url>
# Normally called through reels-sync.sh / posts-sync.sh, which set GDL_TAB.
#
# Also run by gdl-cron.sh (profiles, full-sweep) once per profile and tab since
# 2026-10-23, with GDL_MAX_KNOWN_ROUNDS set -- see TOOLING.md "Posts grid by browser".
# Exits non-zero if either step does. Everything is logged.
set -eu
RAW="${1:?usage: GDL_TAB=reels|posts chrome-sync.sh <username-or-profile-url>}"
TAB="${GDL_TAB:?set GDL_TAB=reels or posts (or use reels-sync.sh / posts-sync.sh)}"
GDL_HOME="${GDL_HOME:-$HOME/gdl}"
GDL_PYTHON="${GDL_PYTHON:-$HOME/.local/share/pipx/venvs/gallery-dl/bin/python3}"
INDEX="${GDL_INDEX:-https://instaarchive.ergosteur.com}"
PUBLISH="${GDL_PUBLISH:-agentapi@10.20.28.200:/volume1/rslsync/sync/Instagram-archive/archives/}"
# Same hand-paced pacing gdl-cron.sh uses -- see its comment for why. Override
# per-run with GDL_SLEEP_REQUEST etc. if you ever need to, but raise them
# rather than lower them.
SLEEP_REQUEST="${GDL_SLEEP_REQUEST:-12 20}"
SLEEP="${GDL_SLEEP:-5 10}"
RATE="${GDL_RATE:-500K}"
SCROLL_PAUSE="${GDL_SCROLL_PAUSE:-2.0 3.5}"
MAX_IDLE_ROUNDS="${GDL_MAX_IDLE_ROUNDS:-3}"
# Empty = instagram-chrome-scrape.py's own default (2 for posts, 0 = scroll to the end for reels).
MAX_KNOWN_ROUNDS="${GDL_MAX_KNOWN_ROUNDS:-}"
PATH="$HOME/.local/bin:$PATH"; export PATH
if [ ! -x "$GDL_PYTHON" ]; then
echo "instagram-chrome-scrape.py needs gallery-dl's own pipx venv python (websocket-client" >&2
echo "was injected there, not into the system python): $GDL_PYTHON not found" >&2
exit 2
fi
# Same username-from-URL parsing gdl-sync.py already does for --urls-file,
# reused rather than re-implemented so the two never drift apart.
PROFILE=$("$GDL_PYTHON" -c "
import re, sys, importlib.util
from pathlib import Path
spec = importlib.util.spec_from_file_location('gdl_sync', Path('$GDL_HOME/gdl-sync.py'))
gdl = importlib.util.module_from_spec(spec)
sys.modules['gdl_sync'] = gdl
spec.loader.exec_module(gdl)
raw = '$RAW'
m = gdl.RE_PROFILE_URL.match(raw)
user = m.group('user') if m else raw.strip('/')
if not user or '/' in user or ' ' in user:
print(f'cannot read a username from {raw!r}', file=sys.stderr)
sys.exit(1)
print(user)
")
mkdir -p "$GDL_HOME/logs"
LOG="$GDL_HOME/logs/$TAB-$PROFILE-$(date +%Y%m%d-%H%M%S)-$$.log"
URLS_FILE="$GDL_HOME/$PROFILE-$TAB.txt"
STAGING="$GDL_HOME/staging-$TAB-$PROFILE"
echo "=== $TAB-sync $PROFILE $(date -Is) ===" | tee -a "$LOG"
# The exit status has to survive the pipe into tee -- see gdl-cron.sh's
# comment on PIPESTATUS for why this needs to be bash, not sh.
set +e
"$GDL_PYTHON" "$GDL_HOME/instagram-chrome-scrape.py" \
--profile "$PROFILE" \
--tab "$TAB" \
--index "$INDEX" \
--scroll-pause $SCROLL_PAUSE \
--max-idle-rounds "$MAX_IDLE_ROUNDS" \
${MAX_KNOWN_ROUNDS:+--max-known-rounds "$MAX_KNOWN_ROUNDS"} \
--out "$URLS_FILE" 2>&1 | tee -a "$LOG"
scrape_status=${PIPESTATUS[0]}
set -e
if [ "$scrape_status" -ne 0 ]; then
echo "=== exit $scrape_status (scrape failed) at $(date -Is) ===" | tee -a "$LOG"
exit "$scrape_status"
fi
if [ ! -s "$URLS_FILE" ]; then
echo "no new $TAB for $PROFILE; nothing to fetch" | tee -a "$LOG"
echo "=== exit 0 at $(date -Is) ===" | tee -a "$LOG"
exit 0
fi
# Staging is wiped every run on purpose -- same reasoning as gdl-cron.sh: what
# we already hold is decided by the archive dedupe in instagram-chrome-scrape.py, not by
# what happens to be sitting in staging.
rm -rf "$STAGING"
set +e
# shellcheck disable=SC2086
"$GDL_HOME/gdl-sync.py" \
--publish "$PUBLISH" \
--staging "$STAGING" \
--post-urls-file "$URLS_FILE" \
--sleep-request $SLEEP_REQUEST \
--sleep $SLEEP \
--rate "$RATE" \
--execute 2>&1 | tee -a "$LOG"
status=${PIPESTATUS[0]}
set -e
echo "=== exit $status at $(date -Is) ===" | tee -a "$LOG"
# Keep the log directory from growing without bound -- scoped to this
# script's own logs so it never touches gdl-cron.sh's rotation.
ls -1t "$GDL_HOME/logs" | grep "^$TAB-[^0-9]" | tail -n +30 | while read -r old; do
rm -f "$GDL_HOME/logs/$old"
done
exit "$status"
-225
View File
@@ -1,225 +0,0 @@
#!/bin/bash
# Unattended wrapper around gdl-sync.py. One argument: the run mode.
#
# stories daily ~6 requests; the only surface that cannot be backfilled
# profiles weekly every surface, --abort 50: stops enumerating a profile
# once it reaches content already held, so it costs
# ~40-60 requests and catches everything NEW
# full-sweep rarely every surface, no abort: walks each profile to the end
# for ~420 requests. The only run that notices posts
# EDITED after we archived them, and by far the most
# expensive thing here -- see TOOLING.md before running.
#
# Posts and reels are LISTED by scrolling the signed-in Chrome (chrome-sync.sh,
# per profile and tab) rather than by gallery-dl's listing API, which broke on
# 2026-10-22 (gh#9829) while single-post fetches kept working. gallery-dl still
# does stories and highlights. GDL_LISTING=api restores the old all-API run.
#
# Exits non-zero if the sync does, so cron mails you. Everything is logged.
set -eu
MODE="${1:?usage: gdl-cron.sh stories|profiles|full-sweep}"
# healthchecks.io (or compatible) ping URL, e.g. https://hc-ping.com/<uuid>.
# Set by the run-*.sh wrapper for whichever mode has a check configured;
# unset here means no ping, silently. A ping failure (network blip, check
# deleted) must never fail the sync itself -- hence `|| true` throughout, and
# a short timeout (-m) so a hung ping can't hang the run.
HC_URL="${HC_URL:-}"
GDL_HOME="${GDL_HOME:-$HOME/gdl}"
INDEX="${GDL_INDEX:-https://instaarchive.ergosteur.com}"
PUBLISH="${GDL_PUBLISH:-agentapi@10.20.28.200:/volume1/rslsync/sync/Instagram-archive/archives/}"
STAGING="$GDL_HOME/staging-$MODE"
# $$ in the name because two runs in the same SECOND would otherwise share a
# log file, and `tee -a` appends -- which made a test see the previous run.
LOG="$GDL_HOME/logs/$MODE-$(date +%Y%m%d-%H%M%S)-$$.log"
PATH="$HOME/.local/bin:$PATH"; export PATH
# Pacing. These are the values the 2026-08-22 runs used by hand, after the
# scraping warning -- roughly double the caution of gdl-sync.py's own defaults
# (6-10s / 3-6s / 1M). An archive sync has no deadline; being slow is free and
# being restricted is not. Override per-run with GDL_SLEEP_REQUEST etc. if you
# ever need to, but raise them rather than lower them.
SLEEP_REQUEST="${GDL_SLEEP_REQUEST:-12 20}"
SLEEP="${GDL_SLEEP:-5 10}"
RATE="${GDL_RATE:-500K}"
LISTING="${GDL_LISTING:-browser}"
case "$LISTING" in
browser|api) ;;
*) echo "GDL_LISTING must be browser or api, not '$LISTING'" >&2; exit 2 ;;
esac
GDL_PYTHON="${GDL_PYTHON:-$HOME/.local/share/pipx/venvs/gallery-dl/bin/python3}"
# Set per mode below when posts/reels go through the browser: empty = no browser step.
BROWSER_KNOWN_ROUNDS=""
case "$MODE" in
# --min-interval 8, not the 20h default. The floor exists to stop an ABORTED
# RESTART re-enumerating profiles -- a minutes-to-hours concern. At 20h the
# daily timer silently did nothing whenever a manual run had happened the
# previous afternoon, which is exactly what happened on 2026-08-21: it fired,
# skipped all six sources and reported success. A stories fetch is one
# request per profile, so the worst case 8h permits is roughly twelve
# requests in a day instead of six.
stories) ARGS="--only stories --min-interval 8" ;;
# Browser listing: stop scrolling a grid after 2 rounds of nothing unarchived,
# the browser's equivalent of --abort 50.
profiles)
if [ "$LISTING" = browser ]; then
ARGS="--only stories,highlights --abort 50"; BROWSER_KNOWN_ROUNDS=2
else
ARGS="--only posts,reels,stories,highlights --abort 50"
fi ;;
# No --abort: the whole point of full-sweep is enumerating to the end, so it
# is the only run that notices carousels edited after we archived them. The
# browser equivalent is scrolling every grid to the bottom (0 = no early stop).
full-sweep)
if [ "$LISTING" = browser ]; then
ARGS="--only stories,highlights"; BROWSER_KNOWN_ROUNDS=0
else
ARGS="--only posts,reels,stories,highlights"
fi ;;
# Renamed 2026-08-22: "full" was misleading (it is the abort-LIMITED run) and
# "sweep" did not say it was the exhaustive one. Catch the old names rather
# than failing with a bare error, in case something still passes them.
full) echo "mode 'full' was renamed to 'profiles'" >&2; exit 2 ;;
sweep) echo "mode 'sweep' was renamed to 'full-sweep'" >&2; exit 2 ;;
*) echo "unknown mode: $MODE (want stories|profiles|full-sweep)" >&2; exit 2 ;;
esac
mkdir -p "$GDL_HOME/logs"
[ -n "$HC_URL" ] && curl -fsS -m 10 --retry 3 -o /dev/null "$HC_URL/start" || true
# Preflight: refuse to scrape logged out. gdl reads the Chrome profile's cookie DB, and the
# session in it can die silently -- from 2026-10-07 every stories run fetched "No results"
# for all six profiles and still exited 0, so nothing alerted and a week of stories (which
# cannot be backfilled) was lost. An open Chrome tab keeps showing its logged-in UI after
# that, so only the cookie DB tells the truth. Exit 76 turns the Healthchecks check red.
SESSION_CHECK="$GDL_HOME/check-session.py"
if [ -x "$SESSION_CHECK" ]; then
if ! SESSION_MSG=$("$SESSION_CHECK" 2>&1); then
echo "=== $MODE run $(date -Is) ===" | tee -a "$LOG"
echo "ABORT (no requests sent): $SESSION_MSG" | tee -a "$LOG" >&2
echo "=== exit 76 at $(date -Is) ===" | tee -a "$LOG"
[ -n "$HC_URL" ] && curl -fsS -m 10 --retry 3 --data-binary "@$LOG" \
-o /dev/null "$HC_URL/76" || true
exit 76
fi
echo "preflight: $SESSION_MSG" | tee -a "$LOG"
fi
# Upgrade gallery-dl/yt-dlp first (PyPI only, no Instagram requests; never fatal). Instagram
# endpoints change and gallery-dl follows within days -- on 1.32.14 the posts listing silently
# returned "No results" after 1.32.17 had already fixed the query. See gdl-upgrade.sh.
UPGRADE="$GDL_HOME/gdl-upgrade.sh"
if [ -x "$UPGRADE" ]; then
"$UPGRADE" 2>&1 | tee -a "$LOG" || true
fi
# Staging is wiped every run ON PURPOSE. What we already hold is decided by the
# skip-archive (--download-archive), never by which files happen to be sitting
# in staging, so starting empty is correct -- and it keeps the publish rsync
# to just the new files instead of re-walking gigabytes each time.
rm -rf "$STAGING"
echo "=== $MODE run $(date -Is) ===" | tee -a "$LOG"
# The exit status has to survive the pipe into tee. The left-hand side of a
# pipeline runs in a SUBSHELL, so an `exit` in there sets the subshell's status
# and the script goes on to return tee's, which is always 0. An earlier version
# of this file did exactly that and reported success no matter what the sync
# did -- which is why the shebang is bash: PIPESTATUS is the fix.
# This run's output only. The check below must never see a previous run's
# lines, so it reads this rather than the (appended-to) log.
RUNOUT=$(mktemp)
trap 'rm -f "$RUNOUT"' EXIT
set +e
# shellcheck disable=SC2086
"$GDL_HOME/gdl-sync.py" \
--index "$INDEX" \
--staging "$STAGING" \
--publish "$PUBLISH" \
--archive-db "$GDL_HOME/artms.db" \
--urls-file "$GDL_HOME/artms_account_links.txt" \
--sleep-request $SLEEP_REQUEST \
--sleep $SLEEP \
--rate "$RATE" \
$ARGS --execute 2>&1 | tee -a "$LOG" "$RUNOUT"
status=${PIPESTATUS[0]}
set -e
# Posts and reels by browser: scrape each profile's grid and Reels tab, fetch only
# the new shortcodes as single posts. chrome-sync.sh logs and publishes on its own;
# its output is copied into this run's log so the Healthchecks body shows it too.
if [ -n "$BROWSER_KNOWN_ROUNDS" ]; then
mapfile -t PROFILES < <("$GDL_PYTHON" -c "
import sys, importlib.util
from pathlib import Path
spec = importlib.util.spec_from_file_location('gdl_sync', Path('$GDL_HOME/gdl-sync.py'))
gdl = importlib.util.module_from_spec(spec)
sys.modules['gdl_sync'] = gdl
spec.loader.exec_module(gdl)
for p in gdl.read_urls_file(Path('$GDL_HOME/artms_account_links.txt')):
print(p)
")
browser_failed=0
for p in "${PROFILES[@]}"; do
for tab in posts reels; do
echo "==> browser: $p / $tab" | tee -a "$LOG"
set +e
GDL_TAB=$tab GDL_MAX_KNOWN_ROUNDS=$BROWSER_KNOWN_ROUNDS \
"$GDL_HOME/chrome-sync.sh" "$p" 2>&1 | tee -a "$LOG" >/dev/null
s=${PIPESTATUS[0]}
set -e
if [ "$s" -ne 0 ]; then
echo " browser $p / $tab FAILED (exit $s)" | tee -a "$LOG" >&2
browser_failed=$((browser_failed + 1))
fi
done
done
echo "browser listing: ${#PROFILES[@]} profile(s), $browser_failed step(s) failed" | tee -a "$LOG"
[ "$browser_failed" -gt 0 ] && [ "$status" -eq 0 ] && status=1
fi
# A stories run that skipped every source is NOT a success. It means the
# min-interval floor blocked the one surface that cannot be backfilled, and
# without this it looks identical to a clean run: exit 0, "0 step(s) failed".
# Note this is not the same as "no stories today" -- that shows up as sources
# being fetched and returning no results, which is normal and stays quiet.
if [ "$MODE" = "stories" ] && grep -q "sources : 0 to sync" "$RUNOUT"; then
echo "WARNING: every stories source was skipped by --min-interval." | tee -a "$LOG" >&2
echo " Nothing was fetched. Stories expire in 24h and cannot be" | tee -a "$LOG" >&2
echo " backfilled, so this is a real loss, not a quiet no-op." | tee -a "$LOG" >&2
[ "$status" -eq 0 ] && status=75
fi
# Every stories source empty for several days running = a session Instagram stopped
# honouring while the cookie is still in the jar (the preflight cannot see that). Stories-only:
# other modes legitimately return few items per source. See stories-streak.py.
if [ "$MODE" = "stories" ] && [ "$status" -eq 0 ] && [ -x "$GDL_HOME/stories-streak.py" ]; then
if ! STREAK_MSG=$("$GDL_HOME/stories-streak.py" "$RUNOUT" "$GDL_HOME/stories-empty-streak" 2>&1); then
echo "WARNING: $STREAK_MSG" | tee -a "$LOG" >&2
status=77
fi
fi
echo "=== exit $status at $(date -Is) ===" | tee -a "$LOG"
# The log becomes the ping body -- healthchecks.io keeps the last ping's body
# and shows it in the dashboard, so a failure is diagnosable without SSHing
# in. The trailing URL segment is the exit code by convention: 0 means
# success, anything else means fail (this is how the "0 skipped" -> 75
# remap above also reaches the dashboard as a failure, not a quiet success).
[ -n "$HC_URL" ] && curl -fsS -m 10 --retry 3 --data-binary "@$LOG" \
-o /dev/null "$HC_URL/$status" || true
# Keep the log directory from growing without bound.
ls -1t "$GDL_HOME/logs" | tail -n +30 | while read -r old; do
rm -f "$GDL_HOME/logs/$old"
done
exit $status
-1132
View File
File diff suppressed because it is too large Load Diff
-34
View File
@@ -1,34 +0,0 @@
#!/bin/bash
# Upgrade gallery-dl and its injected yt-dlp (both pipx). Instagram changes its
# endpoints often and gallery-dl follows within days -- 1.32.17 (2026-10-17)
# "update user & posts query parameters" landed while we were still on 1.32.14
# and the posts listing silently returned "No results".
#
# Called by gdl-cron.sh before every scrape and weekly by gdl-upgrade.timer.
# Never fatal: a failed upgrade (offline, PyPI down) must not stop a stories
# run, which cannot be backfilled -- it just scrapes with what is installed.
# Costs no Instagram requests (PyPI only).
set -u
PATH="$HOME/.local/bin:$PATH"; export PATH
VENV_PY="${GDL_VENV_PY:-$HOME/.local/share/pipx/venvs/gallery-dl/bin/python}"
versions() {
printf 'gallery-dl %s, yt-dlp %s' \
"$(gallery-dl --version 2>/dev/null || echo '?')" \
"$("$VENV_PY" -c 'import yt_dlp; print(yt_dlp.version.__version__)' 2>/dev/null || echo '?')"
}
before=$(versions)
if ! timeout 180 pipx upgrade --quiet --include-injected gallery-dl >/dev/null 2>&1; then
echo "upgrade: WARNING pipx upgrade failed; continuing with $before" >&2
exit 0
fi
after=$(versions)
if [ "$before" = "$after" ]; then
echo "upgrade: up to date ($after)"
else
echo "upgrade: CHANGED $before -> $after"
fi
-306
View File
@@ -1,306 +0,0 @@
#!/usr/bin/env python3
"""
Find a profile's reels (or posts) by scrolling the real page, not calling the API.
gallery-dl's dedicated reels extractor POSTs to /api/v1/clips/user/, which
Instagram now 302-redirects to the home page for this account -- confirmed on
2026-08-26/27 across multiple profiles, hours apart, with a freshly-warmed
session and a correct X-IG-WWW-Claim header. The reels tab itself loads fine
in a real browser, so this drives the SAME logged-in Chrome (already exposed
on loopback CDP for MCP automation -- see TOOLING.md) via the DevTools
protocol, scrolls it like a person would, and scrapes `/reel/<code>/` links
out of the rendered page instead.
`--tab posts` does the same for the main profile grid (`/<user>/`, both `/p/` and
`/reel/` links), for when gallery-dl's posts listing breaks -- on 2026-10-22,
1.32.14 returned "No results" and 1.32.17 "Requested user could not be found"
(upstream gh#9829), while single-post fetches kept working. Since the grid
runs newest-first, it stops scrolling once rounds turn up nothing but
already-archived posts (`--max-known-rounds`) instead of walking a whole
profile's history.
This only finds shortcodes; it never downloads anything itself. What comes
out the other end is deduped against the archive (via the same --index used
elsewhere) and printed as plain post URLs -- feed them to gdl-sync.py:
./scripts/instagram-chrome-scrape.py --profile someuser \\
--index https://instaarchive.ergosteur.com > /tmp/someuser-reels.txt
./scripts/gdl-sync.py --publish user@host:/path --staging /var/tmp/gdl \\
--post-urls-file /tmp/someuser-reels.txt \\
--sleep-request 12 20 --sleep 5 10 --rate 500K --execute
Requires the `websocket-client` package (only imported inside scrape_reels,
so everything else here stays importable -- and testable -- without it).
"""
from __future__ import annotations
import argparse
import itertools
import json
import random
import re
import sys
import time
from pathlib import Path
from urllib.request import urlopen
RE_POST_HREF = re.compile(r"/(p|reel)/([^/?#]+)")
# tab -> (page path under the profile, link kinds to collect)
TABS = {
"reels": ("reels/", ("reel",)),
"posts": ("", ("p", "reel")),
}
# The reels tab has finished loading once a scroll round finds no NEW
# shortcodes this many times in a row -- lazy-loaded feeds sometimes stall
# for a round or two before producing more, so one dry round is not enough.
DEFAULT_MAX_IDLE_ROUNDS = 3
DEFAULT_MAX_SCROLLS = 200
DEFAULT_SCROLL_PAUSE = (2.0, 3.5)
# Posts tab only: stop after this many consecutive scroll rounds whose new
# shortcodes were all already archived. The first round is exempt -- it holds
# the pinned posts, which are old by nature and sit above the new ones.
DEFAULT_MAX_KNOWN_ROUNDS = 2
def scrape_js(kinds: tuple[str, ...]) -> str:
selector = ", ".join(f'a[href*="/{k}/"]' for k in kinds)
return (
"(() => {"
"window.scrollTo(0, document.body.scrollHeight);"
f"return Array.from(document.querySelectorAll({json.dumps(selector)}))"
".map(a => a.getAttribute('href'));"
"})()"
)
def extract_shortcodes(hrefs: list[str],
kinds: tuple[str, ...] = ("reel",)) -> list[str]:
codes = []
for href in hrefs:
if (m := RE_POST_HREF.search(href)) and m.group(1) in kinds:
codes.append(m.group(2))
return codes
class CDPError(RuntimeError):
pass
class CDP:
"""
A deliberately minimal synchronous DevTools Protocol client: one request
in flight at a time, which is all a linear scroll-and-scrape loop needs.
Anything fancier (concurrent requests, event subscriptions) is scope this
script has no reason to carry.
"""
def __init__(self, ws_url: str, timeout: float = 30.0):
import websocket # local: keep this importable without the package
self.ws = websocket.create_connection(ws_url, timeout=timeout)
self._ids = itertools.count(1)
def send(self, method: str, params: dict | None = None,
session_id: str | None = None) -> dict:
msg_id = next(self._ids)
payload = {"id": msg_id, "method": method, "params": params or {}}
if session_id:
payload["sessionId"] = session_id
self.ws.send(json.dumps(payload))
while True:
msg = json.loads(self.ws.recv())
if msg.get("id") != msg_id:
continue # an event notification, not our reply -- ignore
if "error" in msg:
raise CDPError(f"{method}: {msg['error']}")
return msg.get("result", {})
def close(self) -> None:
self.ws.close()
def should_stop_known(round_no: int, new: list[str], have: set[str],
streak: int, max_known_rounds: int) -> tuple[bool, int]:
"""
Early-stop bookkeeping for a newest-first grid: returns (stop, streak).
A round counts toward the streak when it found new shortcodes and every
one is already archived; a round with anything unarchived resets it. An
empty round leaves it alone (that is what max_idle_rounds is for), and
round 1 never counts -- pinned posts come first.
"""
if max_known_rounds <= 0 or round_no == 1 or not new:
return False, streak
if all(c in have for c in new):
streak += 1
else:
streak = 0
return streak >= max_known_rounds, streak
def scrape_grid(user: str, tab: str = "reels", cdp_port: int = 9222,
scroll_pause: tuple[float, float] = DEFAULT_SCROLL_PAUSE,
max_idle_rounds: int = DEFAULT_MAX_IDLE_ROUNDS,
max_scrolls: int = DEFAULT_MAX_SCROLLS,
have: set[str] | None = None,
max_known_rounds: int = 0,
log=lambda msg: None) -> list[str]:
"""
Open the profile's reels tab (or main grid) in a NEW tab of the
already-signed-in Chrome, scroll it to the bottom repeatedly, and collect
every unique shortcode that appears. Closes the tab when done either way.
"""
path, kinds = TABS[tab]
js = scrape_js(kinds)
version = json.loads(urlopen(f"http://localhost:{cdp_port}/json/version",
timeout=10).read())
browser = CDP(version["webSocketDebuggerUrl"])
target_id = None
try:
target = browser.send("Target.createTarget", {
"url": f"https://www.instagram.com/{user}/{path}"})
target_id = target["targetId"]
attach = browser.send("Target.attachToTarget", {
"targetId": target_id, "flatten": True})
session_id = attach["sessionId"]
browser.send("Page.enable", session_id=session_id)
browser.send("Runtime.enable", session_id=session_id)
time.sleep(4.0) # initial page load, before the first scroll
seen: set[str] = set()
idle = 0
known_streak = 0
for i in range(max_scrolls):
result = browser.send(
"Runtime.evaluate",
{"expression": js, "returnByValue": True},
session_id=session_id)
hrefs = result.get("result", {}).get("value") or []
codes = extract_shortcodes(hrefs, kinds)
new = list(dict.fromkeys(c for c in codes if c not in seen))
seen.update(new)
log(f" scroll {i + 1}: {len(seen)} unique {tab} so far (+{len(new)})")
stop, known_streak = should_stop_known(
i + 1, new, have or set(), known_streak, max_known_rounds)
if stop:
log(f" {known_streak} round(s) of already-archived {tab} in a row; "
f"stopping")
break
if new:
idle = 0
else:
idle += 1
if idle >= max_idle_rounds:
break
time.sleep(random.uniform(*scroll_pause))
return sorted(seen)
finally:
if target_id:
try:
browser.send("Target.closeTarget", {"targetId": target_id})
except CDPError:
pass # best-effort cleanup; a leftover tab is harmless
browser.close()
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--profile", required=True,
help="Instagram username to scrape")
ap.add_argument("--tab", choices=sorted(TABS), default="reels",
help="reels tab, or the main posts grid (default: reels)")
ap.add_argument("--index", required=True,
help="existing archive listing, same as gdl-sync.py's "
"--index: a local root, or the viewer's base URL. "
"Used only to dedupe -- already-archived shortcodes "
"are dropped before anything is printed")
ap.add_argument("--cdp-port", type=int, default=9222,
help="Chrome's loopback DevTools port (see TOOLING.md)")
ap.add_argument("--scroll-pause", nargs=2, type=float,
default=list(DEFAULT_SCROLL_PAUSE), metavar=("MIN", "MAX"),
help="random pause between scrolls, seconds")
ap.add_argument("--max-idle-rounds", type=int, default=DEFAULT_MAX_IDLE_ROUNDS,
help="stop after this many consecutive scrolls with no "
"new reels")
ap.add_argument("--max-scrolls", type=int, default=DEFAULT_MAX_SCROLLS,
help="hard ceiling on scroll rounds, in case a page never "
"goes idle")
ap.add_argument("--max-known-rounds", type=int, default=None,
help="stop after this many consecutive scroll rounds of "
"only already-archived shortcodes; 0 disables "
f"(default: {DEFAULT_MAX_KNOWN_ROUNDS} for posts, "
"0 for reels)")
ap.add_argument("--out", type=Path,
help="write new reel URLs here, one per line (default: "
"stdout)")
args = ap.parse_args()
import importlib.util
spec = importlib.util.spec_from_file_location(
"gdl_sync", Path(__file__).with_name("gdl-sync.py"))
gdl = importlib.util.module_from_spec(spec)
sys.modules["gdl_sync"] = gdl
spec.loader.exec_module(gdl)
index = gdl.ArchiveIndex(args.index)
# Deduped against the WHOLE archive, not just this profile's own
# directories: a shortcode is globally unique, and a reels tab commonly
# surfaces reposts/collabs by OTHER tracked accounts. Missing that let a
# 2026-08-27 run re-fetch 9 reels already held under their true owner's
# directory -- gallery-dl filed them correctly there (by the post's real
# `username`, not the scraped profile), so nothing was lost, but it spent
# 9 avoidable Instagram requests to find that out. All of this is local
# requests to our own viewer, never to instagram.com, so checking every
# profile costs nothing on the budget that actually matters.
profiles = index.profiles()
have: set[str] = set()
for profile in profiles:
have.update(code for code, _ in gdl.index_existing(index.listing(profile)))
print(f"archive already holds {len(have)} shortcode(s) across "
f"{len(profiles)} profile(s)", file=sys.stderr)
max_known = args.max_known_rounds
if max_known is None:
max_known = DEFAULT_MAX_KNOWN_ROUNDS if args.tab == "posts" else 0
path, _ = TABS[args.tab]
print(f"scraping https://www.instagram.com/{args.profile}/{path} ...",
file=sys.stderr)
codes = scrape_grid(args.profile, tab=args.tab, cdp_port=args.cdp_port,
scroll_pause=tuple(args.scroll_pause),
max_idle_rounds=args.max_idle_rounds,
max_scrolls=args.max_scrolls,
have=have, max_known_rounds=max_known,
log=lambda msg: print(msg, file=sys.stderr))
new_codes = [c for c in codes if c not in have]
print(f"found {len(codes)} {args.tab} on the page, {len(codes) - len(new_codes)} "
f"already archived, {len(new_codes)} new", file=sys.stderr)
# /p/<code>/ resolves reels too, so the posts grid needs no per-kind URL.
if args.tab == "posts":
urls = [f"https://www.instagram.com/p/{c}/" for c in new_codes]
else:
urls = [f"https://www.instagram.com/{args.profile}/reel/{c}/"
for c in new_codes]
text = "\n".join(urls)
if args.out:
args.out.write_text(text + ("\n" if text else ""))
print(f"wrote {len(urls)} URL(s) to {args.out}", file=sys.stderr)
else:
if text:
print(text)
return 0
if __name__ == "__main__":
sys.exit(main())
-277
View File
@@ -1,277 +0,0 @@
/**
* Generate JDownloader2 .crawljob files for the archives on disk.
*
* The manual flow is: paste a profile URL into JDownloader, paste the /reels
* URL separately (the profile page misses some reels), and set the output
* folder by hand — times however many profiles you keep. This emits one
* crawljob per source with the folder already pointed at the right directory,
* so JDownloader's folder-watch picks the whole batch up at once.
*
* Profiles and their sidecar directories are derived with the same grouping
* logic the server uses, so the output folders always match what the viewer
* expects to find.
*
* Only posts and reels are emitted. Story and highlight URLs can't be rebuilt
* from a directory name — highlights need their numeric id and stories expire —
* so those stay manual.
*
* Crawljob format verified against JDownloader's own docs for the extension:
* src/org/jdownloader/extensions/folderwatchV2/explain.txt. JDownloader
* develops on SVN; read it via the daily mirror at
* https://github.com/mycodedoesnotcompile2/jdownloader_mirror (svn_trunk/),
* not one of the abandoned GitHub copies — several are a decade stale.
*
* Entries are separated by `->NEW ENTRY<-` and any property may be omitted.
* There is also a `setBeforePackagizerEnabled` companion to
* `overwritePackagizerEnabled`, if the Packagizer ever needs to see these
* values before they're applied.
*
* Usage:
* npx tsx scripts/jd2-sync.ts --archives <dir> [options]
*
* --archives <dir> Archive root to scan (default: $ARCHIVES_DIR)
* --out <dir> JDownloader folder-watch directory to write into
* --download-base <dir> Root path as *JDownloader* sees it, when it runs on
* a different machine than this script (e.g. a mapped
* drive). Defaults to --archives.
* --user <name> Only this profile (repeatable)
* --skip <name> Never emit jobs for this directory (repeatable).
* Also read from a `.jd2ignore` file in the archive
* root, one name per line.
* --chunks <n> Connections per file (default 1: multi-chunk ranged
* requests are the one CDN pattern that doesn't look
* like a browser)
* --auto-start Start downloads immediately instead of parking them
* in the LinkGrabber for review
* --all-reels Emit a reels job even where no reels directory
* exists yet
* --dry-run Print the crawljob instead of writing it
*/
import fs from 'fs';
import path from 'path';
import { groupArchiveDirectories, ArchiveSource } from '../src/lib/archive-grouping.js';
interface Options {
archives: string;
out: string | null;
downloadBase: string;
users: string[];
skip: Set<string>;
chunks: number;
autoStart: boolean;
allReels: boolean;
dryRun: boolean;
}
const parseArgs = (argv: string[]): Options => {
const opts: Options = {
archives: process.env.ARCHIVES_DIR ?? '',
out: null,
downloadBase: '',
users: [],
skip: new Set(),
chunks: 1,
autoStart: false,
allReels: false,
dryRun: false,
};
for (let i = 0; i < argv.length; i++) {
const arg = argv[i];
const next = () => argv[++i];
switch (arg) {
case '--archives': opts.archives = path.resolve(next()); break;
case '--out': opts.out = path.resolve(next()); break;
case '--download-base': opts.downloadBase = next(); break;
case '--user': opts.users.push(next()); break;
case '--skip': opts.skip.add(next()); break;
case '--chunks': opts.chunks = parseInt(next(), 10); break;
case '--auto-start': opts.autoStart = true; break;
case '--all-reels': opts.allReels = true; break;
case '--dry-run': opts.dryRun = true; break;
case '--help': case '-h': printUsage(); process.exit(0);
default:
console.error(`Unknown argument: ${arg}`);
process.exit(1);
}
}
if (!opts.archives) {
console.error('No archive root. Pass --archives <dir> or set ARCHIVES_DIR.');
process.exit(1);
}
if (!opts.downloadBase) opts.downloadBase = opts.archives;
if (!opts.out && !opts.dryRun) {
console.error('No destination. Pass --out <folder-watch dir>, or --dry-run to preview.');
process.exit(1);
}
return opts;
};
const printUsage = () => {
const header = readHeaderComment();
console.log(header);
};
/** Print the usage block from this file's own header comment. */
const readHeaderComment = () => {
try {
const self = fs.readFileSync(new URL(import.meta.url), 'utf8');
const usage = self.slice(self.indexOf(' * Usage:'), self.indexOf(' */'));
return usage.split('\n').map(l => l.replace(/^ \* ?/, '')).join('\n');
} catch {
return 'See the comment at the top of scripts/jd2-sync.ts';
}
};
/**
* JDownloader escapes nothing in crawljob values, so a stray newline would
* silently split a property. Paths with spaces are fine as-is.
*/
const sanitise = (value: string) => value.replace(/[\r\n]+/g, ' ').trim();
/**
* Instagram usernames are 1–30 characters of letters, digits, dots and
* underscores. Archive roots also collect directories that aren't profiles at
* all — tool output, exports from other services — and pointing a crawl at
* those spends requests on instagram.com to be told the profile doesn't exist.
* That's the exact traffic worth not spending.
*/
const USERNAME_RE = /^[A-Za-z0-9._]{1,30}$/;
/** Directory names to skip, from `.jd2ignore` in the archive root. */
const readIgnoreFile = (archives: string): string[] => {
try {
return fs.readFileSync(path.join(archives, '.jd2ignore'), 'utf8')
.split('\n').map(l => l.trim()).filter(l => l && !l.startsWith('#'));
} catch {
return [];
}
};
interface Job {
user: string;
kind: 'posts' | 'reels';
url: string;
packageName: string;
downloadFolder: string;
fileCount: number | null;
}
const buildJobs = (opts: Options): Job[] => {
const dirNames = fs.readdirSync(opts.archives, { withFileTypes: true })
.filter(e => e.isDirectory() && !/^[.@_]/.test(e.name))
.map(e => e.name);
const groups = groupArchiveDirectories(dirNames);
const jobs: Job[] = [];
const skipped: string[] = [];
for (const name of readIgnoreFile(opts.archives)) opts.skip.add(name);
const countFiles = (dir: string): number | null => {
try {
return fs.readdirSync(path.join(opts.archives, dir)).length;
} catch {
return null;
}
};
// JDownloader must be given the path *it* can see, which differs from the
// scan path whenever the archive lives on a share.
const downloadFolderFor = (dir: string) =>
opts.downloadBase.includes('\\')
? `${opts.downloadBase.replace(/\\$/, '')}\\${dir}`
: path.posix.join(opts.downloadBase, dir);
for (const [user, sources] of [...groups].sort(([a], [b]) => a.localeCompare(b))) {
if (opts.users.length && !opts.users.includes(user)) continue;
if (opts.skip.has(user)) { skipped.push(`${user} (ignored)`); continue; }
if (!USERNAME_RE.test(user)) { skipped.push(`${user} (not a username)`); continue; }
const has = (kind: ArchiveSource['kind']) => sources.find(s => s.kind === kind);
const base = has('posts');
if (!base) continue; // sidecar-only group: nothing sensible to point a URL at
jobs.push({
user, kind: 'posts',
url: `https://www.instagram.com/${encodeURIComponent(user)}/`,
packageName: base.dir,
downloadFolder: downloadFolderFor(base.dir),
fileCount: countFiles(base.dir),
});
const reels = has('reels');
if (reels || opts.allReels) {
const dir = reels?.dir ?? `${user} - reels`;
jobs.push({
user, kind: 'reels',
url: `https://www.instagram.com/${encodeURIComponent(user)}/reels/`,
packageName: dir,
downloadFolder: downloadFolderFor(dir),
fileCount: reels ? countFiles(dir) : null,
});
}
}
if (skipped.length) {
console.error(`Skipped ${skipped.length} director${skipped.length === 1 ? 'y' : 'ies'}:`);
for (const s of skipped) console.error(` - ${s}`);
console.error('');
}
return jobs;
};
const renderCrawljob = (jobs: Job[], opts: Options): string =>
jobs.map(job => [
`text=${sanitise(job.url)}`,
`packageName=${sanitise(job.packageName)}`,
`downloadFolder=${sanitise(job.downloadFolder)}`,
`chunks=${opts.chunks}`,
// Without this a Packagizer rule can override downloadFolder and scatter
// files away from the directory the viewer reads.
'overwritePackagizerEnabled=TRUE',
`autoStart=${opts.autoStart ? 'TRUE' : 'FALSE'}`,
`autoConfirm=${opts.autoStart ? 'TRUE' : 'FALSE'}`,
'enabled=TRUE',
`comment=instaarchive jd2-sync (${job.kind})`,
].join('\n')).join('\n->NEW ENTRY<-\n');
const main = () => {
const opts = parseArgs(process.argv.slice(2));
const jobs = buildJobs(opts);
if (!jobs.length) {
console.error('No profiles matched.');
process.exit(1);
}
console.error(`Archive root : ${opts.archives}`);
console.error(`JD sees root : ${opts.downloadBase}`);
console.error(`Jobs : ${jobs.length} (${new Set(jobs.map(j => j.user)).size} profiles)\n`);
for (const job of jobs) {
const count = job.fileCount === null ? 'new' : `${job.fileCount} files`;
console.error(` ${job.kind.padEnd(5)} ${job.user.padEnd(24)} -> ${job.packageName} (${count})`);
}
console.error('');
const body = renderCrawljob(jobs, opts);
if (opts.dryRun || !opts.out) {
console.log(body);
return;
}
fs.mkdirSync(opts.out, { recursive: true });
const file = path.join(opts.out, `instaarchive-${new Date().toISOString().replace(/[:.]/g, '-')}.crawljob`);
fs.writeFileSync(file, body, 'utf8');
console.error(`Wrote ${file}`);
console.error(opts.autoStart
? 'Downloads will start automatically.'
: 'Links land in the LinkGrabber for review; start them when ready.');
};
main();
-11
View File
@@ -1,11 +0,0 @@
#!/bin/bash
# chrome-sync.sh for the main profile grid: scroll /<user>/ in the signed-in
# Chrome, collect /p/ and /reel/ links, fetch only the new ones as single
# posts. A by-hand fallback for when gallery-dl's posts listing is broken
# (2026-10-22: upstream gh#9829) while single-post fetches still work -- see
# TOOLING.md. Stops scrolling once it reaches already-archived posts.
#
# Usage: ./posts-sync.sh <username-or-profile-url>
set -eu
export GDL_TAB=posts
exec "$(dirname "$0")/chrome-sync.sh" "$@"
-8
View File
@@ -1,8 +0,0 @@
#!/bin/bash
# chrome-sync.sh for a profile's reels tab. See TOOLING.md ("Reels: the API is
# blocked, scrape by scrolling instead").
#
# Usage: ./reels-sync.sh <username-or-profile-url>
set -eu
export GDL_TAB=reels
exec "$(dirname "$0")/chrome-sync.sh" "$@"
-8
View File
@@ -1,8 +0,0 @@
#!/bin/bash
# Rarely: every surface, no --abort -- walks each profile to the end, ~420
# requests. The only run that notices posts EDITED after they were archived
# (e.g. a caption added later -- see TOOLING.md "Reels: the API is blocked"
# for how that showed up), and by far the most expensive thing here. Read
# TOOLING.md before running this; prefer running it by hand, not on a timer.
set -eu
exec "$(dirname "$0")/gdl-cron.sh" full-sweep "$@"
-9
View File
@@ -1,9 +0,0 @@
#!/bin/bash
# Routine run: every surface, --abort 50 (only gdl-cron.sh's "profiles" mode
# does this), ~5 min. Covers everything but is no substitute for stories.sh --
# a story posted and expired between two runs of this is simply gone.
set -eu
# healthchecks.io check "instagram-archiver-posts". Override or unset (e.g.
# HC_URL= ./run-profiles.sh) to run without pinging it.
export HC_URL="${HC_URL:-https://hc-ping.com/3f40abd8-e9bd-43af-a105-c583efaad88c}"
exec "$(dirname "$0")/gdl-cron.sh" profiles "$@"
-56
View File
@@ -1,56 +0,0 @@
#!/bin/bash
# Run reels-sync.sh for every tracked ARTMS profile, one after another.
# TOOLING.md ("Reels: the API is blocked, scrape by scrolling instead")
# covers what this actually does and why it exists; this is just the loop
# version of running reels-sync.sh six times by hand, which is what the
# 2026-09-03 run was before this script existed.
#
# A by-hand full sweep of every Reels tab (no early stop). The weekly
# gdl-cron.sh profiles run lists reels through chrome-sync.sh itself, with an
# early stop, since 2026-10-23 -- see TOOLING.md "Posts grid by browser".
#
# Usage: ./reels-all.sh
# GDL_ARTMS_LINKS=/path/to/list ./reels-all.sh # override the profile list
set -eu
GDL_HOME="${GDL_HOME:-$HOME/gdl}"
LINKS="${GDL_ARTMS_LINKS:-$GDL_HOME/artms_account_links.txt}"
GDL_PYTHON="${GDL_PYTHON:-$HOME/.local/share/pipx/venvs/gallery-dl/bin/python3}"
if [ ! -f "$LINKS" ]; then
echo "profile list not found: $LINKS" >&2
exit 2
fi
# Same parser gdl-sync.py uses for --urls-file (URLs or bare usernames, #
# comments, blank lines, duplicates dropped), reused so this never drifts
# from what a routine profiles.sh run actually tracks.
mapfile -t PROFILES < <("$GDL_PYTHON" -c "
import sys, importlib.util
from pathlib import Path
spec = importlib.util.spec_from_file_location('gdl_sync', Path('$GDL_HOME/gdl-sync.py'))
gdl = importlib.util.module_from_spec(spec)
sys.modules['gdl_sync'] = gdl
spec.loader.exec_module(gdl)
for p in gdl.read_urls_file(Path('$LINKS')):
print(p)
")
if [ "${#PROFILES[@]}" -eq 0 ]; then
echo "no usable profiles in $LINKS" >&2
exit 2
fi
echo "=== reels-all: ${#PROFILES[@]} profile(s) $(date -Is) ==="
status=0
for p in "${PROFILES[@]}"; do
echo "=== starting $p ==="
if ! "$(dirname "$0")/reels-sync.sh" "$p"; then
status=1
echo "=== $p FAILED ===" >&2
fi
echo "=== finished $p ==="
done
echo "=== reels-all: exit $status at $(date -Is) ==="
exit "$status"
-12
View File
@@ -1,12 +0,0 @@
#!/bin/bash
# Daily run: stories only, ~1 min. The one surface that cannot be backfilled
# -- do this often. See TOOLING.md "Quick reference" for the full picture.
#
# Just a fixed-mode entry point so you don't have to remember gdl-cron.sh's
# mode argument; override pacing/paths the same way gdl-cron.sh does, with
# GDL_* env vars (see gdl-cron.sh's own comments for what's available).
set -eu
# healthchecks.io check "instagram-archiver-stories". Override or unset (e.g.
# HC_URL= ./run-stories.sh) to run without pinging it.
export HC_URL="${HC_URL:-https://hc-ping.com/9fe7347f-95be-47aa-a46b-affd27691554}"
exec "$(dirname "$0")/gdl-cron.sh" stories "$@"
-57
View File
@@ -1,57 +0,0 @@
#!/usr/bin/env python3
"""
Detect a silently dead Instagram session from a stories run's output.
A dead session (killed server-side while the sessionid is still in the cookie jar, so
check-session.py passes) shows up as every fetched stories source returning
"No results" -- the daily run still exits 0. From 2026-10-07 all six profiles came back
empty every day and nothing alerted. One all-empty day is normal (nobody posted a story);
several in a row across ALL sources is not.
Usage: stories-streak.py <this run's output file> [state file]
Counts at most one day per calendar date (manual re-runs don't inflate it). Prints a message
and exits 1 once the streak reaches GDL_EMPTY_STREAK_FAIL days (default 3); exits 0 otherwise.
Runs that fetched nothing (all skipped / no sources) leave the streak untouched.
"""
import datetime as dt
import os
import re
import sys
from pathlib import Path
THRESHOLD = int(os.environ.get("GDL_EMPTY_STREAK_FAIL", "3"))
def main() -> int:
out = Path(sys.argv[1]).read_text(errors="replace")
state = Path(sys.argv[2] if len(sys.argv) > 2
else Path.home() / "gdl" / "stories-empty-streak")
m = re.search(r"sources\s*:\s*(\d+) to sync", out)
to_sync = int(m.group(1)) if m else 0
if to_sync == 0:
return 0 # nothing fetched: no evidence either way
empty = len(re.findall(r"No results for \S*/stories/", out))
today = dt.date.today().isoformat()
last_date, streak = "", 0
try:
last_date, s = state.read_text().split()
streak = int(s)
except (OSError, ValueError):
pass
if empty >= to_sync: # every fetched source came back empty
if last_date != today:
streak += 1
else:
streak = 0
state.write_text(f"{today} {streak}\n")
if streak >= THRESHOLD:
print(f"SESSION LIKELY DEAD: stories came back empty for ALL {to_sync} profile(s) on "
f"{streak} consecutive day(s). The cookie is present but Instagram is probably "
"no longer honouring it -- log in again in the Chrome window (reload the tab to "
"check) and re-run.")
return 1
return 0
if __name__ == "__main__":
sys.exit(main())
-16
View File
@@ -1,16 +0,0 @@
[Unit]
# One templated service for all three modes; the instance name (%i) is the
# mode: stories, full or sweep.
Description=Instagram archive sync (%i)
Documentation=file:%h/gdl/TOOLING.md
After=network-online.target
Wants=network-online.target
[Service]
Type=oneshot
ExecStart=%h/gdl/run-%i.sh
# A sync has no deadline and the pacing is deliberately slow; a full sweep can
# run for hours. Never let systemd kill one midway -- a half-published run is
# the one state the publish step is designed to avoid.
TimeoutStartSec=infinity
Nice=10
-18
View File
@@ -1,18 +0,0 @@
[Unit]
Description=Full Instagram archive sweep, no abort (~420 requests)
# The only run that enumerates every profile to the end, and so the only one
# that notices carousels edited after we archived them (test case 15).
#
# It is also by far the most expensive thing here: ~420 requests to
# instagram.com, the same order as the run that preceded the 2026-08-21
# scraping warning, spent to catch a handful of retroactively edited posts.
# Consider running it by hand when you mean to, rather than on a timer.
[Timer]
# Month names are not valid in OnCalendar's date field -- numeric only.
OnCalendar=*-01,04,07,10-07 04:00:00
RandomizedDelaySec=45m
Persistent=true
[Install]
WantedBy=timers.target
-18
View File
@@ -1,18 +0,0 @@
[Unit]
Description=Weekly Instagram profile sync (all surfaces, --abort 50)
[Timer]
# Weekly, but deliberately not pinned to a day: OnCalendar anchors to Monday
# 00:00:00 and RandomizedDelaySec spreads the actual fire up to 6 days later,
# so which day it lands on varies week to week -- same "don't look like a
# machine" reasoning as the stories timer's 45m jitter, just applied to the
# day instead of the minute. See TOOLING.md's healthchecks.io section: the
# matching instagram-archiver-posts check uses Simple Schedule (period +
# grace) rather than Cron Schedule for exactly this reason -- a fixed cron
# day/time would fight this jitter instead of tolerating it.
OnCalendar=weekly
RandomizedDelaySec=6d
Persistent=true
[Install]
WantedBy=timers.target
-15
View File
@@ -1,15 +0,0 @@
[Unit]
Description=Daily Instagram stories sync
# Stories expire in 24h and cannot be backfilled. This is the only timer whose
# missed run means content is gone for good, which is what Persistent= is for.
[Timer]
OnCalendar=*-*-* 09:00:00
# Not a fixed time: a job firing at exactly 09:00 every day is obviously a
# machine, and the whole safety model is about not looking like one.
RandomizedDelaySec=45m
# Catch up after the host was asleep or off. cron would silently skip.
Persistent=true
[Install]
WantedBy=timers.target
-9
View File
@@ -1,9 +0,0 @@
[Unit]
Description=Upgrade gallery-dl and yt-dlp (pipx)
After=network-online.target
Wants=network-online.target
[Service]
Type=oneshot
ExecStart=%h/gdl/gdl-upgrade.sh
Nice=10
-12
View File
@@ -1,12 +0,0 @@
[Unit]
Description=Weekly gallery-dl and yt-dlp upgrade
# gdl-cron.sh also upgrades before every scrape; this keeps the tools current
# on weeks when no run happens to call it.
[Timer]
OnCalendar=weekly
RandomizedDelaySec=1d
Persistent=true
[Install]
WantedBy=timers.target
-385
View File
@@ -1,385 +0,0 @@
#!/usr/bin/env python3
"""
Tests for the request-budget logic in gdl-sync.py.
python3 -m unittest discover -s scripts -p 'test_*.py'
Deliberately stdlib-only, so it runs anywhere the sync itself runs. What is
covered here is the part that decides whether to spend a request — the part
whose absence got the archive's Instagram account suspended.
"""
import datetime as dt
import importlib.util
import json
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
from unittest import mock
_spec = importlib.util.spec_from_file_location(
"gdl_sync", Path(__file__).with_name("gdl-sync.py"))
gdl = importlib.util.module_from_spec(_spec)
sys.modules["gdl_sync"] = gdl
_spec.loader.exec_module(gdl)
NOW = dt.datetime(2026, 8, 18, 12, 0, tzinfo=dt.timezone.utc)
NOW_TS = NOW.timestamp()
def ago(hours: float) -> str:
return (NOW - dt.timedelta(hours=hours)).isoformat()
class SourceSelection(unittest.TestCase):
def test_only_stories_is_a_single_cheap_source(self):
srcs = gdl.Profile("u").sources(gdl.STORIES_ONLY)
self.assertEqual([s.kind for s in srcs], ["stories"])
self.assertEqual(srcs[0].directory, "story - u")
def test_full_sync_covers_every_surface(self):
srcs = gdl.Profile("u").sources(set(gdl.ALL_KINDS))
self.assertEqual([s.kind for s in srcs], list(gdl.ALL_KINDS))
def test_reels_and_stories_go_to_their_own_directories(self):
by_kind = {s.kind: s for s in gdl.Profile("u").sources(set(gdl.ALL_KINDS))}
self.assertEqual(by_kind["posts"].directory, "u")
self.assertEqual(by_kind["reels"].directory, "u - reels")
# Highlights derive their directory from the title mid-extraction.
self.assertEqual(by_kind["highlights"].directory, "")
class PlanSource(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.state = gdl.SyncState(Path(self.tmp.name) / "state.json")
self.posts = gdl.Profile("u").sources({"posts"})[0]
self.stories = gdl.Profile("u").sources({"stories"})[0]
def tearDown(self):
self.tmp.cleanup()
def test_first_run_seeds(self):
fetch, seed, _ = gdl.plan_source(self.posts, self.state, NOW_TS, 20)
self.assertTrue(fetch)
self.assertTrue(seed)
def test_seeding_happens_only_once(self):
self.state.mark_seeded(self.posts.url, ago(720))
fetch, seed, reason = gdl.plan_source(self.posts, self.state, NOW_TS, 20)
self.assertTrue(fetch)
self.assertFalse(seed, "a seeded source must never be re-probed")
self.assertIn("already seeded", reason)
def test_stories_never_seed(self):
# A story cannot be in the archive before it is fetched, so probing
# would double the cost of the cheapest surface for no benefit.
_, seed, reason = gdl.plan_source(self.stories, self.state, NOW_TS, 20)
self.assertFalse(seed)
self.assertIn("no seed", reason)
def test_recent_fetch_is_refused(self):
self.state.mark_fetched(self.posts.url, ago(3))
fetch, _, reason = gdl.plan_source(self.posts, self.state, NOW_TS, 20)
self.assertFalse(fetch)
self.assertIn("under the", reason)
def test_an_old_fetch_is_allowed_again(self):
self.state.mark_fetched(self.posts.url, ago(30))
fetch, _, _ = gdl.plan_source(self.posts, self.state, NOW_TS, 20)
self.assertTrue(fetch)
def test_daily_stories_pass_a_20h_floor(self):
# The cadence this is built for: once a day, every day.
self.state.mark_fetched(self.stories.url, ago(24))
fetch, _, _ = gdl.plan_source(self.stories, self.state, NOW_TS, 20)
self.assertTrue(fetch)
def test_force_disables_the_floor(self):
self.state.mark_fetched(self.posts.url, ago(1))
fetch, _, _ = gdl.plan_source(self.posts, self.state, NOW_TS, 0.0)
self.assertTrue(fetch)
def test_the_aborted_run_scenario(self):
"""
Yesterday's failure: a run died mid-way and the restart re-enumerated
every profile. Seeded-but-not-fetched must not re-probe.
"""
self.state.mark_seeded(self.posts.url, ago(0.5))
fetch, seed, _ = gdl.plan_source(self.posts, self.state, NOW_TS, 20)
self.assertTrue(fetch, "the fetch still needs to happen")
self.assertFalse(seed, "but the listing pass must not be paid for twice")
class StatePersistence(unittest.TestCase):
def test_state_survives_a_reload(self):
with tempfile.TemporaryDirectory() as d:
path = Path(d) / "state.json"
a = gdl.SyncState(path)
a.mark_seeded("https://x/", ago(1))
a.mark_fetched("https://x/", ago(1))
a.save()
b = gdl.SyncState(path)
self.assertFalse(b.needs_seed("https://x/"))
self.assertEqual(b.last_fetch("https://x/"), ago(1))
def test_a_corrupt_state_file_never_blocks_a_sync(self):
with tempfile.TemporaryDirectory() as d:
path = Path(d) / "state.json"
path.write_text("{ not json")
self.assertTrue(gdl.SyncState(path).needs_seed("https://x/"))
class ProbeCaching(unittest.TestCase):
def test_fresh_entries_are_reused_and_stale_ones_are_not(self):
with tempfile.TemporaryDirectory() as d:
cache = gdl.ProbeCache(Path(d) / "p.json", ttl_hours=24)
cache.put("https://x/", [{"shortcode": "A", "post_shortcode": "A",
"num": 1, "media_id": "1"}], ago(1))
self.assertEqual(len(cache.get("https://x/", NOW_TS)), 1)
cache.put("https://y/", [{"shortcode": "B", "post_shortcode": "B",
"num": 1, "media_id": "2"}], ago(48))
self.assertIsNone(cache.get("https://y/", NOW_TS))
def test_cache_keeps_only_the_fields_seeding_needs(self):
with tempfile.TemporaryDirectory() as d:
path = Path(d) / "p.json"
cache = gdl.ProbeCache(path, ttl_hours=24)
cache.put("https://x/", [{"shortcode": "A", "post_shortcode": "A",
"num": 1, "media_id": "1",
"description": "x" * 5000}], ago(0))
cache.save()
self.assertNotIn("description", path.read_text())
def test_a_miss_is_reported_rather_than_guessed(self):
with tempfile.TemporaryDirectory() as d:
cache = gdl.ProbeCache(Path(d) / "p.json", ttl_hours=24)
self.assertIsNone(cache.get("https://never-seen/", NOW_TS))
class Seeding(unittest.TestCase):
"""The bug that seeded 5 of 2275: matching the wrong shortcode field."""
def test_posts_are_keyed_by_post_shortcode(self):
item = {"shortcode": "childcode", "post_shortcode": "POSTCODE",
"num": 2, "media_id": "9"}
self.assertEqual(gdl.live_key(item, "posts"), ("POSTCODE", 2))
def test_stories_are_keyed_by_the_per_item_shortcode(self):
item = {"shortcode": "ITEMCODE", "post_shortcode": "reelid",
"num": 3, "media_id": "9"}
self.assertEqual(gdl.live_key(item, "stories"), ("ITEMCODE", 1))
self.assertEqual(gdl.live_key(item, "highlights"), ("ITEMCODE", 1))
def test_index_existing_normalises_a_missing_index_to_one(self):
held = gdl.index_existing([
"u/2023-04-19_u - ABC.mp4",
"u/2023-04-12_u - DEF - 3.jpg",
"u/2023-04-12_u - DEF.txt", # sidecars are not media
"u/2023-04-12_u - DEF.json",
])
self.assertEqual(held, {("ABC", 1), ("DEF", 3)})
def test_seeding_marks_only_what_is_already_held(self):
with tempfile.TemporaryDirectory() as d:
db = Path(d) / "a.db"
live = [
{"post_shortcode": "HELD", "shortcode": "x", "num": 1, "media_id": "11"},
{"post_shortcode": "NEW", "shortcode": "y", "num": 1, "media_id": "22"},
]
n = gdl.seed_archive_db(db, {("HELD", 1)}, live, "posts")
self.assertEqual(n, 1)
import sqlite3
rows = {r[0] for r in sqlite3.connect(db).execute(
"SELECT entry FROM archive")}
self.assertEqual(rows, {"instagram11"})
class Publishing(unittest.TestCase):
def test_publish_only_ever_adds(self):
cmd = gdl.rsync_command(Path("/stage"), "host:/archives", dry_run=False)
self.assertIn("--ignore-existing", cmd)
self.assertNotIn("--delete", cmd)
def test_tooling_files_are_excluded_from_the_archive(self):
cmd = " ".join(gdl.rsync_command(Path("/stage"), "/dest", dry_run=True))
for pattern in ("gdl-sync*.json", "*.db"):
self.assertIn(pattern, cmd)
self.assertIn("--dry-run", cmd)
def _publish_with(self, returncode, stderr=""):
with tempfile.TemporaryDirectory() as d:
staging = Path(d)
(staging / "file.txt").write_text("x") # publish() skips empty dirs
completed = subprocess.CompletedProcess(
args=["rsync"], returncode=returncode, stdout="", stderr=stderr)
with mock.patch("subprocess.run", return_value=completed):
return gdl.publish(staging, "host:/dest", dry_run=False)
def test_a_clean_rsync_is_success(self):
self.assertEqual(self._publish_with(0), 0)
def test_the_expected_chown_warning_is_not_a_failure(self):
# The exact shape seen on every real run: the archive share is never
# owned by the ssh user, so rsync's own chown step fails -- exit 23,
# "some files/attrs were not transferred" -- even though every file
# landed. TOOLING.md documents this as something to read past; the
# exit CODE needs to agree, or every healthcheck ping reports a
# clean run as a failure.
stderr = (
'rsync: failed to set permissions on "/dest/.": Operation not permitted (1)\n'
'rsync: failed to set permissions on "/dest/user": Operation not permitted (1)\n'
'rsync error: some files/attrs were not transferred (see previous errors) '
'(code 23) at main.c(1347) [sender=3.4.1]\n'
)
self.assertEqual(self._publish_with(23, stderr), 0)
def test_a_real_transfer_failure_still_fails(self):
# Same exit code (23) can also mean a real problem -- e.g. a file
# vanishing mid-transfer -- so only the SPECIFIC permission-warning
# shape above is masked, not the exit code in general.
stderr = (
'rsync: [sender] readlink_stat("/stage/broken") failed: '
'No such file or directory (2)\n'
'rsync error: some files/attrs were not transferred (see previous errors) '
'(code 23) at main.c(1347) [sender=3.4.1]\n'
)
self.assertEqual(self._publish_with(23, stderr), 23)
def test_an_unrelated_exit_code_still_fails(self):
self.assertEqual(self._publish_with(1, "rsync: some other error\n"), 1)
class PostUrl(unittest.TestCase):
"""--post-url: a one-off fetch outside the tracked profile list, whose
owning account is only known mid-extraction -- same reasoning as
highlights, so it must be exempted from the same forced-destination rule."""
def test_config_keys_a_username_directory(self):
config = gdl.build_config("1M", [6.0, 10.0], [3.0, 6.0])
for kind in ("post", "reel"):
self.assertEqual(
config["extractor"]["instagram"][kind]["directory"],
["{username}"])
def test_destination_is_not_forced_like_posts_and_reels(self):
staging = Path("/stage")
for subcategory in ("post", "reel"):
src = gdl.Source(subcategory, "https://www.instagram.com/p/ABC/",
"", subcategory)
cmd = gdl.gdl_command(src, staging, Path("/cfg.json"), "chrome:x",
None)
self.assertIn(str(staging), cmd)
self.assertNotIn(str(staging / subcategory), cmd)
class MetadataFields(unittest.TestCase):
"""Extra fields captured from the raw API response, verified against two
saved real examples on 2026-09-01 (see docs/gallery-dl.md)."""
def test_post_level_json_captures_coauthors(self):
config = gdl.build_config("1M", [6.0, 10.0], [3.0, 6.0])
pps = config["extractor"]["instagram"]["posts"]["postprocessors"]
post_json = next(pp for pp in pps if pp.get("filename", "").endswith(".json"))
self.assertIn("coauthors", post_json["include"])
def test_per_item_dimensions_and_tags_get_their_own_sidecar(self):
config = gdl.build_config("1M", [6.0, 10.0], [3.0, 6.0])
pps = config["extractor"]["instagram"]["posts"]["postprocessors"]
# No "filename" of its own -- unlike the other two, which are keyed
# by POST_STEM -- because it must vary per carousel item, not per post.
media_pp = next(pp for pp in pps if "filename" not in pp)
for field in ("width", "height", "width_original", "height_original",
"tagged_users", "shortcode", "num"):
self.assertIn(field, media_pp["include"])
# `owner` is a full user object (profile pic URLs, privacy flags) for
# whoever posted that item -- deliberately excluded, same reasoning
# as `audio_user` on the post-level json.
self.assertNotIn("owner", media_pp["include"])
def test_reels_get_the_same_capture_as_posts(self):
config = gdl.build_config("1M", [6.0, 10.0], [3.0, 6.0])
posts_pps = config["extractor"]["instagram"]["posts"]["postprocessors"]
reels_pps = config["extractor"]["instagram"]["reels"]["postprocessors"]
self.assertEqual(posts_pps, reels_pps)
class UrlsFile(unittest.TestCase):
def test_reads_every_form_a_person_might_paste(self):
with tempfile.TemporaryDirectory() as d:
p = Path(d) / "urls.txt"
p.write_text(
"# comment\n"
"https://www.instagram.com/a/\n"
"https://instagram.com/b\n"
"www.instagram.com/c/\n"
"d\n"
" e # trailing\n"
"\n"
"https://www.instagram.com/a/\n" # duplicate
"https://www.instagram.com/p/ABC123/\n" # a post, not a profile
"not a username\n")
self.assertEqual(gdl.read_urls_file(p), ["a", "b", "c", "d", "e"])
class PostUrlsFile(unittest.TestCase):
"""The format instagram-chrome-scrape.py writes: whole URLs, not usernames."""
def test_skips_comments_blanks_and_duplicates(self):
with tempfile.TemporaryDirectory() as d:
p = Path(d) / "reels.txt"
p.write_text(
"# scraped 2026-08-27\n"
"https://www.instagram.com/u/reel/AAA/\n"
"\n"
"https://www.instagram.com/u/reel/BBB/\n"
"https://www.instagram.com/u/reel/AAA/\n") # duplicate
self.assertEqual(gdl.read_post_urls_file(p), [
"https://www.instagram.com/u/reel/AAA/",
"https://www.instagram.com/u/reel/BBB/",
])
class ProgressOutput(unittest.TestCase):
"""The heartbeat that stops a healthy paced run (12-20s between requests, no per-file
output) from looking hung."""
def test_elapsed_formatting(self):
self.assertEqual(gdl._fmt_elapsed(5), "5s")
self.assertEqual(gdl._fmt_elapsed(125), "2m05s")
self.assertEqual(gdl._fmt_elapsed(7300), "2h01m")
def test_count_new_counts_media_written_since_and_ignores_sidecars(self):
with tempfile.TemporaryDirectory() as d:
root = Path(d)
(root / "sub").mkdir()
old = root / "old.jpg"
old.write_text("x")
import os
os.utime(old, (1, 1)) # long before `since`
(root / "sub" / "new.jpg").write_text("x")
(root / "sub" / "new.mp4").write_text("x")
for sidecar in ("new.txt", "new.json", "half.jpg.part"):
(root / "sub" / sidecar).write_text("x")
n, newest = gdl._count_new(root, since=1000)
self.assertEqual(n, 2)
self.assertIn(newest, ("new.jpg", "new.mp4"))
def test_heartbeat_prints_while_running_and_stops_cleanly(self):
with tempfile.TemporaryDirectory() as d:
with mock.patch("builtins.print") as pr:
with gdl.Heartbeat("fetching", watch=Path(d), every=0.05) as hb:
(Path(d) / "a.jpg").write_text("x")
import time
time.sleep(0.2)
lines = [str(c.args[0]) for c in pr.call_args_list]
self.assertTrue(any("fetching" in s and "new file(s) so far" in s for s in lines))
self.assertFalse(hb._thread.is_alive())
self.assertGreater(hb.elapsed, 0.1)
if __name__ == "__main__":
unittest.main(verbosity=2)
-71
View File
@@ -1,71 +0,0 @@
#!/usr/bin/env python3
"""
Tests for the pure parts of instagram-chrome-scrape.py -- everything except the actual
CDP session, which needs a live signed-in Chrome and is exercised by hand.
python3 -m unittest discover -s scripts -p 'test_*.py'
"""
import importlib.util
import sys
import unittest
from pathlib import Path
_spec = importlib.util.spec_from_file_location(
"instagram_chrome_scrape", Path(__file__).with_name("instagram-chrome-scrape.py"))
scrape = importlib.util.module_from_spec(_spec)
sys.modules["instagram_chrome_scrape"] = scrape
_spec.loader.exec_module(scrape)
class ExtractShortcodes(unittest.TestCase):
def test_reads_relative_and_absolute_hrefs(self):
hrefs = [
"/someuser/reel/ABC123/",
"https://www.instagram.com/someuser/reel/DEF456/",
"/reel/GHI789/?img_index=1",
]
self.assertEqual(scrape.extract_shortcodes(hrefs),
["ABC123", "DEF456", "GHI789"])
def test_ignores_non_reel_links(self):
hrefs = ["/someuser/", "/someuser/p/ABC123/", "/explore/tags/foo/"]
self.assertEqual(scrape.extract_shortcodes(hrefs), [])
def test_empty_input(self):
self.assertEqual(scrape.extract_shortcodes([]), [])
def test_posts_kinds_read_both_post_and_reel_links(self):
hrefs = ["/someuser/p/ABC123/", "/p/DEF456/?img_index=2",
"/someuser/reel/GHI789/", "/explore/tags/foo/"]
self.assertEqual(scrape.extract_shortcodes(hrefs, ("p", "reel")),
["ABC123", "DEF456", "GHI789"])
class ShouldStopKnown(unittest.TestCase):
have = {"OLD1", "OLD2", "OLD3", "OLD4"}
def step(self, round_no, new, streak=0, max_known=2):
return scrape.should_stop_known(round_no, new, self.have, streak, max_known)
def test_first_round_never_counts(self):
# pinned posts: old, but on top of the grid
self.assertEqual(self.step(1, ["OLD1", "OLD2"]), (False, 0))
def test_stops_after_consecutive_known_rounds(self):
stop, streak = self.step(2, ["OLD1"])
self.assertEqual((stop, streak), (False, 1))
self.assertEqual(self.step(3, ["OLD2"], streak), (True, 2))
def test_unarchived_resets_streak(self):
self.assertEqual(self.step(3, ["OLD3", "NEW1"], streak=1), (False, 0))
def test_empty_round_leaves_streak(self):
self.assertEqual(self.step(3, [], streak=1), (False, 1))
def test_disabled(self):
self.assertEqual(self.step(5, ["OLD1"], streak=9, max_known=0), (False, 9))
if __name__ == "__main__":
unittest.main(verbosity=2)
-160
View File
@@ -1,160 +0,0 @@
"""
Tests for the two guards that stop a dead Instagram session going unnoticed:
check-session.py preflight: is there a usable `sessionid` in the Chrome cookie DB?
stories-streak.py postflight: did every stories source come back empty for days running?
Both are run as subprocesses (the way gdl-cron.sh runs them); neither touches the network.
python3 -I -m unittest scripts/test_session_guards.py
"""
import datetime as dt
import os
import sqlite3
import subprocess
import sys
import tempfile
import time
import unittest
from pathlib import Path
HERE = Path(__file__).resolve().parent
CHROME_EPOCH = 11644473600 # seconds between 1601-01-01 and 1970-01-01
def run(script, *args, env=None):
e = {**os.environ, **(env or {})}
return subprocess.run([sys.executable, str(HERE / script), *args],
capture_output=True, text=True, env=e)
class CheckSession(unittest.TestCase):
def profile(self, rows):
"""A fake Chrome profile whose cookie DB holds `rows` = [(host, name, expires_unix|None)]."""
tmp = tempfile.TemporaryDirectory()
self.addCleanup(tmp.cleanup)
d = Path(tmp.name) / "Default"
d.mkdir()
con = sqlite3.connect(d / "Cookies")
con.execute("CREATE TABLE cookies (host_key TEXT, name TEXT, expires_utc INTEGER, has_expires INTEGER)")
for host, name, exp in rows:
con.execute("INSERT INTO cookies VALUES (?,?,?,?)",
(host, name, 0 if exp is None else (exp + CHROME_EPOCH) * 1_000_000,
0 if exp is None else 1))
con.commit()
con.close()
return tmp.name
def check(self, rows):
return run("check-session.py", env={"GDL_COOKIE_PROFILE": self.profile(rows)})
def test_valid_sessionid_passes(self):
r = self.check([(".instagram.com", "sessionid", time.time() + 200 * 86400)])
self.assertEqual(r.returncode, 0, r.stdout)
self.assertIn("session ok", r.stdout)
self.assertNotIn("WARNING", r.stdout)
def test_missing_sessionid_fails_even_with_other_instagram_cookies(self):
# What the real failure looked like: csrftoken/ds_user_id/mid linger after logout.
r = self.check([(".instagram.com", "csrftoken", time.time() + 86400),
(".instagram.com", "ds_user_id", time.time() + 86400),
(".instagram.com", "mid", time.time() + 86400)])
self.assertEqual(r.returncode, 1)
self.assertIn("NOT LOGGED IN", r.stdout)
def test_expired_sessionid_fails(self):
r = self.check([(".instagram.com", "sessionid", time.time() - 86400)])
self.assertEqual(r.returncode, 1)
self.assertIn("expired", r.stdout)
def test_near_expiry_warns_but_passes(self):
r = self.check([(".instagram.com", "sessionid", time.time() + 3 * 86400)])
self.assertEqual(r.returncode, 0)
self.assertIn("WARNING", r.stdout)
def test_session_cookie_without_expiry_passes(self):
r = self.check([(".instagram.com", "sessionid", None)])
self.assertEqual(r.returncode, 0)
def test_other_sites_sessionid_does_not_count(self):
r = self.check([(".example.com", "sessionid", time.time() + 86400)])
self.assertEqual(r.returncode, 1)
def test_missing_profile_fails_loudly(self):
r = run("check-session.py", env={"GDL_COOKIE_PROFILE": "/nonexistent/profile"})
self.assertEqual(r.returncode, 1)
self.assertIn("no cookie DB", r.stdout)
def test_never_prints_cookie_values(self):
# The fake DB has no value column at all; the script must not need one.
r = self.check([(".instagram.com", "sessionid", time.time() + 90 * 86400)])
self.assertNotIn("sessionid=", r.stdout + r.stderr)
class StoriesStreak(unittest.TestCase):
def setUp(self):
tmp = tempfile.TemporaryDirectory()
self.addCleanup(tmp.cleanup)
self.dir = Path(tmp.name)
self.state = self.dir / "streak"
self.today = dt.date.today().isoformat()
self.yesterday = (dt.date.today() - dt.timedelta(days=1)).isoformat()
def out(self, to_sync, empty, skipped=0):
lines = [f"sources : {to_sync} to sync, {skipped} skipped"]
lines += [f"[instagram][info] No results for https://www.instagram.com/stories/user{i}/"
for i in range(empty)]
p = self.dir / "run.out"
p.write_text("\n".join(lines) + "\n")
return p
def streak(self, to_sync, empty, skipped=0):
return run("stories-streak.py", str(self.out(to_sync, empty, skipped)), str(self.state))
def stored(self):
return self.state.read_text().split()
def test_all_empty_for_three_days_fails(self):
self.state.write_text(f"{self.yesterday} 2\n")
r = self.streak(6, 6)
self.assertEqual(r.returncode, 1)
self.assertIn("SESSION LIKELY DEAD", r.stdout)
self.assertEqual(self.stored(), [self.today, "3"])
def test_first_empty_day_is_quiet(self):
r = self.streak(6, 6)
self.assertEqual(r.returncode, 0)
self.assertEqual(self.stored(), [self.today, "1"])
def test_same_day_rerun_does_not_inflate_the_streak(self):
self.state.write_text(f"{self.today} 2\n")
r = self.streak(6, 6)
self.assertEqual(r.returncode, 0)
self.assertEqual(self.stored(), [self.today, "2"])
def test_any_story_resets_the_streak(self):
self.state.write_text(f"{self.yesterday} 2\n")
r = self.streak(6, 5) # one profile had a story
self.assertEqual(r.returncode, 0)
self.assertEqual(self.stored(), [self.today, "0"])
def test_run_that_fetched_nothing_leaves_state_alone(self):
self.state.write_text(f"{self.yesterday} 2\n")
r = self.streak(0, 0, skipped=6)
self.assertEqual(r.returncode, 0)
self.assertEqual(self.stored(), [self.yesterday, "2"])
def test_threshold_is_configurable(self):
r = run("stories-streak.py", str(self.out(6, 6)), str(self.state),
env={"GDL_EMPTY_STREAK_FAIL": "1"})
self.assertEqual(r.returncode, 1)
def test_corrupt_state_file_is_treated_as_no_history(self):
self.state.write_text("garbage\n")
r = self.streak(6, 6)
self.assertEqual(r.returncode, 0)
self.assertEqual(self.stored(), [self.today, "1"])
if __name__ == "__main__":
unittest.main()
+4 -26
View File
@@ -16,23 +16,7 @@ const PORT = process.env.PORT || 3001;
const ARCHIVES_DIR = path.resolve(process.env.ARCHIVES_DIR || path.join(__dirname, '_sample-archives'));
console.log(`[Server] Initializing...`);
/**
* Describe the running user without assuming it exists in /etc/passwd.
*
* `os.userInfo()` throws ENOENT for a UID with no passwd entry, which is
* exactly what happens when the container is started with `--user 1234:1234`
* (as the deployment docs suggest) — previously crashing the server at boot.
*/
const describeUser = () => {
try {
const info = os.userInfo();
return `${info.username} (UID: ${info.uid}, GID: ${info.gid})`;
} catch {
return `UID: ${typeof process.getuid === 'function' ? process.getuid() : '?'}, GID: ${typeof process.getgid === 'function' ? process.getgid() : '?'} (no passwd entry)`;
}
};
console.log(`[Server] Running as user: ${describeUser()}`);
console.log(`[Server] Running as user: ${os.userInfo().username} (UID: ${os.userInfo().uid}, GID: ${os.userInfo().gid})`);
console.log(`[Server] Environment ARCHIVES_DIR: ${process.env.ARCHIVES_DIR}`);
console.log(`[Server] Resolved ARCHIVES_DIR: ${ARCHIVES_DIR}`);
@@ -80,16 +64,10 @@ app.use((req, res, next) => {
"default-src 'self'",
"img-src 'self' blob: data:",
"media-src 'self' blob: data:",
// 'wasm-unsafe-eval' permits WebAssembly compilation without allowing
// eval() of JavaScript. The xz decompressor used for Instaloader's
// .json.xz sidecars is WebAssembly, embedded as a data: URL it fetches at
// startup — so connect-src must allow data: too. Without both, decoding
// fails with a bare "TypeError: Failed to fetch" and every archive silently
// loses its captions, story flags and profile metadata.
"script-src 'self' 'wasm-unsafe-eval'",
"script-src 'self'",
"style-src 'self' 'unsafe-inline'",
"font-src 'self'",
"connect-src 'self' data:",
"connect-src 'self'",
"worker-src 'self' blob:",
"frame-ancestors 'self'",
"object-src 'none'",
@@ -136,7 +114,7 @@ app.get('/api/archives', (req, res) => {
res.json(archives);
} catch (err: any) {
if (err.code === 'EACCES') {
console.error(`[API] Permission Denied! The server (${describeUser()}) cannot read ${ARCHIVES_DIR}.`);
console.error(`[API] Permission Denied! The server (UID ${os.userInfo().uid}) cannot read ${ARCHIVES_DIR}.`);
console.error(`[API] Hint: If using Linux/Docker, check folder permissions (chmod 755) or SELinux context (append :z to your volume mount).`);
} else {
console.error('[API] Error listing archives:', err);
+75 -122
View File
@@ -17,9 +17,6 @@ import {
import { motion, AnimatePresence } from 'motion/react';
import { cn } from './lib/utils';
import { PRESS, prefersReducedMotion } from './lib/motion';
import { buildPath, findPostBySlug, parseRoute, postSlug, tabForSource } from './lib/routing';
import { postsForTab } from './lib/post-tabs';
import { LocalArchiveFile, RemoteArchiveFile } from './lib/archive-files';
import {
deleteCachedArchive,
@@ -39,8 +36,6 @@ import { CacheData, Post, ServerArchive, ServerArchiveFile } from './types';
import { ArchiveDashboard } from './components/ArchiveDashboard';
import { StoryViewer } from './components/StoryViewer';
import { PostModal } from './components/PostModal';
import { PostFeed } from './components/PostFeed';
import { useIsMobile } from './hooks/useIsMobile';
import { PostThumbnail } from './components/PostThumbnail';
import { useArchiveScanner } from './hooks/useArchiveScanner';
import { useThumbnailQueue } from './hooks/useThumbnailQueue';
@@ -65,15 +60,14 @@ export default function App() {
const [hasInitialLoaded, setHasInitialLoaded] = useState(false);
/**
* The route as it was when the app booted.
* The query string as it was when the app booted.
*
* Captured during the first render because the URL is rewritten from app
* state as soon as anything loads; reading `window.location` later would see
* the rewritten value rather than the link the user actually followed.
*/
const initialRouteRef = useRef(parseRoute(window.location.pathname, window.location.search));
const initialParamsRef = useRef(new URLSearchParams(window.location.search));
const isMobile = useIsMobile();
const fileInputRef = useRef<HTMLInputElement>(null);
const profilePicInputRef = useRef<HTMLInputElement>(null);
@@ -112,30 +106,6 @@ export default function App() {
const [lastLoadedScanningImage, setLastLoadedScanningImage] = useState<string | null>(null);
/**
* Blurred backdrops behind the scanning UI, newest last.
*
* Each new image is stacked *over* the previous one and fades in; the one
* underneath stays fully opaque until it's covered. Cross-fading by swapping
* a single element left the pale backdrop showing through mid-transition,
* which read as a white flash between every image.
*/
const [scanBackdrops, setScanBackdrops] = useState<string[]>([]);
useEffect(() => {
if (!lastLoadedScanningImage) return;
setScanBackdrops(prev =>
prev[prev.length - 1] === lastLoadedScanningImage
? prev
: [...prev, lastLoadedScanningImage].slice(-3),
);
}, [lastLoadedScanningImage]);
// Don't carry one archive's backdrops into the next scan.
useEffect(() => {
if (!isScanning) { setScanBackdrops([]); setLastLoadedScanningImage(null); }
}, [isScanning]);
const {
username,
fullName,
@@ -173,18 +143,20 @@ export default function App() {
const clearCache = async (name: string) => { await deleteCachedArchive(name); await refreshCachedArchives(); };
/**
* The grid shows everything, reels included, and the Reels tab is a filtered
* view of the same set — see src/lib/post-tabs.ts for the reel test and for
* why a reel can arrive on disk twice.
* Archives with a `- reels` sidecar directory say outright which posts are
* reels; only fall back to the "lone video" heuristic for archives that have
* no such directory.
*/
const filteredPosts = useMemo(() => postsForTab(allPosts, activeTab), [allPosts, activeTab]);
const hasReelSource = useMemo(() => allPosts.some(p => p.source === 'reels'), [allPosts]);
const isReel = useCallback((p: Post) => (
hasReelSource ? p.source === 'reels' : p.media.length === 1 && p.media[0].type === 'video'
), [hasReelSource]);
/**
* Instagram's "N posts" counter equals what its grid holds, so count the grid
* rather than `allPosts` — the raw list still holds both copies of any post
* fetched into two directories.
*/
const totalPosts = useMemo(() => postsForTab(allPosts, 'posts').length, [allPosts]);
const filteredPosts = useMemo(() => {
if (activeTab === 'reels') return allPosts.filter(isReel);
if (activeTab === 'posts') return allPosts.filter(p => !isReel(p));
return [];
}, [allPosts, activeTab, isReel]);
/** Story highlights, grouped into the circles shown under the bio. */
const highlightGroups = useMemo(() => {
@@ -367,28 +339,41 @@ export default function App() {
// loader below is waiting to read.
if (!hasInitialLoaded) return;
const archive = currentArchive?.name ?? (allPosts.length > 0 ? username : null) ?? null;
const nextPath = buildPath({
archive,
tab: activeTab,
post: selectedPost ? postSlug(selectedPost) : null,
});
const params = new URLSearchParams(window.location.search);
if (currentArchive) params.set('a', currentArchive.name);
else if (allPosts.length > 0 && username) params.set('a', username);
else params.delete('a');
if (nextPath !== window.location.pathname + window.location.search) {
console.log(`[Permalink] Updating URL to: ${nextPath}`);
window.history.replaceState(null, '', nextPath);
if (activeTab !== 'posts') params.set('t', activeTab);
else params.delete('t');
if (selectedPost) params.set('p', selectedPost.id);
else params.delete('p');
const newSearch = params.toString();
const currentSearch = new URLSearchParams(window.location.search).toString();
if (newSearch !== currentSearch) {
console.log(`[Permalink] Updating URL to: ?${newSearch}`);
const newUrl = window.location.pathname + (newSearch ? `?${newSearch}` : '');
window.history.replaceState(null, '', newUrl);
}
}, [hasInitialLoaded, currentArchive?.name, username, allPosts.length, activeTab, selectedPost?.id]);
useEffect(() => {
if (hasInitialLoaded) return;
const route = initialRouteRef.current;
console.log('[Permalink] Initial route:', route);
const params = initialParamsRef.current;
const archiveName = params.get('a');
const tab = params.get('t');
console.log('[Permalink] Initial read from URL:', {
archiveName, tab, postId: params.get('p'),
});
if (route.tab !== 'posts') setActiveTab(route.tab);
if (tab && ['posts', 'reels', 'saved'].includes(tab)) {
setActiveTab(tab as 'posts' | 'reels' | 'saved');
}
if (!route.archive) {
if (!archiveName) {
setHasInitialLoaded(true);
return;
}
@@ -396,12 +381,12 @@ export default function App() {
// Wait for the archive list before deciding the link is unresolvable.
if (!archivesFetched) return;
const archive = serverArchives.find(a => a.name === route.archive);
const archive = serverArchives.find(a => a.name === archiveName);
if (archive) {
console.log(`[Permalink] Auto-loading archive: ${route.archive}`);
console.log(`[Permalink] Auto-loading archive: ?a=${archiveName}`);
loadServerArchive(archive);
} else {
console.warn(`[Permalink] No archive named "${route.archive}".`);
console.warn(`[Permalink] No archive named "${archiveName}".`);
}
setHasInitialLoaded(true);
}, [serverArchives, archivesFetched, hasInitialLoaded, loadServerArchive]);
@@ -421,14 +406,10 @@ export default function App() {
if (appliedPostParamRef.current === archiveKey) return;
appliedPostParamRef.current = archiveKey;
const slug = initialRouteRef.current.post;
if (!slug) return;
const post = findPostBySlug(allPosts, slug);
if (!post) return;
// A /p/<code>/ link carries no tab, so derive the one that contains it —
// otherwise next/prev would page through the wrong list.
setActiveTab(tabForSource(post.source));
setSelectedPost(post);
const postId = initialParamsRef.current.get('p');
if (!postId) return;
const post = allPosts.find(p => p.id === postId);
if (post) setSelectedPost(post);
}, [allPosts, currentArchive?.name, username]);
return (
@@ -483,28 +464,17 @@ export default function App() {
onLoad={() => setLastLoadedScanningImage(currentScanningImage)}
/>
)}
{/*
The 0.4 lives on the group, not the images: two layers overlap
during a cross-fade, and fading them individually would darken the
backdrop as they cross. Inside the group each layer goes to full
opacity, so the stack is always completely covered.
*/}
<div className="absolute inset-0 z-0 opacity-40">
{scanBackdrops.map(src => (
<motion.img
key={src}
src={src}
<div className="absolute inset-0 z-0">
<AnimatePresence initial={false}>
<motion.img
key={lastLoadedScanningImage}
src={lastLoadedScanningImage || undefined}
initial={{ opacity: 0 }}
animate={{ opacity: 1 }}
transition={prefersReducedMotion() ? { duration: 0 } : { duration: 0.9, ease: 'easeInOut' }}
onAnimationComplete={() => setScanBackdrops(prev => {
// Once this layer is opaque it hides everything below it.
const i = prev.indexOf(src);
return i > 0 ? prev.slice(i) : prev;
})}
animate={{ opacity: 0.4 }}
transition={{ duration: 1.5 }}
className="absolute inset-0 w-full h-full object-cover blur-[60px] scale-110"
/>
))}
</AnimatePresence>
</div>
<div className="absolute inset-0 bg-white/40 z-1" />
<div className="relative z-10 w-full max-w-4xl px-4 flex flex-col items-center gap-8 text-black">
@@ -526,7 +496,7 @@ export default function App() {
{allProfilePics.length > 1 && <button onClick={cycleProfilePic} className="bg-gray-100 hover:bg-gray-200 px-4 py-1.5 rounded-lg text-sm font-semibold transition-colors flex items-center gap-2 text-black"><Layers size={16} />Next Profile Pic</button>}
</div>
</div>
<div className="flex justify-center md:justify-start gap-10 text-sm md:text-base text-black"><div><span className="font-semibold text-black/80 text-black">{totalPosts.toLocaleString()}</span> posts</div><div><span className="font-semibold text-black/80 text-black">{(followerCount || 0).toLocaleString()}</span> followers</div><div><span className="font-semibold text-black/80 text-black">{(followingCount || 0).toLocaleString()}</span> following</div></div>
<div className="flex justify-center md:justify-start gap-10 text-sm md:text-base text-black"><div><span className="font-semibold text-black/80 text-black">{allPosts.length}</span> posts</div><div><span className="font-semibold text-black/80 text-black">{(followerCount || 0).toLocaleString()}</span> followers</div><div><span className="font-semibold text-black/80 text-black">{(followingCount || 0).toLocaleString()}</span> following</div></div>
<div className="space-y-1 text-black/80 text-black"><div className="font-semibold text-black">{fullName || `@${username}`}</div><div className="text-gray-600 whitespace-pre-wrap max-w-sm mx-auto md:mx-0 text-sm md:text-base text-black">{bio || 'Archived profile viewer for local files.'}</div>{externalUrl && <a href={externalUrl} target="_blank" rel="noopener noreferrer" className="text-blue-900 font-semibold text-sm block hover:underline truncate max-w-[250px] text-black">{externalUrl.replace(/^https?:\/\/(www\.)?/, '')}</a>}</div>
</div>
</header>
@@ -534,11 +504,9 @@ export default function App() {
{highlightGroups.length > 0 && (
<div className="flex gap-6 md:gap-8 overflow-x-auto scrollbar-hide px-4 pb-2">
{highlightGroups.map(group => (
<motion.button
<button
key={group.title}
onClick={() => setActiveHighlight(group.title)}
whileTap={{ scale: 0.94 }}
transition={PRESS}
className="flex flex-col items-center gap-2 shrink-0 group/hl"
title={`${group.title} — ${group.items.length} item${group.items.length === 1 ? '' : 's'}`}
>
@@ -554,7 +522,7 @@ export default function App() {
</div>
</div>
<span className="text-[11px] max-w-[80px] truncate text-gray-700">{group.title}</span>
</motion.button>
</button>
))}
</div>
)}
@@ -570,7 +538,7 @@ export default function App() {
<div className="grid grid-cols-3 gap-[2px] md:gap-[2px] text-black">
{activeTab === 'posts' && Array.from({ length: gridOffset }).map((_, i) => (<div key={`blank-${i}`} className={cn("bg-gray-100/50 border border-dashed border-gray-200 flex items-center justify-center text-[10px] font-bold text-gray-300 uppercase tracking-tighter text-black", gridAspectRatio === '1:1' ? "aspect-square" : "aspect-[3/4]")}>Blank</div>))}
{visiblePosts.map((post) => (
<motion.div key={post.id} layoutId={post.id} onClick={() => setSelectedPost(post)} whileTap={{ scale: 0.97 }} transition={PRESS} className={cn("relative group cursor-pointer overflow-hidden bg-gray-200 transition-all duration-300 text-black", activeTab === 'reels' ? "aspect-[9/16]" : (gridAspectRatio === '1:1' ? "aspect-square" : "aspect-[3/4]"))}>
<motion.div key={post.id} layoutId={post.id} onClick={() => setSelectedPost(post)} className={cn("relative group cursor-pointer overflow-hidden bg-gray-200 transition-all duration-300 text-black", activeTab === 'reels' ? "aspect-[9/16]" : (gridAspectRatio === '1:1' ? "aspect-square" : "aspect-[3/4]"))}>
<PostThumbnail
post={post}
thumbnailUrl={cacheHits.get(post.id)}
@@ -586,36 +554,21 @@ export default function App() {
)}
</main>
{/*
Mobile opens a real scrolling feed page, the way Instagram does; desktop
keeps the modal, where a centred sheet with side arrows fits the pointer.
*/}
{selectedPost && isMobile ? (
<PostFeed
posts={filteredPosts}
initialPostId={selectedPost.id}
profilePic={profilePic}
title={activeTab === 'reels' ? 'Reels' : 'Posts'}
onClose={() => setSelectedPost(null)}
onActivePostChange={setSelectedPost}
/>
) : (
<AnimatePresence>
{selectedPost && (
<PostModal
post={selectedPost}
nextPost={postIndex < filteredPosts.length - 1 ? filteredPosts[postIndex + 1] : undefined}
prevPost={postIndex > 0 ? filteredPosts[postIndex - 1] : undefined}
onClose={() => setSelectedPost(null)}
onNextPost={onNextPost}
onPrevPost={onPrevPost}
hasNextPost={postIndex < filteredPosts.length - 1}
hasPrevPost={postIndex > 0}
profilePic={profilePic}
/>
)}
</AnimatePresence>
)}
<AnimatePresence>
{selectedPost && (
<PostModal
post={selectedPost}
nextPost={postIndex < filteredPosts.length - 1 ? filteredPosts[postIndex + 1] : undefined}
prevPost={postIndex > 0 ? filteredPosts[postIndex - 1] : undefined}
onClose={() => setSelectedPost(null)}
onNextPost={onNextPost}
onPrevPost={onPrevPost}
hasNextPost={postIndex < filteredPosts.length - 1}
hasPrevPost={postIndex > 0}
profilePic={profilePic}
/>
)}
</AnimatePresence>
<AnimatePresence>{showStoryViewer && allStories.length > 0 && <StoryViewer stories={allStories} onClose={() => setShowStoryViewer(false)} profilePic={profilePic} />}</AnimatePresence>
<AnimatePresence>
{activeHighlight && (
@@ -631,7 +584,7 @@ export default function App() {
{!isScanning && (
<footer className="max-w-5xl mx-auto px-4 py-12 text-center text-xs text-gray-400 space-y-4 text-black">
<div className="flex flex-wrap justify-center gap-x-4 gap-y-2 uppercase tracking-tight text-black"><span>Meta</span><span>About</span><span>Blog</span><span>Jobs</span><span>Help</span><span>API</span><span>Privacy</span><span>Terms</span><span>Locations</span><span>Instagram Lite</span><span>Threads</span><span>Contact Uploading & Non-Users</span><span>Meta Verified</span></div>
<div className="text-black/40 text-black">© 2026 InstaArchive Viewer · v{__APP_VERSION__}</div>
<div className="text-black/40 text-black">© 2026 InstaArchive Viewer</div>
</footer>
)}
</div>
-67
View File
@@ -1,67 +0,0 @@
import React from 'react';
import { Bookmark, Heart, MessageCircle, MoreHorizontal, Send } from 'lucide-react';
import { Post } from '../types';
import { formatDateSafe } from '../lib/utils';
import { MediaCarousel } from './MediaCarousel';
interface FeedPostProps {
post: Post;
profilePic: string | null;
/** Off-screen posts keep their video paused. */
paused: boolean;
}
/**
* One post in the mobile feed, laid out like Instagram's: header, media,
* action row, then caption.
*
* Media is capped below full viewport height so the next post always peeks in
* at the bottom — that overlap is what tells you the page scrolls rather than
* pages.
*/
export const FeedPost: React.FC<FeedPostProps> = ({ post, profilePic, paused }) => (
<article className="bg-white border-b border-gray-200">
<header className="flex items-center justify-between px-3 py-2.5">
<div className="flex items-center gap-2.5 min-w-0">
<div className="w-8 h-8 rounded-full bg-gradient-to-tr from-yellow-400 to-purple-600 p-0.5 shrink-0">
<div className="w-full h-full rounded-full bg-white p-0.5">
<div className="w-full h-full rounded-full bg-gray-200 overflow-hidden flex items-center justify-center text-[10px] font-bold uppercase">
{profilePic
? <img src={profilePic} alt="" className="w-full h-full object-cover" referrerPolicy="no-referrer" />
: <span>{post.username[0]}</span>}
</div>
</div>
</div>
<span className="font-semibold text-sm truncate">{post.username}</span>
</div>
<MoreHorizontal size={20} className="text-gray-500 shrink-0" />
</header>
{/*
Taller ceiling than the modal so ordinary portrait media (9:16 reels,
4:5 photos) fills the feed width instead of sitting in side bars, while
still stopping anything extreme from swallowing the screen.
*/}
<MediaCarousel post={post} paused={paused} heightCap="max-h-[85vh]" fillWidth className="bg-black" />
<div className="px-3 pt-3 pb-1 flex items-center justify-between">
<div className="flex items-center gap-4">
<Heart size={24} className="cursor-pointer" />
<MessageCircle size={24} className="cursor-pointer" />
<Send size={24} className="cursor-pointer" />
</div>
<Bookmark size={24} className="cursor-pointer" />
</div>
{post.caption && (
<div className="px-3 pb-1 text-sm">
<span className="font-semibold mr-2">{post.username}</span>
<span className="whitespace-pre-wrap">{post.caption}</span>
</div>
)}
<div className="px-3 pb-3 pt-1 text-[10px] uppercase tracking-wide text-gray-400">
{formatDateSafe(post.date, 'MMMM d, yyyy')}
</div>
</article>
);
-112
View File
@@ -1,112 +0,0 @@
import React, { useEffect, useState } from 'react';
import { ChevronLeft, ChevronRight } from 'lucide-react';
import { motion, AnimatePresence } from 'motion/react';
import { Post } from '../types';
import { cn } from '../lib/utils';
import { NAVIGATE, prefersReducedMotion, withVelocity } from '../lib/motion';
import { MediaRenderer } from './MediaRenderer';
interface MediaCarouselProps {
post: Post;
/** Pause video even when this slide is on screen (feed: only one plays). */
paused?: boolean;
/** Override the media height ceiling (the feed allows taller media). */
heightCap?: string;
/** Size video to the container width (see MediaRenderer). */
fillWidth?: boolean;
className?: string;
}
/**
* The horizontal slide strip for one post.
*
* Shared by the desktop modal and the mobile feed so a carousel behaves the
* same in both. Horizontal drag belongs to the carousel and never navigates
* between posts — vertical movement is the page's to handle.
*/
export const MediaCarousel: React.FC<MediaCarouselProps> = ({ post, paused, heightCap, fillWidth, className }) => {
const [index, setIndex] = useState(0);
const [slide, setSlide] = useState<{ dir: number; velocity: number }>({ dir: 0, velocity: 0 });
const reduceMotion = prefersReducedMotion();
useEffect(() => setIndex(0), [post.id]);
const paginate = (dir: number, velocity = 0) => {
const next = index + dir;
if (next < 0 || next >= post.media.length) return;
setSlide({ dir, velocity });
setIndex(next);
};
const variants = {
enter: (d: number) => ({ x: d > 0 ? '100%' : '-100%', opacity: 1, zIndex: 0 }),
center: { x: 0, opacity: 1, zIndex: 1 },
exit: (d: number) => ({ x: d < 0 ? '100%' : '-100%', opacity: 1, zIndex: 0 }),
};
const swipePower = (offset: number, velocity: number) => Math.abs(offset) * velocity;
const current = post.media[index];
return (
<div className={cn('relative bg-black flex items-center justify-center group overflow-hidden w-full', className)}>
<div className="w-full grid grid-cols-1 grid-rows-1">
<AnimatePresence initial={false} custom={slide.dir}>
<motion.div
key={`${post.id}-${index}`}
custom={slide.dir}
variants={variants}
initial="enter"
animate="center"
exit="exit"
transition={reduceMotion ? { duration: 0 } : withVelocity(slide.velocity, NAVIGATE)}
drag={post.media.length > 1 ? 'x' : false}
dragDirectionLock
dragConstraints={{ left: 0, right: 0 }}
dragElastic={0.5}
onDragEnd={(e, { offset, velocity }) => {
const power = swipePower(offset.x, velocity.x);
if (power < -15000) paginate(1, velocity.x);
else if (power > 15000) paginate(-1, velocity.x);
}}
className="col-start-1 row-start-1 w-full flex items-center justify-center relative touch-pan-y"
>
{current && <MediaRenderer file={current} isFullView paused={paused} heightCap={heightCap} fillWidth={fillWidth} />}
</motion.div>
</AnimatePresence>
</div>
{post.media.length > 1 && (
<>
{index > 0 && (
<button
aria-label="Previous photo"
onClick={(e) => { e.stopPropagation(); paginate(-1); }}
className="hidden md:block absolute left-4 top-1/2 -translate-y-1/2 bg-white/20 hover:bg-white/40 text-white p-2 rounded-full backdrop-blur-md transition-all opacity-0 group-hover:opacity-100 z-30"
>
<ChevronLeft size={24} />
</button>
)}
{index < post.media.length - 1 && (
<button
aria-label="Next photo"
onClick={(e) => { e.stopPropagation(); paginate(1); }}
className="hidden md:block absolute right-4 top-1/2 -translate-y-1/2 bg-white/20 hover:bg-white/40 text-white p-2 rounded-full backdrop-blur-md transition-all opacity-0 group-hover:opacity-100 z-30"
>
<ChevronRight size={24} />
</button>
)}
<div className="absolute bottom-4 left-1/2 -translate-x-1/2 flex gap-1.5 z-30">
{post.media.map((_, i) => (
<div
key={i}
className={cn(
'w-1.5 h-1.5 rounded-full transition-all',
i === index ? 'bg-blue-500 scale-125' : 'bg-white/40 shadow-sm',
)}
/>
))}
</div>
</>
)}
</div>
);
};
+6 -65
View File
@@ -1,71 +1,12 @@
import React, { useState, useEffect, useRef } from 'react';
import React, { useState } from 'react';
import { Play, Volume2, VolumeX } from 'lucide-react';
import { MediaFile } from '../types';
import { cn } from '../lib/utils';
interface MediaRendererProps {
file: MediaFile;
className?: string;
isFullView?: boolean;
/** Hold playback: the feed keeps every off-screen video paused. */
paused?: boolean;
/** Cap media height to this instead of the default full-view ceiling. */
heightCap?: string;
/**
* Size video to the container width rather than its own intrinsic size.
*
* A <video> reports 300x150 until metadata loads, so `w-auto` makes it render
* narrow and then jump to full width. The feed needs a stable width more than
* it needs a snug fit.
*/
fillWidth?: boolean;
}
export const MediaRenderer = ({ file, className, isFullView, paused, heightCap, fillWidth }: MediaRendererProps) => {
// Try to play with sound: opening the modal is a user gesture, so browsers
// generally allow it. If this particular browser still refuses, the effect
// below falls back to muted playback rather than leaving a stalled video.
const [isMuted, setIsMuted] = useState(false);
const videoRef = useRef<HTMLVideoElement>(null);
useEffect(() => {
const video = videoRef.current;
if (!video || file.type !== 'video') return;
if (paused) {
video.pause();
return;
}
let cancelled = false;
video.muted = false;
video.play().catch(() => {
if (cancelled) return;
setIsMuted(true);
video.muted = true;
video.play().catch(() => { /* user can start it from the controls */ });
});
return () => { cancelled = true; };
}, [file.url, file.type, paused]);
/**
* In full view the media must never outgrow the viewport.
*
* Video is sized to its own aspect within the cap (`w-auto`) so a portrait
* clip doesn't sit in a wide letterbox, while images keep filling the modal
* width and only gain a height ceiling — `object-contain` stops the cap from
* distorting anything that hits it.
*
* The desktop cap subtracts the modal's own padding (md:p-10 = 2.5rem each
* side); mobile leaves room for the caption panel stacked underneath.
*/
const fullViewCap = `${heightCap ?? 'max-h-[70vh] md:max-h-[calc(100vh-5rem)]'} object-contain`;
const videoFullView = fillWidth
? `block w-full h-auto ${fullViewCap}`
: `block w-auto max-w-full ${fullViewCap}`;
const videoSizing = isFullView ? videoFullView : "w-full h-full object-cover";
const imageSizing = isFullView ? `block w-full h-auto ${fullViewCap}` : "w-full h-full object-cover";
const sizingClass = file.type === 'video' ? videoSizing : imageSizing;
export const MediaRenderer = ({ file, className, isFullView }: { file: MediaFile; className?: string; isFullView?: boolean }) => {
// Start muted so autoplay is not blocked by Safari/Firefox policy.
const [isMuted, setIsMuted] = useState(true);
const sizingClass = isFullView ? "w-full h-auto block" : "w-full h-full object-cover";
const mediaStyle = { transform: 'translateZ(0)' };
if (!file.url) return <div className={cn("bg-gray-100 flex items-center justify-center text-black", sizingClass)}><Play size={24} className="text-gray-300" /></div>;
@@ -73,7 +14,7 @@ export const MediaRenderer = ({ file, className, isFullView, paused, heightCap,
if (file.type === 'video') {
return (
<div className="relative w-full h-full flex items-center justify-center group/video text-black">
<video ref={videoRef} src={file.url} className={cn("transition-all duration-300", sizingClass, className)} style={mediaStyle} playsInline autoPlay muted={isMuted} loop controls />
<video src={file.url} className={cn("transition-all duration-300", sizingClass, className)} style={mediaStyle} playsInline autoPlay muted={isMuted} loop controls />
<button onClick={(e) => { e.stopPropagation(); setIsMuted(!isMuted); }} className="absolute bottom-16 right-4 z-30 bg-black/40 hover:bg-black/60 text-white p-2 rounded-full backdrop-blur-md transition-all md:opacity-0 md:group-hover/video:opacity-100">
{isMuted ? <VolumeX size={20} /> : <Volume2 size={20} />}
</button>
-150
View File
@@ -1,150 +0,0 @@
import React, { useCallback, useEffect, useLayoutEffect, useMemo, useRef, useState } from 'react';
import { ChevronLeft } from 'lucide-react';
import { Post } from '../types';
import { FeedPost } from './FeedPost';
interface PostFeedProps {
posts: Post[];
/** Post the feed should open at. */
initialPostId: string;
profilePic: string | null;
onClose: () => void;
/** Fires as the post crossing the viewport centre changes. */
onActivePostChange: (post: Post) => void;
title?: string;
}
/** Posts added each time the feed grows in either direction. */
const BATCH = 6;
/** Render this many ahead of the entry point so the first scroll is smooth. */
const LOOKAHEAD = 3;
/**
* The mobile post view: a real scrolling feed, not a modal.
*
* Only a window of posts around the entry point is mounted — a profile can hold
* thousands, and mounting them all would mean thousands of full-size images.
* The window grows in both directions as you scroll; growing *upwards* shifts
* everything below it, so the scroll position is corrected in the same frame to
* keep the content under your thumb still.
*/
export const PostFeed: React.FC<PostFeedProps> = ({
posts, initialPostId, profilePic, onClose, onActivePostChange, title,
}) => {
const initialIndex = useMemo(() => {
const found = posts.findIndex(p => p.id === initialPostId);
return found === -1 ? 0 : found;
}, [posts, initialPostId]);
const [range, setRange] = useState(() => ({
start: initialIndex,
end: Math.min(posts.length, initialIndex + LOOKAHEAD + 1),
}));
const [activeId, setActiveId] = useState(initialPostId);
const scrollRef = useRef<HTMLDivElement>(null);
const topSentinelRef = useRef<HTMLDivElement>(null);
const bottomSentinelRef = useRef<HTMLDivElement>(null);
/** Distance from the bottom of the content, captured before a prepend. */
const anchorRef = useRef<number | null>(null);
const visible = posts.slice(range.start, range.end);
const extendDown = useCallback(() => {
setRange(r => (r.end >= posts.length ? r : { ...r, end: Math.min(posts.length, r.end + BATCH) }));
}, [posts.length]);
const extendUp = useCallback(() => {
const el = scrollRef.current;
if (!el) return;
setRange(r => {
if (r.start === 0) return r;
// Measure from the bottom: prepending changes scrollHeight, but the
// distance between our position and the end of the content does not.
anchorRef.current = el.scrollHeight - el.scrollTop;
return { ...r, start: Math.max(0, r.start - BATCH) };
});
}, []);
// Restore the scroll position in the same frame the prepended posts appear,
// before the browser paints, so nothing visibly jumps.
useLayoutEffect(() => {
const el = scrollRef.current;
if (el && anchorRef.current !== null) {
el.scrollTop = el.scrollHeight - anchorRef.current;
anchorRef.current = null;
}
}, [range.start]);
// Grow the window when either end comes into view.
useEffect(() => {
const root = scrollRef.current;
if (!root) return;
const observer = new IntersectionObserver(entries => {
for (const entry of entries) {
if (!entry.isIntersecting) continue;
if (entry.target === bottomSentinelRef.current) extendDown();
if (entry.target === topSentinelRef.current) extendUp();
}
}, { root, rootMargin: '600px 0px' });
if (topSentinelRef.current) observer.observe(topSentinelRef.current);
if (bottomSentinelRef.current) observer.observe(bottomSentinelRef.current);
return () => observer.disconnect();
}, [extendDown, extendUp]);
// Track the post crossing the viewport centre. The negative margins collapse
// the root to a thin band, so exactly one post qualifies at a time.
useEffect(() => {
const root = scrollRef.current;
if (!root) return;
const observer = new IntersectionObserver(entries => {
for (const entry of entries) {
if (!entry.isIntersecting) continue;
const id = (entry.target as HTMLElement).dataset.postId;
if (id) setActiveId(id);
}
}, { root, rootMargin: '-45% 0px -45% 0px', threshold: 0 });
root.querySelectorAll('[data-post-id]').forEach(el => observer.observe(el));
return () => observer.disconnect();
}, [visible.length, range.start]);
useEffect(() => {
const post = posts.find(p => p.id === activeId);
if (post) onActivePostChange(post);
}, [activeId, posts, onActivePostChange]);
// Escape closes, matching the modal it replaces.
useEffect(() => {
const onKey = (e: KeyboardEvent) => { if (e.key === 'Escape') onClose(); };
window.addEventListener('keydown', onKey);
return () => window.removeEventListener('keydown', onKey);
}, [onClose]);
return (
<div className="fixed inset-0 z-50 bg-white flex flex-col">
<header className="flex items-center gap-3 px-2 h-12 border-b border-gray-200 bg-white/95 backdrop-blur-md shrink-0">
<button onClick={onClose} aria-label="Back" className="p-2 -ml-1 active:opacity-60">
<ChevronLeft size={24} />
</button>
<span className="font-semibold text-base">{title ?? 'Posts'}</span>
</header>
<div ref={scrollRef} className="flex-1 overflow-y-auto overscroll-contain">
<div ref={topSentinelRef} aria-hidden />
{visible.map(post => (
<div key={post.id} data-post-id={post.id}>
<FeedPost post={post} profilePic={profilePic} paused={post.id !== activeId} />
</div>
))}
<div ref={bottomSentinelRef} aria-hidden />
{range.end >= posts.length && (
<div className="py-10 text-center text-xs uppercase tracking-widest text-gray-400">
End of {title?.toLowerCase() ?? 'posts'}
</div>
)}
</div>
</div>
);
};
+27 -82
View File
@@ -12,7 +12,6 @@ import {
import { motion, AnimatePresence } from 'motion/react';
import { Post } from '../types';
import { cn, formatDateSafe } from '../lib/utils';
import { FADE, NAVIGATE, PRESENT, prefersReducedMotion, withVelocity } from '../lib/motion';
import { MediaRenderer } from './MediaRenderer';
interface PostModalProps {
@@ -31,14 +30,7 @@ export const PostModal: React.FC<PostModalProps> = ({
post, nextPost, prevPost, onClose, onNextPost, onPrevPost, hasNextPost, hasPrevPost, profilePic
}) => {
const [currentIndex, setCurrentIndex] = useState(0);
/**
* How the next slide/post should enter: along which axis, in which direction,
* and carrying how much velocity from the gesture that triggered it.
*/
const [slideMotion, setSlideMotion] = useState<{ axis: 'x' | 'y'; dir: number; velocity: number }>(
{ axis: 'x', dir: 0, velocity: 0 },
);
const reduceMotion = prefersReducedMotion();
const [direction, setDirection] = useState(0);
// Preloading Logic
useEffect(() => {
@@ -82,102 +74,55 @@ export const PostModal: React.FC<PostModalProps> = ({
useEffect(() => setCurrentIndex(0), [post.id]);
useEffect(() => {
// Arrows page within the carousel — the thing the arrows visually point at.
// Moving between posts stays on the side buttons, with , and . as keyboard
// equivalents.
const handleKeyDown = (e: KeyboardEvent) => {
if (e.key === 'ArrowRight') paginate(1);
else if (e.key === 'ArrowLeft') paginate(-1);
else if (e.key === '.') goToPost(1, 'x');
else if (e.key === ',') goToPost(-1, 'x');
if (e.key === 'ArrowRight') onNextPost?.();
else if (e.key === 'ArrowLeft') onPrevPost?.();
else if (e.key === '.') paginate(1);
else if (e.key === ',') paginate(-1);
else if (e.key === 'Escape') onClose();
};
window.addEventListener('keydown', handleKeyDown);
return () => window.removeEventListener('keydown', handleKeyDown);
}, [onNextPost, onPrevPost, currentIndex, post.media.length, onClose]);
const paginate = (newDirection: number, velocity = 0) => {
const paginate = (newDirection: number) => {
const nextIndex = currentIndex + newDirection;
if (nextIndex >= 0 && nextIndex < post.media.length) {
setSlideMotion({ axis: 'x', dir: newDirection, velocity });
setCurrentIndex(nextIndex);
}
if (nextIndex >= 0 && nextIndex < post.media.length) { setDirection(newDirection); setCurrentIndex(nextIndex); }
};
/**
* Move between posts, animating along the axis the input implies: vertical
* for a touch swipe, horizontal for the desktop arrows and arrow keys.
*/
const goToPost = (dir: 1 | -1, axis: 'x' | 'y', velocity = 0) => {
if (dir > 0 ? !hasNextPost : !hasPrevPost) return;
setSlideMotion({ axis, dir, velocity });
if (dir > 0) onNextPost?.(); else onPrevPost?.();
};
type SlideMotion = { axis: 'x' | 'y'; dir: number };
const offscreen = (dir: number) => (dir > 0 ? '100%' : '-100%');
const variants = {
enter: ({ axis, dir }: SlideMotion) =>
axis === 'y'
? { y: offscreen(dir), x: 0, opacity: 1, zIndex: 0 }
: { x: offscreen(dir), y: 0, opacity: 1, zIndex: 0 },
center: { zIndex: 1, x: 0, y: 0, opacity: 1 },
exit: ({ axis, dir }: SlideMotion) =>
axis === 'y'
? { zIndex: 0, y: offscreen(-dir), x: 0, opacity: 1 }
: { zIndex: 0, x: offscreen(-dir), y: 0, opacity: 1 },
const variants = {
enter: (d: number) => ({ x: d > 0 ? '100%' : '-100%', opacity: 1, zIndex: 0 }),
center: { zIndex: 1, x: 0, opacity: 1 },
exit: (d: number) => ({ zIndex: 0, x: d < 0 ? '100%' : '-100%', opacity: 1 })
};
const swipePower = (offset: number, velocity: number) => Math.abs(offset) * velocity;
/**
* Desktop-only surface: mobile opens PostFeed instead, so the only vertical
* gesture left here is drag-to-dismiss.
*/
const handleVerticalDragEnd = (offset: { y: number }, velocity: { y: number }) => {
if (offset.y > 200 || velocity.y > 800) onClose();
};
/*
* Horizontal padding on the overlay reserves a gutter for the prev/next
* arrows so they always sit *outside* the modal. Without it the modal grows
* until it sits under them and a white chevron lands on the white caption
* panel, leaving the control invisible until hovered.
*
* overscroll-contain stops wheel events chaining through to the very long
* post grid behind the overlay.
*/
return (
<motion.div initial={{ opacity: 0 }} animate={{ opacity: 1 }} exit={{ opacity: 0 }} transition={FADE} className="fixed inset-0 z-50 flex items-start justify-center bg-[#0c1014]/95 md:bg-[#0c1014]/70 p-0 md:py-10 md:px-16 lg:px-24 overflow-y-auto overscroll-contain text-black" onClick={onClose}>
<motion.div initial={{ opacity: 0 }} animate={{ opacity: 1 }} exit={{ opacity: 0 }} className="fixed inset-0 z-50 flex items-start justify-center bg-[#0c1014]/95 md:bg-[#0c1014]/70 p-0 md:p-10 overflow-y-auto text-black" onClick={onClose}>
<div className="min-h-full w-full flex items-center justify-center md:py-0">
<button onClick={onClose} className="fixed top-4 right-4 text-white hover:text-gray-300 z-50 p-2 md:p-3 bg-black/20 rounded-full backdrop-blur-sm"><X size={24} className="md:w-8 md:h-8" /></button>
{/* Solid pill so the arrows read against whatever sits behind them. */}
{hasPrevPost && onPrevPost && <button aria-label="Previous post" onClick={(e) => { e.stopPropagation(); goToPost(-1, 'x'); }} className="hidden md:flex items-center justify-center fixed md:left-3 lg:left-6 top-1/2 -translate-y-1/2 z-50 p-2 rounded-full bg-white/90 hover:bg-white text-gray-800 shadow-lg transition-transform hover:scale-110 active:scale-90"><ChevronLeft size={28} strokeWidth={2} /></button>}
{hasNextPost && onNextPost && <button aria-label="Next post" onClick={(e) => { e.stopPropagation(); goToPost(1, 'x'); }} className="hidden md:flex items-center justify-center fixed md:right-3 lg:right-6 top-1/2 -translate-y-1/2 z-50 p-2 rounded-full bg-white/90 hover:bg-white text-gray-800 shadow-lg transition-transform hover:scale-110 active:scale-90"><ChevronRight size={28} strokeWidth={2} /></button>}
<motion.div drag="y" dragDirectionLock dragConstraints={{ top: 0, bottom: 0 }} dragElastic={0.15} onDragEnd={(e, { offset, velocity }) => handleVerticalDragEnd(offset, velocity)} initial={reduceMotion ? false : { opacity: 0, scale: 0.96 }} animate={{ opacity: 1, scale: 1 }} exit={reduceMotion ? { opacity: 0 } : { opacity: 0, scale: 0.96 }} transition={PRESENT} className="bg-black flex flex-col md:flex-row w-full max-w-6xl h-auto md:rounded-sm overflow-hidden shadow-2xl relative text-black" onClick={e => e.stopPropagation()}>
{hasPrevPost && onPrevPost && <button onClick={(e) => { e.stopPropagation(); onPrevPost(); }} className="hidden md:block fixed left-4 md:left-10 top-1/2 -translate-y-1/2 text-white hover:text-gray-300 z-50 transition-transform hover:scale-110 active:scale-90"><ChevronLeft size={48} strokeWidth={1.5} /></button>}
{hasNextPost && onNextPost && <button onClick={(e) => { e.stopPropagation(); onNextPost(); }} className="hidden md:block fixed right-4 md:right-10 top-1/2 -translate-y-1/2 text-white hover:text-gray-300 z-50 transition-transform hover:scale-110 active:scale-90"><ChevronRight size={48} strokeWidth={1.5} /></button>}
<motion.div drag="y" dragDirectionLock dragConstraints={{ top: 0, bottom: 0 }} dragElastic={0.15} onDragEnd={(e, { offset, velocity }) => { if (offset.y > 200 || velocity.y > 800) onClose(); }} className="bg-black flex flex-col md:flex-row w-full max-w-6xl h-auto md:rounded-sm overflow-hidden shadow-2xl relative text-black" onClick={e => e.stopPropagation()}>
<div className="relative bg-black flex items-center justify-center group overflow-hidden w-full h-auto text-black">
<div className="w-full grid grid-cols-1 grid-rows-1 text-black">
<AnimatePresence initial={false} custom={slideMotion}>
<motion.div
key={`${post.id}-${currentIndex}`}
custom={slideMotion}
variants={variants}
initial="enter"
animate="center"
exit="exit"
transition={reduceMotion
? { duration: 0 }
: { x: withVelocity(slideMotion.velocity, NAVIGATE), y: withVelocity(slideMotion.velocity, NAVIGATE) }}
<AnimatePresence initial={false} custom={direction}>
<motion.div
key={`${post.id}-${currentIndex}`}
custom={direction}
variants={variants}
initial="enter"
animate="center"
exit="exit"
transition={{ x: { type: "spring", stiffness: 200, damping: 26, bounce: 0 } }}
drag="x"
dragDirectionLock
dragConstraints={{ left: 0, right: 0 }}
dragElastic={0.5}
onDragEnd={(e, { offset, velocity }) => {
// Carousel only. Crossing into the next post from the last
// slide made a horizontal flick mean two different things.
const s = swipePower(offset.x, velocity.x);
if (s < -15000) paginate(1, velocity.x);
else if (s > 15000) paginate(-1, velocity.x);
if (s < -15000) { if (currentIndex < post.media.length - 1) paginate(1); else if (hasNextPost && onNextPost && s < -40000) onNextPost(); }
else if (s > 15000) { if (currentIndex > 0) paginate(-1); else if (hasPrevPost && onPrevPost && s > 40000) onPrevPost(); }
}}
className="col-start-1 row-start-1 w-full flex items-center justify-center cursor-grab active:cursor-grabbing relative text-black"
>
@@ -193,7 +138,7 @@ export const PostModal: React.FC<PostModalProps> = ({
</>
)}
</div>
<div className="w-full md:w-80 lg:w-96 bg-white flex flex-col border-l border-gray-200 overflow-hidden shrink-0 text-black">
<div className="w-full md:w-96 bg-white flex flex-col border-l border-gray-200 overflow-hidden shrink-0 text-black">
<div className="p-3 md:p-4 border-b border-gray-100 flex items-center justify-between shrink-0 text-black">
<div className="flex items-center gap-3 text-black">
<div className="w-8 h-8 rounded-full bg-gradient-to-tr from-yellow-400 to-purple-600 p-0.5 text-black"><div className="w-full h-full rounded-full bg-white p-0.5 text-black"><div className="w-full h-full rounded-full bg-gray-200 flex items-center justify-center overflow-hidden text-[10px] font-bold uppercase text-black">{profilePic ? <img src={profilePic} alt="" className="w-full h-full object-cover text-black" referrerPolicy="no-referrer" /> : <span className="text-black">{post.username[0]}</span>}</div></div></div>
+6 -30
View File
@@ -9,7 +9,6 @@ import {
import { motion } from 'motion/react';
import { Post } from '../types';
import { cn, formatDateSafe } from '../lib/utils';
import { FADE, PRESENT, prefersReducedMotion } from '../lib/motion';
interface StoryViewerProps {
stories: Post[];
@@ -27,12 +26,10 @@ export const StoryViewer: React.FC<StoryViewerProps> = ({
}) => {
const [currentStoryIndex, setCurrentStoryIndex] = useState(0);
const [progress, setProgress] = useState(0);
// Opening the reel is a user gesture, so try for sound; the effect below
// falls back to muted if the browser refuses, which would otherwise stall
// the progress bar on the first video.
const [isMuted, setIsMuted] = useState(false);
// Start muted: Safari and Firefox refuse to autoplay audible media, which
// would stall the reel on its first video.
const [isMuted, setIsMuted] = useState(true);
const videoRef = useRef<HTMLVideoElement>(null);
const reduceMotion = prefersReducedMotion();
const story = stories[currentStoryIndex];
const primary = story?.media?.[0];
@@ -64,22 +61,6 @@ export const StoryViewer: React.FC<StoryViewerProps> = ({
return () => clearInterval(timer);
}, [currentStoryIndex, primary]);
useEffect(() => {
const video = videoRef.current;
if (!video || primary?.type !== 'video') return;
let cancelled = false;
video.muted = false;
video.play().catch(() => {
if (cancelled) return;
setIsMuted(true);
video.muted = true;
video.play().catch(() => { /* leave it to the controls */ });
});
return () => { cancelled = true; };
}, [primary]);
useEffect(() => {
if (progress >= 100) {
if (currentStoryIndex < stories.length - 1) {
@@ -108,11 +89,10 @@ export const StoryViewer: React.FC<StoryViewerProps> = ({
if (!story || !primary) return null;
return (
<motion.div
<motion.div
initial={{ opacity: 0 }}
animate={{ opacity: 1 }}
exit={{ opacity: 0 }}
transition={FADE}
className="fixed inset-0 z-[100] bg-[#1a1a1a] flex items-center justify-center overflow-hidden text-white"
onClick={onClose}
>
@@ -141,11 +121,7 @@ export const StoryViewer: React.FC<StoryViewerProps> = ({
<ChevronRight size={32} strokeWidth={1.5} />
</button>
<motion.div
initial={reduceMotion ? false : { scale: 0.94, opacity: 0 }}
animate={{ scale: 1, opacity: 1 }}
exit={reduceMotion ? { opacity: 0 } : { scale: 0.94, opacity: 0 }}
transition={PRESENT}
<div
className="relative w-full h-full md:h-[90vh] md:max-w-[45vh] bg-black overflow-hidden md:rounded-lg shadow-2xl z-10 text-white"
onClick={e => e.stopPropagation()}
>
@@ -230,7 +206,7 @@ export const StoryViewer: React.FC<StoryViewerProps> = ({
{story.caption}
</div>
)}
</motion.div>
</div>
</motion.div>
);
};
+4 -42
View File
@@ -4,8 +4,6 @@ import { XzReadableStream } from 'xz-decompress';
import { ArchiveFile, CacheData, Post, ServerArchive } from '../types';
import { setCachedArchive, getDirectoryHandle } from '../lib/archive-cache';
import { parseArchiveFilename, scopedPostId, EXPORT_RE, INSTALOADER_RE } from '../lib/archive-patterns';
import { isGalleryDlSidecar, sidecarDate, sidecarIsReel } from '../lib/gallery-dl-sidecar';
import { DateSource, shouldReplaceDate } from '../lib/post-dates';
const hasDirectoryHandle = async (name: string) => Boolean(await getDirectoryHandle(name));
@@ -109,23 +107,11 @@ export const useArchiveScanner = (
/** Stable identity for a media file, used to rehydrate URLs after a reload. */
const mediaPath = (file: ArchiveFile) => file.webkitRelativePath || file.name;
/**
* Decompress an `.xz` metadata sidecar.
*
* Read fully into memory first rather than handing the live HTTP body to
* the decompressor. These sidecars are a few KB, so buffering costs
* nothing, and streaming was actively harmful: the decompressor stops
* reading at the end of the xz member, leaving the response body neither
* drained nor cancelled. Across a couple of hundred sidecars that exhausts
* the connection pool and every later fetch fails with "Failed to fetch" —
* which silently cost Instaloader archives their captions, story flags and
* profile metadata, since all of it lives in these files.
*/
const parseXZFile = async (file: ArchiveFile) => {
try {
const compressed = await file.arrayBuffer();
const stream = new XzReadableStream(new Blob([compressed]).stream());
return await new Response(stream).json();
const stream = new XzReadableStream(file.stream());
const response = new Response(stream);
return await response.json();
} catch (e) { console.error(`[Scanner] XZ Parse Error:`, file.name, e); return null; }
};
@@ -144,17 +130,6 @@ export const useArchiveScanner = (
try {
const postsMap = new Map<string, Partial<Post>>();
/**
* Which source supplied each post's date, so a better one can replace it.
* Sidecar beats filename beats mtime — see src/lib/post-dates.ts.
*/
const dateSources = new Map<string, DateSource>();
const applyDate = (postId: string, post: Partial<Post>, date: string, source: DateSource) => {
const current = post.date ? { date: post.date, source: dateSources.get(postId) ?? 'mtime' } : undefined;
if (!shouldReplaceDate(current, { date, source })) return;
post.date = date;
dateSources.set(postId, source);
};
const mediaFilesMap = new Map<string, ArchiveFile>();
const discoveredProfilePics: { name: string, url: string }[] = [];
const allImageFiles: ArchiveFile[] = [];
@@ -313,26 +288,13 @@ export const useArchiveScanner = (
}
else if (isStory) post.isStory = true;
// Files describing one post are scanned in directory order, not in
// order of trustworthiness, so every date goes through the ranking
// in post-dates.ts rather than last-write-wins.
applyDate(postId, post, date, parsed.dateFromMtime ? 'mtime' : 'filename');
const lowerExt = ext.toLowerCase();
if (lowerExt === 'txt') {
try { post.caption = await file.text(); } catch(e) {}
} else if (lowerExt === 'json' || lowerName.endsWith('.json.xz')) {
try {
const data = lowerName.endsWith('.xz') ? await parseXZFile(file) : JSON.parse(await file.text());
if (isGalleryDlSidecar(data)) {
// The only format that states what a post is rather than
// leaving it to be inferred from filenames.
if (data.description) post.caption = data.description;
const reel = sidecarIsReel(data);
if (reel !== undefined) post.isReel = reel;
if (data.type === 'story') post.isStory = true;
applyDate(postId, post, sidecarDate(data), 'sidecar');
} else if (data) {
if (data) {
const node = data.node || data; const iphone = node.iphone_struct || {};
const captionText = node.edge_media_to_caption?.edges?.[0]?.node?.text || node.caption?.text || iphone.caption?.text || '';
if (captionText) post.caption = captionText;
-27
View File
@@ -1,27 +0,0 @@
import { useEffect, useState } from 'react';
/** Matches Tailwind's `md` breakpoint, the point where the layout splits. */
const MOBILE_QUERY = '(max-width: 767px)';
/**
* True on phone-sized viewports.
*
* Drives more than styling: mobile opens posts as a scrollable feed page while
* desktop uses the modal, so this needs to be real state rather than a CSS
* media query.
*/
export const useIsMobile = () => {
const [isMobile, setIsMobile] = useState(
() => typeof window !== 'undefined' && window.matchMedia(MOBILE_QUERY).matches,
);
useEffect(() => {
const query = window.matchMedia(MOBILE_QUERY);
const onChange = (e: MediaQueryListEvent) => setIsMobile(e.matches);
query.addEventListener('change', onChange);
setIsMobile(query.matches);
return () => query.removeEventListener('change', onChange);
}, []);
return isMobile;
};
+10
View File
@@ -14,6 +14,7 @@ export class LocalArchiveFile implements ArchiveFile {
get size() { return this.file.size; }
text() { return this.file.text(); }
arrayBuffer() { return this.file.arrayBuffer(); }
stream() { return this.file.stream(); }
/**
* A blob: URL backed directly by the on-disk File.
@@ -54,6 +55,15 @@ export class RemoteArchiveFile implements ArchiveFile {
const res = await fetch(this.url);
return res.arrayBuffer();
}
stream() {
const transform = new TransformStream();
fetch(this.url).then(res => {
if (res.body) res.body.pipeTo(transform.writable);
else transform.writable.getWriter().close();
});
return transform.readable;
}
createObjectUrl() {
return this.url;
}
+28 -28
View File
@@ -3,39 +3,39 @@ import { classifyDirectory, groupArchiveDirectories } from './archive-grouping';
describe('classifyDirectory', () => {
it('treats a bare profile directory as the base', () => {
expect(classifyDirectory('4utumn07')).toEqual({
owner: '4utumn07',
source: { kind: 'posts', dir: '4utumn07' },
expect(classifyDirectory('0ct0ber19')).toEqual({
owner: '0ct0ber19',
source: { kind: 'posts', dir: '0ct0ber19' },
});
});
it('recognises a reels sidecar', () => {
expect(classifyDirectory('4utumn07 - reels')).toEqual({
owner: '4utumn07',
source: { kind: 'reels', dir: '4utumn07 - reels' },
expect(classifyDirectory('0ct0ber19 - reels')).toEqual({
owner: '0ct0ber19',
source: { kind: 'reels', dir: '0ct0ber19 - reels' },
});
});
it('recognises a stories sidecar', () => {
expect(classifyDirectory('story - dawn_petal')).toEqual({
owner: 'dawn_petal',
source: { kind: 'stories', dir: 'story - dawn_petal' },
expect(classifyDirectory('story - cher_ryppo')).toEqual({
owner: 'cher_ryppo',
source: { kind: 'stories', dir: 'story - cher_ryppo' },
});
});
it('splits highlight owner from title', () => {
const { owner, source } = classifyDirectory('story highlights - 4utumn07 - Sunstory');
expect(owner).toBe('4utumn07');
const { owner, source } = classifyDirectory('story highlights - 0ct0ber19 - Heestory');
expect(owner).toBe('0ct0ber19');
expect(source.kind).toBe('highlight');
expect(source.title).toBe('Sunstory');
expect(source.title).toBe('Heestory');
});
it.each([
['story highlights - theoldlyricmuseinsta - 💙1999-2005 era', 'theoldlyricmuseinsta', '💙1999-2005 era'],
['story highlights - member_theworld - [Bracket]', 'member_theworld', '[Bracket]'],
['story highlights - official_band - Tour Schedule', 'official_band', 'Tour Schedule'],
['story highlights - 4utumn07 - Sketching⠀', '4utumn07', 'Sketching⠀'],
['story highlights - official_band - A.B.C', 'official_band', 'A.B.C'],
['story highlights - theoldtaylorswiftinsta - 💙2014-1989 era', 'theoldtaylorswiftinsta', '💙2014-1989 era'],
['story highlights - heejin_theworld - [Dall]', 'heejin_theworld', '[Dall]'],
['story highlights - official_artms - Cosmo Schedule', 'official_artms', 'Cosmo Schedule'],
['story highlights - 0ct0ber19 - Drawheeing⠀', '0ct0ber19', 'Drawheeing⠀'],
['story highlights - official_artms - G.C.I', 'official_artms', 'G.C.I'],
])('handles real-world title %s', (dir, owner, title) => {
const result = classifyDirectory(dir);
expect(result.owner).toBe(owner);
@@ -57,25 +57,25 @@ describe('classifyDirectory', () => {
describe('groupArchiveDirectories', () => {
const dirs = [
'4utumn07',
'4utumn07 - reels',
'story - 4utumn07',
'story highlights - 4utumn07 - Sunstory',
'story highlights - 4utumn07 - Sketching⠀',
'kestrelsings',
'0ct0ber19',
'0ct0ber19 - reels',
'story - 0ct0ber19',
'story highlights - 0ct0ber19 - Heestory',
'story highlights - 0ct0ber19 - Drawheeing⠀',
'carlyraejepsen',
];
it('folds sidecars into their base profile', () => {
const groups = groupArchiveDirectories(dirs);
expect([...groups.keys()].sort()).toEqual(['4utumn07', 'kestrelsings']);
expect(groups.get('4utumn07')).toHaveLength(5);
expect(groups.get('kestrelsings')).toHaveLength(1);
expect([...groups.keys()].sort()).toEqual(['0ct0ber19', 'carlyraejepsen']);
expect(groups.get('0ct0ber19')).toHaveLength(5);
expect(groups.get('carlyraejepsen')).toHaveLength(1);
});
it('orders sources posts, reels, stories, then highlights by title', () => {
const sources = groupArchiveDirectories(dirs).get('4utumn07')!;
const sources = groupArchiveDirectories(dirs).get('0ct0ber19')!;
expect(sources.map(s => s.kind)).toEqual(['posts', 'reels', 'stories', 'highlight', 'highlight']);
expect(sources.slice(3).map(s => s.title)).toEqual(['Sketching⠀', 'Sunstory']);
expect(sources.slice(3).map(s => s.title)).toEqual(['Drawheeing⠀', 'Heestory']);
});
it('still groups a sidecar whose base profile is missing', () => {
+4 -4
View File
@@ -18,10 +18,10 @@ export interface ArchiveSource {
/**
* Sidecar directories sit next to the profile directory they belong to:
*
* 4utumn07 -> posts (base)
* 4utumn07 - reels -> reels
* story - 4utumn07 -> stories
* story highlights - 4utumn07 - Sunstory -> highlight "Sunstory"
* 0ct0ber19 -> posts (base)
* 0ct0ber19 - reels -> reels
* story - 0ct0ber19 -> stories
* story highlights - 0ct0ber19 - Heestory -> highlight "Heestory"
*
* Instagram usernames cannot contain spaces, so matching the username as a
* run of non-space characters reliably separates it from a highlight title
-34
View File
@@ -1,34 +0,0 @@
import { describe, expect, it } from 'vitest';
import { isSystemDirectory } from './archive-index';
describe('isSystemDirectory', () => {
it.each([
['@eaDir', 'Synology thumbnail/index metadata, written inside every folder'],
['@tmp', 'Synology scratch'],
['.sync', 'Resilio state'],
['.DS_Store', 'macOS'],
['#recycle', 'Synology deletions'],
['#snapshot', 'Synology snapshots'],
])('skips %s (%s)', name => {
expect(isSystemDirectory(name)).toBe(true);
});
it.each([
'4utumn07',
'4utumn07 - reels',
'story - dawn_petal',
'story highlights - official_band - A.B.C',
'story highlights - theoldlyricmuseinsta - 💙1999-2005 era',
'Heejin_Bubble heejinmedia',
'gallery-dl',
'posts',
])('keeps %s', name => {
expect(isSystemDirectory(name)).toBe(false);
});
it('does not treat a leading underscore as a system directory', () => {
// `_gemini-plans` is filtered separately at the archive root only; nothing
// below the root should be excluded just for starting with an underscore.
expect(isSystemDirectory('_gemini-plans')).toBe(false);
});
});
+1 -17
View File
@@ -33,21 +33,6 @@ interface DirIndex {
const MEDIA_RE = /\.(jpg|jpeg|png|webp|gif|bmp|tiff|mp4|webm|ogv|mov)$/i;
const STAT_CONCURRENCY = 16;
/**
* Directories the walk must never descend into.
*
* NAS filesystems scatter sidecar metadata *inside* every folder, not just at
* the share root: Synology writes `@eaDir` (thumbnails and indexing data),
* `#recycle` holds deletions, and `.sync` is Resilio's state. Indexing those
* would count NAS thumbnails as archive media and spend a stat on each one —
* measured on a real share, `@eaDir` accounted for 12,516 of 123,023 files.
*
* The archive root is already filtered by prefix; this is the same rule applied
* at every level below it.
*/
export const isSystemDirectory = (name: string): boolean =>
name.startsWith('@') || name.startsWith('.') || name === '#recycle' || name === '#snapshot';
export class ArchiveIndex {
private dirs = new Map<string, DirIndex>();
private inFlight = new Map<string, Promise<DirIndex>>();
@@ -58,7 +43,7 @@ export class ArchiveIndex {
/** Visible (non-system) directories at the archive root. */
private listRootDirs(): string[] {
return fs.readdirSync(this.archivesDir, { withFileTypes: true })
.filter(e => e.isDirectory() && !isSystemDirectory(e.name) && !e.name.startsWith('_'))
.filter(e => e.isDirectory() && !/^[.@_]/.test(e.name))
.map(e => e.name);
}
@@ -84,7 +69,6 @@ export class ArchiveIndex {
return out;
}
for (const entry of entries) {
if (isSystemDirectory(entry.name)) continue;
const rel = base ? `${base}/${entry.name}` : entry.name;
if (entry.isDirectory()) out = out.concat(this.walk(path.join(absDir, entry.name), rel));
else if (entry.isFile()) out.push(rel);
+12 -121
View File
@@ -1,21 +1,20 @@
import { describe, expect, it } from 'vitest';
import { canonicalItemId, parseArchiveFilename, scopedPostId } from './archive-patterns';
import { parseArchiveFilename, scopedPostId } from './archive-patterns';
describe('parseArchiveFilename — Instagram export format', () => {
it('parses a single-image post', () => {
expect(parseArchiveFilename('2023-04-19_4utumn07 - CrORBIcJJbM.mp4')).toEqual({
expect(parseArchiveFilename('2023-04-19_0ct0ber19 - CrORBIcJJbM.mp4')).toEqual({
postId: 'CrORBIcJJbM',
date: '2023-04-19',
username: '4utumn07',
username: '0ct0ber19',
index: 1,
ext: 'mp4',
isStory: false,
dateFromMtime: false,
});
});
it('parses a carousel slide index', () => {
const parsed = parseArchiveFilename('2023-04-12_4utumn07 - Cq8LrxSJAJE - 3.jpg');
const parsed = parseArchiveFilename('2023-04-12_0ct0ber19 - Cq8LrxSJAJE - 3.jpg');
expect(parsed).toMatchObject({ postId: 'Cq8LrxSJAJE', index: 3, ext: 'jpg' });
});
@@ -27,7 +26,7 @@ describe('parseArchiveFilename — Instagram export format', () => {
});
it('parses caption sidecar files', () => {
expect(parseArchiveFilename('2023-04-12_4utumn07 - Cq8LrxSJAJE.txt')).toMatchObject({
expect(parseArchiveFilename('2023-04-12_0ct0ber19 - Cq8LrxSJAJE.txt')).toMatchObject({
postId: 'Cq8LrxSJAJE',
ext: 'txt',
});
@@ -39,8 +38,8 @@ describe('parseArchiveFilename — Instagram export format', () => {
it('parses the story sidecar layout (date_user - N - shortcode)', () => {
// Files in `story - <user>` carry a per-day ordinal before the shortcode.
const parsed = parseArchiveFilename('2025-10-26_4utumn07 - 2 - DQRuDx9iW5Q.jpg', 'stories');
expect(parsed).toMatchObject({ date: '2025-10-26', username: '4utumn07', ext: 'jpg' });
const parsed = parseArchiveFilename('2025-10-26_0ct0ber19 - 2 - DQRuDx9iW5Q.jpg', 'stories');
expect(parsed).toMatchObject({ date: '2025-10-26', username: '0ct0ber19', ext: 'jpg' });
expect(parsed!.postId).toContain('DQRuDx9iW5Q');
});
@@ -71,9 +70,9 @@ describe('parseArchiveFilename — Instaloader format', () => {
describe('parseArchiveFilename — story highlights', () => {
it('parses the dateless highlight layout', () => {
expect(parseArchiveFilename('4utumn07 - C5dQPEYpd9W.mp4', 'highlight')).toMatchObject({
expect(parseArchiveFilename('0ct0ber19 - C5dQPEYpd9W.mp4', 'highlight')).toMatchObject({
postId: 'C5dQPEYpd9W',
username: '4utumn07',
username: '0ct0ber19',
ext: 'mp4',
isStory: false,
});
@@ -95,7 +94,7 @@ describe('parseArchiveFilename — story highlights', () => {
});
describe('parseArchiveFilename — non-matching files', () => {
it.each(['4utumn07.jpg', 'profile_pic.jpg', 'README.md', 'no-separator.png'])(
it.each(['0ct0ber19.jpg', 'profile_pic.jpg', 'README.md', 'no-separator.png'])(
'returns null for %s',
name => expect(parseArchiveFilename(name)).toBeNull(),
);
@@ -107,8 +106,8 @@ describe('scopedPostId', () => {
});
it('namespaces sidecar ids by directory', () => {
expect(scopedPostId('C5dQ', 'highlight', 'story highlights - u - Sunstory'))
.toBe('story highlights - u - Sunstory/C5dQ');
expect(scopedPostId('C5dQ', 'highlight', 'story highlights - u - Heestory'))
.toBe('story highlights - u - Heestory/C5dQ');
});
it('keeps the same shortcode distinct across sources', () => {
@@ -117,111 +116,3 @@ describe('scopedPostId', () => {
expect(inPosts).not.toBe(inHighlight);
});
});
/**
* gallery-dl is replacing JDownloader as the fetcher (docs/gallery-dl.md).
* Its naming differs cosmetically, and these cases pin down that the two
* interoperate so a mixed archive parses identically.
*/
describe('gallery-dl / JDownloader naming interop', () => {
it('treats a single-media post the same with or without an index', () => {
const jd2 = parseArchiveFilename('2023-04-19_4utumn07 - CrORBIcJJbM.mp4')!;
const gdl = parseArchiveFilename('2023-04-19_4utumn07 - CrORBIcJJbM - 1.mp4')!;
expect(jd2.postId).toBe(gdl.postId);
expect(jd2.index).toBe(gdl.index);
expect(jd2.index).toBe(1);
});
it('normalises zero-padded carousel indices', () => {
// JD2 pads to the width of the media count (10+ items -> "01"), and
// gallery-dl's count can be one higher, so the same post may be padded
// by one tool and not the other.
expect(parseArchiveFilename('2024-04-17_4utumn07 - C53YPQzp7Wj - 09.jpg')!.index).toBe(9);
expect(parseArchiveFilename('2024-04-17_4utumn07 - C53YPQzp7Wj - 9.jpg')!.index).toBe(9);
expect(parseArchiveFilename('2023-11-03_4utumn07 - CzM8Uf6B6H_ - 01.jpg')!.index).toBe(1);
});
it('reads a gallery-dl story name, which carries a per-item shortcode', () => {
const p = parseArchiveFilename('2026-08-16_official_band - DcF9OyhBJ1H.jpg', 'stories')!;
expect(p.postId).toBe('DcF9OyhBJ1H');
expect(p.date).toBe('2026-08-16');
});
it('gives a dated highlight a real date instead of the mtime fallback', () => {
const mtime = Date.parse('2026-08-17T00:00:00Z');
const undated = parseArchiveFilename('4utumn07 - C-IImhvpFuk.jpg', 'highlight', mtime)!;
const dated = parseArchiveFilename('2024-08-04_4utumn07 - C-IImhvpFuk.jpg', 'highlight', mtime)!;
// Same item either way, so re-fetching cannot split it into two posts.
expect(dated.postId).toBe(undated.postId);
expect(undated.date).toBe('2026-08-17');
expect(dated.date).toBe('2024-08-04');
});
});
/**
* Highlights are the only files with no date in the name, so they fall back to
* mtime — which is when the file was written, not when it was posted. Callers
* need to know the difference to let a real date win.
*/
describe('dateFromMtime', () => {
const mtime = Date.parse('2026-08-17T00:00:00Z');
it('flags an undated highlight name as mtime-dated', () => {
const p = parseArchiveFilename('4utumn07 - C-IImhvpFuk.jpg', 'highlight', mtime)!;
expect(p.date).toBe('2026-08-17');
expect(p.dateFromMtime).toBe(true);
});
it('does not flag a highlight that carries its own date', () => {
const p = parseArchiveFilename('2024-08-04_4utumn07 - C-IImhvpFuk.jpg', 'highlight', mtime)!;
expect(p.date).toBe('2024-08-04');
expect(p.dateFromMtime).toBe(false);
});
it('never flags ordinary post or Instaloader names', () => {
expect(parseArchiveFilename('2023-04-19_u - ABC.mp4', 'posts', mtime)!.dateFromMtime).toBe(false);
expect(parseArchiveFilename('2024-01-01_12-00-00_UTC.jpg', 'posts', mtime)!.dateFromMtime).toBe(false);
});
it('leaves the date empty rather than guessing when no mtime is given', () => {
const p = parseArchiveFilename('4utumn07 - C-IImhvpFuk.jpg', 'highlight')!;
expect(p.date).toBe('');
expect(p.dateFromMtime).toBe(false);
});
});
/**
* JDownloader wrote story-shaped names for highlights during one period, so
* the same item exists under two conventions. They must be one post.
*/
describe('canonicalItemId', () => {
it('collapses the two highlight naming conventions onto one id', () => {
const dir = 'story highlights - 4utumn07 - Sunstory';
const undated = parseArchiveFilename('4utumn07 - C5dQPEYpd9W.mp4', 'highlight', 1)!;
const dated = parseArchiveFilename('2024-04-07_4utumn07 - 01 - C5dQPEYpd9W.mp4', 'highlight')!;
expect(scopedPostId(dated.postId, 'highlight', dir))
.toBe(scopedPostId(undated.postId, 'highlight', dir));
});
it('does the same for stories', () => {
const a = parseArchiveFilename('2025-10-26_u - 2 - DQRuDx9iW5Q.jpg', 'stories')!;
expect(scopedPostId(a.postId, 'stories', 'story - u')).toBe('story - u/DQRuDx9iW5Q');
});
it('keeps distinct story items distinct', () => {
const a = parseArchiveFilename('2026-08-13_u - 1 - Db-UTJcCUUr.mp4', 'stories')!;
const b = parseArchiveFilename('2026-08-13_u - 2 - Db-oNJ1CWQ4.mp4', 'stories')!;
expect(scopedPostId(a.postId, 'stories', 'story - u'))
.not.toBe(scopedPostId(b.postId, 'stories', 'story - u'));
});
it('leaves a shortcode that merely starts with digits alone', () => {
expect(canonicalItemId('4utumn07')).toBe('4utumn07');
expect(canonicalItemId('C5dQPEYpd9W')).toBe('C5dQPEYpd9W');
expect(canonicalItemId('12345')).toBe('12345');
});
it('does not touch posts, whose ids are permalinks', () => {
expect(scopedPostId('01 - ABC', 'posts')).toBe('01 - ABC');
});
});
+2 -38
View File
@@ -30,15 +30,6 @@ export interface ParsedFilename {
index: number;
ext: string;
isStory: boolean;
/**
* True when `date` is the file's mtime rather than anything Instagram said.
*
* Only highlights fetched by JDownloader lack a date in the filename, and
* their mtime is just when the file was written. Callers should let any real
* date win over this one — the same item is often also present under a
* gallery-dl name that does carry the date.
*/
dateFromMtime: boolean;
}
/**
@@ -63,7 +54,6 @@ export const parseArchiveFilename = (
index: indexStr ? parseInt(indexStr, 10) : 1,
ext,
isStory: Boolean(story),
dateFromMtime: false,
};
}
@@ -77,7 +67,6 @@ export const parseArchiveFilename = (
index: indexStr ? parseInt(indexStr, 10) : 1,
ext,
isStory: Boolean(story),
dateFromMtime: false,
};
}
@@ -92,7 +81,6 @@ export const parseArchiveFilename = (
index: 1,
ext,
isStory: false,
dateFromMtime: Boolean(mtime),
};
}
}
@@ -100,36 +88,12 @@ export const parseArchiveFilename = (
return null;
};
/**
* A leading per-day ordinal on a story or highlight id: `01 - C5dQPEYpd9W`.
*
* JDownloader wrote story-shaped names for highlights during one period of its
* life, so the same item exists as both `user - CODE.jpg` and
* `date_user - 01 - CODE.jpg`. Those parse to different ids and the viewer
* shows the item twice. The ordinal carries no information the shortcode does
* not — it is a position within a day's stories, and the shortcode is already
* unique — so it is dropped.
*/
const LEADING_ORDINAL = /^\d+ - (?=[A-Za-z0-9_-]+$)/;
/** Strip the ordinal so both naming conventions land on the same post. */
export const canonicalItemId = (postId: string): string =>
postId.replace(LEADING_ORDINAL, '');
/**
* Namespace a post ID by its source directory.
*
* Base-profile IDs are left untouched so existing permalinks keep working;
* sidecar IDs are prefixed so a shortcode appearing in both the profile and a
* highlight stays two distinct posts.
*
* Story and highlight ids are canonicalised first, so an item fetched under
* two different naming conventions is one post rather than two.
*/
export const scopedPostId = (postId: string, kind: SourceKind, dir?: string): string => {
if (kind === 'posts') return postId;
const id = (kind === 'stories' || kind === 'highlight')
? canonicalItemId(postId)
: postId;
return `${dir ?? kind}/${id}`;
};
export const scopedPostId = (postId: string, kind: SourceKind, dir?: string): string =>
kind === 'posts' ? postId : `${dir ?? kind}/${postId}`;
-82
View File
@@ -1,82 +0,0 @@
import { describe, expect, it } from 'vitest';
import {
GalleryDlSidecar, isGalleryDlSidecar, sidecarDate, sidecarIsReel, sidecarSource,
} from './gallery-dl-sidecar';
// Trimmed from real files published to the archive on 2026-08-16.
const REEL: GalleryDlSidecar = {
post_shortcode: 'Db-lNCoib9m', post_id: '3962768346034323302', type: 'reel',
date: '2026-08-13 11:00:44', post_date: '2026-08-13 11:00:44',
username: 'official_band', fullname: 'Official ARTMS',
description: 'Dancing in the spotlight', count: 1, likes: 22914,
};
const FEED_VIDEO: GalleryDlSidecar = { ...REEL, post_shortcode: 'DbdG9L9jU4m', type: 'post', count: 2 };
const HIGHLIGHT: GalleryDlSidecar = {
post_shortcode: 'BATVdRZi_3', post_id: '18099435932626935', type: 'highlight',
date: '2026-08-08 16:22:09', username: 'official_band', count: 154,
};
describe('isGalleryDlSidecar', () => {
it('accepts a real sidecar', () => {
expect(isGalleryDlSidecar(REEL)).toBe(true);
expect(isGalleryDlSidecar(HIGHLIGHT)).toBe(true);
});
it('rejects an Instaloader GraphQL payload', () => {
expect(isGalleryDlSidecar({ node: { __typename: 'GraphVideo', shortcode: 'x' } })).toBe(false);
expect(isGalleryDlSidecar({ __typename: 'GraphImage', post_shortcode: 'x', type: 'post' })).toBe(false);
});
it('rejects an Instagram export manifest', () => {
expect(isGalleryDlSidecar({ media: [{ uri: 'a.jpg' }] })).toBe(false);
expect(isGalleryDlSidecar([{ media: [] }])).toBe(false);
});
it('rejects junk', () => {
for (const v of [null, undefined, 0, '', 'string', {}, { post_shortcode: 'x' }]) {
expect(isGalleryDlSidecar(v)).toBe(false);
}
});
});
describe('sidecarDate', () => {
it('takes the day from the timestamp', () => {
expect(sidecarDate(REEL)).toBe('2026-08-13');
});
it('falls back to post_date', () => {
expect(sidecarDate({ post_shortcode: 'x', post_date: '2024-01-02 03:04:05' })).toBe('2024-01-02');
});
it('returns empty when there is no usable date', () => {
expect(sidecarDate({ post_shortcode: 'x' })).toBe('');
expect(sidecarDate({ post_shortcode: 'x', date: 'not a date' })).toBe('');
});
});
describe('sidecarIsReel', () => {
it('distinguishes a reel from an ordinary feed video', () => {
// Both are single mp4s -- the lone-video heuristic cannot tell them apart.
expect(sidecarIsReel(REEL)).toBe(true);
expect(sidecarIsReel(FEED_VIDEO)).toBe(false);
});
it('declines to answer for stories and highlights', () => {
expect(sidecarIsReel(HIGHLIGHT)).toBeUndefined();
expect(sidecarIsReel({ post_shortcode: 'x', type: 'story' as const })).toBeUndefined();
expect(sidecarIsReel({ post_shortcode: 'x' })).toBeUndefined();
});
});
describe('sidecarSource', () => {
it('maps type onto the archive source kinds', () => {
expect(sidecarSource(REEL)).toBe('reels');
expect(sidecarSource(FEED_VIDEO)).toBe('posts');
expect(sidecarSource(HIGHLIGHT)).toBe('highlight');
expect(sidecarSource({ post_shortcode: 'x', type: 'story' as const })).toBe('stories');
});
it('is undefined for an unknown type', () => {
expect(sidecarSource({ post_shortcode: 'x' })).toBeUndefined();
});
});
-87
View File
@@ -1,87 +0,0 @@
import { SourceKind } from '../types';
/**
* gallery-dl `.json` metadata sidecars.
*
* Written one per post next to the media (see docs/gallery-dl.md). This is the
* only source in any archive format that states outright what a post *is* —
* `type` is Instagram's own classification, the `product_type: "clips"` signal
* carried through the listing response. Everything else the viewer knows about
* reels is guesswork from filenames and directory names.
*
* Deliberately separate from the two older JSON shapes the scanner reads:
*
* Instagram export `posts_1.json`, an array of entries with `media`
* Instaloader `.json.xz`, a GraphQL node under `node`
* gallery-dl this — flat, no wrapper
*/
export interface GalleryDlSidecar {
post_shortcode: string;
post_id?: string;
/** Instagram's own classification of the post. */
type?: 'post' | 'reel' | 'story' | 'highlight';
/** Local-time "YYYY-MM-DD HH:MM:SS" — gallery-dl is configured to emit local. */
date?: string;
post_date?: string;
username?: string;
fullname?: string;
description?: string;
count?: number;
likes?: number;
post_url?: string;
}
/**
* Recognise a gallery-dl sidecar.
*
* Checked structurally rather than by filename, because the older formats are
* also plain `.json`. `node` and `__typename` are what an Instaloader or
* export payload carries, and their absence is what makes this shape
* unambiguous.
*/
export const isGalleryDlSidecar = (data: unknown): data is GalleryDlSidecar => {
if (!data || typeof data !== 'object' || Array.isArray(data)) return false;
const o = data as Record<string, unknown>;
return typeof o.post_shortcode === 'string'
&& typeof o.type === 'string'
&& o.node === undefined
&& o.__typename === undefined
&& o.media === undefined;
};
/** The ISO date (YYYY-MM-DD) a sidecar reports, or '' if it carries none. */
export const sidecarDate = (s: GalleryDlSidecar): string => {
const raw = s.date || s.post_date || '';
const day = raw.slice(0, 10);
return /^\d{4}-\d{2}-\d{2}$/.test(day) ? day : '';
};
/**
* Whether the sidecar says this post is a reel.
*
* Returns undefined rather than false for stories and highlights: those are
* neither reels nor grid posts, and answering "no" would let them be counted
* as ordinary posts.
*/
export const sidecarIsReel = (s: GalleryDlSidecar): boolean | undefined => {
if (s.type === 'reel') return true;
if (s.type === 'post') return false;
return undefined;
};
/**
* Which source kind the sidecar implies, for cross-checking the directory.
*
* A reel shared to the profile grid legitimately appears under `posts`, so a
* disagreement is not an error — the directory says where the file was
* fetched from, `type` says what Instagram considers it.
*/
export const sidecarSource = (s: GalleryDlSidecar): SourceKind | undefined => {
switch (s.type) {
case 'reel': return 'reels';
case 'post': return 'posts';
case 'story': return 'stories';
case 'highlight': return 'highlight';
default: return undefined;
}
};
-44
View File
@@ -1,44 +0,0 @@
import type { Transition } from 'motion/react';
/**
* Shared motion vocabulary, tuned to feel like a native iOS app.
*
* Two rules do most of the work:
* - UIKit animates with springs, not fixed-duration easing, so gestures hand
* their exit velocity to the animation and motion continues rather than
* restarting.
* - iOS springs are critically damped. They settle firmly with no visible
* bounce; overshoot reads as "web animation", not "native".
*/
/** The curve UIKit uses for sheet presentation. */
export const IOS_EASE = [0.32, 0.72, 0, 1] as const;
/** Moving between peers: carousel slides, next/previous post. */
export const NAVIGATE: Transition = { type: 'spring', stiffness: 420, damping: 40, mass: 1 };
/** Presenting or dismissing a surface. Slightly softer than navigation. */
export const PRESENT: Transition = { type: 'spring', stiffness: 320, damping: 34, mass: 1 };
/** Backdrops and cross-fades, where a spring would feel fussy. */
export const FADE: Transition = { duration: 0.28, ease: IOS_EASE };
/** Touch-down feedback. Fast enough to feel like a direct response. */
export const PRESS: Transition = { type: 'spring', stiffness: 600, damping: 30 };
/**
* Continue a drag into its animation.
*
* Handing the gesture's exit velocity to the spring is what separates "the
* sheet kept moving because I flicked it" from "the sheet started a new
* animation once I let go".
*/
export const withVelocity = (velocity: number, base: Transition = NAVIGATE): Transition => ({
...base,
velocity,
});
/** True when the viewer has asked the OS to reduce motion. */
export const prefersReducedMotion = () =>
typeof window !== 'undefined' &&
window.matchMedia('(prefers-reduced-motion: reduce)').matches;
-56
View File
@@ -1,56 +0,0 @@
import { describe, expect, it } from 'vitest';
import { DatedValue, preferDate, shouldReplaceDate } from './post-dates';
const sidecar: DatedValue = { date: '2024-04-07', source: 'sidecar' };
const filename: DatedValue = { date: '2024-04-08', source: 'filename' };
const mtime: DatedValue = { date: '2026-08-17', source: 'mtime' };
describe('date precedence', () => {
it('ranks sidecar above filename above mtime', () => {
expect(preferDate(mtime, filename)).toEqual(filename);
expect(preferDate(filename, sidecar)).toEqual(sidecar);
expect(preferDate(mtime, sidecar)).toEqual(sidecar);
});
it('never lets a weaker source overwrite a stronger one', () => {
expect(preferDate(sidecar, filename)).toEqual(sidecar);
expect(preferDate(sidecar, mtime)).toEqual(sidecar);
expect(preferDate(filename, mtime)).toEqual(filename);
});
it('keeps the incumbent on a tie, so scan order cannot flip the date', () => {
const other: DatedValue = { date: '2020-01-01', source: 'filename' };
expect(preferDate(filename, other)).toEqual(filename);
expect(preferDate(other, filename)).toEqual(other);
});
it('accepts anything when nothing is held yet', () => {
expect(preferDate(undefined, mtime)).toEqual(mtime);
expect(shouldReplaceDate(undefined, mtime)).toBe(true);
});
it('ignores an empty date regardless of source', () => {
const empty: DatedValue = { date: '', source: 'sidecar' };
expect(shouldReplaceDate(filename, empty)).toBe(false);
expect(preferDate(filename, empty)).toEqual(filename);
});
it('replaces a held-but-empty date', () => {
const empty: DatedValue = { date: '', source: 'filename' };
expect(preferDate(empty, mtime)).toEqual(mtime);
});
it('is order-independent for the full three-source case', () => {
const orders = [
[mtime, filename, sidecar],
[sidecar, mtime, filename],
[filename, sidecar, mtime],
[mtime, sidecar, filename],
];
for (const order of orders) {
const won = order.reduce<DatedValue | undefined>(
(acc, next) => preferDate(acc, next), undefined);
expect(won).toEqual(sidecar);
}
});
});
-46
View File
@@ -1,46 +0,0 @@
/**
* Where a post's date came from, and which source wins.
*
* A post is usually described by several files — media, a caption `.txt`, a
* `.json` sidecar, sometimes the same item under two naming conventions — and
* they are scanned in directory order, not in order of trustworthiness. Without
* an explicit ranking the date is decided by whichever file happened to be
* reached first.
*
* Ranked best to worst:
*
* sidecar what Instagram reported, straight from a gallery-dl `.json`
* filename a date the fetcher wrote into the name; correct, but derived
* mtime when the file was written to disk — unrelated to when it was
* posted, and only ever a last resort for JDownloader highlights,
* whose filenames carry no date at all
*/
export type DateSource = 'sidecar' | 'filename' | 'mtime';
const RANK: Record<DateSource, number> = { sidecar: 0, filename: 1, mtime: 2 };
export interface DatedValue {
date: string;
source: DateSource;
}
/**
* Whether `next` should replace the date currently held.
*
* Ties keep the incumbent, so scanning stays stable: two files of equal
* authority cannot flip a post's date back and forth by scan order.
*/
export const shouldReplaceDate = (
current: DatedValue | undefined,
next: DatedValue,
): boolean => {
if (!next.date) return false;
if (!current || !current.date) return true;
return RANK[next.source] < RANK[current.source];
};
/** Apply `next` if it outranks `current`, otherwise keep what we have. */
export const preferDate = (
current: DatedValue | undefined,
next: DatedValue,
): DatedValue => (shouldReplaceDate(current, next) ? next : (current ?? next));
-141
View File
@@ -1,141 +0,0 @@
import { describe, expect, it } from 'vitest';
import { dedupePostCopies, hasReelSource, makeIsReel, postsForTab } from './post-tabs';
import { MediaFile, Post } from '../types';
const media = (type: MediaFile['type'], index = 1): MediaFile => ({
name: `f${index}.${type === 'video' ? 'mp4' : 'jpg'}`,
path: `d/f${index}`, url: '', type, index,
});
const post = (id: string, opts: Partial<Post> = {}): Post => ({
id, date: '2024-01-01', username: 'u', caption: '', media: [media('image')], thumbnail: '', ...opts,
});
const video = (id: string, opts: Partial<Post> = {}) => post(id, { media: [media('video')], ...opts });
const carousel = (id: string, opts: Partial<Post> = {}) =>
post(id, { media: [media('image', 1), media('video', 2)], ...opts });
describe('hasReelSource', () => {
it('is false for an archive with no reels directory', () => {
expect(hasReelSource([post('A'), video('B')])).toBe(false);
});
it('is true once any post came from a reels directory', () => {
expect(hasReelSource([post('A'), video('u - reels/B', { source: 'reels' })])).toBe(true);
});
});
describe('makeIsReel', () => {
it('believes the reels directory when there is one', () => {
const posts = [video('A'), video('u - reels/B', { source: 'reels' })];
const isReel = makeIsReel(posts);
// A is a lone video too, but the archive states which posts are reels.
expect(isReel(posts[0])).toBe(false);
expect(isReel(posts[1])).toBe(true);
});
it('falls back to the lone-video heuristic without one', () => {
const posts = [post('A'), video('B'), carousel('C')];
const isReel = makeIsReel(posts);
expect(posts.map(isReel)).toEqual([false, true, false]);
});
});
describe('dedupePostCopies', () => {
it('leaves distinct posts alone', () => {
const posts = [post('A'), video('B')];
expect(dedupePostCopies(posts).map(p => p.id)).toEqual(['A', 'B']);
});
it('collapses a reel fetched into both the profile and the reels directory', () => {
const posts = [video('B'), video('u - reels/B', { source: 'reels' })];
const deduped = dedupePostCopies(posts);
expect(deduped).toHaveLength(1);
// The reels copy wins, so the survivor is still recognised as a reel.
expect(deduped[0].source).toBe('reels');
});
it('picks the reels copy regardless of scan order', () => {
const profileCopy = video('B');
const reelCopy = video('u - reels/B', { source: 'reels' });
expect(dedupePostCopies([profileCopy, reelCopy])[0].source).toBe('reels');
expect(dedupePostCopies([reelCopy, profileCopy])[0].source).toBe('reels');
});
it('keeps the position of the first copy seen', () => {
const posts = [post('A'), video('B'), post('C'), video('u - reels/B', { source: 'reels' })];
expect(dedupePostCopies(posts).map(p => p.id.split('/').pop())).toEqual(['A', 'B', 'C']);
});
});
describe('postsForTab', () => {
it('shows reels in the profile grid, as Instagram does', () => {
const posts = [post('A'), video('u - reels/B', { source: 'reels' })];
expect(postsForTab(posts, 'posts').map(p => p.id)).toEqual(['A', 'u - reels/B']);
});
it('shows the same reel in both tabs', () => {
const posts = [post('A'), video('u - reels/B', { source: 'reels' })];
const inGrid = postsForTab(posts, 'posts').map(p => p.id);
const inReels = postsForTab(posts, 'reels').map(p => p.id);
expect(inReels).toEqual(['u - reels/B']);
expect(inGrid).toContain('u - reels/B');
});
it('shows a duplicated reel once in the grid, not twice', () => {
const posts = [post('A'), video('B'), video('u - reels/B', { source: 'reels' })];
expect(postsForTab(posts, 'posts')).toHaveLength(2);
expect(postsForTab(posts, 'reels')).toHaveLength(1);
});
it('treats lone videos as reels for archives with no reels directory', () => {
const posts = [post('A'), video('B'), carousel('C')];
expect(postsForTab(posts, 'posts').map(p => p.id)).toEqual(['A', 'B', 'C']);
expect(postsForTab(posts, 'reels').map(p => p.id)).toEqual(['B']);
});
it('has nothing saved', () => {
expect(postsForTab([post('A')], 'saved')).toEqual([]);
});
});
/**
* Once an archive carries gallery-dl sidecars, the guesswork above is replaced
* by Instagram's own classification. These are the cases the heuristic got
* wrong (see docs/gallery-dl.md).
*/
describe('explicit isReel from a sidecar', () => {
it('beats the lone-video heuristic for an ordinary feed video', () => {
// A single mp4 that Instagram calls a post, not a reel — indistinguishable
// by shape alone.
const posts = [video('DbdG9L9jU4m', { isReel: false })];
expect(postsForTab(posts, 'reels')).toEqual([]);
expect(postsForTab(posts, 'posts')).toHaveLength(1);
});
it('recognises a reel that lives in the profile grid', () => {
// Shared to feed, so it sits in the base directory with source 'posts'.
const posts = [post('A'), video('C8FHM6EJl15', { source: 'posts', isReel: true })];
expect(postsForTab(posts, 'reels').map(p => p.id)).toEqual(['C8FHM6EJl15']);
expect(postsForTab(posts, 'posts')).toHaveLength(2);
});
it('beats the directory when both are present', () => {
const posts = [
video('u - reels/A', { source: 'reels', isReel: false }),
video('u - reels/B', { source: 'reels' }),
];
// A is a feed video that the reels tab happened to return; B is unlabelled
// and falls back to its directory.
expect(postsForTab(posts, 'reels').map(p => p.id)).toEqual(['u - reels/B']);
});
it('falls back per post, so a mixed archive still works', () => {
const posts = [
video('labelled', { isReel: true }),
video('unlabelled'),
carousel('C'),
];
expect(postsForTab(posts, 'reels').map(p => p.id)).toEqual(['labelled', 'unlabelled']);
});
});
-108
View File
@@ -1,108 +0,0 @@
import { Post, SourceKind } from '../types';
import { Tab } from './routing';
/**
* Which posts each profile tab shows.
*
* Instagram's profile grid holds everything the account posted — photos,
* carousels and reels alike — and the Reels tab is a *filtered view* of that
* same set rather than a separate one. So a reel belongs in both tabs, and
* only the Reels tab does any filtering.
*
* Kept pure and separate from App.tsx so the reel heuristic and the
* duplicate-copy rules can be tested directly.
*/
/**
* The shortcode shared by every copy of a post, regardless of which source
* directory it came from. Sidecar ids are directory-scoped
* (`4utumn07 - reels/Cq8LrxSJAJE`); the trailing segment is the shortcode.
*/
const shortcode = (post: Post): string => post.id.split('/').pop() ?? post.id;
/**
* True when the archive has a `- reels` sidecar directory, i.e. it states
* outright which posts are reels.
*/
export const hasReelSource = (posts: Post[]): boolean => posts.some(p => p.source === 'reels');
/**
* Build the reel test for an archive.
*
* Instagram's own marker is `product_type: "clips"` on the post's GraphQL
* node, but only Instaloader archives carry that metadata, and only on newer
* captures — JDownloader grabs are media plus a caption `.txt` and nothing
* else (see docs/jdownloader.md). So:
*
* - archives with a `- reels` directory are believed outright;
* - everything else falls back to treating a lone video as a reel.
*
* The fallback is a guess: it cannot tell a reel from an ordinary feed video
* or an old IGTV upload, all three of which are plain `GraphVideo` nodes
* distinguished only by `product_type`.
*/
export const makeIsReel = (posts: Post[]): ((post: Post) => boolean) => {
const guess = hasReelSource(posts)
? (post: Post) => post.source === 'reels'
: (post: Post) => post.media.length === 1 && post.media[0]?.type === 'video';
// `isReel` comes from a gallery-dl sidecar and is Instagram's own answer, so
// it beats both fallbacks — per post, since an archive is usually a mix of
// files fetched before and after sidecars existed.
return (post: Post) => post.isReel ?? guess(post);
};
/** Preference order when the same post was fetched into more than one directory. */
const SOURCE_RANK: Record<SourceKind, number> = { reels: 0, posts: 1, stories: 2, highlight: 3 };
const rankOf = (post: Post): number => SOURCE_RANK[post.source ?? 'posts'];
/**
* Collapse copies of one post that were fetched into more than one directory.
*
* The JDownloader flow crawls a profile URL and its `/reels/` URL separately
* because the profile page misses some reels — so the two overlap, and a reel
* present in both lands on disk twice. Those become two posts with distinct
* directory-scoped ids, which the grid would happily render side by side.
*
* The reels-source copy wins, so the surviving post still reports
* `source: 'reels'` and both the Reels tab and `tabForSource` recognise it.
*
* Only safe because callers pass the grid's posts, which exclude stories and
* highlights — a shortcode may legitimately appear in both the profile and a
* highlight, and those must stay distinct.
*/
export const dedupePostCopies = (posts: Post[]): Post[] => {
const winners = new Map<string, Post>();
for (const post of posts) {
const code = shortcode(post);
const existing = winners.get(code);
if (!existing || rankOf(post) < rankOf(existing)) winners.set(code, post);
}
// Preserve input order, keyed on the winner so ordering does not depend on
// which copy happened to be scanned first.
const emitted = new Set<string>();
const result: Post[] = [];
for (const post of posts) {
const code = shortcode(post);
if (emitted.has(code)) continue;
emitted.add(code);
result.push(winners.get(code)!);
}
return result;
};
/**
* The posts a tab displays.
*
* `posts` must already exclude stories and highlights (App passes `allPosts`).
*/
export const postsForTab = (posts: Post[], tab: Tab): Post[] => {
if (tab === 'saved') return [];
const unique = dedupePostCopies(posts);
if (tab === 'posts') return unique;
return unique.filter(makeIsReel(posts));
};
-114
View File
@@ -1,114 +0,0 @@
import { describe, expect, it } from 'vitest';
import { buildPath, findPostBySlug, parseRoute, postSlug, tabForSource } from './routing';
import { Post } from '../types';
const post = (id: string, source?: Post['source']): Post => ({
id, date: '2024-01-01', username: 'u', caption: '', media: [], thumbnail: '', source,
});
describe('parseRoute', () => {
it('reads the explorer root', () => {
expect(parseRoute('/')).toEqual({ archive: null, tab: 'posts', post: null });
});
it('reads a profile', () => {
expect(parseRoute('/4utumn07/')).toEqual({ archive: '4utumn07', tab: 'posts', post: null });
});
it('reads a profile without a trailing slash', () => {
expect(parseRoute('/4utumn07')).toEqual({ archive: '4utumn07', tab: 'posts', post: null });
});
it('reads a tab', () => {
expect(parseRoute('/4utumn07/reels/').tab).toBe('reels');
expect(parseRoute('/4utumn07/saved/').tab).toBe('saved');
});
it('reads a post in Instagram form', () => {
expect(parseRoute('/4utumn07/p/Db5tIoRCcvm/')).toEqual({
archive: '4utumn07', tab: 'posts', post: 'Db5tIoRCcvm',
});
});
it('decodes archive names containing spaces', () => {
expect(parseRoute('/Heejin_Bubble%20heejinmedia/').archive).toBe('Heejin_Bubble heejinmedia');
});
it('does not treat reserved prefixes as archives', () => {
for (const path of ['/api/archives', '/archives/x/y.jpg', '/assets/index.js']) {
expect(parseRoute(path).archive).toBeNull();
}
});
it('still understands the legacy query form', () => {
expect(parseRoute('/', '?a=4utumn07&t=reels&p=ABC')).toEqual({
archive: '4utumn07', tab: 'reels', post: 'ABC',
});
});
it('ignores an unknown tab', () => {
expect(parseRoute('/', '?a=u&t=bogus').tab).toBe('posts');
});
});
describe('buildPath', () => {
it.each([
[{ archive: null, tab: 'posts', post: null }, '/'],
[{ archive: '4utumn07', tab: 'posts', post: null }, '/4utumn07/'],
[{ archive: '4utumn07', tab: 'reels', post: null }, '/4utumn07/reels/'],
[{ archive: '4utumn07', tab: 'posts', post: 'Db5tIoRCcvm' }, '/4utumn07/p/Db5tIoRCcvm/'],
] as const)('builds %j', (route, expected) => {
expect(buildPath(route as any)).toBe(expected);
});
it('omits the tab from a post URL, matching Instagram', () => {
expect(buildPath({ archive: 'u', tab: 'reels', post: 'ABC' })).toBe('/u/p/ABC/');
});
it('encodes archive names with spaces', () => {
expect(buildPath({ archive: 'a b', tab: 'posts', post: null })).toBe('/a%20b/');
});
it('round-trips through parseRoute', () => {
for (const route of [
{ archive: '4utumn07', tab: 'posts' as const, post: null },
{ archive: '4utumn07', tab: 'reels' as const, post: null },
{ archive: 'Heejin_Bubble heejinmedia', tab: 'posts' as const, post: null },
]) {
expect(parseRoute(buildPath(route))).toEqual(route);
}
});
});
describe('postSlug / findPostBySlug', () => {
it('uses the bare shortcode for base posts', () => {
expect(postSlug(post('Db5tIoRCcvm'))).toBe('Db5tIoRCcvm');
});
it('strips the sidecar directory from the slug', () => {
expect(postSlug(post('story highlights - u - Sunstory/C5dQPEYpd9W'))).toBe('C5dQPEYpd9W');
});
it('resolves a slug back to its post', () => {
const posts = [post('AAA'), post('4utumn07 - reels/BBB', 'reels')];
expect(findPostBySlug(posts, 'BBB')?.id).toBe('4utumn07 - reels/BBB');
expect(findPostBySlug(posts, 'AAA')?.id).toBe('AAA');
});
it('prefers an exact id match over a shortcode match', () => {
const posts = [post('x/ABC'), post('ABC')];
expect(findPostBySlug(posts, 'ABC')?.id).toBe('ABC');
});
it('returns undefined for an unknown slug', () => {
expect(findPostBySlug([post('AAA')], 'ZZZ')).toBeUndefined();
});
});
describe('tabForSource', () => {
it('sends reels to the reels tab and everything else to posts', () => {
expect(tabForSource('reels')).toBe('reels');
expect(tabForSource('posts')).toBe('posts');
expect(tabForSource(undefined)).toBe('posts');
});
});
-85
View File
@@ -1,85 +0,0 @@
import { Post, SourceKind } from '../types';
/**
* Instagram-shaped paths.
*
* / the archive explorer
* /<archive>/ a profile, posts tab
* /<archive>/reels/ a profile, reels tab
* /<archive>/saved/
* /<archive>/p/<shortcode>/ a single post
*
* The older `?a=&t=&p=` query form is still parsed so existing links keep
* working; it is never written back.
*/
export type Tab = 'posts' | 'reels' | 'saved';
const TABS: Tab[] = ['posts', 'reels', 'saved'];
/**
* Path prefixes the app must never treat as an archive name, or a profile
* called "api" would shadow the backend.
*/
const RESERVED = new Set(['api', 'archives', 'assets', 'p', 'fonts', 'sw.js', 'manifest.webmanifest']);
export interface Route {
archive: string | null;
tab: Tab;
/** Post shortcode, i.e. the trailing segment of a post id. */
post: string | null;
}
/**
* A post's URL slug.
*
* Sidecar posts carry a directory-scoped id (`story highlights - u - H/ABC`)
* so ids stay unique across sources, but only the shortcode belongs in a URL.
*/
export const postSlug = (post: Pick<Post, 'id'>): string => {
const tail = post.id.split('/').pop() ?? post.id;
return encodeURIComponent(tail);
};
/** Find the post a slug refers to, preferring an exact id match. */
export const findPostBySlug = (posts: Post[], slug: string): Post | undefined => {
const decoded = decodeURIComponent(slug);
return posts.find(p => p.id === decoded)
?? posts.find(p => (p.id.split('/').pop() ?? p.id) === decoded);
};
/** Which tab shows a given post, so a deep link lands on the right one. */
export const tabForSource = (source?: SourceKind): Tab => (source === 'reels' ? 'reels' : 'posts');
export const parseRoute = (pathname: string, search = ''): Route => {
const segments = pathname.split('/').filter(Boolean).map(decodeURIComponent);
if (segments.length && !RESERVED.has(segments[0])) {
const [archive, second, third] = segments;
if (second === 'p' && third) return { archive, tab: 'posts', post: third };
if (second && TABS.includes(second as Tab)) return { archive, tab: second as Tab, post: null };
return { archive, tab: 'posts', post: null };
}
// Legacy query form: ?a=<archive>&t=<tab>&p=<post id>
const params = new URLSearchParams(search);
const archive = params.get('a');
const tab = params.get('t');
return {
archive: archive || null,
tab: tab && TABS.includes(tab as Tab) ? (tab as Tab) : 'posts',
post: params.get('p'),
};
};
export const buildPath = ({ archive, tab, post }: Route): string => {
if (!archive) return '/';
const base = `/${encodeURIComponent(archive)}`;
// A post URL omits the tab, matching Instagram; the tab is re-derived from
// the post itself when the link is opened.
if (post) return `${base}/p/${post}/`;
if (tab !== 'posts') return `${base}/${tab}/`;
return `${base}/`;
};
+1 -1
View File
@@ -14,7 +14,7 @@ const updateSW = registerSW({
setInterval(() => {
r.update();
}, 60 * 60 * 1000);
console.log(`[PWA] v${__APP_VERSION__} registered; hourly update checks enabled.`);
console.log('[PWA] Service Worker registered and update interval set.');
}
},
onNeedRefresh() {
-9
View File
@@ -1,9 +0,0 @@
/**
* Build-time constants.
*
* This file deliberately has no imports or exports: that keeps it an ambient
* script rather than a module, so the declarations below are global.
*/
/** Release version, injected by `define` in vite.config.ts. */
declare const __APP_VERSION__: string;
+1 -6
View File
@@ -37,12 +37,6 @@ export interface Post {
isStory?: boolean;
/** Defaults to 'posts' for archives without sidecar directories. */
source?: SourceKind;
/**
* Instagram's own answer to "is this a reel", from a gallery-dl `.json`
* sidecar. Undefined when the archive carries no such sidecar, which is when
* the viewer has to fall back to guessing — see src/lib/post-tabs.ts.
*/
isReel?: boolean;
/** Highlight this post belongs to, for source === 'highlight'. */
highlightTitle?: string;
}
@@ -56,6 +50,7 @@ export interface ArchiveFile {
size: number;
text(): Promise<string>;
arrayBuffer(): Promise<ArrayBuffer>;
stream(): ReadableStream<Uint8Array>;
url?: string;
/**
* A URL pointing at this file's contents. Local files mint a disk-backed
-15
View File
@@ -3,24 +3,9 @@ import react from '@vitejs/plugin-react';
import path from 'path';
import {defineConfig} from 'vite';
import { VitePWA } from 'vite-plugin-pwa';
import { createRequire } from 'module';
const { version } = createRequire(import.meta.url)('./package.json');
export default defineConfig(() => {
return {
/**
* The release version, compiled into the client.
*
* This is load-bearing, not cosmetic. The service worker precaches
* index.html *including its response headers*, so a server-side header
* change (a CSP fix, say) never reaches an installed PWA: nothing in the
* client build changed, the precache manifest is byte-identical, and the
* worker has no reason to update. Baking the version in means every release
* changes the bundle hash, which changes index.html, which invalidates the
* precache and re-fetches the shell with current headers.
*/
define: { __APP_VERSION__: JSON.stringify(version) },
plugins: [
react(),
tailwindcss(),