feat: read gallery-dl sidecars for reel type and post dates

The .json sidecars published with the ARTMS fetch were inert: the scanner
fed them through the Instaloader path, where `node.edge_media_to_caption`
and `checkIsStory`'s `product_type` are both absent, so nothing happened.

They are now recognised structurally -- flat, with post_shortcode and
type, and none of the markers the other two JSON shapes carry -- and used
for three things:

- `type` sets post.isReel, which post-tabs prefers over every fallback.
  This is Instagram's own classification and it disagrees with ours a
  lot: of 781 items in "official_artms - reels", the sidecars say only
  360 are reels. The other 421 are feed videos the clips endpoint returns
  via include_feed_video, and the directory-based rule counted them all.
- `description` fills the caption where no .txt exists.
- `date` dates a post whose filename could not.

Also fixes date precedence. Only JDownloader highlights lack a date in
the filename, so parseArchiveFilename now marks those as mtime-derived
and the scanner lets any real date replace them -- previously the date
depended on which file the scan reached first.

Verified against real published files: a directory of three type=post and
three type=reel renders 6 in the grid and exactly the 3 reels in the
Reels tab.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-17 12:42:54 -04:00
co-authored by Claude Opus 5
parent f151a33ef2
commit 2beb0f0ec7
9 changed files with 308 additions and 5 deletions
+26 -1
View File
@@ -4,6 +4,7 @@ import { XzReadableStream } from 'xz-decompress';
import { ArchiveFile, CacheData, Post, ServerArchive } from '../types';
import { setCachedArchive, getDirectoryHandle } from '../lib/archive-cache';
import { parseArchiveFilename, scopedPostId, EXPORT_RE, INSTALOADER_RE } from '../lib/archive-patterns';
import { isGalleryDlSidecar, sidecarDate, sidecarIsReel } from '../lib/gallery-dl-sidecar';
const hasDirectoryHandle = async (name: string) => Boolean(await getDirectoryHandle(name));
@@ -142,6 +143,8 @@ export const useArchiveScanner = (
try {
const postsMap = new Map<string, Partial<Post>>();
// Posts whose date is only a file mtime, so a real date can replace it.
const weakDates = new Set<string>();
const mediaFilesMap = new Map<string, ArchiveFile>();
const discoveredProfilePics: { name: string, url: string }[] = [];
const allImageFiles: ArchiveFile[] = [];
@@ -297,16 +300,38 @@ export const useArchiveScanner = (
if (!post) {
post = { id: postId, date, username: user, caption: '', media: [], isStory, source: kind, highlightTitle: file.source?.title };
postsMap.set(postId, post);
if (parsed.dateFromMtime) weakDates.add(postId);
}
else if (isStory) post.isStory = true;
// A JDownloader highlight filename carries no date, so it is dated
// by mtime — but the same item is usually also present under a
// gallery-dl name that does carry one. Let the real date win
// regardless of which file the scan happened to reach first.
if (!parsed.dateFromMtime && date && weakDates.has(postId)) {
post.date = date;
weakDates.delete(postId);
}
const lowerExt = ext.toLowerCase();
if (lowerExt === 'txt') {
try { post.caption = await file.text(); } catch(e) {}
} else if (lowerExt === 'json' || lowerName.endsWith('.json.xz')) {
try {
const data = lowerName.endsWith('.xz') ? await parseXZFile(file) : JSON.parse(await file.text());
if (data) {
if (isGalleryDlSidecar(data)) {
// The only format that states what a post is rather than
// leaving it to be inferred from filenames.
if (data.description) post.caption = data.description;
const reel = sidecarIsReel(data);
if (reel !== undefined) post.isReel = reel;
if (data.type === 'story') post.isStory = true;
const day = sidecarDate(data);
if (day && weakDates.has(postId)) {
post.date = day;
weakDates.delete(postId);
}
} else if (data) {
const node = data.node || data; const iphone = node.iphone_struct || {};
const captionText = node.edge_media_to_caption?.edges?.[0]?.node?.text || node.caption?.text || iphone.caption?.text || '';
if (captionText) post.caption = captionText;