feat: capture coauthors and per-item width/height/tagged_users
Two additions to the metadata sidecars, following up on the 2026-09-01 collab/dedup investigation already documented here: - coauthors added to the post-level JSON's include list -- a direct, native signal for collab detection instead of inferring it from the filename/directory identity mismatch also documented that day. - width, height, width_original, height_original and tagged_users get their own per-file sidecar (event: "file", one JSON per downloaded media item, named "<filename>.json"), since a carousel's items can each have different dimensions and tags -- data the existing post-level JSON has no way to represent. owner is deliberately left out, same reasoning as audio_user's existing exclusion. Verified against a live re-fetch of an already-archived carousel: correct per-item data came back, zero media re-downloaded (skip-archive still applies; only the new sidecars are new files). 3 new tests. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011qAds5qr7nZRq5R4yAuxUk
This commit is contained in:
@@ -325,6 +325,32 @@ def build_config(rate: str, sleep_request: list[float],
|
||||
"post_shortcode", "post_id", "type", "date", "post_date",
|
||||
"username", "fullname", "owner_id", "description", "count",
|
||||
"likes", "post_url", "sidecar_shortcode",
|
||||
# A Collab's `username`/`fullname`/`owner_id` above are always the
|
||||
# original poster's, never the scraped account's -- see
|
||||
# docs/gallery-dl.md. `coauthors` is the API's own list of every
|
||||
# OTHER collaborator (it excludes the post's owner), a direct
|
||||
# signal instead of inferring a collab from identity mismatches.
|
||||
"coauthors",
|
||||
],
|
||||
}
|
||||
# Per-carousel-item fields (`width`/`height`/`tagged_users`/`owner`) live
|
||||
# on the per-FILE kwdict, not the per-post one `meta_pp` reads -- a
|
||||
# carousel's items can each have different dimensions and tags, which one
|
||||
# post-level JSON can't represent. `owner` is deliberately left out: it is
|
||||
# a full user object (profile pic URLs, privacy flags) for whoever posted
|
||||
# that specific item, the same reason `audio_user` is excluded above.
|
||||
media_pp = {
|
||||
"name": "metadata",
|
||||
"mode": "json",
|
||||
# event defaults to "file" when omitted -- runs once per downloaded
|
||||
# file, appending ".json" to that file's own full name, e.g.
|
||||
# "... - 01.jpg" gets "... - 01.jpg.json" alongside it. Never
|
||||
# collides with the post-level "....json" above, which has no
|
||||
# per-item number.
|
||||
"include": [
|
||||
"media_id", "shortcode", "num",
|
||||
"width", "height", "width_original", "height_original",
|
||||
"tagged_users",
|
||||
],
|
||||
}
|
||||
|
||||
@@ -363,6 +389,7 @@ def build_config(rate: str, sleep_request: list[float],
|
||||
"postprocessors": [
|
||||
{**caption_pp, "filename": stem + ".txt"},
|
||||
{**meta_pp, "filename": stem + ".json"},
|
||||
media_pp,
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user