ergosteurandClaude Sonnet 5
2f46123022
gallery-dl's dedicated reels extractor POSTs to /api/v1/clips/user/, which now 302-redirects for this account -- confirmed across multiple profiles, hours apart, with a freshly-warmed session and a correct X-IG-WWW-Claim header (ruled out as the cause). The reels tab itself loads fine in a real, already-signed-in browser, so reels-scrape.py drives that same Chrome via its loopback CDP port, scrolls the reels tab like a person would, and scrapes /reel/<code>/ links out of the rendered page instead of calling the blocked endpoint at all. It only finds shortcodes -- deduped against the archive via the same --index gdl-sync.py already uses -- and prints new post URLs. Feeding many of those into gdl-sync.py needed two small additions: a --post-urls-file so the list doesn't have to become a giant argv, and inter-item pacing in run_post_urls (each --post-url was its own subprocess with nothing pacing the gap between them). Verified end to end against zindoriyam: 26 reels found, 16 already archived, 10 new ones fetched and published cleanly. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011qAds5qr7nZRq5R4yAuxUk
41 lines
1.2 KiB
Python
41 lines
1.2 KiB
Python
#!/usr/bin/env python3
|
|
"""
|
|
Tests for the pure parts of reels-scrape.py -- everything except the actual
|
|
CDP session, which needs a live signed-in Chrome and is exercised by hand.
|
|
|
|
python3 -m unittest discover -s scripts -p 'test_*.py'
|
|
"""
|
|
|
|
import importlib.util
|
|
import sys
|
|
import unittest
|
|
from pathlib import Path
|
|
|
|
_spec = importlib.util.spec_from_file_location(
|
|
"reels_scrape", Path(__file__).with_name("reels-scrape.py"))
|
|
scrape = importlib.util.module_from_spec(_spec)
|
|
sys.modules["reels_scrape"] = scrape
|
|
_spec.loader.exec_module(scrape)
|
|
|
|
|
|
class ExtractShortcodes(unittest.TestCase):
|
|
def test_reads_relative_and_absolute_hrefs(self):
|
|
hrefs = [
|
|
"/someuser/reel/ABC123/",
|
|
"https://www.instagram.com/someuser/reel/DEF456/",
|
|
"/reel/GHI789/?img_index=1",
|
|
]
|
|
self.assertEqual(scrape.extract_shortcodes(hrefs),
|
|
["ABC123", "DEF456", "GHI789"])
|
|
|
|
def test_ignores_non_reel_links(self):
|
|
hrefs = ["/someuser/", "/someuser/p/ABC123/", "/explore/tags/foo/"]
|
|
self.assertEqual(scrape.extract_shortcodes(hrefs), [])
|
|
|
|
def test_empty_input(self):
|
|
self.assertEqual(scrape.extract_shortcodes([]), [])
|
|
|
|
|
|
if __name__ == "__main__":
|
|
unittest.main(verbosity=2)
|