feat: add healthchecks.io pings to run-stories.sh and run-profiles.sh
Start/finish pings wired through gdl-cron.sh via HC_URL, finish ping carries the run's log as its body (visible in the dashboard, no SSH needed to diagnose a failure). Exit code is the trailing URL segment by convention, so the existing "stories: 0 to sync" -> exit 75 remap already reaches the dashboard as a failure too. Also fixes the actual bug that would have made this useless: the routine chown-permission rsync warning (the archive share is never owned by the ssh user) made gdl-sync.py return 1 on every clean run, not just failing ones -- harmless when a human reads the log, but it would have pinged every successful run as a failure. publish() now only masks that one specific, verified-benign error shape; a real transfer failure sharing the same exit code (23) still fails loudly. 6 new tests. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011qAds5qr7nZRq5R4yAuxUk
This commit is contained in:
+26
@@ -159,6 +159,32 @@ files, it is host-local runtime config, and it embeds the same real
|
|||||||
Chrome-profile path already documented above. Recreate it by hand (or from
|
Chrome-profile path already documented above. Recreate it by hand (or from
|
||||||
this section) after a fresh `mattellite` setup.
|
this section) after a fresh `mattellite` setup.
|
||||||
|
|
||||||
|
## Healthchecks.io
|
||||||
|
|
||||||
|
`run-stories.sh` and `run-profiles.sh` each ping a healthchecks.io check —
|
||||||
|
`instagram-archiver-stories` / `instagram-archiver-posts` — via `HC_URL`,
|
||||||
|
plumbed through to `gdl-cron.sh`. A start ping fires before the sync, a
|
||||||
|
finish ping fires after with the run's own log attached as the body (visible
|
||||||
|
in the healthchecks.io dashboard, so a failure is diagnosable without SSHing
|
||||||
|
in). The trailing URL segment is the exit code: `0` means success, anything
|
||||||
|
else fails the check — including the `stories: 0 to sync` → exit 75 remap
|
||||||
|
above, so a fully-skipped stories run alerts, not silently passes.
|
||||||
|
|
||||||
|
Unset `HC_URL` (e.g. `HC_URL= ./run-stories.sh`) to run without pinging.
|
||||||
|
`run-full-sweep.sh` and the reels scripts have no check configured yet.
|
||||||
|
|
||||||
|
**This is also why `gdl-sync.py`'s `publish()` no longer treats the routine
|
||||||
|
chown-permission warning as a failure (2026-09-14).** The archive share is
|
||||||
|
never owned by the ssh user, so rsync's own `chown` step fails on *every*
|
||||||
|
run — exit 23, "some files/attrs were not transferred" — even though every
|
||||||
|
file lands; "Reading the output" above has documented this as cosmetic since
|
||||||
|
this file was created. Before this fix the exit *code* didn't know that, so
|
||||||
|
`gdl-sync.py` returned 1 on every clean run too — harmless when a human was
|
||||||
|
reading the log anyway, but it would have made every healthcheck ping report
|
||||||
|
a normal run as a failure. Only that specific permission-warning shape is
|
||||||
|
masked; a real transfer failure sharing the same exit code (23) still fails
|
||||||
|
loudly, verified with both a real captured rsync failure and unit tests.
|
||||||
|
|
||||||
## Remotes
|
## Remotes
|
||||||
|
|
||||||
| remote | what goes there |
|
| remote | what goes there |
|
||||||
|
|||||||
@@ -15,6 +15,13 @@ set -eu
|
|||||||
|
|
||||||
MODE="${1:?usage: gdl-cron.sh stories|profiles|full-sweep}"
|
MODE="${1:?usage: gdl-cron.sh stories|profiles|full-sweep}"
|
||||||
|
|
||||||
|
# healthchecks.io (or compatible) ping URL, e.g. https://hc-ping.com/<uuid>.
|
||||||
|
# Set by the run-*.sh wrapper for whichever mode has a check configured;
|
||||||
|
# unset here means no ping, silently. A ping failure (network blip, check
|
||||||
|
# deleted) must never fail the sync itself -- hence `|| true` throughout, and
|
||||||
|
# a short timeout (-m) so a hung ping can't hang the run.
|
||||||
|
HC_URL="${HC_URL:-}"
|
||||||
|
|
||||||
GDL_HOME="${GDL_HOME:-$HOME/gdl}"
|
GDL_HOME="${GDL_HOME:-$HOME/gdl}"
|
||||||
INDEX="${GDL_INDEX:-https://instaarchive.ergosteur.com}"
|
INDEX="${GDL_INDEX:-https://instaarchive.ergosteur.com}"
|
||||||
PUBLISH="${GDL_PUBLISH:-agentapi@10.20.28.200:/volume1/rslsync/sync/Instagram-archive/archives/}"
|
PUBLISH="${GDL_PUBLISH:-agentapi@10.20.28.200:/volume1/rslsync/sync/Instagram-archive/archives/}"
|
||||||
@@ -57,6 +64,8 @@ esac
|
|||||||
|
|
||||||
mkdir -p "$GDL_HOME/logs"
|
mkdir -p "$GDL_HOME/logs"
|
||||||
|
|
||||||
|
[ -n "$HC_URL" ] && curl -fsS -m 10 --retry 3 -o /dev/null "$HC_URL/start" || true
|
||||||
|
|
||||||
# Staging is wiped every run ON PURPOSE. What we already hold is decided by the
|
# Staging is wiped every run ON PURPOSE. What we already hold is decided by the
|
||||||
# skip-archive (--download-archive), never by which files happen to be sitting
|
# skip-archive (--download-archive), never by which files happen to be sitting
|
||||||
# in staging, so starting empty is correct -- and it keeps the publish rsync
|
# in staging, so starting empty is correct -- and it keeps the publish rsync
|
||||||
@@ -104,6 +113,14 @@ fi
|
|||||||
|
|
||||||
echo "=== exit $status at $(date -Is) ===" | tee -a "$LOG"
|
echo "=== exit $status at $(date -Is) ===" | tee -a "$LOG"
|
||||||
|
|
||||||
|
# The log becomes the ping body -- healthchecks.io keeps the last ping's body
|
||||||
|
# and shows it in the dashboard, so a failure is diagnosable without SSHing
|
||||||
|
# in. The trailing URL segment is the exit code by convention: 0 means
|
||||||
|
# success, anything else means fail (this is how the "0 skipped" -> 75
|
||||||
|
# remap above also reaches the dashboard as a failure, not a quiet success).
|
||||||
|
[ -n "$HC_URL" ] && curl -fsS -m 10 --retry 3 --data-binary "@$LOG" \
|
||||||
|
-o /dev/null "$HC_URL/$status" || true
|
||||||
|
|
||||||
# Keep the log directory from growing without bound.
|
# Keep the log directory from growing without bound.
|
||||||
ls -1t "$GDL_HOME/logs" | tail -n +30 | while read -r old; do
|
ls -1t "$GDL_HOME/logs" | tail -n +30 | while read -r old; do
|
||||||
rm -f "$GDL_HOME/logs/$old"
|
rm -f "$GDL_HOME/logs/$old"
|
||||||
|
|||||||
+21
-1
@@ -748,7 +748,27 @@ def publish(staging: Path, dest: str, dry_run: bool) -> int:
|
|||||||
return 0
|
return 0
|
||||||
cmd = rsync_command(staging, dest, dry_run)
|
cmd = rsync_command(staging, dest, dry_run)
|
||||||
print(" " + " ".join(cmd))
|
print(" " + " ".join(cmd))
|
||||||
return subprocess.run(cmd).returncode
|
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||||
|
sys.stdout.write(result.stdout)
|
||||||
|
sys.stderr.write(result.stderr)
|
||||||
|
if result.returncode == 0:
|
||||||
|
return 0
|
||||||
|
|
||||||
|
# The archive share is never owned by the ssh user, so rsync's `chown`
|
||||||
|
# step fails on every single run -- exit 23 ("some files/attrs were not
|
||||||
|
# transferred"), even though every file landed. This is long documented
|
||||||
|
# (TOOLING.md "Reading the output") as something to read past, not a
|
||||||
|
# real failure -- but until now the exit CODE didn't know that, so it
|
||||||
|
# was 1 on every clean run too, silently. Only mask it when every rsync
|
||||||
|
# error line is that specific permission warning; anything else (a real
|
||||||
|
# transfer failure sharing the same exit code) still fails loudly.
|
||||||
|
err_lines = [l for l in result.stderr.splitlines() if l.startswith("rsync:")]
|
||||||
|
if (result.returncode == 23 and err_lines
|
||||||
|
and all("failed to set permissions" in l for l in err_lines)):
|
||||||
|
print(" (rsync exit 23 was only the expected chown-permission "
|
||||||
|
"warning; files landed)")
|
||||||
|
return 0
|
||||||
|
return result.returncode
|
||||||
|
|
||||||
|
|
||||||
def run_post_urls(args) -> int:
|
def run_post_urls(args) -> int:
|
||||||
|
|||||||
@@ -3,4 +3,7 @@
|
|||||||
# does this), ~5 min. Covers everything but is no substitute for stories.sh --
|
# does this), ~5 min. Covers everything but is no substitute for stories.sh --
|
||||||
# a story posted and expired between two runs of this is simply gone.
|
# a story posted and expired between two runs of this is simply gone.
|
||||||
set -eu
|
set -eu
|
||||||
|
# healthchecks.io check "instagram-archiver-posts". Override or unset (e.g.
|
||||||
|
# HC_URL= ./run-profiles.sh) to run without pinging it.
|
||||||
|
export HC_URL="${HC_URL:-https://hc-ping.com/3f40abd8-e9bd-43af-a105-c583efaad88c}"
|
||||||
exec "$(dirname "$0")/gdl-cron.sh" profiles "$@"
|
exec "$(dirname "$0")/gdl-cron.sh" profiles "$@"
|
||||||
|
|||||||
@@ -6,4 +6,7 @@
|
|||||||
# mode argument; override pacing/paths the same way gdl-cron.sh does, with
|
# mode argument; override pacing/paths the same way gdl-cron.sh does, with
|
||||||
# GDL_* env vars (see gdl-cron.sh's own comments for what's available).
|
# GDL_* env vars (see gdl-cron.sh's own comments for what's available).
|
||||||
set -eu
|
set -eu
|
||||||
|
# healthchecks.io check "instagram-archiver-stories". Override or unset (e.g.
|
||||||
|
# HC_URL= ./run-stories.sh) to run without pinging it.
|
||||||
|
export HC_URL="${HC_URL:-https://hc-ping.com/9fe7347f-95be-47aa-a46b-affd27691554}"
|
||||||
exec "$(dirname "$0")/gdl-cron.sh" stories "$@"
|
exec "$(dirname "$0")/gdl-cron.sh" stories "$@"
|
||||||
|
|||||||
@@ -12,10 +12,12 @@ whose absence got the archive's Instagram account suspended.
|
|||||||
import datetime as dt
|
import datetime as dt
|
||||||
import importlib.util
|
import importlib.util
|
||||||
import json
|
import json
|
||||||
|
import subprocess
|
||||||
import sys
|
import sys
|
||||||
import tempfile
|
import tempfile
|
||||||
import unittest
|
import unittest
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
from unittest import mock
|
||||||
|
|
||||||
_spec = importlib.util.spec_from_file_location(
|
_spec = importlib.util.spec_from_file_location(
|
||||||
"gdl_sync", Path(__file__).with_name("gdl-sync.py"))
|
"gdl_sync", Path(__file__).with_name("gdl-sync.py"))
|
||||||
@@ -208,6 +210,48 @@ class Publishing(unittest.TestCase):
|
|||||||
self.assertIn(pattern, cmd)
|
self.assertIn(pattern, cmd)
|
||||||
self.assertIn("--dry-run", cmd)
|
self.assertIn("--dry-run", cmd)
|
||||||
|
|
||||||
|
def _publish_with(self, returncode, stderr=""):
|
||||||
|
with tempfile.TemporaryDirectory() as d:
|
||||||
|
staging = Path(d)
|
||||||
|
(staging / "file.txt").write_text("x") # publish() skips empty dirs
|
||||||
|
completed = subprocess.CompletedProcess(
|
||||||
|
args=["rsync"], returncode=returncode, stdout="", stderr=stderr)
|
||||||
|
with mock.patch("subprocess.run", return_value=completed):
|
||||||
|
return gdl.publish(staging, "host:/dest", dry_run=False)
|
||||||
|
|
||||||
|
def test_a_clean_rsync_is_success(self):
|
||||||
|
self.assertEqual(self._publish_with(0), 0)
|
||||||
|
|
||||||
|
def test_the_expected_chown_warning_is_not_a_failure(self):
|
||||||
|
# The exact shape seen on every real run: the archive share is never
|
||||||
|
# owned by the ssh user, so rsync's own chown step fails -- exit 23,
|
||||||
|
# "some files/attrs were not transferred" -- even though every file
|
||||||
|
# landed. TOOLING.md documents this as something to read past; the
|
||||||
|
# exit CODE needs to agree, or every healthcheck ping reports a
|
||||||
|
# clean run as a failure.
|
||||||
|
stderr = (
|
||||||
|
'rsync: failed to set permissions on "/dest/.": Operation not permitted (1)\n'
|
||||||
|
'rsync: failed to set permissions on "/dest/user": Operation not permitted (1)\n'
|
||||||
|
'rsync error: some files/attrs were not transferred (see previous errors) '
|
||||||
|
'(code 23) at main.c(1347) [sender=3.4.1]\n'
|
||||||
|
)
|
||||||
|
self.assertEqual(self._publish_with(23, stderr), 0)
|
||||||
|
|
||||||
|
def test_a_real_transfer_failure_still_fails(self):
|
||||||
|
# Same exit code (23) can also mean a real problem -- e.g. a file
|
||||||
|
# vanishing mid-transfer -- so only the SPECIFIC permission-warning
|
||||||
|
# shape above is masked, not the exit code in general.
|
||||||
|
stderr = (
|
||||||
|
'rsync: [sender] readlink_stat("/stage/broken") failed: '
|
||||||
|
'No such file or directory (2)\n'
|
||||||
|
'rsync error: some files/attrs were not transferred (see previous errors) '
|
||||||
|
'(code 23) at main.c(1347) [sender=3.4.1]\n'
|
||||||
|
)
|
||||||
|
self.assertEqual(self._publish_with(23, stderr), 23)
|
||||||
|
|
||||||
|
def test_an_unrelated_exit_code_still_fails(self):
|
||||||
|
self.assertEqual(self._publish_with(1, "rsync: some other error\n"), 1)
|
||||||
|
|
||||||
|
|
||||||
class PostUrl(unittest.TestCase):
|
class PostUrl(unittest.TestCase):
|
||||||
"""--post-url: a one-off fetch outside the tracked profile list, whose
|
"""--post-url: a one-off fetch outside the tracked profile list, whose
|
||||||
|
|||||||
Reference in New Issue
Block a user