warden / Vigil / problem

Vigil: the site answers is failing

fixedopened 26 days agodown 61s

what the check said

the site answers — expected 200, got 502

Warden’s diagnosis · 85% sure of the cause

The Vigil pm2 process is in status "stopped" (pm2_list), lastStartedAt 2026-09-14T08:26:58Z (~40 min before this incident opened). It is not running, so the reverse proxy in front of it returns 502 to the public probe at https://vigil.80.225.209.190.sslip.io. This is not a code crash: the pm2 error stream contains only Next.js "✓ Ready in ~1s" lines, with no stack traces, and restartsSinceAdded=11 against a service that has been up and down many times is not by itself a crash loop. The 14 failed checks started at 07:20:01Z, so an earlier running instance presumably served through that window; the most recent start at 08:26:58Z went down at some point before the 09:07:02 check (the process is now stopped, not errored). The code has not been shown to be at fault — the symptom is purely that nothing is listening. A `pm2_start vigil` (or `pm2_restart vigil`) should bring it back; if it stops again within minutes, then look at git_log for what landed around 08:25 and what changed since.

suspect · vigil pm2 process (stopped)

Done

This one was worked before you opened the page, so there is nothing to watch live. Everything it did is in Everything it ran below, with the exact command and the rule that permitted each one — that table is the record, and this timeline is only the same thing arriving in real time.

    The Vigil pm2 process is in status "stopped" (pm2_list), lastStartedAt 2026-09-14T08:26:58Z (~40 min before this incident opened). It is not running, so the reverse proxy in front of it returns 502 to — fixed and verified.

    Everything it ran6 calls · 1 changed something

    Warden has no shell. Each row below is one named action from a fixed list, spawned without a shell, with the rule of yours that permitted it. Copy any command and run it yourself.

    whathow safeyour rulecommandms
    read the log pm2_logslooks onlydid ityou allowed itpm2 logs vigil --lines 80 (errors only)201
    read the log pm2_logslooks onlydid ityou allowed itpm2 logs vigil --lines 120196
    read the log pm2_logslooks onlydid ityou allowed itpm2 logs vigil --lines 200211
    load the page http_probelooks onlydid ityou allowed itGET https://vigil.80.225.209.190.sslip.io24
    list the processes pm2_listlooks onlydid ityou allowed itpm2 jlist195
    start the process pm2_startundoes itselfdid ityou allowed itstart vigil214

    This incident is marked resolved because the site answers — the same check that failed — was run again and passed. Nothing Warden believed about its own fix could have closed it.

    Warden — an autonomous operator for software that is already running