warden / Vigil / problem

Vigil: the site answers is failing

fixedopened 26 days agodown 3m

what the check said

the site answers — expected 200, got 502

Warden’s diagnosis · 85% sure of the cause

vigil is down because the Next.js server has exited — pm2 shows `vigil` as status `stopped`, lastStartedAt 2026-09-14T09:19:54.206Z (about 6 minutes before the incident opened), with restartsSinceAdded 14 since it was added (cumulative, not a crash loop). The error stream is empty and the out log only shows the normal "Ready in …" startup banners, so the process did not crash with a stack trace — it just isn't running. Visitors therefore get 502 from whatever is upstream (the check went from passing at 09:25:25 to failing at 09:25:41; the process had already stopped by 09:19, so the upstream is now returning 502 because nothing is listening on :3100). The 07:20:01 start of the failures aligns with the process being stopped then and only intermittently restarted — the owner's prior "approve: vigil is back up; close this out" entries show this same pattern has been resolving via `pm2_start`. Restarting `vigil` (pm2_start is allowed without asking per the policy) will bring the same code back and clear the 502.

suspect · vigil process — status `stopped` per pm2 jlist, lastStartedAt 2026-09-14T09:19:54.206Z

Done

This one was worked before you opened the page, so there is nothing to watch live. Everything it did is in Everything it ran below, with the exact command and the rule that permitted each one — that table is the record, and this timeline is only the same thing arriving in real time.

    The probe that failed came back clean.

    What you decided

    Everything it ran11 calls · 0 changed something

    Warden has no shell. Each row below is one named action from a fixed list, spawned without a shell, with the rule of yours that permitted it. Copy any command and run it yourself.

    whathow safeyour rulecommandms
    read recent commits git_loglooks onlydid ityou allowed itgit log -n 159
    read the log pm2_logslooks onlydid ityou allowed itpm2 logs vigil --lines 80194
    read the log pm2_logslooks onlydid ityou allowed itpm2 logs vigil --lines 200 (errors only)178
    list the processes pm2_listlooks onlydid ityou allowed itpm2 jlist179
    read a file read_filelooks onlydid ityou allowed itread_file0
    read a file read_filelooks onlydid ityou allowed itread /home/ubuntu/vigil/package.json16
    read a file read_filelooks onlydid ityou allowed itread /home/ubuntu/vigil/CLAUDE.md9
    search the code grep_repolooks onlydid ityou allowed itgrep -rn "vigil" /home/ubuntu/vigil/ecosystem.config.js11
    search the code grep_repolooks onlydid ityou allowed itgrep -rn "pm2" /home/ubuntu/vigil24
    read a file read_filelooks onlydid ityou allowed itread /home/ubuntu/vigil/ecosystem.config.cjs9
    start the process pm2_startundoes itselfasked youstill cooling down from the last actWarden acted on this service 7 minutes ago and your cooldown is 10. Restarting in a loop is not a fix.—

    This incident is marked resolved because the site answers — the same check that failed — was run again and passed. Nothing Warden believed about its own fix could have closed it.

    Warden — an autonomous operator for software that is already running