Vigil: the site answers is failing
fixedopened 26 days agodown 67s
what the check said
the site answers — expected 200, got 502
Warden’s diagnosis · 70% sure of the cause
vigil is stopped, not crashed. `pm2_list`: status "stopped", lastStartedAt 11:06:25 (one minute before this incident). Error stream is empty — no stack trace. Out stream only shows Next.js startup banners ("✓ Ready in ~900ms") repeating. `grep_repo` for `process.on` returned nothing in the repo, so the app has no SIGTERM/SIGINT handler of its own. Failures began 07:20 UTC, intermittent (26/108 looks failed), 17 cumulative restarts. The 502 is the reverse proxy saying "no upstream" because nothing is listening on :3100. Per owner's standing rule ("Restart it if it is just stopped"), pm2_restart vigil is the natural next step. If it stops again immediately after restart, the cause is external (cron, vigilante-sweep, warden-sweep) and warrants a second look.
suspect · vigil process — lastStartedAt 2026-09-14T11:06:25.567Z, exits cleanly with no error output
This one was worked before you opened the page, so there is nothing to watch live. Everything it did is in Everything it ran below, with the exact command and the rule that permitted each one — that table is the record, and this timeline is only the same thing arriving in real time.
vigil is stopped, not crashed. Fixed, and the check that failed passes again.
Everything it ran11 calls · 1 changed something
Warden has no shell. Each row below is one named action from a fixed list, spawned without a shell, with the rule of yours that permitted it. Copy any command and run it yourself.
| what | how safe | your rule | command | ms |
|---|---|---|---|---|
list the processes pm2_list | looks only | did ityou allowed it | pm2 jlist | 206 |
read the log pm2_logs | looks only | did ityou allowed it | pm2 logs vigil --lines 100 (errors only) | 253 |
read the log pm2_logs | looks only | did ityou allowed it | pm2 logs vigil --lines 150 | 239 |
read recent commits git_log | looks only | did ityou allowed it | git log -n 15 | 10 |
load the page http_probe | looks only | did ityou allowed it | GET https://vigil.80.225.209.190.sslip.io | 16 |
read a file read_file | looks only | did ityou allowed it | read /home/ubuntu/vigil/package.json | 12 |
search the code grep_repo | looks only | did ityou allowed it | grep -rn "vigil|process.on\\(|SIGTERM|SIGINT|stop\\(\\)" /home/ubuntu/vigil | 15 |
search the code grep_repo | looks only | did ityou allowed it | grep -rn "vigil" /home/ubuntu/vigil | 32 |
search the code grep_repo | looks only | did ityou allowed it | grep -rn "process\\.on" /home/ubuntu/vigil | 27 |
search the code grep_repo | looks only | did ityou allowed it | grep -rn "owed" /home/ubuntu/vigil | 129 |
restart the process pm2_restart | undoes itself | did ityou allowed it | restart vigil | 251 |
This incident is marked resolved because the site answers — the same check that failed — was run again and passed. Nothing Warden believed about its own fix could have closed it.