Alerts
The server checks every machine every 5 seconds. An alert opens when a problem appears and closes by itself when it goes away. Both are shown under Alerts and, if SMTP is configured, sent by e-mail. Changes found in one check are combined into a single e-mail.
What is checked
| Alert | Fires when | Setting |
|---|---|---|
| Offline | No report for longer than the threshold | offline (40s) |
| CPU | Above the threshold for the whole sustain period | cpu_percent (90), sustain (5m) |
| Memory | Above the threshold for the whole sustain period | mem_percent (90), sustain (5m) |
| Disk | A mount point is fuller than the threshold | disk_percent (90) |
| Inodes | A mount point uses more inodes than the threshold | inode_percent (90) |
| Service | A systemd unit is in the failed state | — |
| Container | A Docker container is unhealthy, restarting, or exited with an error | — |
| WireGuard | A peer that has connected before has not completed a handshake for longer than the threshold | wg_handshake (5m, 0 disables) |
| IP change | A machine connects from a different address (a one-off notice) | — |
Thresholds are set under alerts: in the server config; see Configuration.
Machines that have been approved but have never reported are not counted as offline, and right after the server starts nobody is reported offline until agents had time to reconnect.
smtp:
host: smtp.example.com
port: 465 # 465 = TLS, 587 = STARTTLS
username: alerts@example.com
password: change-me
from: alerts@example.com
to: [you@example.com]
Without smtp.host, alerts are only shown in the admin page and written to the server log.
Language
Alert texts and e-mails are in English by default. Set language: zh in the server config for Chinese. The admin page's own language is chosen per browser.
History
Resolved alerts are kept for 90 days. Metrics are kept for 7 days.