Alert when a run fails
Say the nightly sync fails now and then, and nobody notices until the morning. Here’s how to post an alert to Slack the moment it fails, using a trigger. The alert includes the error and a link to the run.
1. Save the Slack token
Section titled “1. Save the Slack token”A workspace owner or admin saves a Slack bot token with permission to
post messages (chat:write), and lets Reminix send it only to
slack.com:
printf %s "$SLACK_BOT_TOKEN" | reminix secrets set SLACK_TOKEN --hosts slack.com2. Write the alert script
Section titled “2. Write the alert script”{ "slug": "alert-on-failure", "name": "Alert on failure", "command": "sh alert.sh", "hosts": ["slack.com"], "secrets": { "SLACK_TOKEN": { "host": "slack.com", "header": "Authorization", "format": "Bearer {secret}" } }, "inputSchema": { "type": "object", "required": ["runId"], "properties": { "runId": { "type": "string", "title": "Failed run" }, "script": { "type": "string", "title": "Script" }, "error": { "type": "string", "title": "Error" } } }}A short shell script does it. It reads its inputs from the JSON file
Reminix names in $REMINIX_INPUTS_FILE:
set -etext=$(jq -r '(.script // "A script") + " failed.\nError: " + (.error // "none given") + "\nhttps://app.reminix.com/runs/" + .runId' "$REMINIX_INPUTS_FILE")answer=$(jq -n --arg text "$text" '{channel: "#alerts", text: $text}' | curl -s https://slack.com/api/chat.postMessage -H "Content-Type: application/json" -d @-)if [ "$(echo "$answer" | jq .ok)" != true ]; then echo "Slack: $(echo "$answer" | jq -r .error)" >&2 exit 1fiecho '{"posted": true}'Publish it:
reminix publish ./alert-on-failure3. Add the trigger
Section titled “3. Add the trigger”On the alert script’s page, choose Triggers → Add trigger. Pick the
event run.failed, run it only when script.slug equals
nightly-sync, and take the inputs from the event. Through the API:
curl -X POST https://api.reminix.com/v1/scripts/alert-on-failure/triggers \ -H "Authorization: Bearer $REMINIX_TOKEN" -H "X-Workspace: acme" \ -H "Content-Type: application/json" \ -d '{ "type": "event", "name": "Nightly sync failed", "event": "run.failed", "filter": { "script.slug": "nightly-sync" }, "inputs": { "runId": "{{event.data.runId}}", "script": "{{event.data.script.slug}}", "error": "{{event.data.error}}" } }'The alert runs as you, within about a minute of the failure. Reminix records the run on the alert script’s page, marked Trigger.
Good to know
Section titled “Good to know”- Reminix stops a chain of triggered runs after 3, so an alert that fails can’t set off an endless loop.
- On the sync’s page, the Used by card lists the alert’s trigger under Automations. Check there before changing or retiring the sync.
Related: Schedules and triggers, Webhooks.