Alert when a run fails

Say the nightly sync fails now and then, and nobody notices until the morning. Here’s how to post an alert the moment it fails, using a trigger.

1. Write an alert script

It posts a message to Slack, using a SLACK_TOKEN secret that may only be sent to slack.com (see Send a report every week for saving it):

{
  "slug": "slack-alert",
  "name": "Slack alert",
  "entry": "index.ts",
  "hosts": ["slack.com"],
  "secrets": {
    "SLACK_TOKEN": {
      "host": "slack.com",
      "header": "Authorization",
      "value": "Bearer {secret}"
    }
  },
  "inputSchema": {
    "type": "object",
    "required": ["text"],
    "properties": {
      "text": { "type": "string" },
      "runId": { "type": "string" },
      "error": { "type": "string" }
    }
  }
}

export default async function run(
  inputs: { text: string; runId?: string; error?: string },
  ctx,
) {
  const lines = [inputs.text];
  if (inputs.error) lines.push(`Error: ${inputs.error}`);
  if (inputs.runId) lines.push(`https://app.reminix.com/runs/${inputs.runId}`);
  const res = await fetch("https://slack.com/api/chat.postMessage", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ channel: "#alerts", text: lines.join("\n") }),
  });
  const answer = await res.json();
  if (!answer.ok) throw new Error(`Slack: ${answer.error}`);
  return { posted: true };
}

Publish it:

reminix scripts publish ./slack-alert --publish

2. Add the trigger

On the alert script’s page, choose Triggers → Add trigger: the event run.failed, only when script.slug equals nightly-orders-sync, with inputs taken from the event. Through the API:

curl -X POST https://api.reminix.com/v1/scripts/slack-alert/triggers \
  -H "Authorization: Bearer $REMINIX_TOKEN" -H "X-Workspace: acme" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Nightly sync failed",
    "eventType": "run.failed",
    "filter": { "script.slug": "nightly-orders-sync" },
    "inputs": {
      "text": "The nightly orders sync failed.",
      "runId": "{{event.data.runId}}",
      "error": "{{event.data.error}}"
    }
  }'

The alert runs within about a minute of the failure, as you, and Reminix records it on its page, marked Trigger.

Good to know

Chains stop after 3 triggered runs, so an alert that fails can’t set off an endless loop.
On the sync’s page, under who uses it, the alert shows as a script that runs after it. Check there before changing or retiring the sync.