Configuring Notification using Cortex Alertmanager
Cortex Alertmanager notification setup follow mostly the syntax of Prometheus Alertmanager since it is based on the same codebase. The following is a description on how to load the configuration setup so that Alertmanager can use for notification when an alert event happened.
Configuring the Cortex Alertmanager storage backend
With the introduction of Cortex 1.8 the storage backend config option shifted to the new pattern #3888. You can find the new configuration here
Note that when using
-alertmanager.sharding-enabled=true, the following storage backends are not supported:
When using the new configuration pattern it is important that any of the old configuration pattern flags are unset (
-alertmanager.storage), as well as
-<prefix>.configs.url. This is because the old pattern still takes precedence over the new one. The old configuration pattern (
-alertmanager.storage) is marked as deprecated and will be removed by Cortex version 1.11. However this change doesn’t apply to
Cortex Alertmanager configuration
Cortex Alertmanager can be uploaded via Cortex Set Alertmanager configuration API or using Grafana Labs Cortex Tools.
Follow the instruction at the
cortextool link above to download or update to the latest version of the tool.
To obtain the full help of how to use
cortextool for all commands and flags, use
The following example shows the steps to upload the configuration to Cortex
1. Create the Alertmanager configuration
The following is
amconfig.yml, an example of a configuration for Cortex
Alertmanager to send notification via email:
global: # The smarthost and SMTP sender used for mail notifications. smtp_smarthost: 'localhost:25' smtp_from: '[email protected]' smtp_auth_username: 'alertmanager' smtp_auth_password: 'password' route: # A default receiver. receiver: send-email receivers: - name: send-email email_configs: - to: '[email protected]'
Example on how to setup Slack to support receiving Alertmanager notification.
2. Upload the Alertmanager configuration
In this example, Cortex
Alertmanager is set to be available via localhost on port 8095 with user/org = 100.
To upload the above configuration
.yml file with
--key to be your Basic Authentication or API key:
cortextool alertmanager load ./amconfig.yml \ --address=http://localhost:8095 \ --id=100 \ --key=<yourKey>
If there is no error reported, the upload is successful.
To upload the configuration for Cortex
Alertmanager using Cortex API and curl - see Cortex Set Alertmanager configuration API.
3. Ensure the configuration has been uploaded successfully
cortextool alertmanager get \ --address=http://localhost:8095 \ --id=100 \ --key=<yourKey>