Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

alerts: add optional triggered for durations #199

Merged
merged 9 commits into from
Jan 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 8 additions & 5 deletions client/metric_conditions.go
Original file line number Diff line number Diff line change
Expand Up @@ -48,14 +48,17 @@ type Expression struct {
}

type SubAlertExpression struct {
Thresholds Thresholds `json:"thresholds"`
Operand string `json:"operand"`
IsNoData bool `json:"enable-no-data-alert,omitempty"`
Thresholds Thresholds `json:"thresholds"`
Operand string `json:"operand"`
IsNoData bool `json:"enable-no-data-alert,omitempty"`
NoDataDurationMs *int `json:"no-data-duration-ms,omitempty"`
}

type Thresholds struct {
Warning *float64 `json:"warning,omitempty"`
Critical *float64 `json:"critical,omitempty"`
Warning *float64 `json:"warning,omitempty"`
WarningDurationMs *int `json:"warning-duration-ms,omitempty"`
Critical *float64 `json:"critical,omitempty"`
CriticalDurationMs *int `json:"critical-duration-ms,omitempty"`
}

type DependencyMapOptions struct {
Expand Down
6 changes: 6 additions & 0 deletions docs/resources/alert.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,7 @@ Optional:
Optional:

- `is_no_data` (Boolean) If true, a notification is sent when the alert query returns no data. If false, notifications aren't sent in this scenario.
- `no_data_duration_ms` (Number) No data must be seen for this duration before the status changes.
- `operand` (String) Required when at least one threshold (Critical, Warning) is defined. Indicates whether the alert triggers when the value is above the threshold or below the threshold.
- `thresholds` (Block List, Max: 1) Optional values defining the thresholds at which this alert transitions into Critical or Warning states. If a particular threshold is not specified, the alert never transitions into that state. (see [below for nested schema](#nestedblock--composite_alert--alert--expression--thresholds))

Expand All @@ -119,7 +120,9 @@ Optional:
Optional:

- `critical` (String) Defines the threshold for the alert to transition to a Critical (more severe) status.
- `critical_duration_ms` (Number) Critical threshold must be breached for this duration before the status changes.
- `warning` (String) Defines the threshold for the alert to transition to a Warning (less severe) status.
- `warning_duration_ms` (Number) Critical threshold must be breached for this duration before the status changes.



Expand Down Expand Up @@ -161,6 +164,7 @@ Optional:

- `is_multi` (Boolean) When false, send a single notification whenever any number of group_by values exceeds the alert threshold. When true, send individual notifications for each distinct group_by value that exceeds the threshold.
- `is_no_data` (Boolean) If true, a notification is sent when the alert query returns no data. If false, notifications aren't sent in this scenario.
- `no_data_duration_ms` (Number) No data must be seen for this duration before the status changes.
- `operand` (String) Required when at least one threshold (Critical, Warning) is defined. Indicates whether the alert triggers when the value is above the threshold or below the threshold.
- `thresholds` (Block List, Max: 1) Optional values defining the thresholds at which this alert transitions into Critical or Warning states. If a particular threshold is not specified, the alert never transitions into that state. (see [below for nested schema](#nestedblock--expression--thresholds))

Expand All @@ -170,7 +174,9 @@ Optional:
Optional:

- `critical` (String) Defines the threshold for the alert to transition to a Critical (more severe) status.
- `critical_duration_ms` (Number) Critical threshold must be breached for this duration before the status changes.
- `warning` (String) Defines the threshold for the alert to transition to a Warning (less severe) status.
- `warning_duration_ms` (Number) Critical threshold must be breached for this duration before the status changes.



Expand Down
3 changes: 3 additions & 0 deletions docs/resources/metric_condition.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,7 @@ Optional:

- `is_multi` (Boolean) When false, send a single notification whenever any number of group_by values exceeds the alert threshold. When true, send individual notifications for each distinct group_by value that exceeds the threshold.
- `is_no_data` (Boolean) If true, a notification is sent when the alert query returns no data. If false, notifications aren't sent in this scenario.
- `no_data_duration_ms` (Number) No data must be seen for this duration before the status changes.
- `operand` (String) Required when at least one threshold (Critical, Warning) is defined. Indicates whether the alert triggers when the value is above the threshold or below the threshold.
- `thresholds` (Block List, Max: 1) Optional values defining the thresholds at which this alert transitions into Critical or Warning states. If a particular threshold is not specified, the alert never transitions into that state. (see [below for nested schema](#nestedblock--expression--thresholds))

Expand All @@ -129,7 +130,9 @@ Optional:
Optional:

- `critical` (String) Defines the threshold for the alert to transition to a Critical (more severe) status.
- `critical_duration_ms` (Number) Critical threshold must be breached for this duration before the status changes.
- `warning` (String) Defines the threshold for the alert to transition to a Warning (less severe) status.
- `warning_duration_ms` (Number) Critical threshold must be breached for this duration before the status changes.



Expand Down
9 changes: 7 additions & 2 deletions lightstep/resource_alert.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package lightstep

import (
"fmt"

"github.com/lightstep/terraform-provider-lightstep/client"
)

Expand Down Expand Up @@ -52,7 +53,7 @@ func getCompositeAlertFromUnifiedConditionResourceData(compositeAlertIn *client.
return nil, err
}

subAlerts = append(subAlerts, map[string]interface{}{
subAlertExpressionMap := map[string]interface{}{
"name": subAlertIn.Name,
"title": subAlertIn.Title,
"expression": []map[string]interface{}{
Expand All @@ -63,7 +64,11 @@ func getCompositeAlertFromUnifiedConditionResourceData(compositeAlertIn *client.
},
},
"query": queries,
})
}
if subAlertIn.Expression.NoDataDurationMs != nil {
subAlertExpressionMap["no_data_duration_ms"] = subAlertIn.Expression.NoDataDurationMs
}
subAlerts = append(subAlerts, subAlertExpressionMap)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops. Good catch

}

return []map[string][]map[string]interface{}{{
Expand Down
101 changes: 50 additions & 51 deletions lightstep/resource_alert_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -62,13 +62,6 @@ resource "lightstep_alert" "test" {
alerting_rule {
id = lightstep_slack_destination.slack.id
update_interval = "1h"

include_filters = [
{
key = "project_name"
value = "catlab"
}
]
}
}
`
Expand Down Expand Up @@ -103,13 +96,6 @@ resource "lightstep_alert" "test" {
alerting_rule {
id = lightstep_slack_destination.slack.id
update_interval = "1h"

include_filters = [
{
key = "project_name"
value = "catlab"
}
]
}
}
`
Expand Down Expand Up @@ -206,13 +192,6 @@ EOT
alerting_rule {
id = lightstep_slack_destination.slack.id
update_interval = "1h"

include_filters = [
{
key = "project_name"
value = "catlab"
}
]
}
}
`, uqlQuery)
Expand Down Expand Up @@ -251,13 +230,6 @@ EOT
alerting_rule {
id = lightstep_slack_destination.slack.id
update_interval = "1h"

include_filters = [
{
key = "project_name"
value = "catlab"
}
]
}
}
`, uqlQuery)
Expand All @@ -282,10 +254,6 @@ EOT
resource.TestCheckResourceAttr(resourceName, "name", "Too many requests"),
resource.TestCheckResourceAttr(resourceName, "description", "A link to a playbook"),
resource.TestCheckResourceAttr(resourceName, "query.0.query_string", uqlQuery+"\n"),
resource.TestCheckTypeSetElemNestedAttrs(resourceName, "alerting_rule.*", map[string]string{
"include_filters.0.key": "project_name",
"include_filters.0.value": "catlab",
}),
resource.TestCheckResourceAttr(resourceName, "expression.0.is_no_data", "true"),
),
},
Expand Down Expand Up @@ -553,11 +521,6 @@ EOT
alerting_rule {
id = lightstep_slack_destination.slack.id
update_interval = "1h"

include_filters = [{
key = "project_name"
value = "catlab"
}]
}
}
`, uqlQuery)
Expand Down Expand Up @@ -706,13 +669,6 @@ resource "lightstep_alert" "errors" {
alerting_rule {
id = lightstep_slack_destination.slack.id
update_interval = "1h"

include_filters = [
{
key = "project_name"
value = "catlab"
}
]
}
}
`
Expand Down Expand Up @@ -769,13 +725,6 @@ resource "lightstep_alert" "test" {
alerting_rule {
id = lightstep_slack_destination.slack.id
update_interval = "1h"

include_filters = [
{
key = "project_name"
value = "catlab"
}
]
}
}
`
Expand Down Expand Up @@ -1172,3 +1121,53 @@ resource "lightstep_alert" "composite_diff_test" {
},
})
}

func TestAccAlertWithThresholdDurations(t *testing.T) {
var condition client.UnifiedCondition

conditionConfig := `
resource "lightstep_alert" "test" {
project_name = "` + testProject + `"
name = "Too many requests"

expression {
is_multi = true
is_no_data = true
no_data_duration_ms = 60000
operand = "above"
thresholds {
critical = 10
critical_duration_ms = 180000
warning = 5
warning_duration_ms = 120000
}
}

query {
query_name = "a"
hidden = false
display = "line"
query_string = "metric requests | rate 1h | filter service_name == frontend | group_by [method], mean"
}
}
`

resourceName := "lightstep_alert.test"
resource.Test(t, resource.TestCase{
PreCheck: func() { testAccPreCheck(t) },
Providers: testAccProviders,
CheckDestroy: testAccMetricConditionDestroy,
Steps: []resource.TestStep{
{
Config: conditionConfig,
Check: resource.ComposeTestCheckFunc(
testAccCheckLightstepAlertExists(resourceName, &condition),
resource.TestCheckResourceAttr(resourceName, "name", "Too many requests"),
resource.TestCheckResourceAttr(resourceName, "expression.0.no_data_duration_ms", "60000"),
resource.TestCheckResourceAttr(resourceName, "expression.0.thresholds.0.warning_duration_ms", "120000"),
resource.TestCheckResourceAttr(resourceName, "expression.0.thresholds.0.critical_duration_ms", "180000"),
),
},
},
})
}
Loading
Loading