Skip to content

Commit

Permalink
r/burn_alert: Add support for budget rate burn alerts (#391)
Browse files Browse the repository at this point in the history
## Short description of the changes
This PR adds support for budget rate burn alerts to the burn alert
resource.

## How to verify that this has the expected result
Pre-requisites:
- You'll need Go 1.20 and the latest Terraform installed.
- You'll need a team in Honeycomb to test against.
- You'll need a dataset, derived column, and SLO for that team.
- You'll need an API key for that team.

#### Setup:
1. Create a new directory called `providers` somewhere you can easily
access (but not inside the same folder as your Terraform config).
1. Build the provider by checking out this branch and running `make
build`
1. Move the executable created to the `providers` directory you created
earlier.
   ```
   mv terraform-provider-honeycombio /PATH/TO/YOUR/DIRECTORY/providers
   ```
1. Add the following to your ~/.terraformrc file:
   ```
   provider_installation {

# Use /PATH/TO/YOUR/DIRECTORY/providers/terraform-provider-honeycombio
# as an overridden package directory for the honeycombio/honeycombio
provider.
# This disables the version and checksum verifications for this provider
and
# forces Terraform to look for the null provider plugin in the given
directory.
     #dev_overrides {
     #  "honeycombio/honeycombio" = "/PATH/TO/YOUR/DIRECTORY/providers"
     #}

# For all other providers, install them directly from their origin
provider
     # registries as normal. If you omit this, Terraform will _only_ use
# the dev_overrides block, and so no other providers will be available.
     direct {}
   }
   ```
1. Create another new directory (all Terraform commands will be run in
this directory)
1. [Download and save this
file](https://github.com/honeycombio/terraform-provider-honeycombio/files/13367011/budget-rate-burn-alerts-config.tf.txt)
as `main.tf` in your new directory. You'll be modifying it in the steps
below.
1. Set the following environment variables:
   ```
   export HONEYCOMB_API_KEY=YOUR_API_KEY_HERE
   export TF_VAR_dataset=YOUR_DATASET_SLUG_HERE
   export TF_VAR_slo_id=YOUR_SLO_ID_HERE
   ```
1. Run `terraform init`.

#### Provision resources for r/burn_alert:
1. Run `terraform apply`. This should succeed and if you check in the
UI, you should see the burn alerts you just created.
1. Run `terraform plan`. There should be no changes.

#### Check that upgrading provider version doesn't break anything:
1. Update your provider version to local build of this branch by editing
your ~/.terraformrc file and remove the comments in front of the
dev_overrides block
   ```
     dev_overrides {
       "honeycombio/honeycombio" = "/PATH/TO/YOUR/DIRECTORY/providers"
     }
   ```
1. Run `terraform plan`. This time you should see a yellow warning block
telling you that development overrides are in place. It should look like
this:
<img width="1293" alt="Screenshot 2023-11-13 at 3 34 46 PM"
src="https://github.com/honeycombio/terraform-provider-honeycombio/assets/12189856/7a1d4383-2581-493d-88ab-923476baafc2">

#### Check that validation works:
1. Uncomment the 2nd and 3rd burn alerts in your config.
1. Run `terraform apply`. This should succeed and if you check in the
UI, you should see the 2 new burn alerts you just created.
1. Try updating each of your burn alerts to be invalid. Examples
include:
   - Invalid `alert_type`
   - Exhaustion time alert without `exhaustion_minutes` specified
   - Exhaustion time alert with `exhaustion_minutes` < 0
   - Exhaustion time alert with budget rate fields specified
   - Budget rate alert without budget rate fields specified
- Budget rate alert with `budget_rate_window_minutes` < 60 or > the
SLO's time period
- Budget rate alert with `budget_rate_decrease_percent` < 0.0001 or >
100
- Budget rate alert with `budget_rate_decrease_percent` with more than 4
non-zero digits past the decimal
   - Budget rate alert with `exhaustion_minutes` specified
1. Run `terraform apply`. This should fail and you should see
descriptive errors about what attributes have been misconfigured.
1. Undo any changes to your config before moving on

#### Check that updating burn alerts works:
1. Update each of your burn alerts by changing any valid combo of
`alert_type`, `exhaustion_minutes`, `budget_rate_window_minutes`, or
`budget_rate_decrease_percent`. Examples of useful scenarios to test
out:
   - Exhaustion time alert to a budget rate alert
   - Budget rate alert to an exhaustion time alert
- Budget rate burn alert to an exhaustion time burn alert **without**
specifying the `alert_type`
   - Exhaustion time alert to have `exhaustion_minutes` = 0
1. Run `terraform apply`. The output should show the changes you made in
the config. After confirming, you should see these changes reflected in
the UI.
1.  Run `terraform plan`. Your plan should show no changes.

#### Check that destroy works
1. Run `terraform destroy`. This should succeed. You should see these
changes reflected in the UI.

#### Check that a fresh create works
1. Run `terraform apply`. This should succeed. You should see these
changes reflected in the UI.
 
## Related links
- [API documentation](https://docs.honeycomb.io/api/tag/Burn-Alerts)
  • Loading branch information
lafentres authored Nov 15, 2023
1 parent 5068dc5 commit 0d3a6b2
Show file tree
Hide file tree
Showing 14 changed files with 1,316 additions and 156 deletions.
31 changes: 25 additions & 6 deletions client/burn_alert.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,12 +41,31 @@ type SLORef struct {
}

type BurnAlert struct {
ID string `json:"id,omitempty"`
ExhaustionMinutes int `json:"exhaustion_minutes"`
SLO SLORef `json:"slo"`
CreatedAt time.Time `json:"created_at,omitempty"`
UpdatedAt time.Time `json:"updated_at,omitempty"`
Recipients []NotificationRecipient `json:"recipients,omitempty"`
ID string `json:"id,omitempty"`
AlertType string `json:"alert_type"`
ExhaustionMinutes *int `json:"exhaustion_minutes,omitempty"`
BudgetRateWindowMinutes *int `json:"budget_rate_window_minutes,omitempty"`
BudgetRateDecreaseThresholdPerMillion *int `json:"budget_rate_decrease_threshold_per_million,omitempty"`
SLO SLORef `json:"slo"`
CreatedAt time.Time `json:"created_at,omitempty"`
UpdatedAt time.Time `json:"updated_at,omitempty"`
Recipients []NotificationRecipient `json:"recipients,omitempty"`
}

// BurnAlertAlertType represents a burn alert alert type
type BurnAlertAlertType string

const (
BurnAlertAlertTypeExhaustionTime BurnAlertAlertType = "exhaustion_time"
BurnAlertAlertTypeBudgetRate BurnAlertAlertType = "budget_rate"
)

// BurnAlertAlertTypes returns a list of valid burn alert alert types
func BurnAlertAlertTypes() []BurnAlertAlertType {
return []BurnAlertAlertType{
BurnAlertAlertTypeExhaustionTime,
BurnAlertAlertTypeBudgetRate,
}
}

func (s *burnalerts) ListForSLO(ctx context.Context, dataset string, sloId string) ([]BurnAlert, error) {
Expand Down
239 changes: 182 additions & 57 deletions client/burn_alert_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package client

import (
"context"
"fmt"
"testing"

"github.com/stretchr/testify/assert"
Expand All @@ -10,7 +11,6 @@ import (
func TestBurnAlerts(t *testing.T) {
ctx := context.Background()

var burnAlert *BurnAlert
var err error

c := newTestClient(t)
Expand Down Expand Up @@ -39,69 +39,194 @@ func TestBurnAlerts(t *testing.T) {
c.DerivedColumns.Delete(ctx, dataset, sli.ID)
})

t.Run("Create", func(t *testing.T) {
data := &BurnAlert{
ExhaustionMinutes: int(24 * 60), // 24 hours
SLO: SLORef{ID: slo.ID},
Recipients: []NotificationRecipient{
{
Type: "email",
Target: "[email protected]",
},
var defaultBurnAlert *BurnAlert
exhaustionMinutes24Hours := 24 * 60
defaultBurnAlertCreateRequest := BurnAlert{
ExhaustionMinutes: &exhaustionMinutes24Hours,
SLO: SLORef{ID: slo.ID},
Recipients: []NotificationRecipient{
{
Type: "email",
Target: "[email protected]",
},
}

burnAlert, err = c.BurnAlerts.Create(ctx, dataset, data)

assert.NoError(t, err, "failed to create BurnAlert")
assert.NotNil(t, burnAlert.ID, "BurnAlert ID is empty")
assert.NotNil(t, burnAlert.CreatedAt, "created at is empty")
assert.NotNil(t, burnAlert.UpdatedAt, "updated at is empty")
// copy dynamic fields before asserting equality
data.ID = burnAlert.ID
data.CreatedAt = burnAlert.CreatedAt
data.UpdatedAt = burnAlert.UpdatedAt
data.Recipients[0].ID = burnAlert.Recipients[0].ID
assert.Equal(t, data, burnAlert)
})

t.Run("Get", func(t *testing.T) {
getBA, err := c.BurnAlerts.Get(ctx, dataset, burnAlert.ID)
assert.NoError(t, err, "failed to get BurnAlert by ID")
assert.Equal(t, burnAlert, getBA)
})

t.Run("Update", func(t *testing.T) {
burnAlert.ExhaustionMinutes = int(4 * 60) // 4 hours
},
}
exhaustionMinutes1Hour := 60
defaultBurnAlertUpdateRequest := BurnAlert{
ExhaustionMinutes: &exhaustionMinutes1Hour,
SLO: SLORef{ID: slo.ID},
Recipients: []NotificationRecipient{
{
Type: "email",
Target: "[email protected]",
},
},
}

result, err := c.BurnAlerts.Update(ctx, dataset, burnAlert)
var exhaustionTimeBurnAlert *BurnAlert
exhaustionMinutes0Minutes := 0
exhaustionTimeBurnAlertCreateRequest := BurnAlert{
AlertType: string(BurnAlertAlertTypeExhaustionTime),
ExhaustionMinutes: &exhaustionMinutes0Minutes,
SLO: SLORef{ID: slo.ID},
Recipients: []NotificationRecipient{
{
Type: "email",
Target: "[email protected]",
},
},
}
exhaustionMinutes4Hours := 4 * 60
exhaustionTimeBurnAlertUpdateRequest := BurnAlert{
AlertType: string(BurnAlertAlertTypeExhaustionTime),
ExhaustionMinutes: &exhaustionMinutes4Hours,
SLO: SLORef{ID: slo.ID},
Recipients: []NotificationRecipient{
{
Type: "email",
Target: "[email protected]",
},
},
}

assert.NoError(t, err, "failed to update BurnAlert")
// copy dynamic field before asserting equality
burnAlert.UpdatedAt = result.UpdatedAt
assert.Equal(t, burnAlert, result)
})
var budgetRateBurnAlert *BurnAlert
budgetRateWindowMinutes1Hour := 60
budgetRateDecreaseThresholdPerMillion1Percent := 10000
budgetRateBurnAlertCreateRequest := BurnAlert{
AlertType: string(BurnAlertAlertTypeBudgetRate),
BudgetRateWindowMinutes: &budgetRateWindowMinutes1Hour,
BudgetRateDecreaseThresholdPerMillion: &budgetRateDecreaseThresholdPerMillion1Percent,
SLO: SLORef{ID: slo.ID},
Recipients: []NotificationRecipient{
{
Type: "email",
Target: "[email protected]",
},
},
}
budgetRateWindowMinutes2Hours := 2 * 60
budgetRateDecreaseThresholdPerMillion5Percent := 10000
budgetRateBurnAlertUpdateRequest := BurnAlert{
AlertType: string(BurnAlertAlertTypeBudgetRate),
BudgetRateWindowMinutes: &budgetRateWindowMinutes2Hours,
BudgetRateDecreaseThresholdPerMillion: &budgetRateDecreaseThresholdPerMillion5Percent,
SLO: SLORef{ID: slo.ID},
Recipients: []NotificationRecipient{
{
Type: "email",
Target: "[email protected]",
},
},
}

t.Run("ListForSLO", func(t *testing.T) {
results, err := c.BurnAlerts.ListForSLO(ctx, dataset, slo.ID)
burnAlert.Recipients = []NotificationRecipient{}
assert.NoError(t, err, "failed to list burn alerts for SLO")
assert.NotZero(t, len(results))
assert.Equal(t, burnAlert.ID, results[0].ID, "newly created BurnAlert not in list of SLO's burn alerts")
})
testCases := map[string]struct {
alertType string
createRequest BurnAlert
updateRequest BurnAlert
burnAlert *BurnAlert
}{
"default - exhaustion_time": {
alertType: string(BurnAlertAlertTypeExhaustionTime),
createRequest: defaultBurnAlertCreateRequest,
updateRequest: defaultBurnAlertUpdateRequest,
burnAlert: defaultBurnAlert,
},
"exhaustion_time": {
alertType: string(BurnAlertAlertTypeExhaustionTime),
createRequest: exhaustionTimeBurnAlertCreateRequest,
updateRequest: exhaustionTimeBurnAlertUpdateRequest,
burnAlert: exhaustionTimeBurnAlert,
},
"budget_rate": {
alertType: string(BurnAlertAlertTypeBudgetRate),
createRequest: budgetRateBurnAlertCreateRequest,
updateRequest: budgetRateBurnAlertUpdateRequest,
burnAlert: budgetRateBurnAlert,
},
}

t.Run("Delete", func(t *testing.T) {
err = c.BurnAlerts.Delete(ctx, dataset, burnAlert.ID)
for testName, testCase := range testCases {
var burnAlert *BurnAlert
var err error

t.Run(fmt.Sprintf("Create: %s", testName), func(t *testing.T) {
data := &testCase.createRequest
burnAlert, err = c.BurnAlerts.Create(ctx, dataset, data)

assert.NoError(t, err, "failed to create BurnAlert")
assert.NotNil(t, burnAlert.ID, "BurnAlert ID is empty")
assert.NotNil(t, burnAlert.CreatedAt, "created at is empty")
assert.NotNil(t, burnAlert.UpdatedAt, "updated at is empty")
assert.Equal(t, testCase.alertType, burnAlert.AlertType)

// copy dynamic fields before asserting equality
data.AlertType = burnAlert.AlertType
data.ID = burnAlert.ID
data.CreatedAt = burnAlert.CreatedAt
data.UpdatedAt = burnAlert.UpdatedAt
data.Recipients[0].ID = burnAlert.Recipients[0].ID
assert.Equal(t, data, burnAlert)
})

t.Run(fmt.Sprintf("Get: %s", testName), func(t *testing.T) {
result, err := c.BurnAlerts.Get(ctx, dataset, burnAlert.ID)
assert.NoError(t, err, "failed to get BurnAlert by ID")
assert.Equal(t, burnAlert, result)
})

t.Run(fmt.Sprintf("Update: %s", testName), func(t *testing.T) {
data := &testCase.updateRequest
data.ID = burnAlert.ID

burnAlert, err = c.BurnAlerts.Update(ctx, dataset, data)

assert.NoError(t, err, "failed to update BurnAlert")

// copy dynamic field before asserting equality
data.AlertType = burnAlert.AlertType
data.ID = burnAlert.ID
data.CreatedAt = burnAlert.CreatedAt
data.UpdatedAt = burnAlert.UpdatedAt
data.Recipients[0].ID = burnAlert.Recipients[0].ID
assert.Equal(t, burnAlert, data)
})

t.Run(fmt.Sprintf("ListForSLO: %s", testName), func(t *testing.T) {
results, err := c.BurnAlerts.ListForSLO(ctx, dataset, slo.ID)

assert.NoError(t, err, "failed to list burn alerts for SLO")
assert.NotZero(t, len(results))
assert.Equal(t, burnAlert.ID, results[0].ID, "newly created BurnAlert not in list of SLO's burn alerts")
})

t.Run(fmt.Sprintf("Delete - %s", testName), func(t *testing.T) {
err = c.BurnAlerts.Delete(ctx, dataset, burnAlert.ID)

assert.NoError(t, err, "failed to delete BurnAlert")
})

t.Run(fmt.Sprintf("Fail to GET a deleted burn alert: %s", testName), func(t *testing.T) {
_, err := c.BurnAlerts.Get(ctx, dataset, burnAlert.ID)

var de DetailedError
assert.Error(t, err)
assert.ErrorAs(t, err, &de)
assert.True(t, de.IsNotFound())
})
}
}

assert.NoError(t, err, "failed to delete BurnAlert")
})
func TestBurnAlerts_BurnAlertAlertTypes(t *testing.T) {
expectedAlertTypes := []BurnAlertAlertType{
BurnAlertAlertTypeExhaustionTime,
BurnAlertAlertTypeBudgetRate,
}

t.Run("Fail to Get deleted Burn Alert", func(t *testing.T) {
_, err := c.BurnAlerts.Get(ctx, dataset, burnAlert.ID)
t.Run("returns expected burn alert alert types", func(t *testing.T) {
actualAlertTypes := BurnAlertAlertTypes()

var de DetailedError
assert.Error(t, err)
assert.ErrorAs(t, err, &de)
assert.True(t, de.IsNotFound())
assert.NotEmpty(t, actualAlertTypes)
assert.Equal(t, len(expectedAlertTypes), len(actualAlertTypes))
assert.ElementsMatch(t, expectedAlertTypes, actualAlertTypes)
})
}
Loading

0 comments on commit 0d3a6b2

Please sign in to comment.