Alerts not triggered for empty environment value

[InterstellarStella]

Environment

SaaS (https://sentry.io/)

Steps to Reproduce

Set up an issue alert with the conditon:

If all of these filters match:
The event's environment value is not set

Then send an event that does not have an environment value.

Expected Result

This event triggers the alert.

Actual Result

The event does not trigger the alert. Links to such events are provided in the shadow ticket.

The workaround for now is to set the alert condition as:

If all of these filters match:
The event's environment value not in (comma separated) existing,environment,values

Product Area

Alerts

Link

No response

DSN

No response

Version

No response

┆Issue is synchronized with this Jira Improvement by Unito

[getsantry]

Auto-routing to @getsentry/product-owners-alerts for triage ⏲️

[rachrwang]

Thanks for filing @InterstellarStella - can you share more about the use case/ intent behind this alert, i.e. why would an event not have an environment?

[InterstellarStella]

Hi @rachrwang I have asked the customer what the intention of this functionality is and will write here when/if I receive a response, thanks for taking a look!

[guilhermedsz]

Thanks for filing @InterstellarStella - can you share more about the use case/ intent behind this alert, i.e. why would an event not have an environment?

@rachrwang @InterstellarStella - thanks for raising this one and investigating it.

In terms of why you'd want to not have an environment set, you're entirely correct in that it is rare for an organisation to ever have no environment set for the Issues/error logging. With that said, ignoring the Slack notifications/Alerts config, we know that Sentry supports logging errors without having an environment set, and this is a valid scenario / configuration, whether desirable for most or not is a different question.

In our case, we actually had this Alert setup as a catch-all, and was off the back of a misconfiguration in our "environment lookup" logic that determined what to set the environment to for the JS SDK, in which we ended up having the environment not set at all despite logic in place to determine it.

As a result we found ourselves flying blind on these issues, and a double edged sword since the Slack Alerts weren't being triggered for these, and further to this, was rare that we would change the Sentry filters to "All Environments", and only then would we identify these issues and as a result uncover the miscofniguration in setting the SDK environment.

As such, its reasonable, given the UI supports it, to use the "environment is not_set" filter, to ensure that all cases are covered for Alerting in Slack (presumably this is not limited to the Slack alert, but the alerting configuration itself) - we have dev, staging, release, prod alerts configured, but this last one is essential to making sure a fail safe is in place.

Hopefully that clears things up, as you can probably appreciate, given the fact that the UI already supports this exact filter as it stands, its reasonable that it works in the way that one would expect. I actually believe this is a regression in the functionality, since we set this up a while ago and at that time it was sending through the Slack alerts without having changed any configuration for that alert since we identified it stopped sending them (been working since Jan 18th when it was configured, having a look the last time we received a filter based off the "Environment IS NOT SET" filter, was on Oct 22nd, after that they stopped being sent.

[rachrwang]

Thank you for this detailed example and explanation. cc @leedongwei as something we could slot in for an upcoming rotation