DevOps Workshop - Canary Insufficient Data

[bensontwu]

The DevOps workshop at the canary deployment step seems to have a misconfiguration for the underlying metric. The workshop guides you to set up an erroring endpoint and then set up a canary alarm to roll back deployments when errors occur. Then it has you set up a local script to ping the endpoint repeatedly during deployment to trigger the rollback. However, after following the steps, my deployment succeeded unexpectedly.

This seems to be due to a misconfiguration on the underlying canary alarm metric. It shows "Insufficient data" even though the application is returning errors.

The metric alarm has the following details:

Region: us-west-2 • Lambda • Errors • FunctionName: sam-app-prod-HelloWorldFunction-XXXX • Resource: sam-app-prod-HelloWorldFunction-XXXXX:live • ExecutedVersion: 3

I did some ad-hoc testing and recreated the metric with the region removed from the dimensions:

Lambda • Errors • FunctionName: sam-app-prod-HelloWorldFunction-XXXX • Resource: sam-app-prod-HelloWorldFunction-XXXXX:live • ExecutedVersion: 3

and it seemed to fix the issue. This metric ended up picking up all of the errors.

Is this an underlying issue with the canary configuration in SAM? Or am I missing something here?