Switch error tracking to Sentry

[rakyi]

• Remove all usage of Bugsnag
• Add Sentry to admin client bundle and all server side processes
  • Move the Sentry initialization to be as early as possible to maximize the chance an error will be captured
  • Include node profiling integration for server side
  • Create a new admin project for the admin client and server processes
• Tell Sentry about:
  • release is the COMMIT_SHA of the currently running code - helps with triage and works with GitHub integration for stuff like comments on PRs and other features
  • environment is good for filtering issues, we are gonna distinguish between prod and staging from now on. Running Sentry on staging should help us catch issues before they get to prod.
• Add GitHub action for creating Sentry releases
  • Telling Sentry about a release before it gets the first event from that version makes some features nicer to use (I think e.g. "Mark as resolved in the next release")
• Add Sentry Vite plugin - used to upload source maps as part of the production build. This is the recommended way to do it and AFAIK the easiest to get client source maps to work also in staging.
• Rework manual capturing of errors:
  • Rename logErrorAndMaybeSendToBugsnag to logErrorAndMaybeCaptureInSentry
  • Where it makes sense, throw an error instead of calling this function and let the Sentry error handler capture it
  • Remove two instances of logging gdoc validation errors and warnings during baking, since we get more than 1k of them in one bake, which makes them useless and consumes a lot of our quota. We should add it back only if we making it smarter or fix most of the errors first.
  • Remove unnecessary warn wrapper for console.warn
  • Change JsonError to Error where it isn't necessary
• Increase Bundlewatch limit substantially - Sentry dependencies were previously not included in the build at all, giving us a false sense of the bundle being smaller than it was

You can see various testing issues and new issues from baking etc. from this branch as well as current ones from production in Sentry.

Resolves #4204

Depends on https://github.com/owid/ops/pull/263

[rakyi]

Bundle watch originally failed here with a large bundle size increase. We are in fact making the bundle smaller from 2.91 MB to 2.87 MB (I compared prod vs staging), but Bundlewatch didn't pick it up, because it seems that rollup is smart enough to remove code that won't run at import time when it's guarded by a falsy condition, i.e. the SENTRY_DSN isn't set in CI, so the code that was using Sentry didn't run, so rollup didn't include in the bundle previously.

We have now removed the condition and initialize Sentry always (which is fine to do and fails gracefully without DSN). So rollup now doesn't remove the code, and Bundlewatch thus detects the bundle increase.

[owidbot]

Quick links (staging server):

Site Dev	Site Preview	Admin	Wizard	Docs

Login: ssh owid@staging-site-remove-bugsnag

• Site-screenshots: https://github.com/owid/site-screenshots/compare/remove-bugsnag
• SVG tester: https://github.com/owid/owid-grapher-svgs/compare/remove-bugsnag

SVG tester:

Number of differences (default views): 0 ✅
Number of differences (all views): 220 (5ec6c0) ❌

Edited: 2025-01-17 11:32:16 UTC
Execution time: 1.21 seconds

[ikesau]

Really nice to have someone go through this all and make it sane.

There's one other random mention of Bugsnag in the codebase (a comment explaining why some object isn't desctructured) which I think we can also remove.

I think doing some tire kicking over a video call would be a good way to test this. Wanna book me for 15 minutes? 🙂

[ikesau]

It would be nice to have sentry and scope in the filename, I think

e.g.

initSentryAdminSiteClient.ts
initSentryServer.ts
initSentrySite.ts

[rakyi]

I'd like to keep the name, because:

• instrument is what is used in Sentry docs, so if you read the docs and then search for such file, you'll easily find all three of them. Scope is in the module path, i.e. the directory name, we don't usually repeat that in nested file names.
• instrument is named by what it does, so if we ever want to add instrumentation of something else, we can.
• We don't have to change the name, when we change providers. We also don't call files/folders as specific as mysql when we don't need to, but rather db.

[ikesau]

Manual testing notes.

1.

This is from trying to add a circular redirect (which is inferrable through the stack trace) but I'm wondering if this is the top-level message given for all the JsonErrors?

2.

OOC, what's up with all these different ADMIN-X instances? Do they represent different staging instances? Is there a way we can name them more clearly?

3.

Here's an error from me adding a malformed gdoc. In Bugsnag, I was able to look at the headers and see the user's email in them, but it looks like they're not here in Sentry? Not essential, but it would be nice to have as I often used this information to quickly follow up with authors who might be stuck on something.

4.

I wonder if we should add some more DSNs - the fact that errors/crashes from baking and indexing are both under admin is possible to understand, but if there's not much cost 2 having 2 additional buckets for those issues then maybe that would make things clearer?