Github Webhook not validating properly

[eBeyond]

Hi,
I've got several webhooks which are not working as expected. When pushing to github, I receive this:

notification-controller-55d78c78c-8ndkp manager {"level":"info","ts":"2023-06-20T05:00:34.187Z","logger":"receiver-server","msg":"handling request: 45df350307cf3420588e3abe2883199fb9230943903494d4de3d1c28e4866230"}
notification-controller-55d78c78c-8ndkp manager {"level":"error","ts":"2023-06-20T05:00:34.191Z","logger":"receiver-server","msg":"unable to validate payload","reconciler kind":"Receiver","name":"my-receiver","namespace":"flux-system","error":"the GitHub signature header is invalid, err: payload signature check failed"}

I've created the receiver and the secret. The secrets token value I've put in the secrets-field of the webhook (base64-decoded). The hook is in the url of the webhook configuration. the one which is shown by
➜ ~ kubectl -n flux-system get receivers.notification.toolkit.fluxcd.io my-receiver NAME AGE READY STATUS my-receiver 25d True Receiver initialized for path: /hook/45df350307cf3420588e3abe2883199fb9230943903494d4de3d1c28e4866230
If I redeliver the payload from github it shows that the webhook is received by the notification controller and it's using the proper receiver (my-receiver) as statet in the logs. But the payload isn't properly validated.
To check that the proper payload is received, I've created a small service which validates the payload on the cluster. And, provided with the correct secret, it's validating the sha1 and the sha256 checksums properly.
As of now, I'm stuck. I think the secret is read correctly, because the webhookPath, which is generated is generated from the same secret where the token lives. As such, it should work, but I've got no idea why the github payload isn't properly validated.

The notification-controller is the ghcr.io/fluxcd/notification-controller:v1.0.0-rc.4

Has anyone an idea?

Thanks
m.

[somtochiama]

Hey @eBeyond ,

I just created a GitHub webhook and it worked okay. There's probably a configuration error somewhere and the secret is usually the first place to check.

The secrets token value I've put in the secrets-field of the webhook (base64-decoded)

The token is set in the token field of the secret. Can you update this to see if it fixes the issue?

[eBeyond]

I've retried several steps after that it works. It was run in automation. As such I've tried to store the token base64 encoded in the secret using data.token = <base64 encoded token>. This shows pretty good if you're looking into it, but for some reason it doesn't work. When using stringData.token = <plain token> it works.

[somtochiama]

Probably there's a new line getting into the base64 encoded secret

echo "foo" | base64 (wrong: adds newline) versus echo -n "foo" | base64 (right: plain value).

[jeff-r-koyaltech]

Thanks @somtochiama! The newline potentially causing a problem is tricky.

[jeff-r-koyaltech]

One "hint" that you're facing the newline issue, is if your base64 string doesn't end in double equals. Examples:

echo -n 'asdf' | base64
YXNkZg==

echo 'asdf' | base64 
YXNkZgo=