Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aws_base64_compute_encoded_len() is now exact, doesn't add 1 extra for null-terminator #1188

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

graebm
Copy link
Contributor

@graebm graebm commented Feb 14, 2025

Issue:

One of the core principles of aws_byte_buf and aws_byte_cursor, is you DO NOT assume there's a null-terminator, because null-terminators, and the resulting confusion between string "length" and "size" have led to so many bugs in the long history of C.

But aws_base64_encode() tried to be "nice" and add a secret null-terminator after .len, but before .capacity. To achieve this, aws_base64_compute_encoded_len() would say it needed 1 more byte than necessary. This caused trouble, catching Dmitriy off guard the other day. Searching for uses, I see aws-crt-cpp also once had a bug due to this (fixed in this PR)

In changing this, I found that aws_hex_encode() did similar, but even worse. It always added a null-terminator, and included the null-terminator in the .len! Bad! Fortunately, this function was never used except in generating random data for tests. When Bret needed a function like this in 2019 for signing, he avoided this function entirely and built his own alternate version that doesn't add a null-terminator (it has other differences too).

Description of changes:

  • aws_hex_encode() doesn't add null-terminator anymore
    • aws_hex_compute_encoded_len() updated to account for this
  • aws_base64_encode() doesn't add null-terminator anymore
    • aws_base64_compute_encoded_len() updated to account for this
  • aws_base64_compute_decoded_len() math adjusted so that overflow is impossible
  • document some math that confused me
  • stop using the term "size" in tests, we care about "len"

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@graebm graebm changed the title Knock it off with this null terminator nonsense aws_base64_compute_encoded_len() is now exact, doesn't add 1 extra for null-terminator Feb 14, 2025
source/encoding.c Show resolved Hide resolved
@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.21%. Comparing base (6401c83) to head (f1676e8).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1188   +/-   ##
=======================================
  Coverage   84.21%   84.21%           
=======================================
  Files          57       57           
  Lines        5979     5973    -6     
=======================================
- Hits         5035     5030    -5     
+ Misses        944      943    -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants