Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add configurable user agent prefix flag for mount-s3 #548

Merged
merged 7 commits into from
Oct 19, 2023

Conversation

dannycjones
Copy link
Contributor

Description of change

If mountpoint-s3 is run as part of a larger application, user/customer may want to configure a prefix for the user agent to indicate S3 requests came from that particular application.

This change adds a new --user-agent-prefix <PREFIX> flag to achieve this.

Relevant issues: #546

Does this change impact existing behavior?

No change to existing behaviour. Users can now add a prefix to the HTTP 'User-Agent' header.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and I agree to the terms of the Developer Certificate of Origin (DCO).

@dannycjones dannycjones had a problem deploying to PR integration tests October 9, 2023 17:23 — with GitHub Actions Failure
@dannycjones dannycjones temporarily deployed to PR integration tests October 9, 2023 17:23 — with GitHub Actions Inactive
@dannycjones dannycjones had a problem deploying to PR integration tests October 9, 2023 17:23 — with GitHub Actions Failure
@dannycjones dannycjones temporarily deployed to PR integration tests October 9, 2023 17:23 — with GitHub Actions Inactive
@dannycjones
Copy link
Contributor Author

Tested this locally by pointing mount-s3 at a local HTTP server. Looks good.

@dannycjones dannycjones linked an issue Oct 9, 2023 that may be closed by this pull request
sauraank
sauraank previously approved these changes Oct 9, 2023
Copy link
Contributor

@sauraank sauraank left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
We just need to see why the tests are failing. The ASAN failure did not repeat when I repeat the Address sanitizer job on my PR. But, both the tests are failing in other PRs as well.

@dannycjones dannycjones temporarily deployed to PR integration tests October 16, 2023 17:17 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests October 16, 2023 17:17 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests October 16, 2023 17:17 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests October 16, 2023 17:17 — with GitHub Actions Inactive
@dannycjones
Copy link
Contributor Author

This change should update the changelog: https://github.com/awslabs/mountpoint-s3/blob/main/mountpoint-s3/CHANGELOG.md

I'd like to update it before merging this.

@dannycjones dannycjones marked this pull request as draft October 16, 2023 17:24
Signed-off-by: Daniel Carl Jones <[email protected]>
@dannycjones dannycjones temporarily deployed to PR integration tests October 17, 2023 14:58 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests October 17, 2023 14:58 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests October 17, 2023 14:58 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests October 17, 2023 14:58 — with GitHub Actions Inactive
@dannycjones dannycjones marked this pull request as ready for review October 17, 2023 14:58
@dannycjones
Copy link
Contributor Author

This will conflict with #556. Prefer to merge in #556, rebase this, and merge.

@dannycjones dannycjones temporarily deployed to PR integration tests October 18, 2023 10:44 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests October 18, 2023 10:44 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests October 18, 2023 10:44 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests October 18, 2023 10:44 — with GitHub Actions Inactive
@dannycjones dannycjones requested a review from sauraank October 18, 2023 10:44
@dannycjones dannycjones enabled auto-merge October 18, 2023 10:44
mountpoint-s3/CHANGELOG.md Outdated Show resolved Hide resolved
value_name = "PREFIX",
help_heading = CLIENT_OPTIONS_HEADER
)]
pub user_agent_prefix: Option<String>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kind of think this should be an undocumented option. It's really only useful for us right now, right? Not sure it helps customers for it to show up in --help. But it would be our first undocumented option. Maybe an alternative is a new help section for "advanced options" or something, to make clear this isn't something most customers who aren't AWS would ever need to use.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking it may be of value if third-parties wanted to include mount-s3 in their project/product, and include the user agent in their request.

I like the idea of moving it under an advanced options, so I'll update with that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually less comfortable with moving it into a new "Advanced options", since someone looking for this kind of option would expect it to be with "Client options". Our intention is also for things like --max-threads <> and --part-size <> to be things most customers don't need to configure either.

At the moment, it shows up at the top of the list (since its based on the order it is defined). How do you feel about moving it to the bottom of the "Client options" list like below?

Client options:
      --maximum-throughput-gbps <N>  Maximum throughput in Gbps [default: auto-detected on EC2 instances, 10 Gbps elsewhere]
      --max-threads <N>              Maximum number of FUSE daemon threads [default: 16]
      --part-size <PART_SIZE>        Part size for multi-part GET and PUT [default: 8388608]
      --user-agent-prefix <PREFIX>   Configure a string to be prepended to the 'User-Agent' HTTP request header for all S3 requests

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is anyone really going to be looking for it? Only AWS really gets to see the user agent. Unlike the other options, this is something a customer would basically never configure — it's really only useful for bundling Mountpoint into something else (and that basically means only useful for us).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, let me move it to an "Advanced options" section at the bottom.

@dannycjones dannycjones temporarily deployed to PR integration tests October 19, 2023 09:17 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests October 19, 2023 09:17 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests October 19, 2023 09:17 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests October 19, 2023 09:17 — with GitHub Actions Inactive
Signed-off-by: Daniel Carl Jones <[email protected]>
@dannycjones dannycjones temporarily deployed to PR integration tests October 19, 2023 14:56 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests October 19, 2023 14:56 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests October 19, 2023 14:56 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests October 19, 2023 14:56 — with GitHub Actions Inactive
@dannycjones dannycjones requested review from jamesbornholt and removed request for sauraank October 19, 2023 15:08
@dannycjones dannycjones temporarily deployed to PR integration tests October 19, 2023 15:09 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests October 19, 2023 15:09 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests October 19, 2023 15:09 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests October 19, 2023 15:09 — with GitHub Actions Inactive
@dannycjones dannycjones added this pull request to the merge queue Oct 19, 2023
Merged via the queue into awslabs:main with commit 17f7de5 Oct 19, 2023
@dannycjones dannycjones deleted the user-agent-prefix-cli-flag branch October 19, 2023 17:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow configurable user agent prefix for mountpoint-s3
3 participants