Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cli: Add s3-min-part-size flag #1206

Merged
merged 2 commits into from
Oct 22, 2024
Merged

cli: Add s3-min-part-size flag #1206

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
08bd60c
allow setting s3 min part size in cli
wiemann Oct 12, 2024
60890fb
Apply suggestions from code review
Acconut Oct 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cmd/tusd/cli/composer.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ func CreateComposer() {
store := s3store.New(Flags.S3Bucket, s3Client)
store.ObjectPrefix = Flags.S3ObjectPrefix
store.PreferredPartSize = Flags.S3PartSize
store.MinPartSize = Flags.S3MinPartSize
store.MaxBufferedParts = Flags.S3MaxBufferedParts
store.DisableContentHashes = Flags.S3DisableContentHashes
store.SetConcurrentPartUploads(Flags.S3ConcurrentPartUploads)
Expand Down
4 changes: 3 additions & 1 deletion cmd/tusd/cli/flags.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ var Flags struct {
S3Bucket string
S3ObjectPrefix string
S3Endpoint string
S3MinPartSize int64
S3PartSize int64
S3MaxBufferedParts int64
S3DisableContentHashes bool
Expand Down Expand Up @@ -129,7 +130,8 @@ func ParseFlags() {
f.StringVar(&Flags.S3Bucket, "s3-bucket", "", "Use AWS S3 with this bucket as storage backend (requires the AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_REGION environment variables to be set)")
f.StringVar(&Flags.S3ObjectPrefix, "s3-object-prefix", "", "Prefix for S3 object names")
f.StringVar(&Flags.S3Endpoint, "s3-endpoint", "", "Endpoint to use S3 compatible implementations like minio (requires s3-bucket to be pass)")
f.Int64Var(&Flags.S3PartSize, "s3-part-size", 50*1024*1024, "Size in bytes of the individual upload requests made to the S3 API. Defaults to 50MiB (experimental and may be removed in the future)")
f.Int64Var(&Flags.S3PartSize, "s3-part-size", 50*1024*1024, "Preferred size in bytes of the individual upload requests made to the S3 API. Defaults to 50MiB (experimental and may be removed in the future)")
f.Int64Var(&Flags.S3MinPartSize, "s3-min-part-size", 5*1024*1024, "Minimum size in bytes of the individual upload requests made to the S3 API. Must not be lower than S3's limit. Defaults to 5MiB.")
f.Int64Var(&Flags.S3MaxBufferedParts, "s3-max-buffered-parts", 20, "Size in bytes of the individual upload requests made to the S3 API. Defaults to 50MiB (experimental and may be removed in the future)")
f.BoolVar(&Flags.S3DisableContentHashes, "s3-disable-content-hashes", false, "Disable the calculation of MD5 and SHA256 hashes for the content that gets uploaded to S3 for minimized CPU usage (experimental and may be removed in the future)")
f.BoolVar(&Flags.S3DisableSSL, "s3-disable-ssl", false, "Disable SSL and only use HTTP for communication with S3 (experimental and may be removed in the future)")
Expand Down