Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why the return of func Read() with a FileReader is not consistent? #329

Open
YYX413 opened this issue Aug 18, 2023 · 0 comments
Open

Why the return of func Read() with a FileReader is not consistent? #329

YYX413 opened this issue Aug 18, 2023 · 0 comments

Comments

@YYX413
Copy link

YYX413 commented Aug 18, 2023

hello colinmarc, I am encountering a problem with read specific amount of bytes ,say 128MB, from a FileReader.

While the file is 1000MB, I tried to read the file by a loop and 128MB for each iteration except the last one;

but I found that each iteration can only returned 128KB;

and I tried to read 100KB by a loop;

but I found that not each iteration return 100KB consitently, some calls only returned 8192KB;

I found an annotation in block_read_stream.go:

// Always align reads to a chunk boundary. This makes the code much simpler,
// and with readers that pick sane read sizes (like io.Copy), should be
// efficient.

I am wondering if the phenomenon of Read inconsitently is related to the chunk boundaring mentioned above, if it is, I aslo wanna know how that make io.Copy more efficient?

I tried ReadAt also, which can make my need satisfied, but I am still curious about the underlying logic of func Read, thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant