-
Notifications
You must be signed in to change notification settings - Fork 39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Socket sources: Improvements for internal buffers #2230
Socket sources: Improvements for internal buffers #2230
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Note: Probably to avoid duplication of the logic using the buffer for UDP/TCP cases would be an improvement.
I've created two tests to demonstrate two issues:
|
237fed6
to
00926c9
Compare
|
* Increase the capacity of the internal buffer in socket byte sources to 1MB instead of the default value 8KB. * Add Checks for socket byte source to avoid filling the internal buffer if the server is sending the data with high frequency because the producer loop will call load on each iteration while some parsers will parse one item only on each iteration. * Call flush on consume when there is not enough writable memory. * Another check if the buffer is filled and return an error if we can't add all the incoming bytes into the buffer. * Provide unit test to ensure socket byte sources won't fill out the internal buffer. * One unit test demonstrates that parsers not consuming all the memory loaded into udp byte source buffer.
00926c9
to
e8b165e
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
just one comment
This PR includes: