Feature/multiprocess #19
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
|
One issue with tensorpipe shared mem channel is that both sides of the pipe busy wait, and repeatedly call sched_yield as mentioned here. The flame graph for one side of the pipe where the TP_SHM_Reactor is eating an entire core busy waiting looks like this: |
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
jishminor
force-pushed
the
feature/multiprocess
branch
2 times, most recently
from
631e840 to
9efcdfe
Compare
Signed-off-by: Josh Minor <[email protected]>
jishminor
force-pushed
the
feature/multiprocess
branch
from
9efcdfe to
0ba5134
Compare
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Signed-off-by: Josh Minor <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a large overhaul of the backend, changing the implementation to be all contained within the tritonserver process, to splitting out and managing a child process per tflite model instance. This PR closes #5 as there will be a copy of the ACL scheduler singelton instance per model now.
Tensorpipe is used to manage the transport of the input tensor data to the respective model instance processes, and as it's written, only one memcpy into a shared memory channel is needed to do this, meaning the overhead is minimal.