Token Object Contracts for policy-based usage of Hugging Face Models #57
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…. Change enables other use cases to re-use the exchange op_initialize method, while passing in use-case specific initiazation arguments via kwargs Signed-off-by: Prakash Narayana Moorthy <[email protected]>
…s. The token object implements the policy for accessing HF hosted models. The token object code inherits several base methods from the exchange contract token object, and only implements asset-use specific methods. In addition, the token object initialization method is used to store asset (HF model) specifc details, so of which are secrets, and others used to provide meta data regarding the asset for a prospective token user Signed-off-by: Prakash Narayana Moorthy <[email protected]>
…s. The Guardian frontend is largely similar to the guardian used as part of the OpenVINO inference contract use-case. The operations folder implements the capability_handler_map unique to the Hugging Face use case. The use_hfmodel module implemented within the operations package enables the guardian server to process inferencing capabilities that invoke REST API calls to the Hugging Face hosted models. Parameters required for the API call are passed as part of the capability package. The module implements support for JSON and binary payloads. The module is model agnostic, and does not implement any model specific pre or post processing steps. The large overlap among the remaining modules of the HF and OpenVINO guardians calls for a future PR that refactors the modules to permit reuse. Currently, simply expecting the HF use case to reuse the OpenVINO guardian frontend python packages is challenging, since OpenVINO guardian demands intallation of dependencies such an tensoflow, opencv, numpy etc, none of which are required for the HF usecase. In a secure deployment, the guardian is ideally deployed with a TEEs, and hence it's best to have a SW footprint that is minimal. Signed-off-by: Prakash Narayana Moorthy <[email protected]>
…rdian plugin is largely same as the guardian plugin for the openvino inference use case. A future PR that permits refactoring, and reuse of modules among the HF and OpenVINO use cases needs to be explored. 2. installation related files such as setup.py, MANIFEST, etc. 3. test script that needs to be manually invoked to test the HF use case. To run the test script, the user must create an HF account, and obtain an HF authentication token, and set HF_AUTH_TOKEN environment variable to the token value. Due to this external depdency, test is currently not integrated as part of the 'make test' automatic test suite. The test is done using the gpt2 opensource model available on Hugging Face. Signed-off-by: Prakash Narayana Moorthy <[email protected]>
Signed-off-by: Prakash Narayana Moorthy <[email protected]>
|
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR provides a PoC for tokenization and policy-based access of (possibly private) models hosted on Hugging Face, and made available for inferencing via Serverless Inference API. The PoC in spirit is similar to what is demonstrated via the inference contract family, where we provided PDO contracts/guardians for policy-based access control of ML models hosted via OpenVINO model server.
Please see hfmodels-contract/README.md for problem statement, solution overview, and details on how to test the PoC.