[fix] Extend when a model repository/directory already has an exported OV model

[tomaarsen]

What does this PR do?

This PR extends when a model repository/directory already has an exported OV model.

This is required to load e.g. openvino_model_qint8_quantized.xml from https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2. The only reason that it's possible to load this model is because this repository also has openvino_model.xml, so the export isn't forcibly set to True.

I didn't update any tests as it's a bit niche.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

---

cc @l-bat @AlexKoff88 as you worked on the Sentence Transformers OV quantization function, which is affected by this issue.

• Tom Aarsen

[eaidova]

@tomaarsen thanks, changes looks good to me. Could you please provide test case for making sure that this behaviour will not break in future?

[eaidova]

Also will be helpful, if you can provide example of usage model with optimum-intel in this PR comment or description for such case If I right understand model initialization will require some additional parameters in this case? Even if there 2 models openvino_model.xml and openvino_model_int8_quant.xml, how we can make sure which of them will be loaded by default?

[tomaarsen]

Certainly, here is how you'd use it:

model = OVModelForFeatureExtraction.from_pretrained(
    "sentence-transformers/all-MiniLM-L6-v2",
    export=False,
    subfolder="openvino",
    file_name="openvino_model_qint8_quantized.xml",
)
# tokenize, model forward, etc. as normal

Indeed, both subfolder and file_name are required. file_name="openvino/openvino_model_qint8_quantized.xml" does not work either. If you omit file_name, then it will default to openvino_model.xml if it exists.

and in Sentence Transformers:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer(
    "sentence-transformers/all-MiniLM-L6-v2",
    backend="openvino",
    model_kwargs={"file_name": "openvino_model_qint8_quantized.xml"},

Hope that clears it up.

• Tom Aarsen

[eaidova]

@tomaarsen thanks, I'm wondering will not usage of file_name (if provided) in file search be more robust in this case? there is only some nonverbal convention to start names with openvino, but on practice it is not obligated. Also do you not tried to use variant instead of file_name?

[AlexKoff88]

@tomaarsen, can you please fix the test?
self = <test_modeling.OVModelIntegrationTest testMethod=test_find_files_matching_pattern_with_quantized_ov_model>

def test_find_files_matching_pattern_with_quantized_ov_model(self):
    # This model only has "openvino/openvino_model_qint8_quantized.xml" and "openvino/openvino_model_qint8_quantized.bin"
    # We want to ensure that this model is found, so the `export` isn't forced to True
    model_id = "sentence-transformers-testing/stsb-bert-tiny-openvino-quantized-only"
    pattern = r"(.*)?openvino(.*)?\_model(.*)?.xml$"
    # hub model
    ov_files = _find_files_matching_pattern(model_id, pattern=pattern)

  self.assertTrue(len(ov_files) == 1)

E AssertionError: False is not true

[tomaarsen]

@AlexKoff88 Done! Apologies, in my rush I forgot about adding the subfolder argument.

@eaidova It's possible that there's a better solution - I just went with one that was closest to the current implementation to avoid large changes while still fixing my issue.

• Tom Aarsen

[AlexKoff88]

@tomaarsen, can you please rebase to make the documentation build green as well? Thank you!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.