Fix tracing dinov2

[amyeroberts]

What does this PR do?

Tracing currently fails for DinoV2 due to an issue when calling torch's torch._C._nn._upsample_bicubic function through nn.functional.interpolate. There is an issue if the passed in scale_factor is (tensor(float), tensor(float)) so we must convert to a tuple of floats (float, float) as per this PR to enable tracing.

I have run the slow tracing tests to make sure everything works.

tests/models/dinov2/test_modeling_dinov2.py::Dinov2ModelTest::test_torchscript_simple <- tests/test_modeling_common.py PASSED                                                                                                                                              [ 33%]
tests/models/dinov2/test_modeling_dinov2.py::Dinov2ModelTest::test_torchscript_output_hidden_state <- tests/test_modeling_common.py PASSED                                                                                                                                 [ 66%]
tests/models/dinov2/test_modeling_dinov2.py::Dinov2ModelTest::test_torchscript_output_attentions <- tests/test_modeling_common.py PASSED                                                                                                                                   [100%]

The following now works:

import torch
from transformers import AutoImageProcessor, AutoModel
from PIL import Image
import requests

url = 'http://images.cocodataset.org/val2017/000000039769.jpg'
image = Image.open(requests.get(url, stream=True).raw)

processor = AutoImageProcessor.from_pretrained('facebook/dinov2-base')
model = AutoModel.from_pretrained('facebook/dinov2-base')

inputs = processor(images=image, return_tensors="pt")
outputs = model(**inputs)
last_hidden_states = outputs[0] #.last_hidden_state

# We have to force return_dict=False for tracing
model.config.return_dict = False

with torch.no_grad():
    traced_model = torch.jit.trace(model, [inputs.pixel_values])
    traced_outputs = traced_model(inputs.pixel_values)

print((last_hidden_states - traced_outputs[0]).abs().max())

Note: although the model outputs are close, they still have a significant absolute difference on the order of ~1e-4.

/Users/amyroberts/code/transformers/src/transformers/models/dinov2/modeling_dinov2.py:162: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if num_channels != self.num_channels:
/Users/amyroberts/code/transformers/src/transformers/models/dinov2/modeling_dinov2.py:94: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if num_patches == num_positions and height == width:
/Users/amyroberts/code/transformers/src/transformers/models/dinov2/modeling_dinov2.py:104: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  patch_pos_embed = patch_pos_embed.reshape(1, int(math.sqrt(num_positions)), int(math.sqrt(num_positions)), dim)
/Users/amyroberts/code/transformers/src/transformers/models/dinov2/modeling_dinov2.py:108: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  scale_factor=(float(height / math.sqrt(num_positions)), float(width / math.sqrt(num_positions))),
/Users/amyroberts/code/transformers/src/transformers/models/dinov2/modeling_dinov2.py:112: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if int(height) != patch_pos_embed.shape[-2] or int(width) != patch_pos_embed.shape[-1]:
/Users/amyroberts/code/transformers/src/transformers/models/dinov2/modeling_dinov2.py:112: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if int(height) != patch_pos_embed.shape[-2] or int(width) != patch_pos_embed.shape[-1]:
/Users/amyroberts/opt/miniconda3/envs/ml/lib/python3.10/site-packages/torch/jit/_trace.py:1093: TracerWarning: Output nr 1. of the traced function does not match the corresponding output of the Python function. Detailed error:
Tensor-likes are not close!

Mismatched elements: 1693 / 197376 (0.9%)
Greatest absolute difference: 0.00012087821960449219 at index (0, 46, 415) (up to 1e-05 allowed)
Greatest relative difference: 0.4337851929092805 at index (0, 206, 249) (up to 1e-05 allowed)
  _check_trace(
/Users/amyroberts/opt/miniconda3/envs/ml/lib/python3.10/site-packages/torch/jit/_trace.py:1093: TracerWarning: Output nr 2. of the traced function does not match the corresponding output of the Python function. Detailed error:
Tensor-likes are not close!

Mismatched elements: 5 / 768 (0.7%)
Greatest absolute difference: 1.6689300537109375e-05 at index (0, 688) (up to 1e-05 allowed)
Greatest relative difference: 0.0002756976223801738 at index (0, 6) (up to 1e-05 allowed)
  _check_trace(
tensor(0.0001, grad_fn=<MaxBackward1>)

Fixes #27537

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests? No - but tests added

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[ArthurZucker]

Nice catch! 🤗
Would it make sense to add the tracing snippet to the dinov2.md ? 🤗

[amyeroberts]

@ArthurZucker - nice idea, I'll add it!

[amyeroberts]

@ArthurZucker WDYT of this addition to the model page?

[ArthurZucker]

Love it!
We often have people asking how xx model can be compiled / traced so great to be careful when we add support for this! 😉