[FX] Dynamic Shapes Support

[anzr299]

Changes

Modify NNCF Graph Builder for FX backend to correctly get and insert the dynamic shapes into NNCFGraph

Reason for changes

To support quantization of Torch FX models exported with dynamic shapes

Tests

test is added to tests/torch/fx/test_models.py in test_quantized_models(). Currently only the synthetic transformer is tested because torch.export.dynamic_shapes.Dim.DYNAMIC is not supported in pytorch but is supported in upcoming releases. https://pytorch.org/tutorials/intermediate/torch_export_tutorial.html#constraints-dynamic-shapes

test_dynamic_edge() is also added in tests/torch/fx/test_models.py to check that the tensor shape in NNCF Graph edge has values only of type int or str and not SymInt.

[daniil-lyakhov]

Looks like some test cases are duplicated: non synthetic transformer models are still being checked twice with the static shapes. Please refactor the test to prevent redundant cases, and please try to avoid pytest.skip calls as well

[anzr299]

Would you suggest I rather remove the graph tests and only retain test_dynamic_edge() test since most of the transformer models in this test are not correctly traced with dynamic shapes

[daniil-lyakhov]

NNCFGraphEdge and NNCF algorithms expect the edge shape to be a List[int] https://github.com/openvinotoolkit/nncf/blob/develop/nncf/common/graph/graph.py. The use of an str as a shape dimension could lead to unexpected errors in the algorithms: what would happened if a channel dim will be "s1" for a per-channel quantization?

Could you please show an example with a quantization of a dynamic model? Perhaps we can cover the case with less drastic change (using -1 as a shape dim, for example)

[anzr299]

NNCFGraphEdge and NNCF algorithms expect the edge shape to be a List[int] https://github.com/openvinotoolkit/nncf/blob/develop/nncf/common/graph/graph.py. The use of an str as a shape dimension could lead to unexpected errors in the algorithms: what would happened if a channel dim will be "s1" for a per-channel quantization?

Could you please show an example with a quantization of a dynamic model? Perhaps we can cover the case with less drastic change (using -1 as a shape dim, for example)

I can add an exception for the case where the in_channel has a non-static .
using integer such as -1 could cause more complex dynamic shape definition (for example s0//2*3+s1) in int to resolve to another value. Would you recommend simply catching the known cases? or to brainstorm on other ways to represent the dynamic shapes