Tensorflow 2.16 support

[jmmanso]

Hi, when can we expect NNCF to support TF 2.16? It's an important version since it comes with Keras 3. Thanks

[MaximProshin]

Hi @jmmanso ! There is no plan yet as we don't see a wide adoption of TF path in NNCF. Can you please share some information on whether you use it or plan to use it and how?
cc @AlexKoff88

[jmmanso]

Hi Maxim. We're Planet, a large provider of satellite imagery and derived data, and use TF + NNCF extensively in production. Most of our computer vision models are small CNNs and we run them on Intel CPUs over large pools of nodes. We use NNCF to compile and quantize our TF models, and it has been more effective than any other approach we've tried, reducing our inference costs very significantly. Thus, if NNCF stops supporting TF, that would be a problem for us.
Thanks

[MaximProshin]

Thanks for your reply! Did I get it right that you specifically use the Quantization-Aware Training of NNCF for TF models and then convert such models to OpenVINO? Just want to better understand your pipeline.

[jmmanso]

we actually just use post-training quantization, mostly the nncf.quantize method on a Keras/TF model

[MaximProshin]

@jmmanso , for PTQ we recommend OV path, i.e. TF->OV->NNCF->int8_OV, in that case there wouldn't be any dependency on TF version in NNCF (see more in https://docs.openvino.ai/2024/openvino-workflow/model-optimization-guide/quantizing-models-post-training/basic-quantization-flow.html). Please consider that option.
Anyway, we also consider to support TF 2.18, that was just released, in future NNCF versions but I don't have clear plans yet.