[QNN EP] Initialization errors - unsupported ops BatchNormalization, Sqrt, Reciprocal

[uros-ms]

When I try to run inference using QNN EP, I get initialization errors (below) which seems to originate from unsupported operations: BatchNormalization, Sqrt, and Reciprocal. According to QNN docs and ONNX-QNN EP docs, these operations are supported (in QNN docs Reciprocal and Div maps to the same operation ElementWiseDivide). The model quantization uses int8 precision for all ops.

The model is trained in Tensorflow 2, then converted to ONNX using TF2ONNX, the unsupported ops correspond to tf.keras.layers.BatchNormalization() and tf.nn.l2_normalize().

Framework ONNX 1.19.0

QNN.backendValidateOpConfig() failed for node StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/align/l2_normalize_1/Rsqrt of type ElementWiseSquareRoot with error code 3110

QNN.backendValidateOpConfig() failed for node StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/align/l2_normalize/Rsqrt of type ElementWiseSquareRoot with error code 3110

Unsupported nodes in QNN EP: { name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/align/l2_normalize_1/Rsqrt, type: Sqrt }{ name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/align/l2_normalize/Rsqrt, type: Sqrt }{ name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/align/l2_normalize_1/Rsqrt__274, type: Reciprocal }{ name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/align/l2_normalize/Rsqrt__317, type: Reciprocal }{ name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/dec07_bn/FusedBatchNormV3, type: BatchNormalization }{ name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/dec06_bn/FusedBatchNormV3, type: BatchNormalization }{ name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/dec05_bn/FusedBatchNormV3, type: BatchNormalization }{ name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/dec04_bn/FusedBatchNormV3, type: BatchNormalization }{ name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19
QNN.backendValidateOpConfig() failed for node StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/align/l2_normalize_1/Rsqrt of type ElementWiseSquareRoot with error code 3110

QNN.backendValidateOpConfig() failed for node StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/align/l2_normalize/Rsqrt of type ElementWiseSquareRoot with error code 3110

Unsupported nodes in QNN EP: { name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/align/l2_normalize_1/Rsqrt, type: Sqrt }{ name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/align/l2_normalize/Rsqrt, type: Sqrt }{ name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/align/l2_normalize_1/Rsqrt__274, type: Reciprocal }{ name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/align/l2_normalize/Rsqrt__317, type: Reciprocal }{ name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/dec07_bn/FusedBatchNormV3, type: BatchNormalization }{ name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/dec06_bn/FusedBatchNormV3, type: BatchNormalization }{ name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/dec05_bn/FusedBatchNormV3, type: BatchNormalization }{ name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/dec04_bn/FusedBatchNormV3, type: BatchNormalization }{ name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19
Starting stage: Graph Preparation Initializing
Completed stage: Graph Preparation Initializing (443 us)
Starting stage: Graph Transformations and Optimizations
Completed stage: Graph Transformations and Optimizations (44459 us)
Starting stage: Graph Sequencing for Target
Completed stage: Graph Sequencing for Target (3326 us)
Starting stage: VTCM Allocation
Completed stage: VTCM Allocation (4740 us)
Starting stage: Parallelization Optimization
Completed stage: Parallelization Optimization (506 us)
Starting stage: Finalizing Graph Sequence
Completed stage: Finalizing Graph Sequence (1060 us)
Starting stage: Completion
Completed stage: Completion (49 us)
Starting stage: Graph Preparation Initializing
Completed stage: Graph Preparation Initializing (75 us)
Starting stage: Graph Transformations and Optimizations
Completed stage: Graph Transformations and Optimizations (240 us)
Starting stage: Graph Sequencing for Target
Completed stage: Graph Sequencing for Target (164 us)
Starting stage: VTCM Allocation
Completed stage: VTCM Allocation (16 us)
Starting stage: Parallelization Optimization
Completed stage: Parallelization Optimization (12 us)
Starting stage: Finalizing Graph Sequence
Completed stage: Finalizing Graph Sequence (33 us)
Starting stage: Completion
Completed stage: Completion (4 us)
Starting stage: Graph Preparation Initializing
Completed stage: Graph Preparation Initializing (159 us)
Starting stage: Graph Transformations and Optimizations
Completed stage: Graph Transformations and Optimizations (397 us)
Failed to finalize QNN graph.
Skip invalid vtcm_mb: 0
failed to initialize onnx model

[uros-ms]

Following HectorSVC's suggestion from issue #22819 to enable verbose log, I got these reasons why validation failed:

• BatchNormalization:
  Validation FAILED for nodes (NodeUnit):
  Operator type: BatchNormalization Node name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/dec07_bn/FusedBatchNormV3 Node index: 939
  REASON : batch_norm_op_builder.cc:466 onnxruntime::qnn::BatchNormOpBuilder::IsOpSupported QNN BatchNorm doesn't support dynamic scale.
• Sqrt:
  Validation FAILED for nodes (NodeUnit):
  Operator type: DequantizeLinear Node name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/align/l2_normalize_1/Maximum:0_DequantizeLinear Node index: 523
  Operator type: Sqrt Node name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/align/l2_normalize_1/Rsqrt Node index: 525
  Operator type: QuantizeLinear Node name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/align/l2_normalize_1/Rsqrt:0_QuantizeLinear Node index: 527
  REASON : base_op_builder.cc:162 onnxruntime::qnn::BaseOpBuilder::ProcessOutputs Failed to add node.
• Reciprocal:
  Validation FAILED for nodes (NodeUnit):
  Operator type: Reciprocal Node name: StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/align/l2_normalize_1/Rsqrt__274 Node index: 531
  REASON : qnn_node_group.cc:39 onnxruntime::qnn::QnnNodeUnitWrapper::IsSupported Operators of type Reciprocal are not supported by QNN EP.Reciprocal node StatefulPartitionedCall/model_v203h2d_combined_r_t20i19o/align/l2_normalize_1/Rsqrt__274 will not be assigned to QNN EP.