ValueError: Please either set all three 'ordered_module_input_names', 'ordered_module_output_names' and 'quant_layers_dict' or none of them.

[fabecode]

Summary

Hello, I am trying to create a custom ScatterElements ONNX node in Concrete ML.

After the conversion of the graph nodes from ONNX to NumPy, I face the following error: ValueError: Please either set all three 'ordered_module_input_names', 'ordered_module_output_names' and 'quant_layers_dict' or none of them. from concrete/ml/quantization/quantized_module.py", line 109. Below is a snippet of where the ValueError was raised:

 all_or_none_params = [
            ordered_module_input_names,
            ordered_module_output_names,
            quant_layers_dict,
        ]
  if not (
            all(v is None or v == {} for v in all_or_none_params)
            or not any(v is None or v == {} for v in all_or_none_params)
        ):
            raise ValueError(
                "Please either set all three 'ordered_module_input_names', "
                "'ordered_module_output_names' and 'quant_layers_dict' or none of them."
            )

Upon inspection, the variables hold the following values:

• ordered_module_input_names: <generator object ONNXConverter.quantize_module.<locals>.<genexpr> at 0x7f0168245660>
• ordered_module_output_names: <generator object ONNXConverter.quantize_module.<locals>.<genexpr> at 0x7f0168245740>
• quant_layers_dict: {}

Do you have any suggestions on what went wrong and any tips on how to debug above? What is the input and output of above supposed to look like? Thank you!

[fd0r]

Hello @fabecode , would you mind sharing a link to the current state of your modifications?

It would make it easier for us to help you with access to the code changes you made.

[fabecode]

Hi @fd0r,

Here is the link to my GitHub repository: fabecode/GraphML-FHE. You could view my quantised GNN model in the GINe_FHE class of models.py, and the the state of my Concrete-ML library modifications in the commit history of my local concrete-ml package. If you would like to run the GNN model, you could follow the quick command in gnn_fhe_pipline README.md.

Upon further investigation of the ValueError in quantized_module.py, it seems that the root cause stems from the _quantize_layers function in post_training.py. The has_variable_inputs in line 580 is False, leading to the if block being skipped over, and quant_ops_dict hence being empty.

Any advice or suggestions would be really appreciated. Thank you so much!

[RomanBredehoft]

Hello @fabecode ,
We are currently taking a look at it, we will get back to you once we have some news about it 🙂

Thanks for your patience !

[fabecode]

Thank you @RomanBredehoft. Appreciate your help!

In relation to my project, I have also just applied for the Zama Grant program at zama-ai/bounty-program#119.

Looking forward to hearing from you again!

[Frollamma]

Any update on this? I have the same problem. @fabecode If you solved the problem, would you mind sharing your solution?

[Frollamma]

If this is helpful to anyone, this is what I found out. This error happens in the function optimize of onnxoptimizer/__init__.py, specifically here:

model_str = model.SerializeToString()
if fixed_point:
    optimized_model_str = C.optimize_fixedpoint(model_str, passes)
else:
    optimized_model_str = C.optimize(model_str, passes)

return onnx.load_from_string(optimized_model_str)

What I noticed is that if you check the model graph before this operation it has a positive number of nodes, inputs and outputs, while after they are all 0... So what I did is just replace this code with return model. It worked, but then I got a different error because some ONNX operators are not yet implemented in concrete ML.

I want just to mention that before getting to this error I also had some warnings and I think some problem with the IR version of the ONNX model that I bypassed, so probably the onnxoptimizer library didn't expect to handle this case.