SIMD back-end for IBM Power and Z

[geert56]

This is not really an issue. I'd like to know if there is an interest in incorporating code to support IBM's Power and Z architectures as a back-end. In-house me and a colleague actually worked on this and we have extensions ready for gemmlowp to run optimized on P and Z depending on compiler flags when these architectures are detected. In principle this does not touch or disrupt any of the existing code.
Please comment on this issue and provide advice as to how best proceed.

[bjacob]

it'd be interesting to hear if there is interest from other users. feel free to use the gemmlowp 'google group' to reach more people.
as far as google is concerned we are in the process of migrating to a successor of gemmlowp, named ruy, currently here as a subdirectory of tensorflow, https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/experimental/ruy , but set to move to its own github tree in the near future. it's not 100% ready for wider contribution as documentation is missing but just as a heads-up.

[geert56]

Thank you @bjacob for the reply. Our interest is mainly driven by the use of gemmlowp for quantized models in TensorFlow Lite. We want to have fast inference on Power and Z CPUs. It was an interesting exercise to code up matrix multiplication in Power and Z vector intrinsics. It also helps us to evaluate our SIMD instruction sets. We would like to share our work with the larger community. Pushing it back into mainstream gemmlowp source would simplify a dedicated TF Lite build for us. Is Google's roadmap to drop gemmlowp and adopt "ruy" for TF Lite? Then we might want to have a look at it and consider whether to port our work there. I am curious to hear more from others.

[bjacob]

TFLite has already switched to ruy on arm64. There is work underway on arm32 and x86. However given the complex landscape of inference backends at the moment it's hard to make guesses as to what tflite will end up using. Over the next few months these things should settle a bit.