How to use a quantized CNN in OpenVX 1.3 implementation for the Raspberry Pi?

OpenVX
1

hi all,

I am trying to use the Khronos sample implementation of OpenVX 1.3 for Raspberry Pi and I have the following questions:

  1. I want to implement on a Raspberry Pi 3B an application (that will be fed with a simple CNN trained in Tensorflow Keras) using the sample implementation of OpenVX 1.3 from KhronosGroup, together with the Neural Network extension. The weights and activations of the Keras model are 32-bit floats and so are not supported in this implementation of OpenVX. I quantized the model using Tensorflow Lite Converter, which resulted in a tflite file with uint8- and int32-typed tensors. My first question is: How can I map (if that’s possible) these tensors to the int8 implementation of OpenVX?

  2. I converted the above tflite model to NNEF format in order to use the parser provided in NNEF-Tools to import the quantized model into the OpenVX-based application. Unfortunately the converted model to NNEF format is not parsed by the parser provided in NNEF-Tools. The resulting graph contains a “tflite_quantize” compound fragment and it seems that NNEF does not support the tflite quantization scheme (see support for QUANTIZE and DEQUANTIZE operations · Issue #118 · KhronosGroup/NNEF-Tools · GitHub). In NNEF-1.0 specs I read that a “linear_quantize” operation is specified (also as compound fragment), so the question is: would it be possible to use this “linear_quantize” operation somehow instead of “tflite_quantize”? I am newbie in quantization, am I missing or misunderstanding something?

  3. Is the above approach viable using current development of both NNEF-tools and this OpenVX sample implementation? Is there a misconception in using OpenVX NN extension with a quantized CNN?

Thanks

2

@antoine_naud while I look into how to run a NN model on RPi. You can look through the sample of how to run NN models with OpenVX.

3

@Kiriti Thank you very much for your answer to my question.
I installed ROCm and the MIVisionX toolkit manually, though I cannot use ROCm since I don’t have AMD GPU, I am not sure if my CPU is supported (Intel Core i7-8750H). I could not convert my model from NNEF to NNIR format due to errors but I converted it to ONNX and then to NNIR without problem, and finally to generated the C code for OpenVX. I built the anntest app but I get an error at runtime:

ERROR: agoCreateDataFromDescription: agoGetDataFromDescription(tensor:4,{3,64,64,0},FLOAT32,0) failed

Anyway, AMD’s sample implementation of OpenVX does not solve my problem as it does not support the Raspberry Pi, and as far as I can see supports 32-bit floats for tensors.

4

@antoine_naud I have created a project to convert NN Models to OpenVX Graph using MIVisionX, the code generated can be linked to OpenVX Sample Implementation.

The Sample Implementation has a FP16 option which is still experimental, I am working on linking the quantized FP16 OpenVX code and run on RPi. This is still work in progress.

5

This topic was automatically closed 183 days after the last reply. New replies are no longer allowed.