QNN or FINN as an IP block


Hi Xilinx/Avnet people!

Could someone please provide us with a configurable QNN/FINN IP block to use in our contest designs? I don’t want to have to use HLS to regenerate the IP cores, when it seems the NN projects are already working quite well. This is especially cumbersome as building the projects on Github are currently requires a Linux system.

I want to be able to drop in Convolutional Neural Network IP, and select the configuration for the network topology (variable numbers of convolutional, max pool and fully connected layers), and the quantization level (cnvW1A1, cnvW1A2, cnvW2A2), and the array sizes. Then I could train my own networks, and load the weights in using the Pynq drivers.

Yes, I understand this is a bit of work (although I think the hard work has been done), but for increasing the adoption rate of FPGA boards, allowing us to just use Neural Networks nearly out-of-the-box would really help. Otherwise, the amount of knowledge to get even a simple project up and running is just too much.
What I mean is proficiency in:

  • OpenCV
  • C/C++
  • HLS
  • FPGA
  • HDL
  • Vivado
  • Linux
  • TCL
  • Python
  • Deep Learning

With a drop-in DNN IP, people who know just Python, deep learning and Vivado can start working on accelerated designs, with greatly expands the talent pool.



There is diet and full ChaiDNN and there is also DeePhi both are configurable at run-time NN engines. Neither have been ported over to directly interface with Python yet. But both have the ability to drop in some already trained popular networks. The prior is open source, the latter is proprietary however it has a free SDK and lite version of the engine with some pretty special pruning optimization capabilities.



Thanks for tips!

I had a look at Deephi, it looks really interesting, but then I saw this thread:
It seems that it only works on the standard Pynq board, and although an Ultra96 port may be on the way, it will be too late for the competition.

The same goes for ChaiDNN, it seems interesting, but the compiled versions for UltraScale+ MPSoC are only available for zcu102 or zcu104, not the Ultra96.

I’ll keep looking though, maybe there’s still a way to get CNN’s running on fabric on the Ultra96.



There is a diet version of ChaiDNN that I mentioned, it will fit in the Ultra96 ZU3EG. You have to dig a little into the github instructions, it’s there.

The source is all there for ChaiDNN as well as instructions for how to build it from scratch. You can use the free SDSoC voucher that comes with the Ultra96 kit to fulfill the tool requirements. Of course you will need to also be proficient enough with Caffe as that is what it uses to define the network topology.

As for both of these, the ZU3EG is a ZU+ same as the parts on the ZCU102 and ZCU104 but it is a smaller part and doesn’t have the hard IP for the video codec (which the NNs don’t use). DeePhi has a lite version suited for the smaller parts. Someone has it working on the Z1 PYNQ board which is an even smaller part (7020). https://github.com/hirayaku/DAC2018-TGIIF