Approximating Deep Learning Accelerators – AdequateDL

A framework for precision exploration that brings together the expertise of INRIA, CEA LIST and LIRMM is being developed. This framework, based on N2D2 (Neural Network Design & Deployment), explores the impact of approximations and precision reduction on training and classification of CNNs. N2D2’s backend will also be adapted and used to generate C++ or OpenCL code amenable for high-level synthesis (HLS) of the approximated hardware accelerators. The generated code will take as inputs a library of reduced-precision operators in floating-point and fixed-point arithmetic and will also be optimized through some reduction precision techniques and source-level transformations. Moreover, AdequateDL will also exploit weight quantization methods during learning phase to bring the lowest accuracy loss for a given precision reduction. Finally, the solution will be synthesized and run on a target hardware platform to demonstrate the gains in performance and energy of the automatically generated accelerators.
The developed framework will be validated through a prototype of an FPGA accelerator for CNN. Precision exploration through controlled approximate computing will be the main focus to increase energy efficiency. Comparison with other platforms such as embedded GPU, and existing FPGA implementation of CNNs will be performed.

A library of custom floating-point (FlP) operators was designed and demonstrated the interest in terms of cost-accuracy trade-off of such reduced-precision FlP operators. The ctfloat library was released as opensource. Initial experiments explored the use of the custom FlP operators inside the N2D2 framework.
We also investigated the opportunity of Weight-Sharing (WS). In particular, we focus on how WS can be exploited to reduce the memory footprint of CNNs, what is the best granularity (i.e., network-, layer-, channel- or kernel-wise) to apply it and finally, to develop a framework to allow design space exploration for approximating CNNs. Thanks to the proposed sensitive metrics, we were able to implement a multi objective design space exploration to identify trade-offs between compression and accuracy.
We also integrated a quantization module in N2D2 for post-training quantization module. This method relies on a dataset of representative data points to calculate an approximation of the range of the outputs for each network layer. Further, new features (weight, activation, and threshold quantization) have been fully-integrated in the open-source repository of N2D2. N2D2 was also released to offer the possibility to generates C/HLS code for a quantized DNN. This code opens the way to automatically estimate the cost of a quantized DNN topology on an FPGA.

We expect the work on reduced-precision floating-point operators to be integrated in the near future to the recently developed quantization module for N2D2. We recently started exploring low precision acceleration of DNN training through a PhD funded by the project since October 2020. We are exploring custom number representation based on floating-point and their adaptation during the epochs of the training. The work is progressing fast and we expect relevant results during 2021.
Our efficient exploration method reduced the number of CNN scoring operations required to optimize a weight-sharing for a CNN to a few hundred with respect to the initial. With promising results on small CNNs, we look forward to applying it to larger, state-of-the-art models, such as ResNet and MobileNet, targeting larger datasets such as CIFAR-10/100 and ImageNet.We will explore the use of heuristic methods to avoid the costly scoring step and will, in particular, take a look at using the inertia(sum of the squared error) of the clustering as a proxy for the accuracy loss.
We plan to extend the compression process to include a pruning step.
We plan to evaluate the benefit with state of the art backend, currently exploring Xilinx’s FINN framework for implementing quantized neural network accelerator on FPGA target
We plan to integrate new modules in N2D2 to apply quantization aware training on network parameters. These new features shall balance the loss induced on passing state of the art DNN toward 8-bits integers numerical precision. Moreover, it will be possible to evaluate the performances of these modules on a more aggressive quantization level (<8bits). Evaluation of the robustness and scalability of these methods will be performed on SOTA DNN like MobileNet, Resnet trained on CIFAR10/100 and ImageNet. An evaluation on a segmentation wise task, trained on cityscapes dataset should give a good idea on the robustness of the method on the industrial field of ADAS.

1. E. Dupuis, D. Novo, I. O’Connor and A. Bosio, «On the Automatic Exploration of Weight Sharing for Deep Neural Network Compression,« IEEE/ACM Design, Automation & Test in Europe Conference (DATE), 2020, pp. 1319-1322.
2. E. Dupuis, D. Novo, I. O’Connor and A. Bosio, «Sensitivity Analysis and Compression Opportunities in DNNs Using Weight Sharing,« 23rd International Symposium on Design and Diagnostics of Electronic Circuits & Systems (DDECS), 2020, pp. 1-6.
3. Alberto Bosio, “Security in an Approximated World”, Tutorial HIPEAC 2020.
4. Alberto Bosio, “Making AI Applications Robust, Secure and Efficient: AxC to the Rescue”, Tutorial HIPEAC 2021.
5. Alberto Bosio, Olivier Sentieys, Daniel Menard, “A Comprehensive Analysis of Approximate Computing Techniques: From Component- to Application-Level«, Tutorial DATE 2019.
6. E. Dupuis, D. Novo, I. O’Connor and A. Bosio, « Fast Exploration of Weight Sharing Opportunities for CNN Compression,« Workshop on System-level Design Methods for Deep Learning on Heterogeneous Architectures (SLOHA 2021) in conjunction with DATE 2021.
7. V. Ha, T. Yuki, and O. Sentieys. “Towards Generic and Scalable Word-Length Optimization”. 23rd IEEE/ACM Design, Automation and Test in Europe (DATE), Mar. 9, 2020, pp. 1–6.
8. V. Ha and O. Sentieys. “Leveraging Bayesian Optimization to Speed Up Automatic Precision Tuning”. 24th IEEE/ACM Design, Automation and Test in Europe (DATE), 2021.

A common book chapter entitled «Approximations in Deep Learning« is being written. It should be published in the summer of 2021 by Springer.

Several softwares have been released as opensource.

The design and implementation of convolutional neural networks for deep learning is currently receiving a lot of attention from both industrials and academics. However, the computational workload involved with CNNs is often out of reach for low power embedded devices and is still very costly when run on datacenter. By relaxing the need for fully precise operations, approximate computing substantially improves performance and energy efficiency. Deep learning is very relevant in this context, since playing with the accuracy to reach adequate computations will significantly enhance performance, while keeping quality of results in a user-constrained range. AdequateDL will explore how approximations can improve performance of hardware accelerator in deep-learning applications. Outcomes include a framework for accuracy exploration, demonstration of performance gains by several orders of magnitude of proposed adequate accelerators with regards to conventional CPU/GPU computing platforms.