Heterogeneous FPGA-based Embedded Systems for Vision IoT Applications

Abstract

Embedded sensor devices provided by processing capabilities are opening novel and exciting opportunities in the era of edge-computing Internet-of-Things (IoT). The workload decentralization leads to a plenty of benefits, including better reactivity and reliability and reduced data transfer costs. These advantages have a strong impact especially in the visual IoT field, for which the large bandwidth required by visual data is one of the most critical challenges. However, bringing vision technologies into smart nodes is not a trivial task, because of the stringent energy and performance requirements, in addition to the need of cost-effective and compact processing units. Heterogeneous architectures may represent the key to address these necessities. Among possible heterogeneous platforms, those based on reconfigurable devices such as Field Programmable Gate Arrays (FPGAs) show a high adaptability to a variety of workloads, which is an important goal for edge-computing. Therefore, their deployment in disparate IoT applications, ranging from video surveillance to autonomous driving, is emerging as a promising solution. This dissertation proposes a study on the suitability of modern heterogeneous FPGA System-on-Chips (SoCs) to implement embedded smart vision sensor nodes. To this purpose, several computer vision algorithms aimed to extract synthetic data from raw input frames have been analysed, and novel hardware-oriented solutions have been proposed to deploy them on heterogeneous SoCs. In all the presented cases, ranging from stereo vision to connected component analysis and deep learning, speed performances and/or energy efficiency are considerably improved with respect to state-of-the-art solutions. As an example, the proposed heterogeneous architecture for convolutional neural networks achieves a power efficiency up to 89.5% higher than competitive prior works, demonstrating its suitability in the scenario of energy-constrained and real-time IoT.