The breakthrough of deep learning has started a technological revolution in various areas such as object identification, image/video recognition and semantic segmentation. Neural network, which is one of representative applications of deep learning, has been widely used and developed many efficient models. However, the edge implementation of neural network inference is restricted because of conflicts between the high computation and storage complexity and resource-limited hardware platforms in applications scenarios. In this paper, we research neural networks which are involved in the acceleration on FPGA-based platforms. The architecture of networks and characteristics of FPGA are analyzed, compared and summarized, as well as their influence on acceleration tasks. Based on the analysis, we generalize the acceleration strategies into five aspects—computing complexity, computing parallelism, data reuse, pruning and quantization. Then previous works on neural network acceleration are introduced following these topics. We summarize how to design a technical route for practical applications based on these strategies. Challenges in the path are discussed to provide guidance for future work.
In urban environments, semantic segmentation using computer vision plays a pivotal role in understanding and interpreting the diverse elements within urban imagery. The Cityscapes dataset, widely used for semantic segmentation in urban scenes, predominantly features urban elements like buildings and vehicles but lacks aquatic elements. Recognizing this limitation, our study introduces a method to enhance the Cityscapes dataset by incorporating aquatic classes, crucial for a comprehensive understanding of coastal urban environments. To achieve this, we employ a dual-model approach using two advanced neural networks. The first network is trained on the standard Cityscapes dataset, while the second focuses on aquatic scenes. We adeptly integrate aquatic features from the marine-focused model into the Cityscapes imagery. This integration is carefully executed to ensure a seamless blend of urban and aquatic elements, thereby creating an enriched dataset that reflects the realities of coastal cities more accurately. Our method is evaluated by comparing the enhanced Cityscapes model with the original on a set of diverse urban images, including aquatic views. The results demonstrate that our approach effectively maintains the high segmentation accuracy of the original Cityscapes dataset for urban elements while successfully integrating marine features. Importantly, this is achieved without necessitating additional training, which is a significant advantage in terms of resource efficiency.
2 August 2022 in Computer Vision and Image Understanding