Only with new sensor concepts in a network, which go far beyond what the current state-of-the-art can offer, can current and future requirements for flexibility, safety, and security be met. The combination of data from many sensors allows a richer representation of the observed phenomenon, e.g., system degradation, which can facilitate analysis and decision-making processes. This work addresses the topic of predictive maintenance by exploiting sensor data fusion and artificial intelligence-based analysis. With a dataset such as vibration and sound from sensors, we focus on studying paradigms that orchestrate the most optimal combination of sensors with deep learning sensor fusion algorithms to enable predictive maintenance. In our experimental setup, we used raw data obtained from two sensors, a microphone, and an accelerometer installed on a brushless direct current (BLDC) motor. The data from each sensor were processed individually and, in a second step, merged to create a solid base for analysis. To diagnose BLDC motor faults, this work proposes to use data-level sensor fusion with deep learning methods such as deep convolutional neural networks (DCNNs) for their ability to automatically extract relevant information from the input data, the long short-term memory method (LSTM), and convolutional long short-term memory (CNN-LSTM), a combination of the two previous methods. The results show that in our setup, sound signals outperform vibrations when used individually for training. However, without any feature selection/extraction step, the accuracy of the models improves with data fusion and reaches 98.8%, 93.5%, and 73.6% for the DCNN, CNN-LSTM, and LSTM methods, respectively, 98.8% being a performance that, according to our reading, has never been reached in the analysis of the faults of a BLDC motor without first going through the extraction of the characteristics and their fusion by traditional methods. These results show that it is possible to work with raw data from multiple sensors and achieve good results using deep learning methods without spending time and resources on selecting appropriate features to extract and methods to use for feature extraction and data fusion.

In this paper, we present a bit stream feature-based energy model that accurately estimates the energy required to decode a given High Efficiency Video Coding-coded bit stream. Therefore, we take a model from literature and extend it by explicitly modeling the in-loop filters, which was not done before. Furthermore, to prove its superior estimation performance, it is compared with seven different energy models from the literature. By using a unified evaluation framework, we show how accurately the required decoding energy for different decoding systems can be approximated. We give thorough explanations on the model parameters and explain how the model variables are derived. To show the modeling capabilities in general, we test the estimation performance for different decoding software and hardware solutions, where we find that the proposed model outperforms the models from the literature by reaching framewise mean estimation errors of less than 7% for software and less than 15% for hardware-based systems.

Predictive maintenance of equipment requires a set of data collected through sensors, from which models will learn behaviors that will allow the automatic detection or prediction of these behaviors. The objective is to anticipate unexpected situations such as sudden equipment stoppages. Industries are noisy environments due to production lines that involve a series of components. As a result, the data will always be obstructed by noise. Noise-robust predictive maintenance models, which include ensemble and deep learning models with and without data fusion, are proposed to enhance the monitoring of industrial equipment. The work reported in this paper is based on two components, a milling tool, and a motor, with sound, vibration, and ultrasound data collected in real experiments. Four main tasks were performed, namely the construction of the datasets, the training of the monitoring models without adding artificial noise to the data, the evaluation of the robustness of the previously trained models by injecting several levels of noise into the test data, and the optimization of the models by a proposed noisy training approach. The results show that the models maintain their performances at over 95% accuracy despite adding noise in the test phase. These performances decrease by only 2% at a considerable noise level of 15dB signal-to-noise ratio (SNR). The noisy training method proved to be an optimal solution for improving the noise robustness and accuracy of convolutional deep learning models, whose performance regression of 2% went from a noise level of 28dB to 15dB like the other models.

Fast and resource-efficient inference in artificial neural networks (ANNs) is of utmost importance and drives many new developments in the area of new hardware architectures, e.g., by means of systolic arrays or algorithmic optimization such as pruning. In this paper, we present a novel method for lowering the computation effort for ANN inference utilizing ideas from information theory.Weight matrices are sliced into submatrices of logarithmic aspect ratios. These slices are then factorized. This reduces the number of required computations without compromising on fully parallel processing. We create a new hardware architecture for this dedicated purpose. We also provide a tool to map these sliced and factorized matrices efficiently to reconfigurable hardware. By comparing to the state of the art FPGA implementations, we can prove our claim by lowering hardware resources measured in look-up-tables (LUTs) by a factor of three to six. Our method does not rely on any particular property of the weight matrices of the ANN. It works for the general task of multiplying an input vector with a constant matrix and is also suitable for digital signal processing beyond ANNs.

The efficient realization of self-organizing systems based on 2D stencil code applications, like our developed Marching Pixel algorithms, is a great challenge. They are data-intensive and also computational-intensive, because often a high number of iterations is required. FPGAs are predestined for the realization of these algorithms. They are very flexible, allow a scalable parallel processing and have a moderate power consumption, even in high-performance versions. Therefore, FPGAs are highly qualified to make these applications also real-time capable. Our goal was to implement an efficient parameterizable buffering and parallel processing scheme for such operations in FPGAs, to process them as fast as possible. We developed a generic VHDL template which allows a scalable parallelization and pipelining of 2D stencil code applications in relation to application and hardware constraints.