In this review, we present an analysis of the most used multi-agent reinforcement learning algorithms. Starting with the single-agent reinforcement learning algorithms, we focus on the most critical issues that must be taken into account in their extension to multi-agent scenarios. The analyzed algorithms were grouped according to their features. We present a detailed taxonomy of the main multi-agent approaches proposed in the literature, focusing on their related mathematical models. For each algorithm, we describe the possible application fields, while pointing out its pros and cons. The described multi-agent algorithms are compared in terms of the most important characteristics for multi-agent reinforcement learning applications—namely, nonstationarity, scalability, and observability. We also describe the most common benchmark environments used to evaluate the performances of the considered methods.

In this paper we propose an efficient hardware architecture that implements the Q-Learning algorithm, suitable for real-time applications. Its main features are low-power, high throughput and limited hardware resources. We also propose a technique based on approximated multipliers to reduce the hardware complexity of the algorithm. We implemented the design on a Xilinx Zynq Ultrascale+ MPSoC ZCU106 Evaluation Kit. The implementation results are evaluated in terms of hardware resources, throughput and power consumption. The architecture is compared to the state of the art of Q-Learning hardware accelerators presented in the literature obtaining better results in speed, power and hardware resources. Experiments using different sizes for the Q-Matrix and different wordlengths for the fixed point arithmetic are presented. With a Q-Matrix of size 8×4 (8 bit data) we achieved a throughput of 222 MSPS (Mega Samples Per Second) and a dynamic power consumption of 37 mW, while with a Q-Matrix of size 256×16 (32 bit data) we achieved a throughput of 93 MSPS and a power consumption 611 mW. Due to the small amount of hardware resources required by the accelerator, our system is suitable for multi-agent IoT applications. Moreover, the architecture can be used to implement the SARSA (State-Action-Reward-State-Action) Reinforcement Learning algorithm with minor modifications.

In the last few years, indoor localization has attracted researchers and commercial developers. Indeed, the availability of systems, techniques and algorithms for localization allows the improvement of existing communication applications and services by adding position information. Some examples can be found in the managing of people and/or robots for internal logistics in very large warehouses (e.g., Amazon warehouses, etc.). In this paper, we study and develop a system allowing the accurate indoor localization of people visiting a museum or any other cultural institution. We assume visitors are equipped with a Bluetooth Low Energy (BLE) device (commonly found in modern smartphones or in a small chipset), periodically transmitting packets, which are received by geolocalized BLE receivers inside the museum area. Collected packets are provided to the locator server to estimate the positions of the visitors inside the museum. The position estimation is based on a feed-forward neural network trained by a measurement campaign in the considered environment and on a non-linear least square algorithm. We also provide a strategy for deploying the BLE receivers in a given area. The performance results obtained from measurements show an achievable position estimate accuracy below 1 m.

In this brief, we propose a FPGA implementation of a novel Variable Fractional Delay (VFD) filter architecture for digital wideband (hundred of MHz or more) beamforming applications. The proposed architecture is based on a multirate structure that allows reaching high performance in terms of amplitude and group delay with a significant reduction in design complexity. Usually, the antenna’s beams are modified by introducing precise delays that can be multiples or fractions of the sampling period. The most used structures for realizing this operation are Farrow filters but, for wideband applications, the classical Farrow structure may require a huge amount of resources. A common solution to simplify the Farrow filter architecture is interpolation. However, this solution requires an increase of the clock frequency which is not always possible. In this context, our multirate approach mitigates the limitations in terms of computational complexity avoiding the requirement to increase of the clock frequency, while still exhibiting great performance. A comparison between a reference design based on the proposed approach and the literature is also provided. The experimental results confirm that the proposed architecture, characterized by a computational complexity similar to other works, exhibits better performance without the need to increase the clock frequency.

This paper proposes a Design Space Exploration for Edge machine learning through the utilization of the novel MathWorks FPGA Deep Learning Processor IP, featured in the HDL Deep Learning toolbox. With the ever-increasing demand for real-time machine learning applications, there is a critical need for efficient and low-latency hardware solutions that can operate at the edge of the network, in close proximity to the data source. The HDL Deep Learning toolbox provides a flexible and customizable platform for deploying deep learning models on FPGAs, enabling effective inference acceleration for embedded IoT applications. In this study, our primary focus lies in investigating the impact of parallel processing elements on the performance and resource utilization of the FPGA-based processor. By analyzing the trade-offs between accuracy, speed, energy efficiency, and hardware resource utilization, we aim to gain valuable insights into making optimal design choices for FPGA-based implementations. Our evaluation is conducted on the AMD-Xilinx ZC706 development board, which serves as the target device for our experiments. We consider all the compatible Convolutional Neural Networks available within the HDL Deep Learning toolbox to comprehensively assess the performances.