In recent years, indoor localization systems have been the object of significant research activity and of growing interest for their great expected social impact and their impressive business potential. Application areas include tracking and navigation, activity monitoring, personalized advertising, Active and Assisted Living (AAL), traceability, Internet of Things (IoT) networks, and Home-land Security. In spite of the numerous research advances and the great industrial interest, no canned solutions have yet been defined. The diversity and heterogeneity of applications, scenarios, sensor and user requirements, make it difficult to create uniform solutions. From that diverse reality, a main problem is derived that consists in the lack of a consensus both in terms of the metrics and the procedures used to measure the performance of the different indoor localization and navigation proposals. This paper introduces the general lines of the EvAAL benchmarking framework, which is aimed at a fair comparison of indoor positioning systems through a challenging competition under complex, realistic conditions. To evaluate the framework capabilities, we show how it was used in the 2016 Indoor Positioning and Indoor Navigation (IPIN) Competition. The 2016 IPIN competition considered three different scenario dimensions, with a variety of use cases: (1) pedestrian versus robotic navigation, (2) smartphones versus custom hardware usage and (3) real-time positioning versus off-line post-processing. A total of four competition tracks were evaluated under the same EvAAL benchmark framework in order to validate its potential to become a standard for evaluating indoor localization solutions. The experience gained during the competition and feedback from track organizers and competitors showed that the EvAAL framework is flexible enough to successfully fit the very different tracks and appears adequate to compare indoor positioning systems.

Mobile CrowdSensing (MCS) is an emerging paradigm in the distributed acquisition of smart city and Internet of Things (IoT) data. MCS requires large number of users to enable access to the built-in sensors in their mobile devices and share sensed data to ensure high value and high veracity of big sensed data. Improving user participation in MCS campaigns requires to boost users effectively, which is a key concern for the success of MCS platforms. As MCS builds on non-dedicated sensors, data trustworthiness cannot be guaranteed as every user attains an individual strategy to benefit from participation. At the same time, MCS platforms endeavor to acquire highly dependable crowd-sensed data at lower cost. This phenomenon introduces a game between users that form the participant pool, as well as between the participant pool and the MCS platform. Research on various game theoretic approaches aims to provide a stable solution to this problem. This article presents a comprehensive review of different game theoretic solutions that address the following issues in MCS such as sensing cost, quality of data, optimal price determination between data requesters and providers, and incentives. We propose a taxonomy of game theory-based solutions for MCS platforms in which problems are mainly formulated based on Stackelberg, Bayesian and Evolutionary games. We present the methods used by each game to reach an equilibrium where the solution for the problem ensures that every participant of the game is satisfied with their utility with no requirement of change in their strategies. The initial criterion to categorize the game theoretic solutions for MCS is based on co-operation and information available among participants whereas a participant could be either a requester or provider. Following a thorough qualitative comparison of the surveyed approaches, we provide insights concerning open areas and possible directions in this active field of research.

Every year, for ten years now, the IPIN competition has aimed at evaluating real-world indoor localisation systems by testing them in a realistic environment, with realistic movement, using the EvAAL framework. The competition provided a unique overview of the state-of-the-art of systems, technologies, and methods for indoor positioning and navigation purposes. Through fair comparison of the performance achieved by each system, the competition was able to identify the most promising approaches and to pinpoint the most critical working conditions. In 2020, the competition included 5 diverse off-site off-site Tracks, each resembling real use cases and challenges for indoor positioning. The results in terms of participation and accuracy of the proposed systems have been encouraging. The best performing competitors obtained a third quartile of error of 1 m for the Smartphone Track and 0.5 m for the Foot-mounted IMU Track. While not running on physical systems, but only as algorithms, these results represent impressive achievements.

IPIN 2019 Competition, sixth in a series of IPIN competitions, was held at the CNR Research Area of Pisa (IT), integrated into the program of the IPIN 2019 Conference. It included two on-site real-time Tracks and three off-site Tracks. The four Tracks presented in this paper were set in the same environment, made of two buildings close together for a total usable area of 1000 m 2 outdoors and and 6000 m 2 indoors over three floors, with a total path length exceeding 500 m. IPIN competitions, based on the EvAAL framework, have aimed at comparing the accuracy performance of personal positioning systems in fair and realistic conditions: past editions of the competition were carried in big conference settings, university campuses and a shopping mall. Positioning accuracy is computed while the person carrying the system under test walks at normal walking speed, uses lifts and goes up and down stairs or briefly stops at given points. Results presented here are a showcase of state-of-the-art systems tested side by side in real-world settings as part of the on-site real-time competition Tracks. Results for off-site Tracks allow a detailed and reproducible comparison of the most recent positioning and tracking algorithms in the same environment as the on-site Tracks.

Several Radio-Frequency technologies have been explored to evaluate the efficacy of localization algorithms in indoor environments, including Received Signal Strength (RSS), Time of Flight (ToF), and Angle of Arrival (AoA). Among these, AoA technique has been gaining interest when adopted with the Bluetooth protocol. In this work, we describe a data collection measurement campaign of AoA and RSS values collected from Bluetooth 5.1 compliant tags and a set of anchor nodes deployed in the environment. The resulting dataset and the adopted software is publicly available to the community. To collect the data, we deploy four anchor nodes and four Bluetooth tags and we reproduce some representative scenarios for indoor localization: calibration, static, mobility, and proximity. Each scenario is annotated with an accurate ground truth (GT). We also assess the quality of the collected data. Specifically, we compute the Mean Absolute Error (MAE) between the AoA estimated by the anchors and the corresponding GT. Additionally, we investigate the packet loss metric which measures the percentage of Bluetooth beacons lost by the anchors.