Visual impairment or even loss seriously affects quality of life. Benefited by the rapid development of sound/laser detection, Global Positioning System (GPS)/Beidou positioning, machine vision and other technologies, the quality of life of blind people is expected to be improved through visual substitution technology. The existing visual substitution devices still have limitations in terms of safety, robustness, and ease of operation. The remote companion system developed here fully utilizes multimodal navigation and remote communication technologies, and the positioning and interaction functions of commercial mobile phones. Together with the accumulated judgment of backend personnel, it can provide real-time, safe, and reliable navigation services for blind people, helping them complete daily activities such as independent travel, circulation, and shopping. The practical results show that the system not only has strong operability and is easy to use, but also can provide users with a strong sense of security and companionship, making it suitable for promotion. In the future, this system can also be promoted for other vulnerable groups such as the elderly.

In this work, we presented a novel encoding method for tactile communication. This approach was based on several tactile sensory characteristics of human skin at different body parts, such as the head and neck, where location coordinates in the three-dimensional (3D) space were clearly mapped in the brain cortex, and gentle stimulations of vibrational touching with varied strengths were received instantly and precisely. For certain applications, such as playing cards or navigating walk paths for blinded people, we demonstrated specifically designed code lists with different patterns of tactile points in varied temporal sequences. By optimizing these codes, we achieved excellent efficiency and accuracy in our test experiments. As this method matched well with the natural habits of tactile sensory, it was easy to learn in a short training period. The results of the present work have offered a silent, efficient and accurate communication solution for visually impaired people or other users.

Remote-sighted assistance (RSA) is a technology designed to provide assistance for visually impaired people (VIPs). In this scene, a remote-sighted agent communicates and sends commands to navigate and assist VIPs via real-time video sent back. However, the latency in real-time video and the deviation in the execution of instructions by VIPs are two important factors that affect the performance of agents to guide them. Therefore, how to enable agents to better guide VIPs under conditions of video transmission latency and deviation in instruction execution is an important issue. In this paper, we utilize Unreal Engine to create a virtual training platform for RSA, which simulates VIPs executing instructions in the real world and resembles the environment in RSA systems. We aim to help remote-sighted agents quickly master the set of vibration commands formed after encoding tactile vibrations and enable them to guide VIPs more effectively. Our experiment results show that, compared with untrained novices, when guiding people through the same path, agents trained on this platform reduce their average time by 32.09% and their average number of contacts with the environment by 57.57%. Our work provides agents with a simple and convenient simulation and training platform designed to enhance their performance by guiding VIPs with less travel time and fewer environmental contacts. Through this platform, agents can more effectively assist the visually impaired.

Visually impaired people face urban mobility hazards such as vehicles, pedestrians, curbs, and steps. Wearable electronic travel aids can support obstacle avoidance only when these hazards fall within sensing coverage. Existing studies have focused mainly on algorithms and human–computer interaction, while the sensing coverage required by real-world hazards remains less examined. We collected 1486 min of street-scene video, built urban mobility hazard samples, and used simulation to compare target pixel counts over time for cameras with different horizontal fields of view. Among 81,353 valid samples, hazards showed concentrated high-frequency categories and a rich long tail, mainly including vehicles, scooters, pedestrians, curbs, bollards, poles, trees, and steps. Simulation showed that a horizontal field of view of approximately $120°$ balanced pixel coverage and sustained observation. We then built a chest-mounted four-camera prototype and conducted a feasibility demonstration at a real intersection. Camera configuration should therefore be guided by real-world hazard visibility, and coordinated multi-camera coverage is a promising design choice for wearable ETAs in urban environments.

As urban environments become increasingly interconnected, the demand for precise and efficient pedestrian solutions in digitalized smart cities has grown significantly. This study introduces a scalable spatial visualization system designed to enhance interactions between individuals and the street in outdoor sidewalk environments. The system operates in two main phases: the spatial prior phase and the target localization phase. In the spatial prior phase, the system captures the user’s perspective using first-person visual data and leverages landmark elements within the sidewalk environment to localize the user’s camera. In the target localization phase, the system detects surrounding objects, such as pedestrians or cyclists, using high-angle closed-circuit television (CCTV) cameras. The system was deployed in a real-world sidewalk environment at an intersection on a university campus. By combining user location data with CCTV observations, a 4D+ virtual monitoring system was developed to present a spatiotemporal visualization of the mobile participants within the user’s surrounding sidewalk space. Experimental results show that the landmark-based localization method achieves a planar positioning error of 0.468 m and a height error of 0.120 m on average. With the assistance of CCTV cameras, the localization of other targets maintains an overall error of 0.24 m. This system establishes the spatial relationship between pedestrians and the street by integrating detailed sidewalk views, with promising applications for pedestrian navigation and the potential to enhance pedestrian-friendly urban ecosystems.

Pedestrian localization is a fundamental sensing task for intelligent outdoor systems. To overcome the limitations of accuracy and efficiency in conventional binocular approaches, this study introduces a trinocular stereo vision framework that integrates template matching with geometric constraint optimization. The system employs a trinocular camera configuration arranged in an equilateral triangle, which enables complementary perspectives beyond a standard horizontal baseline. Based on this setup, an initial depth estimate is obtained through multi-scale template matching on the primary binocular pair. The additional vertical viewpoint is then incorporated by enforcing three-view geometric consistency, yielding refined and more reliable depth estimates. We evaluate the method on a custom outdoor trinocular dataset. Experimental results demonstrate that the proposed approach achieves a mean absolute error of 0.435 m with an average processing time of 3.13 ms per target. This performance surpasses both the binocular Semi-Global Block Matching (0.536 m) and RAFT-Stereo (0.623 m for the standard model and 0.621 m for the real-time model without fine-tuning). When combined with the YOLOv8-s detector, the system can localize pedestrians in 7.52 ms per frame, maintaining real-time operation (>30 Hz) for up to nine individuals, with a total end-to-end latency of approximately 32.56 ms.