Image dehazing methods can restore clean images from hazy images and are popularly used as a preprocessing step to improve performance in various image analysis tasks. In recent times, deep learning-based methods have been used to sharply increase the visual quality of restored images, but they require a long computation time. The processing time of image-dehazing methods is one of the important factors to be considered in order not to affect the latency of the main image analysis tasks such as detection and segmentation. We propose an end-to-end network model for real-time image dehazing. We devised a zoomed convolution group that processes computation-intensive operations with low resolution to decrease the processing time of the network model without performance degradation. Additionally, the zoomed convolution group adopts an efficient channel attention module to improve the performance of the network model. Thus, we designed a network model using a zoomed convolution group to progressively recover haze-free images using a coarse-to-fine strategy. By adjusting the sampling ratio and the number of convolution blocks that make up the convolution group, we distributed small and large computational complexities respectively in the early and later operational stages. The experimental results with the proposed method on a public dataset showed a real-time performance comparable to that of another state-of-the-art (SOTA) method. The proposed network’s peak-signal-to-noise ratio was 0.8 dB lower than that of the SOTA method, but the processing speed was 10.4 times faster.

Visual-auditory sensory substitution has demonstrated great potential to help visually impaired and blind groups to recognize objects and to perform basic navigational tasks. However, the high latency between visual information acquisition and auditory transduction may contribute to the lack of the successful adoption of such aid technologies in the blind community; thus far, substitution methods have remained only laboratory-scale research or pilot demonstrations. This high latency for data conversion leads to challenges in perceiving fast-moving objects or rapid environmental changes. To reduce this latency, prior analysis of auditory sensitivity is necessary. However, existing auditory sensitivity analyses are subjective because they were conducted using human behavioral analysis. Therefore, in this study, we propose a cross-modal generative adversarial network-based evaluation method to find an optimal auditory sensitivity to reduce transmission latency in visual-auditory sensory substitution, which is related to the perception of visual information. We further conducted a human-based assessment to evaluate the effectiveness of the proposed model-based analysis in human behavioral experiments. We conducted experiments with three participant groups, including sighted users (SU), congenitally blind (CB) and late-blind (LB) individuals. Experimental results from the proposed model showed that the temporal length of the auditory signal for sensory substitution could be reduced by 50%. This result indicates the possibility of improving the performance of the conventional vOICe method by up to two times. We confirmed that our experimental results are consistent with human assessment through behavioral experiments. Analyzing auditory sensitivity with deep learning models has the potential to improve the efficiency of sensory substitution.

Recent studies have applied the superior performance of deep learning to mobile devices, and these studies have enabled the running of the deep learning model on a mobile device with limited computing power. However, there is performance degradation of the deep learning model when it is deployed in mobile devices, due to the different sensors of each device. To solve this issue, it is necessary to train a network model specific to each mobile device. Therefore, herein, we propose an acceleration method for on-device learning to mitigate the device heterogeneity. The proposed method efficiently utilizes unified memory for reducing the latency of data transfer during network model training. In addition, we propose the layer-wise processor selection method to consider the latency generated by the difference in the processor performing the forward propagation step and the backpropagation step in the same layer. The experiments were performed on an ODROID-XU4 with the ResNet-18 model, and the experimental results indicate that the proposed method reduces the latency by at most 28.4% compared to the central processing unit (CPU) and at most 21.8% compared to the graphics processing unit (GPU). Through experiments using various batch sizes to measure the average power consumption, we confirmed that device heterogeneity is alleviated by performing on-device learning using the proposed method.

The visual sensory organ (VSO) serves as the primary channel for transmitting external information to the brain; therefore, damage to the VSO can severely limit daily activities. Visual-to-Auditory Sensory Substitution (V2A-SS), an innovative approach to restoring vision, offers a promising solution by leveraging neuroplasticity to convey visual information via auditory channels. Advances in information technology and artificial intelligence mitigate technical challenges such as low resolution and limited bandwidth, thereby enabling broader applicability of V2A-SS. Despite these advances, integrating V2A-SS effectively into everyday life necessitates extensive training and adaptation. Therefore, alongside addressing technical challenges, investigating effective learning strategies to accelerate the acceptance of V2A-SS is crucial. This study introduces a V2A-SS learning model based on the Information Processing Learning Theory (IPLT), encompassing the stages of “concept acquisition, rehearsal, assessment” to reduce the learning curve and enhance adaptation. The experimental results show that the proposed learning model improves recognition rates, achieving an 11% increase over simple random repetition learning. This improvement is significantly higher than the gain of 2.72% achieved by optimizing the V2A-SS algorithm with Mel-Scaled Frequency Mapping. This study suggests that a structured learning model for sensory substitution technologies can contribute to bridging gaps between technical feasibility and practical application. This underscores the need to develop effective learning models, alongside technological advancements, to accelerate the adoption of V2A-SS and neuroplasticity.

Introduction: Visual-to-auditory sensory substitution converts visual information into structured sound patterns when visual input is unavailable, impaired, or experimentally restricted, thereby supporting access to information otherwise conveyed visually. Its perceptual usability depends jointly on the sensory-conversion scheme that encodes visual information into sound and on the learning conditions under which learners acquire sound-visual mappings through learning and experience. Although prior visual-to-auditory sensory-substitution studies have often evaluated substituted coding schemes or learning approaches primarily through endpoint learnability or recognition accuracy, endpoint performance alone does not characterize the temporal cost and efficiency with which usable perceptual decoding emerges during learning. Methods: This study examined whether structured learning based on information-processing principles was associated with more efficient early acquisition of sound-visual mappings in a controlled perceptual-learning model with a constant visual-to-auditory mapping framework. Acquisition-phase behavioral records from 63 blindfolded sighted adults who completed a randomized visual-to-auditory sensory-substitution learning experiment were analyzed, with 31 and 32 participants in the structured-learning and control conditions, respectively. The outcomes included correct-trial response time, total learning time, response-time variability, and inverse efficiency score. Results: Compared with the control condition, structured learning was associated with shorter correct-trial response time, lower total learning time, and lower inverse efficiency score, without a meaningful between-group difference in response-time variability. These patterns remained evident across robustness analyses and after adjustment for baseline pre-test performance. Exploratory schedule-derived analyses suggested that structured learning provided denser repeated exposure and a more staged introduction of sensory mappings. Discussion: Together, these behavioral-log findings indicate that structured learning was associated with more efficient early acquisition of sound-visual mappings and suggest that acquisition efficiency in sensory-substitution learning may be related to how those mappings are sequenced and repeated during learning.