30 July 2024 in Expert Systems with Applications
5 January 2022 in IEEE Transactions on Neural Networks and Learning Systems
Recent deep neural networks (DNNs) with several layers of feature representations rely on some form of skip connections to simultaneously circumnavigate optimization problems and improve generalization performance. However, the operations of these models are still not clearly understood, especially in comparison to DNNs without skip connections referred to as plain networks (PlainNets) that are absolutely untrainable beyond some depth. As such, the exposition of this article is the theoretical analysis of the role of skip connections in training very DNNs using concepts from linear algebra and random matrix theory. In comparison with PlainNets, the results of our investigation directly unravel the following: 1) why DNNs with skip connections are easier to optimize and 2) why DNNs with skip connections exhibit improved generalization. Our investigation results concretely show that the hidden representations of PlainNets progressively suffer from information loss via singularity problems with depth increase, thus making their optimization difficult. In contrast, as model depth increases, the hidden representations of DNNs with skip connections circumnavigate singularity problems to retain full information that reflects in improved optimization and generalization. For theoretical analysis, this article studies in relation to PlainNets two popular skip connection-based DNNs that are residual networks (ResNets) and residual network with aggregated features (ResNeXt).
6 October 2023 in Journal of Space Safety Engineering
2 June 2017 in IEEE Transactions on Pattern Analysis and Machine Intelligence
In this paper, we introduce a deformation based representation space for curved shapes in Rn. Given an ordered set of points sampled from a curved shape, the proposed method represents the set as an element of a finite dimensional matrix Lie group. Variation due to scale and location are filtered in a preprocessing stage, while shapes that vary only in rotation are identified by an equivalence relationship. The use of a finite dimensional matrix Lie group leads to a similarity metric with an explicit geodesic solution. Subsequently, we discuss some of the properties of the metric and its relationship with a deformation by least action. Furthermore, invariance to reparametrization or estimation of point correspondence between shapes is formulated as an estimation of sampling function. Thereafter, two possible approaches are presented to solve the point correspondence estimation problem. Finally, we propose an adaptation of k-means clustering for shape analysis in the proposed representation space. Experimental results show that the proposed representation is robust to uninformative cues, e.g. local shape perturbation and displacement. In comparison to state of the art methods, it achieves a high precision on the Swedish and the Flavia leaf datasets and a comparable result on MPEG-7, Kimia99 and Kimia216 datasets.
24 November 2022 in Applied Intelligence
1 July 2019 in Computer Methods and Programs in Biomedicine
With the increase in the number of stroke survivors, there is an urgent need for designing appropriate home-based rehabilitation tools to reduce health-care costs. The objective is to empower the rehabilitation of post-stroke patients at the comfort of their homes by supporting them while exercising without the physical presence of the therapist. A novel low-cost home-based training system is introduced. This system is designed as a composition of two linked applications: one for the therapist and another one for the patient. The therapist prescribes personalized exercises remotely, monitors the home-based training and re-adapts the exercises if required. On the other side, the patient loads the prescribed exercises, trains the prescribed exercise while being guided by color-based visual feedback and gets updates about the exercise performance. To achieve that, our system provides three main functionalities, namely: 1) Feedback proposals guiding a personalized exercise session, 2) Posture monitoring optimizing the effectiveness of the session, 3) Assessment of the quality of the motion. The proposed system is evaluated on 10 healthy participants without any previous contact with the system. To analyze the impact of the feedback proposals, we carried out two different experimental sessions: without and with feedback proposals. The obtained results give preliminary assessments about the interest of using such feedback. Obtained results on 10 healthy participants are promising. This encourages to test the system in a realistic clinical context for the rehabilitation of stroke survivors.