The early diagnosis of cerebral palsy is an area which has recently seen significant multi-disciplinary research. Diagnostic tools such as the General Movements Assessment (GMA), have produced some very promising results. However, the prospect of automating these processes may improve accessibility of the assessment and also enhance the understanding of movement development of infants. Previous works have established the viability of using pose-based features extracted from RGB video sequences to undertake classification of infant body movements based upon the GMA. In this paper, we propose a series of new and improved features, and a feature fusion pipeline for this classification task. We also introduce the RVI-38 dataset, a series of videos captured as part of routine clinical care. By utilising this challenging dataset we establish the robustness of several motion features for classification, subsequently informing the design of our proposed feature fusion framework based upon the GMA. We evaluate our proposed framework’s classification performance using both the RVI-38 dataset and the publicly available MINI-RGBD dataset. We also implement several other methods from the literature for direct comparison using these two independent datasets. Our experimental results and feature analysis show that our proposed pose-based method performs well across both datasets. The proposed features afford us the opportunity to include finer detail than previous methods, and further model GMA specific body movements. These new features also allow us to take advantage of additional body-part specific information as a means of improving the overall classification performance, whilst retaining GMA relevant, interpretable, and shareable features.

Knowledge about individual body shape has numerous applications in various domains such as healthcare, fashion and personalized entertainment. Most of the depth based whole body scanners need multiple cameras surrounding the user and requiring the user to keep a canonical pose strictly during capturing depth images. These scanning devices are expensive and need professional knowledge for operation. In order to make 3D scanning as easy-to-use and fast as possible, there is a great demand to simplify the process and to reduce the hardware requirements. In this paper, we propose a deep learning algorithm, dubbed 3DBodyNet, to rapidly reconstruct the 3D shape of human bodies using a single commodity depth camera. As easy-to-use as taking a photo using a mobile phone, our algorithm only needs two depth images of the front-facing and back-facing bodies. The proposed algorithm has strong operability since it is insensitive to the pose and the pose variations between the two depth images. It can also reconstruct an accurate body shape for users under tight/loose clothing. Another advantage of our method is the ability to generate an animatable human body model. Extensive experimental results show that the proposed method enables robust and easy-to-use animatable human body reconstruction, and outperforms the state-of-the-art methods with respect to running time and accuracy.

Estimating the 3D human body shape under clothing is important for many applications. Existing body shape estimation methods formulate this task as an optimization problem by fitting a parametric body model to a single dressed-human scan or a sequence of dressed-human meshes for better accuracy. This is impractical for many applications that require fast acquisition such as gaming and virtual try-on due to the expensive computation. In this paper, we propose the first learning-based approach to estimating human body shape from a single dressed-human scan, dubbed Body PointNet. The Body PointNet operates directly on raw point clouds and predicts the undressed body in a coarse-to-fine manner. We propose a novel method to synthesize the dressed-human pseudo-scans and corresponding ground truth bodies. A new large-scale dataset, BUG (Body Under virtual Garments) is presented. Comprehensive evaluations show that Body PointNet outperforms state-of-the-art methods in terms of both accuracy and running time.

The appearance of 3-D scanners, generating point clouds, has revolutionized anthropometric data collection systems and their applications. Anthropometric data are of paramount importance in several applications, including fashion design, medical diagnosis, and virtual character modeling, all of which require a fully automatic anthropometric measurement extraction method. 3-D-based methods for anthropometric measurement extraction becomes more and more popular due to their improved accuracy compared to classical image-based approaches. Existing 3-D methods can be mainly classified into two categories: landmark and template-based methods. The former is highly dependent on the estimated landmarks which are highly sensitive to noise in the input or missing data. The latter has to iteratively solve an objective function to deform a body template to fit the scan, which is time-consuming while being also sensitive to noise and missing data. In this study, we propose the first approach for automatic contact-less anthropometric measurements extraction based on deep-learning (AM-DL). A novel module dubbed multiscale EdgeConv is proposed to learn local features from point clouds at multiple scales. Multiscale EdgeConv can be directly integrated with other neural networks for various tasks, e.g., classification of point clouds. We exploit this module to design an encoder–decoder architecture that learns to deform a template model to fit a given scan. The measurement values are then calculated on the deformed template model. To evaluate the proposed method, 27 female and 25 male subjects were scanned using a photogrametry-based scanner and measured by an experienced tailor. Experimental results on the synthetic ModelNet40 dataset and on the real scans demonstrate that the proposed method outperforms state-of-the-art methods, and performs sufficiently close to a professional tailor.

3D Anthropometric measurement extraction is of paramount importance for several applications such as clothing design, online garment shopping, and medical diagnosis, to name a few. State-of-the-art 3D anthropometric measurement extraction methods estimate the measurements either through some landmarks found on the input scan or by fitting a template to the input scan using optimization-based techniques. Finding landmarks is very sensitive to noise and missing data. Template-based methods address this problem, but the employed optimization-based template fitting algorithms are computationally very complex and time-consuming. To address the limitations of existing methods, we propose a deep neural network architecture which fits a template to the input scan and outputs the reconstructed body as well as the corresponding measurements. Unlike existing template-based anthropocentric measurement extraction methods, the proposed approach does not need to transfer and refine the measurements from the template to the deformed template, thereby being faster and more accurate. A novel loss function, especially developed for 3D anthropometric measurement extraction is introduced. Additionally, two large datasets of complete and partial front-facing scans are proposed and used in training. This results in two models, dubbed Anet-complete and Anet-partial, which extract the body measurements from complete and partial front-facing scans, respectively. Experimental results on synthesized data as well as on real 3D scans captured by a photogrammetry-based scanner, an Azure Kinect sensor, and the very recent TrueDepth camera system demonstrate that the proposed approach systematically outperforms the state-of-the-art methods in terms of accuracy and robustness.