Emotion recognition plays an important role in the field of human–computer interaction (HCI). An electroencephalogram (EEG) is widely used to estimate human emotion owing to its convenience and mobility. Deep neural network (DNN) approaches using an EEG for emotion recognition have recently shown remarkable improvement in terms of their recognition accuracy. However, most studies in this field still require a separate process for extracting handcrafted features despite the ability of a DNN to extract meaningful features by itself. In this paper, we propose a novel method for recognizing an emotion based on the use of three-dimensional convolutional neural networks (3D CNNs), with an efficient representation of the spatio-temporal representations of EEG signals. First, we spatially reconstruct raw EEG signals represented as stacks of one-dimensional (1D) time series data to two-dimensional (2D) EEG frames according to the original electrode position. We then represent a 3D EEG stream by concatenating the 2D EEG frames to the time axis. These 3D reconstructions of the raw EEG signals can be efficiently combined with 3D CNNs, which have shown a remarkable feature representation from spatio-temporal data. Herein, we demonstrate the accuracy of the emotional classification of the proposed method through extensive experiments on the DEAP (a Dataset for Emotion Analysis using EEG, Physiological, and video signals) dataset. Experimental results show that the proposed method achieves a classification accuracy of 99.11%, 99.74%, and 99.73% in the binary classification of valence and arousal, and, in four-class classification, respectively. We investigate the spatio-temporal effectiveness of the proposed method by comparing it to several types of input methods with 2D/3D CNN. We then verify the best performing shape of both the kernel and input data experimentally. We verify that an efficient representation of an EEG and a network that fully takes advantage of the data characteristics can outperform methods that apply handcrafted features.

Split Federated Learning (SFL) has recently emerged as a promising distributed learning technology that leverages the strengths of both federated and split learning. In this approach, clients are responsible for training only part of the model, termed the client-side model, thereby alleviating their computational burden. Then, clients can enhance their convergence speed by synchronizing these client-side models. Consequently, SFL has received significant attention from both industry and academia, with diverse applications in 6G networks. However, while offering considerable benefits, SFL introduces additional communication overhead when interacting with servers. Moreover, the current SFL method presents several privacy concerns during frequent interactions. In this context, the choice of the cut layer in SFL, which splits the model into both client- and server-side models, can substantially impact the energy consumption of clients and their privacy because it influences the training burden and output of the client-side models. Correspondingly, extensive research is required to analyze the impact of cut layer selection, and careful consideration should be given to this aspect. Therefore, this study provides a comprehensive overview of the SFL process and reviews its state-of-the-art. We thoroughly analyze the energy consumption and privacy regarding the cut layer selection in SFL by considering the influence of various system parameters on the selection strategy. Moreover, we provide an illustrative example of cut layer selection to minimize the clients’ risk of reconstructing raw data at the server. This is done while sustaining energy consumption within the required energy budget, which involves trade-offs. We also discuss other control variables that can be optimized in conjunction with the cut layer selection. Finally, we highlight open challenges in this field. These are promising avenues for future research and development.

As artificial intelligence (AI)- or deep-learning-based technologies become more popular, the main research interest in the field is not only on their accuracy, but also their efficiency, e.g., the ability to give immediate results on the users’ inputs. To achieve this, there have been many attempts to embed deep learning technology on intelligent sensors. However, there are still many obstacles in embedding a deep network in sensors with limited resources. Most importantly, there is an apparent trade-off between the complexity of a network and its processing time, and finding a structure with a better trade-off curve is vital for successful applications in intelligent sensors. In this paper, we propose two strategies for designing a compact deep network that maintains the required level of performance even after minimizing the computations. The first strategy is to automatically determine the number of parameters of a network by utilizing group sparsity and knowledge distillation (KD) in the training process. By doing so, KD can compensate for the possible losses in accuracy caused by enforcing sparsity. Nevertheless, a problem in applying the first strategy is the unclarity in determining the balance between the accuracy improvement due to KD and the parameter reduction by sparse regularization. To handle this balancing problem, we propose a second strategy: a feedback control mechanism based on the proportional control theory. The feedback control logic determines the amount of emphasis to be put on network sparsity during training and is controlled based on the comparative accuracy losses of the teacher and student models in the training. A surprising fact here is that this control scheme not only determines an appropriate trade-off point, but also improves the trade-off curve itself. The results of experiments on CIFAR-10, CIFAR-100, and ImageNet32 × 32 datasets show that the proposed method is effective in building a compact network while preventing performance degradation due to sparsity regularization much better than other baselines.

Polyp segmentation is challenging because the boundary between polyps and mucosa is ambiguous. Several models have considered the use of attention mechanisms to solve this problem. However, these models use only finite information obtained from a single type of attention. We propose a new dual-attention network based on shallow and reverse attention modules for colon polyps segmentation called SRaNet. The shallow attention mechanism removes background noise while emphasizing the locality by focusing on the foreground. In contrast, reverse attention helps distinguish the boundary between polyps and mucous membranes more clearly by focusing on the background. The two attention mechanisms are adaptively fused using a “Softmax Gate”. Combining the two types of attention enables the model to capture complementary foreground and boundary features. Therefore, the proposed model predicts the boundaries of polyps more accurately than other models. We present the results of extensive experiments on polyp benchmarks to show that the proposed method outperforms existing models on both seen and unseen data. Furthermore, the results show that the proposed dual attention module increases the explainability of the model.

The detection of abnormal postures, such as that of a reclining person, is a crucial part of visual surveillance. Further, even regular poses can appear rotated because of incongruity between the image and the angle of a pre-installed camera. However, most existing human pose estimation methods focus on small rotational changes, i.e., those less than 50 degrees, and they seldom consider robust human pose estimation for more drastic rotational changes. To the best of our knowledge, there have been no reports on the robustness of human pose estimation for rotational changes through large angles. In this study, we propose a robust human pose estimation method by creating a path for learning new rotational changes based on a self-supervised method and by combining the results with those obtained from a path based on a supervised method. Furthermore, a combination module composed of a convolutional layer is trained complementarily by both paths of the network to produce robust results for various rotational changes. We demonstrate the robustness of the proposed method with extensive experiments on images generated by rotating the elements of standard benchmark datasets. We fully analyze the rotational characteristics of the state-of-the-art human pose estimators and the proposed method. On the COCO Keypoint Detection dataset, the proposed method attains more than 15% improvement in the mean of average precision compared to the state-of-the-art method, and the standard deviation of the performance is improved by more than 4.7 times.

In the weakly supervised temporal video grounding study, previous methods use predetermined single Gaussian proposals which lack the ability to express diverse events described by the sentence query. To enhance the expression ability of a proposal, we propose a Gaussian mixture proposal (GMP) that can depict arbitrary shapes by learning importance, centroid, and range of every Gaussian in the mixture. In learning GMP, each Gaussian is not trained in a feature space but is implemented over a temporal location. Thus the conventional feature-based learning for Gaussian mixture model is not valid for our case. In our special setting, to learn moderately coupled Gaussian mixture capturing diverse events, we newly propose a pull-push learning scheme using pulling and pushing losses, each of which plays an opposite role to the other. The effects of components in our scheme are verified in-depth with extensive ablation studies and the overall scheme achieves state-of-the-art performance. Our code is available at https://github.com/sunoh-kim/pps.