With the impact of the COVID-19 pandemic, the number of patients suffering from depression is rising around the world. It is important to diagnose depression early so that it may be treated as soon as possible. The self-response questionnaire, which has been used to diagnose depression in hospitals, is impractical since it requires active patient engagement. Therefore, it is vital to have a system that predicts depression automatically and recommends treatment. In this paper, we propose a smartphone-based depression prediction system. In addition, we propose depressive features based on multimodal sensor data for predicting depressive mood. The multimodal depressive features were designed based on depression symptoms defined in the Diagnostic and Statistical Manual of Mental Disorders (DSM-5). The proposed system comprises a “Mental Health Protector” application that collects data from smartphones and a big data-based cloud platform that processes large amounts of data. We recruited 106 mental patients and collected smartphone sensor data and self-reported questionnaires from their smartphones using the proposed system. Finally, we evaluated the performance of the proposed system’s prediction of depression. As the test dataset, 27 out of 106 participants were selected randomly. The proposed system showed 76.92% on an f1-score for 16 patients with depression disease, and in particular, 15 patients, 93.75%, were successfully predicted. Unlike previous studies, the proposed method has high adaptability in that it uses only smartphones and has a distinction of evaluating prediction accuracy based on the diagnosis.

Multimodal emotion recognition (MER) aims to enhance the understanding of human emotions by integrating visual, auditory, and textual modalities. However, previous MER approaches often depend on a dominant modality rather than considering all modalities, leading to poor generalization. To address this, we propose Causal Inference in Multimodal Emotion Recognition (CausalMER), which leverages counterfactual reasoning and causal graphs to capture relationships between modalities and reduce direct modality effects contributing to bias. This allows CausalMER to make unbiased predictions while being easily applied to existing MER methods in a model-agnostic manner, without requiring any architectural modifications. We evaluate CausalMER on the IEMOCAP and CMU-MOSEI datasets, widely used benchmarks in MER, and compare it with existing methods. On the IEMOCAP dataset with the MulT backbone, CausalMER achieves an average accuracy of 83.4%. On the CMU-MOSEI dataset, the average accuracies with MulT, PMR, and DMD backbones are 50.1%, 48.8%, and 48.8%, respectively. Experimental results demonstrate that CausalMER is robust in missing modality scenarios, as shown by its low standard deviation in performance drop gaps. Additionally, we evaluate modality contributions and show that CausalMER achieves balanced contributions from each modality, effectively mitigating direct biases from individual modalities.

Recent advances in artificial intelligence undeniably depend on vast amounts of high-quality data. However, a persistent global challenge is the restricted budgets allocated for data labeling. To address this, active learning emerges as a prominent and efficient strategy. It involves iterative selections of valuable data for labeling through a model and updating the model based on these selections. Nonetheless, the limited data available in each iteration renders the model susceptible to bias, resulting in potentially overconfident predictions. To mitigate this issue, we propose the Overcoming Overconfidence for Active Learning (OO4AL) framework. This framework comprises two parts: Cross-Mix-and-Mix, an augmentation strategy aimed at broadening the training distribution to calibrate the model, and Ranked Margin Sampling, a selection strategy that prevents the selection of overconfidence-inducing data by evaluating predictions. Through comprehensive experiments and analyses, we demonstrate that our framework facilitates efficient data selection by reducing overconfidence, though it can be readily implemented.

Text–Video Retrieval (TVR) retrieves videos that match a natural language query, but extending image–text models such as CLIP to videos is fundamentally limited by the lack of temporal modeling. Videos exhibit frame-wise heterogeneity in appearance and motion, and compressing all frames into a single representation often obscures temporal structure and semantic transitions. To address this, we propose Temporal-Aware Mixture-of-Experts for Text-Video Retrieval (TAME), a CLIP-based framework that jointly models frame-level structure and temporal relations. First, we integrate sparse Mixture-of-Experts (MoE) layers into both CLIP encoders and apply frame-consistent routing on the vision branch so that experts specialize according to frame-level visual patterns while preserving the original vision–language alignment. Second, we introduce Frame–Temporal (FT) tokens that aggregate global cross-frame information and feed it back to each frame, enabling the visual encoder to capture long-range temporal dependencies without harming local details. Third, we design a Cross-Temporal Interaction and Aggregation (CTIA) module that refines frame-wise sentence–video similarities through staged temporal filtering and fusion. Experiments on standard TVR benchmarks show that TAME consistently improves over CLIP-based baselines. On MSR-VTT, it yields a +4.0 R@1 improvement over CLIP4Clip, and also achieves consistent gains on DiDeMo, MSVD, LSMDC and ActivityNet. The code is available at https://github.com/sejong-rcv/TAME.

Recent advances in vision-language models (VLMs), have expanded the scope of scene graph generation by enabling open-world visual relation detection. Nevertheless, VLMs still suffer from inherent limitations in spatial reasoning, particularly in modeling spatial configurations and attributive cues. Prior studies on open-vocabulary scene graph generation (OV-SGG) have tended to rely on adjustments of the VLM backbone, while leaving the fundamental challenges of the task largely unaddressed. We argue that the core bottleneck of OV-SGG lies in the combinatorial explosion of possible \textless subject-predicate-object\textgreater \, triplets, combined with the need for adaptability to novel environments. To overcome this issue, we introduce a hyperbolic geometry based framework, exploiting its exponential expansion property as a natural remedy for the combinatorial complexity. Furthermore, we propose a visual–semantic hierarchical supervision scheme that explicitly aligns visual features with textual semantics across multiple levels of granularity, thereby enhancing relational reasoning. Extensive experiments on widely used SGG benchmarks, evaluated in both closed- and open-vocabulary settings, demonstrate consistent improvements in accuracy and generalization over state-of-the-art methods.