Yong Rui

Yong Rui

Auto-generated profile, based on publicly available information

Yong Rui

School of Computer Science and Engineering, Southeast University, Nanjing, China

Research Focus & Metrics

Academic Career Length

24Years (2003-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 August 2018 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
DLDong Li
Dong Li
TYTing Yao
Ting Yao
LDLing-Yu Duan
Ling-Yu Duan
TMTao Mei
Tao Mei
YRYong Rui
Yong Rui
Recognizing actions in videos is not a trivial task, as video is an information-intensive media and includes multiple modalities. Moreover, on each modality, an action may only appear at some spatial regions or only a part of temporal video segments contain the action. A valid question is how to locate the attended spatial areas and selective video segments for action recognition. In this paper, we devise a general attention neural cell, named $AttCell$ , that estimates attention probability not only at each spatial location but also of each video segment in a temporal sequence. With $AttCell$ , a unified Spatio-Temporal Attention Networks (STAN) is proposed in the context of multiple modalities. Specifically, STAN extracts feature map of one convolutional layer as the local descriptors on each modality and pools the extracted descriptors with spatial attention measured by $AttCell$ as representation of each segment. Then, we concatenate the representation on each modality to seek a consensus on temporal attention, as a priori, to holistically fuse the combined representation of video segments to video representation for recognition. Our model differs from conventional deep networks focusing on attention mechanism as our temporal attention provides a principled and global guidance across different modalities and video segments. Extensive experiments are conducted on three public datasets of UCF101, CCV and THUMOS14, and our STAN consistently achieves superior results over several state-of-the-art techniques. More remarkably, we validate and demonstrate the effectiveness of our proposal when capitalizing on different number of modalities.

5 October 2017 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
Weiqing MinWeiqing Min
Weiqing Min
BBBing-Kun Bao
Bing-Kun Bao
SMShuhuan Mei
Shuhuan Mei
YZYaohui Zhu
Yaohui Zhu
YRYong Rui
Yong Rui
SJShuqiang Jiang
Shuqiang Jiang
Cuisine is a style of cooking and usually associated with a specific geographic region. Recipes from different cuisines shared on the web are an indicator of culinary cultures in different countries. Therefore, analysis of these recipes can lead to deep understanding of food from the cultural perspective. In this paper, we perform the first cross-region recipe analysis by jointly using the recipe ingredients, food images and attributes such as the cuisine and course (e.g., main dish and dessert). For that solution, we propose a Multi-modal Multi-attribute Culinary Culture Analysis (M2C2A) framework to discover the themes of ingredient bases and visualize them to enable various applications. We firstly propose a probabilistic theme model to discover cuisine-course-specific themes. The manifold ranking method is then utilized to incorporate deep visual features to retrieve food images for theme visualization. At last, we applied the theme modeling and visualization method for three applications: (1) multi-modal cuisine summarization with both recipe ingredients and images, (2) cuisine-course pattern analysis including theme-specific cuisine distribution and cuisine-specific course distribution of themes, and (3) cuisine recommendation for both cuisine-oriented and ingredient-oriented queries. Through the three applications, we can analyze the culinary cultures at both macro and micro levels. We conduct the experiment on a recipe database Yummly- 66K with 66,615 recipes from 10 cuisines in Yummly. Qualitative and quantitative evaluation results have validated the effectiveness of theme modeling and visualization, and demonstrated the advantage of the framework in utilizing the multi-modal and multi-attribute information to analyze and interpret the culinary cultures from different regions.

4 January 2018 in IEEE Transactions on Knowledge and Data Engineering
Verified
Journal Article
Closed access
DLDefu Lian
Defu Lian
YGYong Ge
Yong Ge
FZFuzheng Zhang
Fuzheng Zhang
NYNicholas Jing Yuan
Nicholas Jing Yuan
XXXing Xie
Xing Xie
YRYong Rui
Yong Rui
Location recommendation plays an essential role in helping people find attractive places. Though recent research has studied how to recommend locations with social and geographical information, few of them addressed the cold-start problem of new users. Because mobility records are often shared on social networks, semantic information can be leveraged to tackle this challenge. A typical method is to feed them into explicit-feedback-based content-aware collaborative filtering, but they require drawing negative samples for better learning performance, as users' negative preference is not observable in human mobility. However, prior studies have empirically shown sampling-based methods do not perform well. To this end, we propose a scalable Implicit-feedback-based Content-aware Collaborative Filtering (ICCF) framework to incorporate semantic content and to steer clear of negative sampling. We then develop an efficient optimization algorithm, scaling linearly with data size and feature size, and quadratically with the dimension of latent space. We further establish its relationship with graph Laplacian regularized matrix factorization. Finally, we evaluate ICCF with a large-scale LBSN dataset in which users have profiles and textual content. The results show that ICCF outperforms several competing baselines, and that user information is not only effective for improving recommendations but also coping with cold-start scenarios.

27 February 2023 in ACM Transactions on Multimedia Computing, Communications, and Applications
Verified
Journal Article
Open access
JYJin Yuan
Jin Yuan
SCShikai Chen
Shikai Chen
YZYao Zhang
Yao Zhang
ZSZhongchao Shi
Zhongchao Shi
Xin GengXin Geng
Xin Geng
YRYong Rui
Yong Rui
Multi-label classification aims to recognize multiple objects or attributes from images. The key to solving this issue relies on effectively characterizing the inter-label correlations or dependencies, which bring the prevailing graph neural network. However, current methods often use the co-occurrence probability of labels based on the training set as the adjacency matrix to model this correlation, which is greatly limited by the dataset and affects the model’s generalization ability. This article proposes a Graph Attention Transformer Network, a general framework for multi-label image classification by mining rich and effective label correlation. First, we use the cosine similarity value of the pre-trained label word embedding as the initial correlation matrix, which can represent richer semantic information than the co-occurrence one. Subsequently, we propose the graph attention transformer layer to transfer this adjacency matrix to adapt to the current domain. Our extensive experiments have demonstrated that our proposed methods can achieve highly competitive performance on three datasets.

11 May 2024
Verified
Conference Paper
Open access
YZYu Zhang
Yu Zhang
JSJingwei Sun
Jingwei Sun
LFLi Feng
Li Feng
CYCen Yao
Cen Yao
MFMingming Fan
Mingming Fan
YRYong Rui
Yong Rui
The proliferation of AI-powered search and recommendation systems has accelerated the formation of “filter bubbles” that reinforce people’s biases and narrow their perspectives. Previous research has attempted to address this issue by increasing the diversity of information exposure, which is often hindered by a lack of user motivation to engage with. In this study, we took a human-centered approach to explore how Large Language Models (LLMs) could assist users in embracing more diverse perspectives. We developed a prototype featuring LLM-powered multi-agent characters that users could interact with while reading social media content. We conducted a participatory design study with 18 participants and found that multi-agent dialogues with gamification incentives could motivate users to engage with opposing viewpoints. Additionally, progressive interactions with assessment tasks could promote thoughtful consideration. Based on these findings, we provided design implications with future work outlooks for leveraging LLMs to help users burst their filter bubbles.

16 January 2023 in ACM Computing Surveys
Verified
Journal Article
Open access
MLMeng Liu
Meng Liu
LNLiqiang Nie
Liqiang Nie
YWYunxiao Wang
Yunxiao Wang
MWMeng Wang
Meng Wang
YRYong Rui
Yong Rui
Video moment localization, also known as video moment retrieval, aims to search a target segment within a video described by a given natural language query. Beyond the task of temporal action localization whereby the target actions are pre-defined, video moment retrieval can query arbitrary complex activities. In this survey paper, we aim to present a comprehensive review of existing video moment localization techniques, including supervised, weakly supervised, and unsupervised ones. We also review the datasets available for video moment localization and group results of related work. In addition, we discuss promising future directions for this field, in particular large-scale datasets and interpretable video moment localization models.