Hongchen Luo

Hongchen Luo

Auto-generated profile, based on publicly available information

Hongchen Luo

College of Information Science and Engineering, Northeastern University, Shenyang, Liaoning, China

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
8 August 2022 in International Journal of Computer Vision
Verified
Journal Article
Closed access
PDF file
WZWei Zhai
Wei Zhai
HLHongchen Luo
Hongchen Luo
JZJing Zhang
Jing Zhang
YCYang Cao
Yang Cao
DTDacheng Tao
Dacheng Tao

11 September 2023 in IEEE Transactions on Neural Networks and Learning Systems
Verified
Journal Article
Closed access
HLHongchen Luo
Hongchen Luo
WZWei Zhai
Wei Zhai
JZJing Zhang
Jing Zhang
YCYang Cao
Yang Cao
DTDacheng Tao
Dacheng Tao
Visual affordance grounding aims to segment all possible interaction regions between people and objects from an image/video, which benefits many applications, such as robot grasping and action recognition. Prevailing methods predominantly depend on the appearance feature of the objects to segment each region of the image, which encounters the following two problems: 1) there are multiple possible regions in an object that people interact with and 2) there are multiple possible human interactions in the same object region. To address these problems, we propose a hand-aided affordance grounding network (HAG-Net) that leverages the aided clues provided by the position and action of the hand in demonstration videos to eliminate the multiple possibilities and better locate the interaction regions in the object. Specifically, HAG-Net adopts a dual-branch structure to process the demonstration video and object image data. For the video branch, we introduce hand-aided attention to enhance the region around the hand in each video frame and then use the long short-term memory (LSTM) network to aggregate the action features. For the object branch, we introduce a semantic enhancement module (SEM) to make the network focus on different parts of the object according to the action classes and utilize a distillation loss to align the output features of the object branch with that of the video branch and transfer the knowledge in the video branch to the object branch. Quantitative and qualitative evaluations on two challenging datasets show that our method has achieved state-of-the-art results for affordance grounding. The source code is available at: https://github.com/lhc1224/HAG-Net .

26 December 2023 in International Journal of Computer Vision
Verified
Journal Article
Closed access
PDF file
HLHongchen Luo
Hongchen Luo
WZWei Zhai
Wei Zhai
JZJing Zhang
Jing Zhang
YCYang Cao
Yang Cao
DTDacheng Tao
Dacheng Tao

16 June 2024
Verified
Conference Paper
Closed access
YYYuhang Yang
Yuhang Yang
WZWei Zhai
Wei Zhai
HLHongchen Luo
Hongchen Luo
YCYang Cao
Yang Cao
ZZZheng-Jun Zha
Zheng-Jun Zha
Learning 3D human-object interaction relation is piv-otal to embodied AI and interaction modeling. Most existing methods approach the goal by learning to predict isolated interaction elements, e.g., human contact, object affordance, and human-object spatial relation, primarily from the perspective of either the human or the object. Which underexploit certain correlations between the interaction counterparts (human and object), and struggle to address the uncertainty in interactions. Actually, objects' functionalities potentially affect humans' interaction intentions, which reveals what the interaction is. Mean-while, the interacting humans and objects exhibit matching geometric structures, which presents how to interact. In light of this, we propose harnessing these inherent correlations between interaction counterparts to mitigate the uncertainty and jointly anticipate the above interaction el-ements in 3D space. To achieve this, we present LEMON (LEarning 3D huMan-Object iNteraction relation), a unified model that mines interaction intentions of the counter-parts and employs curvatures to guide the extraction of ge-ometric correlations, combining them to anticipate the interaction elements. Besides, the 3D Interaction Relation dataset (3DIR) is collected to serve as the test bed for training and evaluation. Extensive experiments demonstrate the superiority of LEMON over methods estimating each element in isolation. The code and dataset are available at https://yyvhang.github.io/LEMON.

17 August 2022 in IEEE Transactions on Artificial Intelligence
Verified
Journal Article
Closed access
LLLiangsheng Lu
Liangsheng Lu
WZWei Zhai
Wei Zhai
HLHongchen Luo
Hongchen Luo
YKYu Kang
Yu Kang
YCYang Cao
Yang Cao
Affordance detection, which refers to perceiving objects with potential action possibilities in images, is a challenging task since the possible affordance depends on the person's purpose in real-world application scenarios. The existing works mainly extract the inherent human-object dependencies from image/video to accommodate affordance properties that change dynamically. In this paper, we explore to perceive affordances from a vision-language perspective, and consider the challenging phrase-based affordance detection task, i.e., given a set of phrases describing the potential actions, all the object regions in a scene with the same affordance should be detected. To this end, we propose a Cyclic Bilateral Consistency Enhancement Network (CBCE-Net) to align language and vision features in a progressive manner. Specifically, the presented CBCE-Net consists of a mutual guided vision-language module that updates the common features of vision and language in a progressive manner, and a Cyclic Interaction Module (CIM) that facilitates the perception of possible interaction with objects in a cyclic manner. In addition, we extend the public Purpose-driven Affordance Dataset (PAD) by annotating affordance categories with short phrases. The extensive contrastive experimental results demonstrate the superior performance of our method over nine typical methods from four relevant fields in terms of both objective metrics and visual quality. The related code and dataset will be released at https://github.com/lulsheng/CBCE-Net. Impact Statement—Affordance learning is concerned with the possible set of actions that an environment can offer to an actor. At present, almost all work focus on utilizing cues of human-object interactions in visual media such as images and videos to learn affordances. Few work explores affordance using language information, which is crucial to building more intelligent agents because we live a multimodal world. In this paper, we propose a phrase-based affordance detection task combining vision and language modalities. We first build a vision-language dataset based on the public Purpose-driven Affordance Dataset (PAD). Then we propose a method to fuse and align multi-modal features. Our method outperforms relevant methods from several other fields in terms of both objective metrics and visual quality. The proposed dataset and method can facilitate building more intelligent agents that can better comprehend the surrounding environment. For instance, the phase-based affordance detection can be used in building more intelligent home service systems. The agents can understand instructions of instructions without specific entities.

22 July 2025 in IEEE Transactions on Automation Science and Engineering
Verified
Journal Article
Closed access
HDHuilin Deng
Huilin Deng
HLHongchen Luo
Hongchen Luo
WZWei Zhai
Wei Zhai
YGYanming Guo
Yanming Guo
YCYang Cao
Yang Cao
YKYu Kang
Yu Kang
Zero-shot anomaly detection (ZSAD) enables the inspection of unseen objects by bridging textual prompts and visual features, showing great potential in flexible manufacturing. While existing ZSAD methods rely on predefined prompts and struggle with unseen defects, Multimodal Large Language Models (MLLMs) offer promising solutions through their generative and interpretative capabilities. However, adapting MLLMs to Industrial Anomaly Detection (IAD) remains challenging due to fine-grained anomaly patterns and subtle visual distinctions. We propose VMAD (Visual-enhanced MLLM Anomaly Detection), a framework that enriches MLLM with visual IAD knowledge through two key components: a Defect-Sensitive Structure Learning scheme that transfers patch-similarities for improved discrimination, and a Locality-enhanced Token Compression that leverages multi-level local features for fine-grained detection. We also introduce RIAD, a comprehensive IAD dataset with detailed anomaly annotations. Extensive experiments on MVTec-AD, Visa, WFDD, and RIAD demonstrate VMAD’s superior performance. The dataset and code will be publicly available at https://github.com/denghuilin-cyber/VMAD.