10 March 2025 in ACM Transactions on Multimedia Computing, Communications, and Applications
Knowledge-aware video recommendation requires the ability of associating external knowledge to capture high-order connectivities between users and videos. One limitation of existing methods is that they only extract user interests at a granular level of relational paths by modeling high-order connectivities, which are coarse-grained in user interest modeling, failing to identify user–video relations at a finer grained level of semantics. In this article, we investigate the utility of semantic scene graphs in video recommendation scenario, which provide detailed, graph-based annotations of social situations depicted in video clips. We propose a new method named Scene-Centric Multi-Level Interest Miner (SCIMiner) , which explicitly models multi-level user interests. Specifically, we construct user–video graph, knowledge graph, and semantic scene graphs as hierarchical heterogeneous graphs. On top of the hierarchical graph representation, we propose two information aggregation strategies to capture user interests from different levels. Knowledge-aware aggregation scheme extracts coarse-grained user interests by aggregating relational paths in high-order connectivities, while scene-aware aggregation scheme models fine-grained user interests by capturing clip-level semantic commonality of user favorite videos. Furthermore, we adaptively distill complementary information about multi-level user interests extracted by different aggregation schemes and encode them into the representations of users and videos. Empirical results show that SCIMiner significantly outperforms the state-of-the-art methods. Further studies verify the complementary benefits of knowledge graph with semantic scene graphs in video recommendation scenario and the finer grained explainability for predictions.