Deep network recommendation is a cutting-edge topic in current recommendation system research, which as a combination of recommendation systems and deep learning theory can effectively improve recommendation accuracy. In a real recommendation scenario, all the effective information in a data set should be extracted, both explicit and implicit, because the comprehensive degree of information is proportional to the recommendation performance. This paper proposes an enhanced multi-modal recommendation based on alternate training with knowledge graph representation (SI-MKR) based on the MKR deep learning recommendation model. Our framework is an enhanced recommendation system based on knowledge graph representation, using valuable external knowledge as multi-modal information. The SI-MKR model solves the problem of ignoring the diversity of data types in the multi-modal knowledge-based recommendation system, which adds user and item attribute information from a knowledge graph as an enhancement recommendation multi-tasking training. By analysing the content of the item and user attributes, the SI-MKR model classifies the attributes of the items and users, processes the text type attributes and multi-value type attributes separately for feature extraction, and other types of attributes are used as inputs to the knowledge graph embedding unit. In addition, the knowledge graph data form a triplet unit, thus continuing the knowledge graph data training process. The feature extraction unit of the knowledge graph and the recommended unit are connected through the cross-compression unit for alternate training. During the deep learning framework training process, the recommendation system’s item has a potential correlation with the head entity in the knowledge graph which embodies the idea of multi-tasking. Through extensive experiments on real-world datasets, we demonstrate that SI-MKR achieves substantial gains in movie recommendation over advanced model baselines. Even user-item interactions are sparse, SI-MKR maintains better performance than the MKR model.

The main idea of a session-based recommendation system is to model the user’s historical click sequence and then summarize user preferences and predict the items the user will interact with. The session recommendation model based on graph neural networks has attracted much attention in recent years because it can accurately obtain the local relationship between items. However, the traditional session recommendation model based on graph neural Networks lack the use of user’s higher-order features or fail to address the impact of item position information on the current session, which are both critical to the recommendation system. In addition, some models proposed the position information while neglects the click frequency information. We propose a graph network recommendation model called GPAN based on position attention in response to the abovementioned problems. Specifically, we propose a novel high-low order session perceptron that uses the perceptron to model undirected and directed graphs separately to obtain high and low order item representations in a session. For position information, we designed a position layer to calculate independently. Finally, the user’s short-term preference and long-term preference are aggregated to obtain the recommendation sequence. The results through a large number of experiments on three real datasets show that the performance of the proposed GPAN model is the best.

Multi-modal image fusion mainly refers to the feature fusion of two or more different images taken from the same perspective range to increase the amount of information contained in an image. This study proposes a multi-modal image fusion deep network called the MFF network. Compared with traditional image fusion models, the MFF network decomposes high-frequency features more finely. In contrast to popular transformer networks, the MFF network utilizes multiple filter networks for the corresponding high and low-frequency feature extraction, thereby improving the model training and inference time. First, GaborNet filtering modules were used by the MFF network to extract high-frequency texture features and invertible neural networks (INN) modules are employed for extracting high-frequency edge features. These two sets of features constitute the high-frequency characteristics of an image. The LEF module is utilized as a low-pass filter to acquire the low-frequency characteristics of an image. The method involving low-frequency feature correlation and high-frequency feature non-correlation was used for image training and fusion purposes. Experiments demonstrate that the MFF model yields good results for visible-infrared image fusion.