Directing at various problems of the traditional Q-Learning algorithm, such as heavy repetition and disequilibrium of explorations, the reinforcement-exploration strategy was used to replace the decayed ε-greedy strategy in the traditional Q-Learning algorithm, and thus a novel self-adaptive reinforcement-exploration Q-Learning (SARE-Q) algorithm was proposed. First, the concept of behavior utility trace was introduced in the proposed algorithm, and the probability for each action to be chosen was adjusted according to the behavior utility trace, so as to improve the efficiency of exploration. Second, the attenuation process of exploration factor ε was designed into two phases, where the first phase centered on the exploration and the second one transited the focus from the exploration into utilization, and the exploration rate was dynamically adjusted according to the success rate. Finally, by establishing a list of state access times, the exploration factor of the current state is adaptively adjusted according to the number of times the state is accessed. The symmetric grid map environment was established via OpenAI Gym platform to carry out the symmetrical simulation experiments on the Q-Learning algorithm, self-adaptive Q-Learning (SA-Q) algorithm and SARE-Q algorithm. The experimental results show that the proposed algorithm has obvious advantages over the first two algorithms in the average number of turning times, average inside success rate, and number of times with the shortest planned route.

In MOOC learning, learners’ emotions have an important impact on the learning effect. In order to solve the problem that learners’ emotions are not obvious in the learning process, we propose a method to identify learner emotion by combining eye movement features and scene features. This method uses an adaptive window to partition samples and enhances sample features through fine-grained feature extraction. Using an adaptive window to partition samples can make the eye movement information in the sample more abundant, and fine-grained feature extraction from an adaptive window can increase discrimination between samples. After adopting the method proposed in this paper, the four-category emotion recognition accuracy of the single modality of eye movement reached 65.1% in MOOC learning scenarios. Both the adaptive window partition method and the fine-grained feature extraction method based on eye movement signals proposed in this paper can be applied to other modalities.

With the rapid development of intelligent transportation systems, the increasing number of connected vehicles and onboard sensing devices has generated substantial data traffic and placed stringent demands on communication resources in the Internet of Vehicles. In three-dimensional urban environments, physical blockage, high mobility, and co-channel interference can cause temporary link disruptions, preventing the timely dissemination of safety-critical vehicular information. This study investigates joint resource allocation for Cellular Vehicle-to-Everything (C-V2X) networks with heterogeneous vehicular platoons. We propose an Age of Information (AoI)- and Fairness-Aware Multi-Agent Resource Allocation (AF-MARL) 1 under imperfect and delayed channel state information. Its novelty lies in combining an AoI-dependent power cost with a cross-platoon AoI-dispersion penalty within a multi-task multi-agent reinforcement learning (MARL) architecture, thereby reducing unnecessary high-power transmission by favorable platoons while explicitly protecting disadvantaged platoons without adding Actor or Critic networks. Simulation results show that AF-MARL reduces interference and persistent AoI outages while maintaining reliable Cooperative Awareness Message (CAM) delivery under different platoon spacings and sizes.

With the rapid development of the Internet of Vehicles (IoV), the explosive growth of data traffic within the system has led to a surge in demand for spectrum resources. However, the strict limitations on spectrum supply make the construction of an efficient and reasonable resource allocation scheme crucial for IoV. To maximize social benefits and improve security in the resource allocation process under IoV spectrum scarcity, this paper proposes a dynamic spectrum allocation (DSA) scheme based on a consortium blockchain framework. In this scheme, we design a demand-based vehicle priority classification method and propose a novel hybrid consensus mechanism—PhDPoR—which integrates practical byzantine fault tolerance (PBFT) and Hierarchical Delegated Proof of Reputation. Furthermore, we construct a multi-leader, multi-follower (MLMF) Stackelberg game model and utilize smart contracts to implement an immutable on-chain record of spectrum resource allocation, thereby deriving the optimal spectrum pricing and purchase strategy. Experimental results show that the proposed scheme not only effectively optimizes the utility of both supply and demand sides and improves overall social benefits while ensuring efficiency, but also significantly outperforms baseline algorithms in identifying and mitigating malicious nodes, thus verifying its feasibility and application value in complex IoV environments.