This paper investigates the application of self-organizing networks in solving the interference problem in underwater acoustic communication networks (UACNs) with the coexistence of multi-node. In this network, each node autonomously adjusts its power based on locally observed information without central controller intervention. Considering the non-convexity of the optimization problem with quality-of-service constraints and the dynamic nature of the underwater environment, we propose a reinforcement learning (RL)-based approach coupled with a distributed coordination mechanism, namely the multi-agent-based transfer Dyna-Q algorithm (MA-TDQ). This algorithm combines Q-learning with Dyna structure and transfer learning, and can quickly obtain optimal intelligent resource management strategies. Furthermore, we rigorously demonstrate the convergence of the MA-TDQ algorithm to Nash equilibrium. Simulation results indicate that the proposed distributed coordination learning algorithm outperforms other existing learning algorithms in terms of learning efficiency, network transmission rate, and communication service quality.

Practice is an essential means by which humans and animals engage in cognitive activities. Intelligent tutoring systems, with a crucial component of modelling learners’ cognitive processes during learning and optimizing their learning strategies, offer an excellent platform to investigate students’ practice-based cognitive processes. In related studies, modelling methods for cognitive processes have demonstrated commendable performance. Furthermore, researchers have extended their investigations using decision-theoretic approaches, such as a partially observable Markov decision process (POMDP), to induce learning strategies by modelling the students’ cognitive processes. However, the existing research has primarily centered around the modelling of macro-level instructional behaviors rather than the specific practice selection made by the students within the intricate realms of cognitive domains. In this paper, we adapt the POMDP model to represent relations between the student’s performance on cognitive tasks and his/her cognitive states. By doing so, we can predict his/her performance while inducing learning strategies. More specifically, we focus on question selection during the student’s real-time learning activities in an intelligent tutoring system. To address the challenges on modelling complex cognitive domains, we exploit the question types to automate parameter learning and subsequently employ information entropy techniques to refine learning strategies in the POMDP. We conduct the experiments in two real-world knowledge concept learning domains. The experimental results show that the performance of the learning strategies induced by our new model is superior to that of other learning strategies. Moreover, the new model has good reliability in predicting the student’s performance. Utilizing an intelligent tutoring system as the research platform, this article addresses the modelling and strategy induction challenges of practice-based cognitive processes with intricate structures, aiming to tutor students effectively. Our work provides a new approach of predicting the students’ performance as well as personalizing their learning strategies.

Reinforcement learning has made great success in recent years. Generally, the learning process requires a huge amount of interaction with the environment before an agent can achieve acceptable performance. This motivates many techniques, such as incorporating prior knowledge which is usually presented as experts’ demonstration, and using a probability distribution to represent state-and-action values, to accelerate the learning process. The methods perform well when the prior knowledge is genuinely correct and no much change occurs to the learning environment. However, the requirement is not perfectly realistic in many complex applications. The demonstration knowledge may not reflect the true environment and even be full of noise. In this paper, we introduce a dynamic distribution merging method to improve knowledge utilization in a general reinforcement learning algorithm, namely Q-learning. The new method adapts a normal distribution to represent state-action values and merges the prior and learned knowledge in a discriminative way. We theoretically analyze the new learning method and demonstrate its empirical performance over multiple problem domains.