The integration of wireless sensor network (WSN) and cognitive radio (CR) technology enables a new paradigm of communication: cognitive radio sensor networks (CRSN). The existing WSN clustering algorithm cannot consider the advantage of channel resource brought by CR function in CRSN, and the CR network (CRN) clustering algorithm is designed based on the infinite energy nodes; thus both algorithms cannot operate with energy efficiency in CRSN. The paper proposes a low-energy adaptive uneven clustering hierarchy for CRSN, which can not only consider the advantage of the channel resource in reducing the energy consumption but also employ uneven clustering method for balancing the energy consumption among the cluster heads under multiple hops transmission means. Simulation results show that compared with the existing several typical clustering algorithms including WSN and CRSN clustering algorithms, low-energy adaptive clustering hierarchy (LEACH), HEED, energy-efficient unequal clustering (EEUC), cognitive LEACH (CogLEACH), and distributed spectrum-aware clustering (DSAC), the proposed algorithm can not only efficiently balance the energy consumption among cluster heads and network load in CRSN but also remarkably prolong the network lifetime.

In order to cope with the resource shortage problem brought by cognitive radio technology in cognitive radio sensor network (CRSN), a new CRSN called heterogeneous CRSN (HCRSN) is proposed, where cognitive nodes (CNs) and sensor nodes (SNs) are separated and undertake different functions. Different from the existing clustering algorithms for homogeneous nodes based WSN, the clustering algorithm for HCRSN needs to consider the distribution of CNs among clusters such that enough high channel detection probability of each cluster can be guaranteed by the lowest deployment cost. Therefore, this paper first proposes a heterogeneous nodes based low energy adaptive clustering hierarchy (HLEACH) algorithm. In the algorithm, the sink node first updates the global information including the optimal number of clusters and average cluster radius and then broadcast it. Each CN calculates its competition radius after receiving the broadcasting information, and then start the competition for CHs based on the proposed competition rules. The elected CHs are finally censored targeting the optimal number of clusters to optimize the distribution of final CHs. In clusters’ formation stage, non-CH CNs and SNs synthetically consider the distance and the connection degree of CHs such that the distribution of CNs among clusters and the energy consumption among CHs can be energy-efficiently balanced. The simulation results show that the proposed algorithm can not only effectively balance the distribution of CNs among clusters, guaranteeing enough high channel detection probability of each cluster and network energy utilization, but also balance the energy consumption among CHs, eventually prolong the network lifetime. Finally, the optimal deployment proportion of numbers and initial energy of the two types of nodes is also theoretical derived to maximize the energy utilization efficiency (i.e. the ratio of the network lifetime to the deployment cost).

As the demand for mobile data traffic continues to grow, offloading data traffic to unlicensed spectrum is a promising approach that can relieve the pressure on cellular systems. Therefore, it is an urgent need to propose an unlicensed spectrum access method to guarantee the harmonious and efficient coexistence between cellular network technologies such as LTE and incumbent users such as WiFi in the unlicensed spectrum. However, existing coexistence schemes such as licensed assisted access (LAA) and LTE-unlicensed (LTE-U) still suffer from inefficient spectrum utilization and unsatisfactory fairness. In the paper, we formulate the optimization problem of the unlicensed spectrum access among multiple small bases (SBSs) as a game, and then solve the Nash Equilibrium (NE) with cooperative and distributed multi-agent deep reinforcement learning (MADRL). Specifically, a two level access framework for the coexistence scenario, which consists of feedback cycle and executive cycle, is first proposed, and then the key elements of MADRL including state, action, reward and Q-network are designed in detail based on the proposed access framework. To overcome the problems of learning divergence and prohibitive computation overhead in the coexistence scenario with multiple SBSs due to the non-stability phenomena, we adopt the mean field technology to solve the NE, which can simplify the process of solving NE by converting the interaction of an agent with the remaining multiple agents into an action with the average effect of them. Simulation results show that 1) the proposed algorithm can overcome the learning divergence problem and converge to the NE quickly, and 2) the proposed algorithm can achieve the bi-objective optimization of total throughput and fairness of the coexistence network, and can achieve better performance in terms of throughput and fairness compared with the baseline methods such as Cat-4 LBT, Cooperative LBT and Random schemes.

The energy detection technology is recommended in the licensed assisted access (LAA) scheme by 3GPP because of its simplicity and low cost. However, due to its inherent limitation, there may exist imperfect channel detection, which can lead to the decrease of the channel utilization efficiency and the deterioration of fairness. The imperfect detection can generally be represented by the detection probability and false alarm probability, which depend on detection time, signal to noise ration (SNR), sampling rate and energy threshold. However, among the parameters, only the energy threshold can be dominated by LAA small base stations (SBSs) in the LAA scheme. Therefore, the energy threshold should be dynamically adjusted in the changeable channel environment such that the detection accuracy can improved as high as possible. Consider the fact that the optimization theory cannot be used to optimize the energy threshold since the expressions of performance indexes about the energy threshold are extremely complex, a Q-learning based energy threshold optimization algorithm (QLET) is thus proposed in the paper, where LAA SBSs act as the agent, the energy threshold is defined as the agent action, the different combinations of fairness and throughput are defined as the agent states, and the fairness and the reward function are also redefined. In order to ensure the smooth implementation of the proposed QLET algorithm, the information exchange mechanism, where the sending-confirmation mechanism and 1- persistent CSMA are used, is also proposed. Based on the proposed QL framework, the agent can learn the optimal energy threshold by repeatedly interacting with the environment, which enables the coexistence system to obtain the best coexistence performance. A large number of simulation results show that the proposed QLET is superior to the traditional fixed energy threshold scheme (FET) in terms of the fairness, WiFi collision probability and transmission delay, and that QLET is almost the same as FET in term of throughput.

In order to make up for the extra energy consumption caused by spectrum sensing and prolong the network lifetime, the Energy Harvesting (EH) module is introduced to the spectrum sensing nodes in CRSN and thus form one new type of network: energy harvesting aided node-Heterogeneous CRSN (EH-HCRSN). Take into account the fact that the existing spectrum sensing algorithms are difficult to be applied to the real practice (due to the high cost of algorithm implementation or network deployment) and may lead to the waste of channel and energy resources, an energy-efficient spectrum sensing algorithm (EESS) is proposed in the paper. The proposed algorithm can maximize the energy utilization efficiency under the premise of detecting enough available time of channels, which is of great significance to reduce the network deployment cost and promote the green communication of WSN and even the large-scale deployment of Internet of Things applications. The simulation results demonstrate that the proposed algorithm can greatly improve the energy utilization efficiency of spectrum sensing nodes and channel utilization, and significantly reduce the deployment cost of network equipments.