With the wide deployments of heterogeneous networks, huge amounts of data with characteristics of high volume, high variety, high velocity, and high veracity are generated. These data, referred to multimodal big data, contain abundant intermodality and cross-modality information and pose vast challenges on traditional data fusion methods. In this review, we present some pioneering deep learning models to fuse these multimodal big data. With the increasing exploration of the multimodal big data, there are still some challenges to be addressed. Thus, this review presents a survey on deep learning for multimodal data fusion to provide readers, regardless of their original community, with the fundamentals of multimodal deep learning fusion method and to motivate new multimodal data fusion techniques of deep learning. Specifically, representative architectures that are widely used are summarized as fundamental to the understanding of multimodal deep learning. Then the current pioneering multimodal data fusion deep learning models are summarized. Finally, some challenges and future topics of multimodal data fusion deep learning models are described.

Deep learning, as the most important architecture of current computational intelligence, achieves super performance to predict the cloud workload for industry informatics. However, it is a non-trivial task to train a deep learning model efficiently since the deep learning model often includes a great number of parameters. In this paper, an efficient deep learning model based on the canonical polyadic decomposition is proposed to predict the cloud workload for industry informatics. In the proposed model, the parameters are compressed significantly by converting the weight matrices to the canonical polyadic format. Furthermore, an efficient learning algorithm is designed to train the parameters. Finally, the proposed efficient deep learning model is applied to the workload prediction of virtual machines on cloud. Experiments are conducted on the datasets collected from PlanetLab to validate the performance of the proposed model by comparing with other machine learning-based approaches for workload prediction of virtual machines. Results indicate that the proposed model achieves a higher training efficiency and workload prediction accuracy than state-of-the-art machine learning-based approaches, proving the potential of the proposed model to provide predictive services for industry informatics.

Due to digitalization, small and medium-sized enterprises (SMEs) have significantly enhanced their efficiency and productivity in the past few years. The process to automate SME transaction execution is getting highly multifaceted as the number of stakeholders of SMEs is connecting, accessing, exchanging, adding, and changing the transactional executions. The balanced lifecycle of SMEs requires partnership exchanges, financial management, manufacturing, and productivity stabilities, along with privacy and security. Interoperability platform issue is another critical challenging aspect while designing and managing a secure distributed Peer-to-Peer industrial development environment for SMEs. However, till now, it is hard to maintain operations of SMEs' integrity, transparency, reliability, provenance, availability, and trustworthiness between two different enterprises due to the current nature of centralized server-based infrastructure. This paper bridges these problems and proposes a novel and secure framework with a standardized process hierarchy/lifecycle for distributed SMEs using collaborative techniques of blockchain, the internet of things (IoT), and artificial intelligence (AI) with machine learning (ML). A blockchain with IoT-enabled permissionless network structure is designed called “B-SMEs” that provides solutions to cross-chain platforms. In this, B-SMEs address the lightweight stakeholder authentication problems as well. For that purpose, three different chain codes are deployed. It handles participating SMEs' registration, day-to-day information management and exchange between nodes, and analysis of partnership exchange-related transaction details before being preserved on the blockchain immutable storage. Whereas AI-enabled ML-based artificial neural networks are utilized, the aim is to handle and optimize day-to-day numbers of SME transactions; so that the proposed B-SMEs consume fewer resources in terms of computational power, network bandwidth, and preservation-related issues during the complete process of SMEs service deliverance. The simulation results present highlight the benefits of B-SMEs, increases the rate of ledger management and optimization while exchanging information between different chains, which is up to 17.3%, and reduces the consumption of the system’s computational resources down to 9.13%. Thus, only 14.11% and 7.9% of B-SME’s transactions use network bandwidth and storage capabilities compared to the current mechanism of SMEs, respectively.

Reducing energy consumption is a vital and challenging problem for the edge computing devices since they are always energy-limited. To tackle this problem, a deep Q-learning model with multiple DVFS algorithms was proposed for energy-efficient scheduling (DQL-EES). However, DQL-EES is highly unstable when using a single stacked auto-encoder to approximate the Q-function. Additionally, it cannot distinguish the continuous system states well since it depends on a Q-table to generate the target values for training parameters. In this paper, a double deep Q-learning model is proposed for energy-efficient edge scheduling (DDQ-EES). Specially, the proposed double deep Q-learning model includes a generated network for producing the Q-value for each DVFS algorithm and a target network for producing the target Q-values to train the parameters. Furthermore, the rectified linear units (ReLU) function is used as the activation function in the double deep Q-learning model, instead of the Sigmoid function in QDL-EES, to avoid gradient vanishing. Finally, a learning algorithm based on experience replay is developed to train the parameters of the proposed model. The proposed model is compared with DQL-EES on EdgeCloudSim in terms of energy saving and training time. Results indicate that our proposed model can save average $2\%-2.4\%$ energy and achieve a higher training efficiency than QQL-EES, proving its potential for energy-efficient edge scheduling.

Energy saving is a critical issue for real-time systems in embedded devices due to their limited energy supply. To reduce the energy consumption, a hybrid dynamic voltage and frequency scaling (DVFS) scheduling based on Q-learning (QL-HDS) was proposed by combining three DVFS techniques. However, QL-HDS discretizes the system state parameters with a certain step size, resulting in a poor distinction of the system states. More importantly, it is difficult for QL-HDS to learn a system for various task sets with a Q-table and limited training sets. In this paper, an energy-efficient scheduling scheme based on deep Q-learning model is proposed for periodic tasks in real-time systems (DQL-EES). Specially, a deep Q-learning model is designed by combining a stacked auto-encoder and a Q-learning model. In the deep Q-learning model, the stacked auto-encoder is used to replace the Q-function for learning the Q-value of each DVFS technology for any system state. Furthermore, a training strategy is devised to learn the parameters of the deep Q-learning model based on the experience replay scheme. The performance of the proposed scheme is evaluated by comparison with QL-HDS on different simulation task sets. Results show that the proposed algorithm can save average 4.2% energy.

Deep computation, as an advanced machine learning model, has achieved the sate-of-the-art performance for feature learning on big data in industrial Internet of Things. However, the current deep computation model usually suffers from over-fitting due to the lack of public available labeled training samples, limiting its performance for big data feature learning. Motivated by the idea of active learning, an adaptive dropout deep computation model with crowdsourcing to cloud is proposed for industrial IoT big data feature learning in this paper. First, a distribution function is designed to set the dropout rate for each hidden layer to prevent over-fitting for the deep computation model. Furthermore, the outsourcing selection algorithm based on maximum entropy is employed to choose appropriate samples from the training set to crowdsource on the cloud platform. Finally, an improved SLME scheme is presented to aggregate answers given by human workers and to update the parameters of the adaptive dropout deep computation model simultaneously. Extensive experiments are conducted to evaluate the performance of the presented model by comparing with the dropout deep computation model and other state-of-the-art crowdsourcing algorithms. The results demonstrate that the proposed model can prevent over-fitting effectively and aggregate the labeled samples to train the parameters of the deep computation model with crowdsouring for industrial IoT big data feature learning.