Deep Learning (DL) is a subfield of machine learning that significantly impacts extracting new knowledge. By using DL, the extraction of advanced data representations and knowledge can be made possible. Highly effective DL techniques help to find more hidden knowledge. Deep learning has a promising future due to its great performance and accuracy. We need to understand the fundamentals and the state‐of‐the‐art of DL to leverage it effectively. A survey on DL ways, advantages, drawbacks, architectures, and methods to have a straightforward and clear understanding of it from different views is explained in the paper. Moreover, the existing related methods are compared with each other, and the application of DL is described in some applications, such as medical image analysis, handwriting recognition, and so on.

The Internet of Medical Things (IoMT), an application of the Internet of Things (IoT) in the medical domain, allows data to be transmitted across communication networks. In particular, IoMT can help improve the quality of life of citizens and older people by monitoring and managing the body’s vital signs, including blood pressure, temperature, heart rate, and others. Since IoMT has become the main platform for information exchange and making high-level decisions, it is necessary to guarantee its reliability and security. The growth of IoMT in recent decades has attracted the interest of many experts. This study provides an in-depth analysis of IoT and IoMT by focusing on security concerns from different points of view, making this comprehensive survey unique compared to other existing studies. A total of 187 articles from 2010 to 2022 are collected and categorized according to the type of applications, year of publications, variety of applications, and other novel perspectives. We compare the current studies based on the above criteria and provide a comprehensive analysis to pave the way for researchers working in this area. In addition, we highlight the trends and future work. We have found that blockchain, as a key technology, has solved many problems of security, authentication, and maintenance of IoT systems due to the decentralized nature of the blockchain. In the current study, this technology is examined from the application fields’ points of view, especially in the health sector, due to its additional importance compared to other fields.

Although the vision-and-language pretraining (VLP) equipped cross-modal image-text retrieval (ITR) has achieved remarkable progress in the past two years, it suffers from a major drawback: the ever-increasing size of VLP models restrict its deployment to real-world search scenarios (where the high latency is unacceptable). To alleviate this problem, we present a novel plug-in dynamic contrastive distillation (DCD) framework to compress the large VLP models for the ITR task. Technically, we face the following two challenges: 1) the typical uni-modal metric learning approach is difficult to directly apply to cross-modal task, due to the limited GPU memory to optimize too many negative samples during handling cross-modal fusion features. 2) it is inefficient to static optimize the student network from different hard samples, which have different effects on distillation learning and student network optimization. We try to overcome these challenges from two points. First, to achieve multi-modal contrastive learning, and balance the training costs and effects, we propose to use a teacher network to estimate the difficult samples for students, making the students absorb the powerful knowledge from pre-trained teachers, and master the knowledge from hard samples. Second, to dynamic learn from hard sample pairs, we propose dynamic distillation to dynamically learn samples of different difficulties, from the perspective of better balancing the difficulty of knowledge and students' self-learning ability. We successfully apply our proposed DCD strategy on two state-of-the-art vision-language pretrained models, i.e. ViLT and METER. Extensive experiments on MS-COCO and Flickr 30 K benchmarks show the effectiveness and efficiency of our DCD framework. Encouragingly, we can speed up the inference at least 129 × compared to the existing ITR models. We further provide in-depth analyses and discussions that explain where the performance improvement comes from. We hope our work can shed light on other tasks that require distillation and contrastive learning.

Recent years have witnessed lots of attacks targeted at the widespread Internet of Things (IoT) devices and malicious activities conducted by compromised IoT devices. After some notorious IoT malware released their source code, many new variants emerge, which are usually more powerful and stealthy. Although numerous existing studies have analyzed some exposed families, there is a lack of systematic study to make full use of them, which can be a fundamental step for provenance, triage, labeling, lineage analysis, and authorship attribution. The key challenge of conducting an IoT malware evolutionary study is how to collect sufficient and accurate information about malware and identify the relationships among them. In this article, we take the first step to investigate the IoT malware evolution by leveraging the information from two sources that complement each other. First, we crawl online articles about IoT malware and employ natural language processing techniques to extract the features of malware samples and their relationships with other malware family, which allow us to form the basic lineage graph. Second, we collect real malware samples through our widely deployed honeypots and design a new classifier to group them into families and identify lineage relationships among them. Such results are used to enhance the basic lineage graph. Eventually, we construct the final lineage graph for 72 IoT malware families by correlating the information from the aforementioned sources, which can help the research community better understand and fight IoT malware now and in the future. Our study has been incorporated into the threat awareness system of NSFOCUS company.