Deep Learning has implemented a wide range of applications and has become increasingly popular in recent years. The goal of multimodal deep learning (MMDL) is to create models that can process and link information using various modalities. Despite the extensive development made for unimodal learning, it still cannot cover all the aspects of human learning. Multimodal learning helps to understand and analyze better when various senses are engaged in the processing of information. This article focuses on multiple types of modalities, i.e., image, video, text, audio, body gestures, facial expressions, physiological signals, flow, RGB, pose, depth, mesh, and point cloud. Detailed analysis of the baseline approaches and an in-depth study of recent advancements during the past five years (2017 to 2021) in multimodal deep learning applications has been provided. A fine-grained taxonomy of various multimodal deep learning methods is proposed, elaborating on different applications in more depth. Last, main issues are highlighted separately for each domain, along with their possible future research directions.

Memes, cultural entities that spread predominantly through social media, have recently seen an influx of harmful content used for trolling and bullying. Detecting such harmful memes presents a significant challenge due to the semantic disparity between their textual and visual components. To address this issue, we propose a Dual-Branch Neural Network (DBNN) for bridging the semantic gap in harmful meme detection. The proposed DBNN framework utilizes global visual and textual embeddings to learn discriminative joint representations. By capturing the context between the textual and visual modalities, DBNN bridges the semantic gap, enabling robust detection of harmful content.We evaluate the performance of the proposed DBNN framework on the Harm-C and Harm-P benchmark datasets, employing Accuracy and Macro-Averaged Mean Absolute Error (MMAE) as evaluation metrics. Our experiments demonstrate that DBNN significantly outperforms state-of-the-art approaches, achieving higher accuracy and robustness in detecting harmful memes. This work highlights the potential of DBNN in effectively identifying harmful memes, contributing to a safer online environment.