With the advent of state of the art nature-inspired pure attention based models i.e. transformers, and their success in natural language processing (NLP), their extension to machine vision (MV) tasks was inevitable and much felt. Subsequently, vision transformers (ViTs) were introduced which are giving quite a challenge to the established deep learning based machine vision techniques. However, pure attention based models/architectures like transformers require huge data, large training times and large computational resources. Some recent works suggest that combinations of these two varied fields can prove to build systems which have the advantages of both these fields. Accordingly, this state of the art survey paper is introduced which hopefully will help readers get useful information about this interesting and potential research area. A gentle introduction to attention mechanisms is given, followed by a discussion of the popular attention based deep architectures. Subsequently, the major categories of the intersection of attention mechanisms and deep learning for machine vision (MV) based are discussed. Afterwards, the major algorithms, issues and trends within the scope of the paper are discussed.

Mathematics lies at the heart of engineering science and is very important for capturing and modeling of diverse processes. These processes may be naturally-occurring or man-made. One important engineering problem in this regard is the modeling of advanced mathematical problems and their analysis. Partial differential equations (PDEs) are important and useful tools to this end. However, solving complex PDEs for advanced problems requires extensive computational resources and complex techniques. Neural networks provide a way to solve complex PDEs reliably. In this regard, large-data models are new generation of techniques, which have large dependency capturing capabilities. Hence, they can richly model and accurately solve such complex PDEs. Some common large-data models include Convolutional neural networks (CNNs) and their derivatives, transformers, etc. In this literature survey, the mathematical background is introduced. A gentle introduction to the area of solving PDEs using large-data models is given. Various state-of-the-art large-data models for solving PDEs are discussed. Also, the major issues and future scope of the area are identified. Through this literature survey, it is hoped that readers will gain an insight into the area of solving PDEs using large-data models and pursue future research in this interesting area.