In recent times, transformer-based deep learning models have risen in prominence in the field of machine learning for a variety of tasks such as computer vision and text generation. Given this increased interest, a historical outlook at the development and rapid progression of transformer-based models becomes imperative in order to gain an understanding of the rise of this key architecture. This paper presents a survey of key works related to the early development and implementation of transformer models in various domains such as generative deep learning and as backbones of large language models. Previous works are classified based on their historical approaches, followed by key works in the domain of text-based applications, image-based applications, and miscellaneous applications. A quantitative and qualitative analysis of the various approaches is presented. Additionally, recent directions of transformer-related research such as those in the biomedical and timeseries domains are discussed. Finally, future research opportunities, especially regarding the multi-modality and optimization of the transformer training process, are identified.

Pandemics can result in large morbidity and mortality rates that can cause significant adverse effects on the social and economic situations of communities. Monitoring and predicting the spread of pandemics helps the concerned authorities manage the required resources, formulate preventive measures, and control the spread effectively. In the specific case of COVID-19, the UAE (United Arab Emirates) has undertaken many initiatives, such as surveillance and contact tracing by introducing mobile apps such as Al Hosn, containment of spread by limiting the gathering of people, online schooling and remote work, sanitation drives, and closure of public places. The aim of this paper is to predict the trends occurring in pandemic outbreak, with COVID-19 in the UAE being a specific case study to investigate. In this paper, a predictive modeling approach is proposed to predict the future number of cases based on the recorded history, taking into consideration the enforced policies and provided vaccinations. Machine learning models such as LASSO Regression and Exponential Smoothing, and deep learning models such as LSTM, LSTM-AE, and bi-directional LSTM-AE, are utilized. The dataset used is publicly available from the UAE government, Federal Competitiveness and Statistics Centre (FCSC) and consists of several attributes, such as the numbers of confirmed cases, recovered cases, deaths, tests, and vaccinations. An additional categorical attribute is manually added to the dataset describing whether an event has taken place, such as a national holiday or a sanitization drive, to study the effect of such events on the pandemic trends. Experimental results showed that the Univariate LSTM model with an input of a five-day history of Confirmed Cases achieved the best performance with an RMSE of 275.85, surpassing the current state of the art related to the UAE by over 30%. It was also found that the bi-directional LSTMs performed relatively well. The approach proposed in the paper can be applied to monitor similar infectious disease outbreaks and thus contribute to strengthening the authorities’ preparedness for future pandemics.

Goal: Respiration-correlated cone-beam computed tomography (4D-CBCT) is an X-ray-based imaging modality that uses reconstruction algorithms to produce time-varying volumetric images of moving anatomy over a cycle of respiratory motion. The quality of the produced images is affected by the number of CBCT projections available for reconstruction. Interpolation techniques have been used to generate intermediary projections to be used, along with the original projections, for reconstruction. Transfer learning is a powerful approach that harnesses the ability to reuse pre-trained models in solving new problems. Methods: Several state-of-the-art pre-trained deep learning models, used for video frame interpolation, are utilized in this work to generate intermediary projections. Moreover, a novel regression predictive modeling approach is also proposed to achieve the same objective. Digital phantom and clinical datasets are used to evaluate the performance of the models. Results: The results show that the Real-Time Intermediate Flow Estimation (RIFE) algorithm outperforms the others in terms of the Structural Similarity Index Method (SSIM): 0.986 $\pm$ 0.010, Peak Signal to Noise Ratio (PSNR): 44.13 $\pm$ 2.76, and Mean Square Error (MSE): 18.86 $\pm$ 206.90 across all datasets. Moreover, the interpolated projections were used along with the original ones to reconstruct a 4D-CBCT image that was compared to that reconstructed from the original projections only. Conclusions: The reconstructed image using the proposed approach was found to minimize the streaking artifacts, thereby enhancing the image quality. This work demonstrates the advantage of using general-purpose transfer learning algorithms in 4D-CBCT image enhancement.

Tuberculosis (TB) is a pulmonary infectious disease causing morbidity and mortality in developing countries. In 2021, an estimated 10.6 million individuals were