Alzheimer’s disease is an irremediable, continuous brain disorder that gradually destroys memory and thinking skills and, eventually, the ability to carry out the simplest tasks. It has become one of the critical diseases throughout the world. Moreover, there is no remedy for Alzheimer’s disease. Machine learning techniques, especially deep learning-based Convolutional Neural Network (CNN), are used to improve the process for the detection of Alzheimer’s disease. In recent days, CNN has achieved major success in MRI image analysis and biomedical research. A lot of research has been carried out for the detection of Alzheimer’s disease based on brain MRI images using CNN. However, one of the fundamental limitations is that proper comparison between a proposed CNN model and pre-trained CNN models (InceptionV3, Xception, MobilenetV2, VGG) was not established. Therefore, in this paper, we present a model based on 12-layer CNN for binary classification and detection of Alzheimer’s disease using brain MRI data. The performance of the proposed model is compared with some existing CNN models in terms of accuracy, precision, recall, F1 score, and ROC curve on the Open Access Series of Imaging Studies (OASIS) dataset. The main contribution of the paper is a 12-layer CNN model with an accuracy of 97.75%, which is higher than any other existing CNN models published on this dataset. The paper also shows side by side comparison between our proposed model and pre-trained CNN models (InceptioV3, Xception, MobilenetV2, VGG). The experimental results show the superiority of the proposed model over the existing models.