Automatic detection and classification of the masses in mammograms are still a big challenge and play a crucial role to assist radiologists for accurate diagnosis. In this paper, we propose a novel computer-aided diagnose (CAD) system based on one of the regional deep learning techniques: a ROI-based Convolutional Neural Network (CNN) which is called You Only Look Once (YOLO). Our proposed YOLO-based CAD system contains four main stages: mammograms preprocessing, feature extraction utilizing multi convolutional deep layers, mass detection with confidence model, and finally mass classification using fully connected neural network (FC-NN). A set of training mammograms with the information of ROI masses and their types are used to train YOLO. The trained YOLO-based CAD system detects the masses and classifies their types into benign or malignant. Our results show that the proposed YOLO-based CAD system detects the mass location with an overall accuracy of 96.33%. The system also distinguishes between benign and malignant lesions with an overall accuracy of 85.52%. Our proposed system seems to be feasible as a CAD system capable of detection and classification at the same time. It also overcomes some challenging breast cancer cases such as the mass existing in the pectoral muscles or dense regions.