Atrial fibrillation and ventricular fibrillation are the two most common cardiac arrhythmia. These cardiac arrhythmias cause heart strokes and other heart complications leading increased risk of heart failure. Early and accurate cardiac arrhythmia detection is vital to preventing various heart-related diseases. Electrocardiogram (ECG) is a popular and reliable method to detect cardiac arrhythmia and heart-related diseases. However, sometimes it becomes difficult and time-consuming to interpret the ECG signal, even for a cardiac expert. This study proposes a deep learning-based method to effectively classify and detect cardiac arrhythmias (atrial fibrillation and ventricular fibrillation) using a time-frequency spectrogram of ECG records. The proposed framework utilizes superlet transform (SLT) to transform the one-dimensional (1-D) ECG signal into a two-dimensional (2-D) time-frequency spectrogram. The last layer of the pre-trained convolutional neural networks, namely, AlexNet, GoogLeNet, and DenseNet, are modified and then used to classify the ECG records into healthy heart, atrial fibrillation, and ventricular fibrillation. The proposed method is tested using ECG signals from the MIT-BIH arrhythmia database, MIT-BIH malignant database, Fantasia database, AF termination challenge database, and CU ventricular tachyarrhythmia database. The proposed method with DenseNet-201 architecture provides the best performance with an overall test accuracy of 96.2%. The proposed method reduces the system complexity as it does not require noise removal, hand-crafted features, and beat detection. Further, the proposed system automatically classifies the ECG signal into healthy heart, atrial fibrillation, and ventricular fibrillation.