This paper presents the design of an optimized hardware-based neural network (NN) called a Shift-Accumulate Binarized Neural Network (SABiNN). SABiNN is employed in detecting respiratory-related diseases such as sleep apnea (SA) among adults. Initially, a 3-hidden layer-based NN model was trained, validated, and tested with open-source apnea PSG datasets collected from the PhysioNET databank. Single lead ECG and pulse oximeter data were collected, pre-processed, and digitized for network training. The NN was later transformed into SABiNN, demonstrating model accuracy of 81.5% (CI: ± 3.5) with an energy efficiency of 5mJ on reprogrammable hardware. The precision rate of the model was further increased by re-designing the XNOR gate of the multiply-accumulate (MAC) operation with NAND gate-based XNOR. This re-design process significantly improved the overall model’s classification and precision. Further expansion of SABiNN was carried out to achieve a higher accuracy rate (over 88%) which was designed on the CMOS platform using a 130 nm open-source process design kit (PDK) developed by Google and Skywater. The proposed model on the CMOS platform utilized a chip area of 0.16 mm ² and showcased an energy efficiency of 1pJ. A total of ~11k CMOS cells with two 16-bit input and one 1-bit output pins were utilized to realize the SABiNN on CMOS. The success of this design technique can be leveraged in developing a fully system-on-a-chip (SoC) integrated wearable system for sleep apnea detection.