Due to its slow degradation rate, polycaprolactone (PCL) is frequently used in biomedical applications. This study deals with the development of antibacterial nanofibers based on PCL and halloysite nanotubes (HNTs). Thanks to a combination with HNTs, the prepared nanofibers can be used as low-cost nanocontainers for the encapsulation of a wide variety of substances, including drugs, enzymes, and DNA. In our work, HNTs were used as a nanocarrier for erythromycin (ERY) as a model antibacterial active compound with a wide range of antibacterial activity. Nanofibers based on PCL and HNT/ERY were prepared by electrospinning. The antibacterial activity was evaluated as a sterile zone of inhibition around the PCL nanofibers containing 7.0 wt.% HNT/ERY. The morphology was observed with SEM and TEM. The efficiency of HNT/ERY loading was evaluated with thermogravimetric analysis. It was found that the nanofibers exhibited outstanding antibacterial properties and inhibited both Gram- (Escherichia coli) and Gram+ (Staphylococcus aureus) bacteria. Moreover, a significant enhancement of mechanical properties was achieved. The potential uses of antibacterial, environmentally friendly, nontoxic, biodegradable PCL/HNT/ERY nanofiber materials are mainly in tissue engineering, wound healing, the prevention of bacterial infections, and other biomedical applications.