We aimed to evaluate the inhibitory effect and mechanism of plantaricin YKX on S. aureus. The mode of action of plantaricin YKX against the cells of S. aureus indicated that plantaricin YKX was able to cause the leakage of cellular content and damage the structure of the cell membranes. Additionally, plantaricin YKX was also able to inhibit the formation of S. aureus biofilms. As the concentration of plantaricin YKX reached 3/4 MIC, the percentage of biofilm formation inhibition was over 50%. Fluorescent dye labeling combined with fluorescence microscopy confirmed the results. Finally, the effect of plantaricin YKX on the AI-2/LuxS QS system was investigated. Molecular docking predicted that the binding energy of AI-2 and plantaricin YKX was −4.7 kcal/mol and the binding energy of bacteriocin and luxS protein was −183.701 kcal/mol. The expression of the luxS gene increased significantly after being cocultured with plantaricin YKX, suggesting that plantaricin YKX can affect the QS system of S. aureus.