To address the requirements for miniaturization and high torque density in clutch systems, this study optimizes a multi-plate shear-mode magnetorheological (MR) clutch employing a dry MR powder. First, a nonlinear magnetic circuit model based on the magnetic potential difference theory is established to predict the radial distribution of magnetic flux density. Simultaneously, a thermal resistance network model is employed to determine the maximum allowable excitation current under temperature rise constraints. To ensure the reliability of these theoretical models, finite element analysis is conducted for both the magnetic and thermal circuits, revealing high consistency between the theoretical and simulation results. Subsequently, an optimization design is performed with the objective of maximizing torque density. An integrated optimization framework is constructed by defining the torque equations, establishing constraints—including mechanical structure, magnetic saturation, maximum current, and peak temperature—and identifying the design parameters. Within this framework, Latin hypercube sampling is utilized for spatial exploration, and an orthogonal polynomial surrogate model is selected for response surface fitting based on accuracy evaluation. The optimal solution is then identified using the NLPQLP algorithm. Finally, prototype fabrication and comparative experiments are conducted to verify the effectiveness of the optimized design in enhancing torque density and ensuring thermal stability. The results demonstrate a maximum torque density of 2.19 N.m kg −1 at 2.16 A, representing a 45.4% improvement over the initial design. With high torque density, compact dimensions, millisecond-range response, and experimentally verified thermal reliability, the proposed MR clutch is particularly suitable for high-performance electromechanical systems that demand an optimal combination of space efficiency, fast response, and operational safety. A preliminary integration of the optimized clutch into a robot wrist joint has been successfully demonstrated, highlighting its potential for enhancing compliance and interactive safety in collaborative and service robots.