A more efficient attention recovery of workers during their break time is essential for achieving higher productivity and wellness. In recent years, the biophilic design that introduces indoor plants has become one of the solutions to these problems. This study aims to determine the impact factors and corresponding levels related to indoor planting design concerning workers’ attention recovery. Firstly, the volume ratio, layout, and leaf size of indoor plants and the corresponding levels were put forward by a focus group study with ten participants. Secondly, the orthogonal experiment method established nine virtual recovery scenarios based on characteristics extraction of staff break areas in the factory. Thirdly, eighteen participants were guided to feel fatigued by experiencing the sustained attention to response test to measure the baseline attention level of participants. Then, participants rested in the virtual scenarios. Lastly, the attention test was conducted again to observe participants’ attention recovery degree. The difference values of participants’ sensitivity, reaction times, and the number of correct responses between the two detection targets were applied to evaluate the attention recovery. Results showed that the volume ratio of indoor plants had the most significant effect on workers’ attention recovery; 3% by volume ratio, mixed floor and wall plants and floor planting with large leaves were demonstrated as the optimal indoor planting design for attention recovery.