In this paper, a new fractional-order seed chaotic generator is designed to solve the problem of the complex operations of single low-dimensional systems and simple high-dimensional systems. The fractional-order chaotic system generated is proven to have better chaotic performance using Lyapunov exponential differential calculus, approximate entropy, 0–1 test and other indicators. On this basis, the “multiple squares nested body scrambling (MSNBS)” model is extended from fractal theory to complete the image scrambling process, and a new algorithm is proposed to be applied to the encryption field in combination with the fractional-order coupled chaotic system. Experimental simulations show that the algorithm can resist common differential attacks and noise attacks and improve the security of the algorithm.