A hybrid framework for solving the non-uniqueness problem in the inverse design of isomorphic metasurfaces is proposed. The framework consists of a representation learning (RL) module and a variational autoencoder-particle swarm optimization (VAE-PSO) algorithm module. The RL module is used to reduce the complex high-dimensional space into a low-dimensional space with obvious features, with the purpose of eliminating the many-to-one relationship between the original design space and response space. The VAE-PSO algorithm first encodes all meta-atoms into a continuous latent space through VAE and then applies PSO to search for an optimized latent vector whose corresponding metasurface fulfills the target response. This framework gives the solution paradigm of the ideal non-uniqueness situation, simplifies the complexity of the network, improves the running speed of the PSO algorithm, and obtains the global optimal solution with 94% accuracy on the test set.