Many functional materials have been designed at the multiscale level. To properly simulate their physical properties, large and sophisticated computer models that can replicate microstructural features with nanometer-scale accuracy are required. This is the case for permanent magnets, which exhibit a long-standing problem of a significant offset between the simulated and experimental coercivities. To overcome this problem and resolve the Brown paradox, we propose an approach to construct large-scale finite element models based on the tomographic data from scanning electron microscopy. Our approach reconstructs a polycrystalline microstructure with actual shape, size, and packing of the grains as well as the individual regions of thin intergranular phase separated by triple junctions. Such a micromagnetic model can reproduce the experimental coercivity of ultrafine-grained Nd-Fe-B magnets along with its mechanism according to the angular dependence of coercivity. Furthermore, a remarkable role of thin triple junctions as nucleation centers for magnetization reversal is revealed. The developed digital twins of Nd-Fe-B permanent magnets can assist their optimization toward the ultimate coercivity, while the proposed tomography-based approach can be applied to a wide range of polycrystalline materials.