We present the ultralow-temperature specific heat and thermal conductivity measurements on single crystals of triangular-lattice compound ${\mathrm{EtMe}}_3\mathrm{Sb}[\mathrm{Pd}(\text{dmit}{)}_2{]}_2$, which has long been considered as a gapless quantum spin liquid candidate. In specific heat measurements, a finite linear term is observed, consistent with the previous work [S. Yamashita et al., Nat. Commun. 2, 275 (2011)]. However, we do not observe a finite residual linear term in the thermal conductivity measurements, and the thermal conductivity does not change in a magnetic field of 6 T. These results are in sharp contrast to previous thermal conductivity measurements on ${\mathrm{EtMe}}_3\mathrm{Sb}[\mathrm{Pd}(\text{dmit}{)}_2{]}_2$ [M. Yamashita et al., Science 328, 1246 (2010)], in which a huge residual linear term was observed and attributed to highly mobile gapless excitations, likely the spinons of a quantum spin liquid. In this context, the true ground state of ${\mathrm{EtMe}}_3\mathrm{Sb}[\mathrm{Pd}(\text{dmit}{)}_2{]}_2$ has to be reconsidered.

An isostructural series of anion radical salts, β'-(Et_xMe_4−xZ)[Pd(dmit)₂]₂ (x = 0–2, Z = P, As, Sb), with a quasi-triangular lattice comprising the dimer unit [Pd(dmit)₂]₂⁻ belong to a strongly correlated electron system with geometrical frustration. Intra and interdimer transfer integrals between the frontier molecular orbitals, which characterize the strength of electron correlation and degree of frustration, can be tuned by selection of the counter cation. We have systematically analyzed the crystal structure with X-ray diffraction method and intermolecular transfer integrals using extended Hückel molecular orbital calculations based on structural data. The variation in the cation affects the unit cell in a manner equivalent to an anisotropic pressure. Increasing the covalent radius of the central atom Z and the number of ethyl groups (x) in the cation leads to slight arching of the Pd(dmit)₂ molecule. This arch-shaped distortion of the Pd(dmit)₂ molecule modifies the interdimer transfer integrals in formation of the regular triangular dimer lattice. On the other hand, the intradimer transfer integral, which is correlated with the effective on-site Coulomb interaction of the dimer, is weakly dependent on the type of cation.

The family of layered organic salts X[Pd(dmit)₂]₂ are Mott insulators and form scalenetriangular spin-1/2 systems with antiferromagnetic exchange interactions J. Among them, EtMe₃Sb[Pd(dmit)₂]₂ has a nearly regular-triangular spin system with J = 220-250 K. Our previous study proposed that a quantum spin-liquid state is realized in this system because this system was revealed to have neither spin ordering/freezing nor an appreciable spin gap down to 1.37 K. In the present study, to confirm the absence of spin ordering/freezing in the low-temperature limit, we cooled this system down to 19.4 mK, which is 4 orders of magnitude smaller than J, and measured ¹³C-NMR spectra. The obtained spectra do not show any critical broadening. Therefore, it is concluded that the spin system of EtMe₃Sb[Pd(dmit)₂]₂ does not have spin ordering/freezing in the low-temperature limit; that is, the spin system is in a quantum-fluctuating spin state.