${\text{YbMgGaO}}_4$, a structurally perfect two-dimensional triangular lattice with an odd number of electrons per unit cell and spin-orbit entangled effective spin-$1/2$ local moments for the ${\mathrm{Yb}}^{3+}$ ions, is likely to experimentally realize the quantum spin liquid ground state. We report the first experimental characterization of single-crystal ${\text{YbMgGaO}}_4$ samples. Because of the spin-orbit entanglement, the interaction between the neighboring ${\mathrm{Yb}}^{3+}$ moments depends on the bond orientations and is highly anisotropic in the spin space. We carry out thermodynamic and the electron spin resonance measurements to confirm the anisotropic nature of the spin interaction as well as to quantitatively determine the couplings. Our result is a first step towards the theoretical understanding of the possible quantum spin liquid ground state in this system and sheds new light on the search for quantum spin liquids in strong spin-orbit coupled insulators.

Quantum spin liquid (QSL) is a novel state of matter which refuses the conventional spin freezing even at 0 K. Experimentally searching for the structurally perfect candidates is a big challenge in condensed matter physics. Here we report the successful synthesis of a new spin-1/2 triangular antiferromagnet YbMgGaO₄ with "Equation missing"<!-- image only, no MathML or LaTex --> symmetry. The compound with an ideal two-dimensional and spatial isotropic magnetic triangular-lattice has no site-mixing magnetic defects and no antisymmetric Dzyaloshinsky-Moriya (DM) interactions. No spin freezing down to 60 mK (despite θ_w ~ −4 K), the power-law temperature dependence of heat capacity and nonzero susceptibility at low temperatures suggest that YbMgGaO₄ is a promising gapless (≤|θ_w|/100) QSL candidate. The residual spin entropy, which is accurately determined with a non-magnetic reference LuMgGaO₄, approaches zero (<0.6%). This indicates that the possible QSL ground state (GS) of the frustrated spin system has been experimentally achieved at the lowest measurement temperatures.

Muon spin relaxation ($\mu \mathrm{SR}$) experiments on single crystals of the structurally perfect triangular antiferromagnet ${\mathrm{YbMgGaO}}_4$ indicate the absence of both static long-range magnetic order and spin freezing down to 0.048 K in a zero field. Below 0.4 K, the ${\mu }^{+}$ spin relaxation rates, which are proportional to the dynamic correlation function of the ${\mathrm{Yb}}^{3+}$ spins, exhibit temperature-independent plateaus. All these $\mu \mathrm{SR}$ results unequivocally support the formation of a gapless U(1) quantum spin liquid ground state in the triangular antiferromagnet ${\mathrm{YbMgGaO}}_4$.

We apply moderate-high-energy inelastic neutron scattering (INS) measurements to investigate ${\mathrm{Yb}}^{3+}$ crystalline electric field (CEF) levels in the triangular spin-liquid candidate ${\mathrm{YbMgGaO}}_4$. Three CEF excitations from the ground-state Kramers doublet are centered at the energies $\hslash \omega =39$, 61, and 97 meV in agreement with the effective spin-$1/2$ $g$ factors and experimental heat capacity, but reveal sizable broadening. We argue that this broadening originates from the site mixing between ${\mathrm{Mg}}^{2+}$ and ${\mathrm{Ga}}^{3+}$ giving rise to a distribution of Yb-O distances and orientations and, thus, of CEF parameters that account for the peculiar energy profile of the CEF excitations. The CEF randomness gives rise to a distribution of the effective spin-$1/2$ $g$ factors and explains the unprecedented broadening of low-energy magnetic excitations in the fully polarized ferromagnetic phase of ${\mathrm{YbMgGaO}}_4$, although a distribution of magnetic couplings due to the $\mathrm{Mg}/\mathrm{Ga}$ disorder may be important as well.

Since its proposal by Anderson, resonating valence bonds (RVB) formed by a superposition of fluctuating singlet pairs have been a paradigmatic concept in understanding quantum spin liquids. Here, we show that excitations related to singlet breaking on nearest-neighbour bonds describe the high-energy part of the excitation spectrum in YbMgGaO₄, the effective spin-1/2 frustrated antiferromagnet on the triangular lattice, as originally considered by Anderson. By a thorough single-crystal inelastic neutron scattering study, we demonstrate that nearest-neighbour RVB excitations account for the bulk of the spectral weight above 0.5 meV. This renders YbMgGaO₄ the first experimental system where putative RVB correlations restricted to nearest neighbours are observed, and poses a fundamental question of how complex interactions on the triangular lattice conspire to form this unique many-body state.