We investigated electronic structure of hexagonal multiferroic ${\mathrm{YMnO}}_3$ using the polarization dependent x-ray absorption spectroscopy (XAS) at O $K$ and Mn $L_{2,3}$ edges. The spectra exhibit strong polarization dependence at both edges, reflecting anisotropic Mn $3d$ orbital occupation. Moreover, the O $K$ edge spectra show that Y $4d$ states are strongly hybridized with O $2p$ ones, resulting in large anomalies in Born effective charges on off-centering Y and O ions. These results manifest that the Y $d^0$-ness with rehybridization is the driving force for the ferroelectricity, and suggest a new approach to understand the multiferroicity in the hexagonal manganites.