Ferroelectric materials exhibit spontaneous and stable polarization, which can usually be reoriented by an applied external electric field. The electrically switchable nature of this polarization is at the core of various ferroelectric devices. The motion of the associated domain walls provides the basis for ferroelectric memory, in which the storage of data bits is achieved by driving domain walls that separate regions with different polarization directions. Here we show the surprising ability to move ferroelectric domain walls of a BaTiO₃ single crystal by varying the polarization angle of a coherent light source. This unexpected coupling between polarized light and ferroelectric polarization modifies the stress induced in the BaTiO₃ at the domain wall, which is observed using in situ confocal Raman spectroscopy. This effect potentially leads to the non-contact remote control of ferroelectric domain walls by light.