Spin-orbit torque (SOT) offers ultrafast magnetic switching in the subnanosecond regime for magnetoresistive random-access memory (MRAM) as an alternative to future cache memories. However, one main obstacle to the development of perpendicular SOT-MRAM is that a small in-plane external field is required to achieve deterministic magnetization switching. Here we eliminate the external magnetic field for deterministic SOT switching by utilizing the stray field in a magnetic tunnel junction (MTJ) structure. The MTJ consists of a perpendicularly magnetized CoFeB free layer and an in-plane magnetized reference layer that provides a stray field. Unlike usual SOT switching in a single ferromagnet, four switching phases as well as unique critical switching current dependence on the applied in-plane magnetic fields have been observed. This letter demonstrates the effectiveness of stray-field assisted field-free SOT switching in MTJ structures that may extend to eliminate the main obstacle of SOT-MRAM in practical applications.