Conventional cancer chemotherapies often exhibit insufficient therapeutic outcomes and dose-limiting toxicity. Therefore, there is a need for novel therapeutics and formulations with higher efficacy, improved safety, and more favorable toxicological profiles. This has promoted the development of nanomedicines, including systems for drug delivery, but also for imaging and diagnostics. Nanoparticles loaded with drugs can be designed to overcome several biological barriers to improving efficiency and reducing toxicity. In addition, stimuli-responsive nanocarriers are able to release their payload on demand at the tumor tissue site, preventing premature drug loss. This review focuses on ultrasound-triggered drug delivery by nanocarriers as a versatile, cost-efficient, non-invasive technique for improving tissue specificity and tissue penetration, and for achieving high drug concentrations at their intended site of action. It highlights aspects relevant for ultrasound-mediated drug delivery, including ultrasound parameters and resulting biological effects. Then, concepts in ultrasound-mediated drug delivery are introduced and a comprehensive overview of several types of nanoparticles used for this purpose is given. This includes an in-depth compilation of the literature on the various in vivo ultrasound-responsive drug delivery systems. Finally, toxicological and safety considerations regarding ultrasound-mediated drug delivery with nanocarriers are discussed.