Tremendous research efforts have been devoted to fabricating high quality quantum dots (QDs) for applications in biology and medicine. Much of this research was pursued with an ultimate goal of using QDs in clinical applications. However, a great deal of concern has been voiced about the potential hazards of QDs due to their heavy-metal content. Many studies have demonstrated toxicity of various QDs in cell culture studies. However, in a smaller number of studies using small animal models (mice and rats), no abnormal behaviour or tissue damage was noticed over periods of months after the systemic administration of QDs. Nevertheless, the correlation of these results with the potential for negative effects of QD on humans remains unclear. Many urgent questions must be answered before the QDs community moves into the clinical research phase. This review provides an overview of the toxicity assessment of QDs, ranging from cell culture studies to animal models and discusses their findings. Guidelines for using various nonhuman primate models for QD toxicity studies are highlighted. This review article is intended to promote the awareness of current developments of QD applications in biology, the potential toxicity of QDs, and approaches to minimizing toxicity.