With a mortality rate over 580,000 per year, cancer is still one of the leading causes of death worldwide. However, the emerging field of microfluidics can potentially shed light on this puzzling disease. Unique characteristics of microfluidic chips (also known as micro-total analysis system) make them excellent candidates for biological applications. The ex vivo approach of tumor-on-a-chip is becoming an indispensable part of personalized medicine and can replace in vivo animal testing as well as conventional in vitro methods. In tumor-on-a-chip, the complex three-dimensional (3D) nature of malignant tumor is co-cultured on a microfluidic chip and high throughput screening tools to evaluate the efficacy of anticancer drugs are integrated on the same chip. In this article, we critically review the cutting edge advances in this field and mainly categorize each tumor-on-a-chip work based on its primary organ. Specifically, design, fabrication and characterization of tumor microenvironment; cell culture technique; transferring mechanism of cultured cells into the microchip; concentration gradient generators for drug delivery; in vitro screening assays of drug efficacy; and pros and cons of each microfluidic platform used in the recent literature will be discussed separately for the tumor of following organs: (1) Lung; (2) Bone marrow; (3) Brain; (4) Breast; (5) Urinary system (kidney, bladder and prostate); (6) Intestine; and (7) Liver. By comparing these microchips, we intend to demonstrate the unique design considerations of each tumor-on-a-chip based on primary organ, e.g., how microfluidic platform of lung-tumor-on-a-chip may differ from liver-tumor-on-a-chip. In addition, the importance of heart–liver–intestine co-culture with microvasculature in tumor-on-a-chip devices for in vitro chemosensitivity assay will be discussed. Such system would be able to completely evaluate the absorption, distribution, metabolism, excretion and toxicity (ADMET) of anticancer drugs and more realistically recapitulate tumor in vivo-like microenvironment.