The promising outcomes of chimeric antigen receptor (CAR) T cell therapy in hematologic malignancies potentiates its capability in the fight against many cancers. Nevertheless, this immunotherapy modality needs significant improvements for the treatment of solid tumors. Researchers have incrementally identified limitations and constantly pursued better CAR designs. However, even if CAR T cells are armed with optimal killer functions, they must overcome and survive suppressive barriers imposed by the tumor microenvironment (TME). In this review, we will discuss in detail the important role of TME in CAR T cell trafficking and how the intrinsic barriers contribute to an immunosuppressive phenotype and cancer progression. It is of critical importance that preclinical models can closely recapitulate the in vivo TME to better predict CAR T activity. Animal models have contributed immensely to our understanding of human diseases, but the intensive care for the animals and unreliable representation of human biology suggest in vivo models cannot be the sole approach to CAR T cell therapy. On the other hand, in vitro models for CAR T cytotoxic assessment offer valuable insights to mechanistic studies at the single cell level, but they often lack in vivo complexities, inter-individual heterogeneity, or physiologically relevant spatial dimension. Understanding the advantages and limitations of preclinical models and their applications would enable more reliable prediction of better clinical outcomes.