Prostate cancer (PC) is the prevalent malignancy widespread among men in the Western World. Prostate specific membrane antigen (PSMA) is an established PC marker and has been considered as a promising biological target for anti-PC drug delivery and diagnostics. The protein was found to be overexpressed in PC cells, including metastatic, and the neovasculature of solid tumors. These properties make PSMA-based approach quite appropriate for effective PC imaging and specific drug therapy. Through the past decade, a variety of PSMA-targeted agents has been systematically evaluated. Small molecule compounds have several advantages over other classes, such as improved pharmacokinetics and rapid blood clearance. These low-weight ligands have similar structure and can be divided into three basic categories in accordance with the type of their zinc-binding core-head. Several PSMA binders are currently undergoing clinical trials generally for PC imaging. The main goal of the present review is to describe the recent progress achieved within the title field and structure activity relationships (SAR) disclosed for different PSMA ligands. Recent in vitro and in vivo studies for each type of the compounds described have also been briefly summarized.