Histone Deacetylase (HDAC) inhibitors are a relatively new class of anti-cancer agents that play important roles in epigenetic or non-epigenetic regulation, inducing death, apoptosis, and cell cycle arrest in cancer cells. Recently, their use has been clinically validated in cancer patients resulting in the approval by the FDA of four HDAC inhibitors, vorinostat, romidepsin, belinostat and panobinostat, used for the treatment of cutaneous/peripheral T-cell lymphoma and multiple myeloma. Many more HDAC inhibitors are at different stages of clinical development for the treatment of hematological malignancies as well as solid tumors. Also, clinical trials of several HDAC inhibitors for use as anti-cancer drugs (alone or in combination with other anti-cancer therapeutics) are ongoing. In the intensifying efforts to discover new, hopefully, more therapeutically efficacious HDAC inhibitors, molecular modelingbased rational drug design has played an important role. In this review, we summarize four major structural classes of HDAC inhibitors (hydroxamic acid derivatives, aminobenzamide, cyclic peptide and short-chain fatty acids) that are in clinical trials and different computer modeling tools available for their structural modifications as a guide to discover additional HDAC inhibitors with greater therapeutic utility.