Recent advances in flexible and stretchable electronics (FSE), a technology diverging from the conventional rigid silicon technology, have stimulated fundamental scientific and technological research efforts. FSE aims at enabling disruptive applications such as flexible displays, wearable sensors, printed RFID tags on packaging, electronics on skin/organs, and Internet-of-things as well as possibly reducing the cost of electronic device fabrication. Thus, the key materials components of electronics, the semiconductor, the dielectric, and the conductor as well as the passive (substrate, planarization, passivation, and encapsulation layers) must exhibit electrical performance and mechanical properties compatible with FSE components and products. In this review, we summarize and analyze recent advances in materials concepts as well as in thin-film fabrication techniques for high-k (or high-capacitance) gate dielectrics when integrated with FSE-compatible semiconductors such as organics, metal oxides, quantum dot arrays, carbon nanotubes, graphene, and other 2D semiconductors. Since thin-film transistors (TFTs) are the key enablers of FSE devices, we discuss TFT structures and operation mechanisms after a discussion on the needs and general requirements of gate dielectrics. Also, the advantages of high-k dielectrics over low-k ones in TFT applications were elaborated. Next, after presenting the design and properties of high-k polymers and inorganic, electrolyte, and hybrid dielectric families, we focus on the most important fabrication methodologies for their deposition as TFT gate dielectric thin films. Furthermore, we provide a detailed summary of recent progress in performance of FSE TFTs based on these high-k dielectrics, focusing primarily on emerging semiconductor types. Finally, we conclude with an outlook and challenges section.