Materials exhibiting high energy/power density are currently needed to meet the growing demand of portable electronics, electric vehicles and large-scale energy storage devices. The highest energy densities are achieved for fuel cells, batteries, and supercapacitors, but conventional dielectric capacitors are receiving increased attention for pulsed power applications due to their high power density and their fast charge-discharge speed. The key to high energy density in dielectric capacitors is a large maximum but small remanent (zero in the case of linear dielectrics) polarization and a high electric breakdown strength. Polymer dielectric capacitors offer high power/energy density for applications at room temperature, but above 100 °C they are unreliable and suffer from dielectric breakdown. For high-temperature applications, therefore, dielectric ceramics are the only feasible alternative. Lead-based ceramics such as La-doped lead zirconate titanate exhibit good energy storage properties, but their toxicity raises concern over their use in consumer applications, where capacitors are exclusively lead free. Lead-free compositions with superior power density are thus required. In this paper, we introduce the fundamental principles of energy storage in dielectrics. We discuss key factors to improve energy storage properties such as the control of local structure, phase assemblage, dielectric layer thickness, microstructure, conductivity, and electrical homogeneity through the choice of base systems, dopants, and alloying additions, followed by a comprehensive review of the state-of-the-art. Finally, we comment on the future requirements for new materials in high power/energy density capacitor applications.