Ultrafast laser technology has moved from ultrafast to ultra-strong due to the development of chirped pulse amplification technology. Ultrafast laser technology, such as femtosecond lasers and picosecond lasers, has quickly become a flexible tool for processing brittle and hard materials and complex micro-components, which are widely used in and developed for medical, aerospace, semiconductor applications and so on. However, the mechanisms of the interaction between an ultrafast laser and brittle and hard materials are still unclear. Meanwhile, the ultrafast laser processing of these materials is still a challenge. Additionally, highly efficient and high-precision manufacturing using ultrafast lasers needs to be developed. This review is focused on the common challenges and current status of the ultrafast laser processing of brittle and hard materials, such as nickel-based superalloys, thermal barrier ceramics, diamond, silicon dioxide, and silicon carbide composites. Firstly, different materials are distinguished according to their bandgap width, thermal conductivity and other characteristics in order to reveal the absorption mechanism of the laser energy during the ultrafast laser processing of brittle and hard materials. Secondly, the mechanism of laser energy transfer and transformation is investigated by analyzing the interaction between the photons and the electrons and ions in laser-induced plasma, as well as the interaction with the continuum of the materials. Thirdly, the relationship between key parameters and ultrafast laser processing quality is discussed. Finally, the methods for achieving highly efficient and high-precision manufacturing of complex three-dimensional micro-components are explored in detail.