Grouted splice sleeve (GSS) connectors are mainly used in precast concrete structures. However, errors in manual operation during construction cause grouted defects in the GSS connector, which can lead to a negative effect on the overall mechanical properties of the structures. Owing to the complex structure of precast concrete members with a GSS connector, it is difficult to detect grouted defects effectively using traditional ultrasonic parameters. In this paper, a wavelet packet analysis algorithm was developed to effectively detect grouted defects using the ultrasonic method, and a verified experiment was carried out. Laboratory detection was performed on the concrete specimens with a GSS connector before grouting, in which the grouted defects were mimicked with five sizes in five GSS connectors of each specimen group. A simple and convenient ultrasonic detection system was developed, and the specimens were detected. According to the proposed grouted defect index, the results demonstrated that when the grouted defects reached certain sizes, the proposed method could detect the grouted defects effectively. The proposed method is effective and easy to implement at a construction site with simple instruments, and so provides an innovative method for grouted defects detection of precast concrete members.