Concrete is the material most produced by humanity. Its popularity is mainly based on its low production cost and great structural design flexibility. Its operational and ambient loadings including environmental effects have a great impact in the performance and overall cost of concrete structures. Thus, the quality control, the structural assessment, the maintenance and the reliable prolongation of the operational service life of the existing concrete structures have become a major issue. In the recent years, non-destructive testing (NDT) is becoming increasingly essential for reliable and affordable quality control and integrity assessment not only during the construction of new concrete structures, but also for the existing ones. Choosing the right inspection technique is always followed by a compromise between its performance and cost. In the present paper, the ultrasonic pulse velocity (UPV) method, which is the most well known and widely accepted ultrasonic concrete NDT method, is thoroughly reviewed and compared with other well-established NDT approaches. Their principles, inherent limitations and reliability are reviewed. In addition, while the majority of the current UPV techniques are based on the use of piezoelectric transducers held on the surface of the concrete, special attention is paid to a very promising technique using low-cost and aggregate-size piezoelectric transducers embedded in the material. That technique has been evaluated based on a series of parameters, such as the ease of use, cost, reliability and performance.