Automatic detection and rapid extraction of condition characteristics of pavement potholes provide important support for road health monitoring and maintenance. In this paper, a method of extracting pavement potholes in vehicle-borne continuous laser point cloud combined with two-dimensional and three-dimensional analysis is proposed. This method mainly includes three parts: Firstly, point cloud data preprocessing is carried out to remove non-road point cloud and road noise points; then the transverse and longitudinal sections of the road are extracted according to the scanning line index of the laser point cloud. The Douglas–Pucker operator is used to analyse the profile contour characteristics, and the integral invariant operator is used to identify the initial feature points of the pothole; finally, the local datum surface of the pavement is constructed by fitting, and the continuous depth image of the pavement is generated. The active contour model operator is used to accurately extract the pothole boundary based on the local depth information, and geometric information such as the depth and area of the pothole is obtained. Selecting some road point cloud data for experiments, the results show that the pothole feature point extraction method in this paper is relatively robust, the extraction precision rate reaches 86.1%, and the recall rate reaches 91.2%. By generating a local depth image for precise extraction of the pothole boundary, it can effectively solve the problem of inaccurate extraction of the pothole boundary based on discrete point clouds. Compared with the manual measurement results, the relative error of the area is less than 7%, and the relative error of the depth is less than 6%, which proves the effectiveness of this method.