Monitoring traffic loads is vital for ensuring bridge safety and overload controlling. Bridge weigh-in-motion (BWIM) technology, which uses an instrumented bridge as a scale platform, has been proven as an efficient and durable vehicle weight identification method. However, there are still challenges with traditional BWIM methods in solving the inverse problem under certain circumstances, such as vehicles running at a non-constant speed, or multiple vehicle presence. For conventional BWIM systems, the velocity of a moving vehicle is usually assumed to be constant. Thus, the positions of loads, which are vital in the identification process, is predicted from the acquired speed and axle spacing by utilizing dedicated axle detectors (installed on the bridge surface or under the bridge soffit). In reality, vehicles may change speed. It is therefore difficult or even impossible for axle detectors to accurately monitor the true position of a moving vehicle. If this happens, the axle loads and bridge response cannot be properly matched, and remarkable errors can be induced to the influence line calibration process and the axle weight identification results. To overcome this problem, a new BWIM method was proposed in this study. This approach estimated the bridge influence line and axle weight by associating the bridge response and axle loads with their accurate positions. Binocular vision technology was used to continuously track the spatial position of the vehicle while it traveled over the bridge. Based on the obtained time–spatial information of the vehicle axles, the ordinate of influence line, axle load, and bridge response were correctly matched in the objective function of the BWIM algorithm. The influence line of the bridge, axle, and gross weight of the vehicle could then be reliably determined. Laboratory experiments were conducted to evaluate the performance of the proposed method. The negative effect of non-constant velocity on the identification result of traditional BWIM methods and the reason were also studied. Results showed that the proposed method predicted bridge influence line and vehicle weight with a much better accuracy than conventional methods under the considered adverse situations, and the stability of BWIM technique also was effectively improved. The proposed method provides a competitive alternative for future traffic load monitoring.