Tool wear is a crucial factor in machining as it directly impacts surface quality and indirectly decreases machining efficiency, which leads to significant economic losses. Hence, monitoring tool wear state is of the utmost importance for achieving high performance and efficient machining. Although monitoring tool wear state using a single sensor has been validated in laboratory settings, it has certain drawbacks such as limited feature information acquisition and inability to learn important features adaptively. These limitations pose challenges to quickly extending the monitoring function of tool wear state of the machine tools. To solve these problems, this paper proposes a double-attention deep learning network based on vibroacoustic signal fusion feature infographics. The first solution is the construction of novel infographics using tool-intrinsic characteristics and multi-domain fusion features of multi-sensor inputs, which includes correlation analysis, principal component analysis, and feature fusion. The second solution is to build a novel deep network with a double-attention module and a spatial pyramid pooling module which can accurately and quickly identify tool wear state by successfully extracting critical spatial data from the infographics at various scales. The validity of the network is examined through an interpretability analysis based on the class activation graph. In terms of the tool wear status recognition task, the F1 score of the double-attention model based on an information graph is 11.61% higher than Resnet18, and peak recognition accuracy reaches 97.98%.