Buildings have now adopted a new dimension: the dimension of smartness. The rapid arrival of connected devices, together with the smart features that they provide, has allowed for the transition of existing buildings towards smart buildings. The assessment of the smartness of the large number of existing buildings could exhaust resources, but some organisations are requesting this regardless (such as the smart readiness indicator of the European Union). To tackle this issue, this work describes a tool that was created to find connected devices to automatically evaluate smartness. The tool, which was given the name SmartWatcher, uses a design-for-purpose natural language processing algorithm that converts verbal information into numerical information. The method was tested on real buildings in four different geographical locations. SmartWatcher is shown to be powerful, as it was capable of obtaining numerical values from verbal descriptions of devices. Additionally, a preliminary comparison of values obtained using the automatic engine and clipboard assessments showed that although the results were still far from being perfect, some visual correlation could be seen. This anticipates that, with the addition of appropriate techniques that refine this algorithm, or with the addition of new ones (with other more advanced natural language processing methods), the accuracy of this tool could be greatly increased.