Tourism recommender systems suggest suitable tourist spots by matching the characteristics of the tourist spots with those of the user. In this paper, we focus on an essential source of these characteristics—geotagged tweets. To solve the problem of associating geotagged tweets to tourist spots, we propose a mapping method that infers the region of a target spot on the basis of two geotagged items. The first is a geotagged tweet, which demonstrates that the tweeter was indeed at the target spot at the time the tweet was posted. We call this a “now-tweet.” The second item is a geotagged photo of the target spot, which we call a “spot-photo.” We regard these now-tweets and spot-photos as training data, and then determine the region of the tourist spot by inferring the geographical distribution of the training data. Next, we map geotagged tweets from the extracted region to the target spot. To improve the accuracy with which the tourist spot is inferred, we apply a clustering algorithm to the training data. Experimental results indicate that photo-based mapping with sophisticated training data produces the most improved performance over baseline methods. When applied to 4,559,643 geotagged tweets, our method maps them to tourist spots with an average granularity of 144.85 m

Recommender systems help users obtain the content they need from massive amounts of information. Artwork recommender systems is a topic that has attracted attention. However, existing art recommender systems rarely consider user preferences and multimodal information at the same time, while utilizing all the information has the potential to help make better personalized recommendations. To better apply recommender systems to the artwork-recommendation scenario, we propose a new neural topic modeling (NTM)-based multimodal artwork recommender system (MultArtRec), that can take all the information into account at the same time and extract effective features representing user preferences from multimodal content. Also, to improve MultArtRec’s performance on monomodal feature extraction, we add a novel topic loss term to the conventional NTM loss. The first two experiments in this study compare the performances of different models with different monomodal inputs. The results show that MultArtRec can improve the performance with image modality inputs by up to 174.8% compared to the second-best model and improve the performance with text modality inputs by up to 10.7% compared to the second-best model. The third experiment is conducted to compare the performance of MultArtRec with monomodal inputs and multimodal inputs. The results show that the performance of MultArtRec with multimodal inputs can be improved by up to 15.9% compared to monomodal inputs. The last experiment preliminarily tests the versatility of MultArtRec on a fashion recommendation scenario that considers clothing image content and user preferences. The results show that MultArtRec outperforms the other methods across all the metrics.

Digital archives (DAs) usually store diverse expert-level materials. Nowadays, access to DAs is increasing for non-expert users, However, they might have difficulties formulating appropriate search queries to find the necessary information. In response to this problem, we propose a query log-based query recommendation algorithm that provides expert knowledge to non-expert users, thus supporting their information seeking in DAs. The use case considered is one where after users enter some general queries, they will be recommended semantically similar expert-level queries in the query logs. The proposed modified conditional restricted Boltzmann machines (M-CRBMs) are capable of utilizing the rich metadata in DAs, thereby alleviating the sparsity problem that conventional restricted Boltzmann machines (RBMs) will face. Additionally, compared with other CRBM models, we drop a large number of model weights. In the experiments, the M-CRBMs outperform the conventional RBMs when using appropriate metadata, and we find that the recommendation results are relevant to the metadata fields that are used in M-CRBMs. Through experiments on the Europeana dataset, we also demonstrate the versatility and scalability of our proposed model.

Nowadays, embodied conversational agents are gradually getting deployed in real-world applications like the guides in museums or exhibitions. In these applications, it is necessary for the agent to identify the addressee of each user utterance to deliberate appropriate responses in interacting with visitor groups. However, as long as the addressee identification mechanism is not completely correct, the agent makes error in its responses. Once there is an error, the agent’s hypothesis collapses and the following decision-making path may go to a totally different direction. We are working on developing the mechanism to detect the error from the users’ reactions and the mechanism to recover the error. This paper presents the first step, a method to detect laughing, surprises, and confused facial expressions after the agent’s wrong responses. This method is machine learning base with the data (user reactions) collected in a WOZ (Wizard of Oz) experiment and reached an accuracy over 90%.