The Historic Oakland Cemetery in downtown Atlanta provides a unique setting for exploring the challenges of location-based mixed-reality experience design. Our objective is to entertain and educate visitors about historically and culturally significant events related to the deceased inhabitants of the cemetery. We worked with the constraints and affordances of the physical environment of the cemetery to design an audio-based dramatic experience. The dramatic narrative is realized through voice actors who play the parts of cemetery residents and tell stories about the time periods in which they lived. The experience provides navigation and linearity through a main narrator who guides visitors to various gravesites. While at each grave, the visitor can choose from several categories of content using a handheld controller. Formative evaluations conducted with users in the cemetery indicate strengths of the current experience and suggest ideas for continued development.
31 December 2021 in IEEE Robotics and Automation Letters
The common paradigm of CNN-based action recognition modelsis to simply use the average of the dense predictions from every frame. However, these dense predictions are inefficient since all frames are evenly utilized regardless of the existence of the action. In real-time action recognition applications where the input video is untrimmed, the dense prediction is even more severely inefficient. Instead of these dense predictions, we propose a lightweight CNN-based sampler that selects frames that are relevant to the action in the video. The proposed sampler is trained by a self-supervised learning method that mixes one irrelevant frame in an input video clip and finds that irrelevant frame. With the trained sampler, we select frames to be used as inputs to the action recognition model. Our sampler can be combined with any action recognition models and works independently of them. Through various experiments, we demonstrate that the proposed sampler is not only time and memory-efficient but also significantly increases performance in the action recognition benchmarks, including HMDB51 and UCF101. Finally, we applied the self-supervised sampler to the real-time surveillance system in the subway station, and improved the action recognition performance by 7% with 80% reduction in computation. Code and dataset are available in https://github.com/seominseok0429/SelSupSampler and https://aihub.or.kr/aidata/34124 .
인공지능 모델 생성은 학습 데이터 가공, 모델 학습, 모델 평가의 단계로 진행된다. 양질의 학습 데이터를 만드는 데이터 전처리 기법은 모델 정확도를 향상시키기 위한 방법 중 많은 기여를 한다. 기존의 전처리 기법은 모델 생성자의 경험에 많이 의존하는 경향이 있다. 경험을 기반으로 전처리를 수행하면 해당 전처리 기법을 선택한 근거를 설명하기 힘들다. 하지만 설계자가 경험에 의존할 수밖에 없는 이유는 학습 모델이 거대해지고 인간이 해석하기 힘든 수준으로 복잡해지기 때문이다. 따라서, 설명 가능한 인공지능을 도입하여 모델의 동작 방식을 설명하는 연구가 진행되고 있다. 본 논문에서는 설명 가능한 인공지능을 사용한 학습 데이터 전처리 시스템을 제안한다. 시스템 동작 과정은 전처리가 수행되지 않은 데이터로 학습시키며, 학습된 모델을 설명 가능한 인공지능 기법을 사용하여 동작 방식을 분석하여 그 정보를 기반으로 데이터 전처리 수행을 반복한다. 최종적으로 모델 성능을 향상시키고 전처리 신뢰성을 설명하며 시스템의 실용성을 보이겠다.
As computer vision and NLP make progress, Vision-Language(VL) is becoming an important area of research. Despite the importance, evaluation metrics of the research domain is still at a preliminary stage of development. In this paper, we propose a quantitative metric "Equivariance Score" and evaluation dataset "Human Puzzle" to assess whether a VL model is understanding an image like a human. We observed that the VL model does not interpret the overall context of an input image but instead shows biases toward a specific object or shape that forms the local context. We aim to quantitatively measure a model's performance in understanding context. To verify the current existing VL model's capability, we sliced the original input image into pieces and randomly placed them, distorting the global context of the image. Our paper discusses each VL model's level of interpretation on global context and addresses how the structural characteristics influenced the results.
Deep convolutional neural networks (CNNs) have shown state-of-the-art performances in various computer vision tasks. Advances on CNN architectures have focused mainly on designing convolutional blocks of the feature extractors, but less on the classifiers that exploit extracted features. In this work, we propose Split-and-Share Module (SSM), a classifier that splits a given feature into parts, which are partially shared by multiple sub-classifiers. Our intuition is that the more the features are shared, the more common they will become, and SSM can encourage such structural characteristics in the split features. SSM can be easily integrated into any architecture without bells and whistles. We have extensively validated the efficacy of SSM on ImageNet-1K classification task, and SSM has shown consistent and significant improvements over baseline architectures. In addition, we analyze the effect of SSM using the Grad-CAM visualization.