In a weakly-supervised scenario object detectors need to be trained using image-level annotation alone. Since bounding-box-level ground truth is not available, most of the solutions proposed so far are based on an iterative, Multiple Instance Learning framework in which the current classifier is used to select the highest-confidence boxes in each image, which are treated as pseudo-ground truth in the next training iteration. However, the errors of an immature classifier can make the process drift, usually introducing many of false positives in the training dataset. To alleviate this problem, we propose in this paper a training protocol based on the self-paced learning paradigm. The main idea is to iteratively select a subset of images and boxes that are the most reliable, and use them for training. While in the past few years similar strategies have been adopted for SVMs and other classifiers, we are the first showing that a self-paced approach can be used with deep-network-based classifiers in an end-to-end training pipeline. Our method is built on the fully-supervised Fast-RCNN architecture and can be applied to similar architectures which represent the input image as a bag of boxes. We show state-of-the-art results on Pascal VOC 2007-2010 and ILSVRC 2013. On ILSVRC 2013 our results based on a low-capacity AlexNet network outperform even those weakly-supervised approaches which are based on much higher-capacity networks.
24 August 2024 in European Journal of Futures Research
This paper explores the potential of a multidisciplinary approach to testing and aligning artificial intelligence (AI), specifically focusing on large language models (LLMs). Due to the rapid development and wide application of LLMs, challenges such as ethical alignment, controllability, and predictability of these models emerged as global risks. This study investigates an innovative simulation-based multi-agent system within a virtual reality framework that replicates the real-world environment. The framework is populated by automated 'digital citizens,' simulating complex social structures and interactions to examine and optimize AI. Application of various theories from the fields of sociology, social psychology, computer science, physics, biology, and economics demonstrates the possibility of a more human-aligned and socially responsible AI. The purpose of such a digital environment is to provide a dynamic platform where advanced AI agents can interact and make independent decisions, thereby mimicking realistic scenarios. The actors in this digital city, operated by the LLMs, serve as the primary agents, exhibiting high degrees of autonomy. While this approach shows immense potential, there are notable challenges and limitations, most significantly the unpredictable nature of real-world social dynamics. This research endeavors to contribute to the development and refinement of AI, emphasizing the integration of social, ethical, and theoretical dimensions for future research.
Monitoring soil organic carbon (SOC) typically assumes conducting a labor-intensive soil sampling campaign, followed by laboratory testing, which is both expensive and impractical for generating useful, spatially continuous data products. The present study leverages the power of machine learning (ML) and, in particular, deep neural networks (DNNs) for segmentation, as well as satellite imagery, to estimate the SOC remotely. We propose a new two-stage pipeline for remote SOC estimation, which relies on using a DNN trained to classify land cover to perform feature extraction, while the SOC estimation is performed by a different ML model. The first stage is an image segmentation DNN with the U-Net architecture, which is trained to estimate the land cover for an observed geographical region, based on multi-spectral images taken by the Sentinel-2 satellite constellation. This estimator is subsequently used to extract the latent feature vector for each of the output pixels, by rolling back from the output (dense) layer of the U-Net and accessing the last available convolutional layer of the same dimension as our desired output. The second stage is trained on a set of feature vectors extracted at the coordinates for which manual SOC measurements exist. We tested a variety of ML models and report on their performance. Using the best extremely randomized trees model, we generated a spatially continuous map of SOC estimations for the region of Tuscany, in Italy, with a resolution of 10 m, to share with the researchers as a means of validating the results and to demonstrate the efficiency of the proposed approach, which can can easily be scaled to create a global continuous SOC map.
Video quality as perceived by human observers is the ground truth when Video Quality Assessment (VQA) is in question. It is dependent on many variables, one of them being the content of the video that is being evaluated. Despite the evidence that content has an impact on the quality score the sequence receives from human evaluators, currently available VQA databases mostly comprise of sequences which fail to take this into account. In this paper, we aim to identify and analyze differences between human cognitive, affective, and conative responses to a set of videos commonly used for VQA and a set of videos specifically chosen to include video content which might affect the judgment of evaluators when perceived video quality is in question. Our findings indicate that considerable differences exist between the two sets on selected factors, which leads us to conclude that videos starring a different type of content than the currently employed ones might be more appropriate for VQA.
Large-scale infrastructure, such as China–Europe Railway Express (CER-Express), which connects countries and regions across Asia and Europe, has a potentially profound effect on land use, as evidenced by changes in land cover along the railway. To ensure sustainable development of such infrastructure and appropriate land administration, effective ways to monitor and assess its impact need to be developed. Remote sensing based on publicly available satellite imagery represents an obvious choice. In the study presented here, we employ a state-of-the-art deep-learning-based approach to automatically detect different types of land cover based on multispectral Sentinel-2 imagery. We then use these data to conduct and present a study of the changes in land use in two geopolitically diverse regions of interest (in Serbia and China and with and without CER-Express infrastructure) for the period of the last three years. Our results show that the standard image-patch-based land cover classification approaches suffer a significant drop in performance in our target scenario in which each pixel needs to be assigned a cove class, but still, validate the applicability of the proposed approach as a remote sensing tool to support the sustainable development of large infrastructure. We discuss the technical limitations of the proposed approach in detail and potential ways in which it can be improved.
For decades, computed tomography (CT) images have been widely used to discover valuable anatomical information. Metallic implants such as dental fillings cause severe streaking artifacts which significantly degrade the quality of CT images. In this paper, we propose a new method for metal‐artifact reduction using complementary magnetic resonance (MR) images. The method exploits the possibilities which arise from the use of emergent trimodality systems. The proposed algorithm corrects reconstructed CT images. The projected data which is affected by dental fillings is detected and the missing projections are replaced with data obtained from a corresponding MR image. A simulation study was conducted in order to compare the reconstructed images with images reconstructed through linear interpolation, which is a common metal‐artifact reduction technique. The results show that the proposed method is successful in reducing severe metal artifacts without introducing significant amount of secondary artifacts.