Machine translation (MT) is being used by millions of people daily, and therefore evaluating the quality of such systems is an important task. While human expert evaluation of MT output remains the most accurate method, it is not scalable by any means. Automatic procedures that perform the task of Machine Translation Quality Estimation (MT-QE) are typically trained on a large corpus of source–target sentence pairs, which are labeled with human judgment scores. Furthermore, the test set is typically drawn from the same distribution as the train. However, recently, interest in low-resource and unsupervised MT-QE has gained momentum. In this paper, we define and study a further restriction of the unsupervised MT-QE setting that we call oblivious MT-QE. Besides having no access no human judgment scores, the algorithm has no access to the test text’s distribution. We propose an oblivious MT-QE system based on a new notion of sentence cohesiveness that we introduce. We tested our system on standard competition datasets for various language pairs. In all cases, the performance of our system was comparable to the performance of the non-oblivious baseline system provided by the competition organizers. Our results suggest that reasonable MT-QE can be carried out even in the restrictive oblivious setting.

Feature selection in high-dimensional studies is conventionally evaluated by the predictive performance of the features it returns, but predictive performance does not indicate whether the same subset would be selected again under a reasonable perturbation of the data. We propose a feature-stability profile, reported as a diagnostic layer beside predictive performance rather than in place of it. The profile is assembled from established quantities: per-feature selection frequency across repeated stratified resamples, a chance-corrected stability summary, recurrent sets reported across a sweep of descriptive cutoffs, a random-selection baseline, and the held-out predictive performance recorded on the same resamples. We examine it in two settings. In a controlled synthetic study, the informative support is planted by construction, so recovery can be measured directly; in an illustrative application across high-dimensional binary datasets, no such support exists, and a broader exploratory roster is reported as supporting results. Under planted support, predictive performance, subset stability, and support recovery can diverge rather than decline together: at an intermediate signal level, performance can remain relatively preserved while exact recovery falls, and some low exact overlap reflects substitution among redundant alternatives. On real data, recurrent features are treated as recurrent candidates, not recovered or validated features; selectors reaching near-equal AUC can differ about twofold in chance-corrected recurrence. The contribution is diagnostic and integrative, not a new selector, a new metric, a benchmark ranking, or an error-controlled procedure, making the reliability of a selected feature set visible rather than assumed.

Feature selection in high-dimensional studies is conventionally evaluated by the predictive performance of the features it returns, but predictive performance does not indicate whether the same subset would be selected again under a reasonable perturbation of the data. We propose a feature-stability profile, reported as a diagnostic layer beside predictive performance rather than in place of it. The profile is assembled from established quantities: per-feature selection frequency across repeated stratified resamples, a chance-corrected stability summary, recurrent sets reported across a sweep of descriptive cutoffs, a random-selection baseline, and the held-out predictive performance recorded on the same resamples. We examine it in two settings. In a controlled synthetic study, the informative support is planted by construction, so recovery can be measured directly; in an illustrative application across high-dimensional binary datasets, no such support exists, and a broader exploratory roster is reported as supporting results. Under planted support, predictive performance, subset stability, and support recovery can diverge rather than decline together: at an intermediate signal level, performance can remain relatively preserved while exact recovery falls, and some low exact overlap reflects substitution among redundant alternatives. On real data, recurrent features are treated as recurrent candidates, not recovered or validated features; selectors reaching near-equal area under the receiver operating characteristic curve (AUC) can differ about twofold in chance-corrected recurrence. The contribution is diagnostic and integrative, not a new selector, a new metric, a benchmark ranking, or an error-controlled procedure, making the reliability of a selected feature set visible rather than assumed.