The color of 15 red wines from several wineries within the renowned wine-producing region Rioja (Northern Spain) was measured by spectrophotometric and spectroradiometric techniques and was visually assessed in a pair-comparison experiment by a panel of 10 experienced observers having normal color vision. Correlation between instrumental color measurements made by spectrophotometric and spectroradiometric techniques was very low, as expected from major differences in the experimental conditions employed (different illumination, path lengths and glass effects). Spectroradiometric measurements at the center of the wine sampler and at positions displaced 1 cm in the horizontal and vertical directions were quite different, mainly because of an increase of the lightness L*, the average color differences between them being high (3.5 and 2.6 CIELAB units, respectively). A 50% acceptance percentage resulted for a color difference of 2.8 CIELAB units, using a reference anchor-pair of wine samples with 4.0 CIELAB units. Thus, a value around 3.0 CIELAB units should be considered a preliminary estimate of the acceptable tolerance by the human eye for red wines poured in standard wine samplers.

Using simulated data sets, we have analyzed some mathematical properties of different statistical measurements that have been employed in previous literature to test the performance of different color-difference formulas. Specifically, the properties of the combined index $PF∕3$ (performance factor obtained as average of three terms), widely employed in current literature, have been considered. A new index named standardized residual sum of squares (STRESS), employed in multidimensional scaling techniques, is recommended. The main difference between $PF∕3$ and STRESS is that the latter is simpler and allows inferences on the statistical significance of two color-difference formulas with respect to a given set of visual data.

The standardized residual sum of squares (STRESS) index was used to reevaluate four experimental datasets employed during the development of CIEDE2000, the current CIE recommended color-difference formula. This index enables statistical inferences not achievable by other metrics used commonly for performance evaluation. It was found that CIEDE2000 was statistically superior at a 95% confidence level to either CIE94, the previous recommended equation by the CIE, or the simple Euclidean distance in CIELAB, $\mathrm{\Delta }{E}_{ab}^{*}$. Recent formulas based on the CIECAM02 color-appearance space and chroma-compressed variants of CIELAB were also evaluated and found to have only slightly reduced performance compared with CIEDE2000. These formulas have the advantage of simplicity and easier interpretation when used for quantifying color accuracy. Finally, each experimental dataset was evaluated separately rather than weight averaged as used during the development of CIEDE2000. Significant differences were found between datasets, suggesting that combining datasets may obscure important differences and that the practice of parameter optimization during formula development using combined data is likely suboptimal.

The standardized residual sum of squares index was proposed to examine the significant merit of a given color-difference formula over another with respect to a given set of visual color-difference data [J. Opt. Soc. Am. A 24, 1823–1829, 2007]. This index can also be employed to determine intra- and inter-observer variability, although the full complexity of this variability cannot be described by just one number. Appropriate utilization of the standardized residual sum of squares index for the assessment of observer variability is described with a view to encourage its use in future color-difference research. The main goal of this paper is to demonstrate that setting the F parameters of the standardized residual sum of squares index to 1 results in a loss of essential properties of the index (for example, symmetry), and is therefore strongly discouraged.

Color-difference formulas modified by power functions provide results in better agreement with visually perceived color differences. Each of the modified color-difference formulas proposed here adds only one relevant parameter to the corresponding original color-difference formula. Results from 16 visual data sets and 11 color-difference formulas indicate that the modified formulas achieve an average decrease of 5.7 STRESS (Standardized Residual Sum of Squares) units with respect to the original formulas, signifying an improvement of 17.3%. In particular, for these 16 visual data sets, the average decrease for the current CIE/ISO recommended color-difference formula CIEDE2000 modified by an exponent 0.70 was 5.4 STRESS units (17.5%). The improvements of all modified color-difference formulas with respect to the original ones held for each of the 16 visual data sets and were statistically significant in most cases, particularly for all data sets with color differences close to the threshold. Results for 2 additional data sets with color pairs in the blue and black regions of the color space confirmed the usefulness of the proposed power functions. The main reason of the improvements found for the modified color-difference formulas with respect to the original color-difference formulas seems to be the compression provided by power functions.

CIELAB-based color-difference formulas are used to improve the prediction of visually perceived color differences through the introduction of various corrections to CIELAB. In our study we analyze the relative importance of these corrections. From the combined dataset employed for the development of CIEDE2000, we found that the improvement of CIE94 over CIELAB was considerably greater than that of CIEDE2000 over CIE94. Chroma-difference correction was the most important correction in both CIE94 and CIEDE2000. With an arbitrary value of 100 assigned to this correction, the score of the hue-difference correction in CIE94 was 21, and the scores of the four remaining corrections in CIEDE2000 were as follows: hue difference, 29; rotation term, 8; lightness difference, 8; and gray correction, 6. At 95% confidence level each of the corrections introduced in CIEDE2000 or CIE94 was statistically significant for the whole combined dataset, in agreement with the results reported by CIE TC 1-47 and 1-29. For the combined dataset, the differences between CMC and CIEDE2000 were found to be statistically significant at 95% confidence level, but the differences between CMC and CIE94 were not. From subsets of the combined dataset it was concluded that further analyses of the lightness-difference and gray corrections proposed by CIEDE2000 would be desirable, using new experimental data.