The error matrix has been adopted as a standard way to report on the thematic accuracy of any remotely sensed data product. A very usual way to perform the thematic accuracy analysis of an error matrix is by means of global indexes (e.g. overall accuracy, Kappa coefficient). But global indices do not allow for a category-wise control. This work proposes a new method for accuracy control of thematic classification based on an application of the chi-square goodness of fit test. By this way we can stablish our preferences of accuracy for each category but also we can consider some degree of misclassification between some categories. A practical example is provided for a 4×4 matrix (16 class combinations). In this example, a total of 13 quality levels (specifications) are imposed for certain class combinations. In this way, much more is controlled than by means of global indexes. In addition, the method allows to be more demanding or flexible in the quality levels of certain classes, according to convenience.

The confusion matrix is the standard way to report on the thematic accuracy of geographic data (spatial databases, topographic maps, thematic maps, classified images, remote sensing products, etc.). Two widely adopted indices for the assessment of thematic quality are derived from the confusion matrix. They are overall accuracy (OA) and the Kappa coefficient (ĸ), which have received some criticism from some authors. Both can be used to test the similarity of two independent classifications by means of a simple statistical hypothesis test, which is the usual practice. Nevertheless, this is not recommended, because different combinations of cell values in the matrix can obtain the same value of OA or ĸ, due to the aggregation of data needed to compute these indices. Thus, not rejecting a test for equality between two index values does not necessarily mean that the two matrices are similar. Therefore, we present a new statistical tool to evaluate the similarity between two confusion matrices. It takes into account that the number of sample units correctly and incorrectly classified can be modeled by means of a multinomial distribution. Thus, it uses the individual cell values in the matrices and not aggregated information, such as the OA or ĸ values. For this purpose, it is considered a test function based on the discrete squared Hellinger distance, which is a measure of similarity between probability distributions. Given that the asymptotic approximation of the null distribution of the test statistic is rather poor for small and moderate sample sizes, we used a bootstrap estimator. To explore how the p-value evolves, we applied the proposed method over several predefined matrices which are perturbed in a specified range. Finally, a complete numerical example of the comparison of two matrices is presented.

The confusion matrix is the standard way to report on the thematic accuracy of geographic information data. Two widely adopted indices for thematic accuracy controls upon error matrix are the overall accuracy and the Kappa coefficient. Provided that a multinomial sampling, this work proposes a new method for testing the homogeneity of two independent thematic classifications which is based on whole error matrices. Specifically, a test function is proposed that uses as a test statistic the discrete Hellinger distance, which null distribution is approximated by bootstrapping. The performance of this approximation is evaluated by simulation, which confirms that it behaves properly, especially for small and moderate sample sizes. Finally an example of application is included.