Selecting the best model from a set of candidates for a given set of data is obviously not an easy task. In this paper, we propose a new criterion that takes into account a larger penalty when adding too many coefficients (or estimated parameters) in the model from too small a sample in the presence of too much noise, in addition to minimizing the sum of squares error. We discuss several real applications that illustrate the proposed criterion and compare its results to some existing criteria based on a simulated data set and some real datasets including advertising budget data, newly collected heart blood pressure health data sets and software failure data.

The number of issues fixed in the current release of the software is one of the factors which decides the next release of the software. The source code files get changed during fixing of these issues. The uncertainty arises due to these changes is quantified using entropy based measures. We developed a Non-Homogeneous Poisson Process model for Open Source Software to understand the fixing of issues across releases. Based on this model, optimal release-updating using entropy and maximizing the active user's satisfaction level subject to fixing of issues up to a desired level, is investigated as well. The proposed models have been validated on five products of the Apache open source project. The optimal release time estimated from the proposed model is close to the observed release time at different active user's satisfaction levels. The proposed decision model can assist management to appropriately determine the optimal release-update time. The proposed entropy based model, for issues estimation shows improvement in performance for 21 releases out of total 23 releases, when compared with well-known traditional software reliability growth models, namely GO model [1] and S-shaped model [2]. The proposed model is also found statistically significant.

Many software reliability growth models (SRGMs) have developed in the past three decades to quantify several reliability measures including the expected number of remaining faults and software reliability. The underlying common assumption of many existing models is that the operating environment and the developing environment are the same. In reality, this is often not the case because the operating environments are unknown due to the uncertainty of environments in the field. In this paper, we present two new software reliability models with considerations of the fault-detection rate based on a Loglog distribution and the testing coverage subject to the uncertainty of operating environments. Examples are included to illustrate the goodness-of-fit test of proposed models and several existing non-homogeneous Poisson process (NHPP) models based on a set of failure data collected from software applications. Three goodness-of-fit test criteria, such as, mean square error, predictive-ratio risk, and predictive power, are used as an example to illustrate the model comparisons. The results show that the proposed models fit significantly better than other existing NHPP models based on the studied criteria. As we know different criteria have different impacts in measuring the software reliability and that no software reliability model is optimal for all contributing criteria. In this paper, we also discuss a method, called normalized criteria distance, to show ways to rank and select the best model from among SRGMs based on a set of criteria taken all together. Examples show that the proposed method offers a promising technique for selecting the best model based on a set of contributing criteria.

When software systems are introduced, these systems are used in field environments that are the same as or close to those used in the development-testing environments; however, they may also be used in many different locations that may differ from the environment in which they were developed and tested. As such, it is difficult to improve software reliability for a variety of reasons, such as a given environment, or a bug location in code. In this paper, we propose a new software reliability model that takes into account the uncertainty of operating environments. The explicit mean value function solution for the proposed model is presented. Examples are presented to illustrate the goodness of fit of the proposed model and several existing non-homogeneous Poisson process (NHPP) models and confidence intervals of all models based on two sets of failure data collected from software applications. The results show that the proposed model fits the data more closely than other existing NHPP models to a significant extent.