The goodness of fit of a statistical model describes how well it fits a set of observations. Measures of goodness of fit typically summarize the discrepancy between observed values and the values expected under the model in question. Such measures can be used in statistical hypothesis testing, e.g. to test for normality of residuals, to test whether two samples are drawn from identical distributions , or whether outcome frequencies follow a specified distribution. In the analysis of variance, one of the components into which the variance is partitioned may be a lack-of-fit sum of squares.
Fit of distributions
In assessing whether a given distribution is suited to a data-set, the following tests and their underlying measures of fit can be used:
uses a measure of goodness of fit which is the sum of differences between observed and expected outcome frequencies, each squared and divided by the expectation: where: The expected frequency is calculated by: where: The resulting value can be compared with a chi-squared distribution to determine the goodness of fit. The chi-squared distribution has degrees of freedom, where k is the number of non-empty cells and c is the number of estimated parameters for the distribution plus one. For example, for a 3-parameter Weibull distribution, c = 4.
For example, to test the hypothesis that a random sample of 100 people has been drawn from a population in which men and women are equal in frequency, the observed number of men and women would be compared to the theoretical frequencies of 50 men and 50 women. If there were 44 men in the sample and 56 women, then If the null hypothesis is true, the test statistic will be drawn from a chi-squared distribution with one degree of freedom. Though one might expect two degrees of freedom, we must take into account that the total number of men and women is constrained, and thus there is only one degree of freedom. In other words, if the male count is known the female count is determined, and vice versa. Consultation of the chi-squared distribution for 1 degree of freedom shows that the probability of observing this difference if men and women are equally numerous in the population is approximately 0.23. This probability is higher than conventional criteria for statistical significance, so normally we would not reject the null hypothesis that the number of men in the population is the same as the number of women Note the assumption that the mechanism that has generated the sample is random, in the sense of independent random selection with the same probability, here 0.5 for both males and females. If, for example, each of the 44 males selected brought a male buddy, and each of the 56 females brought a female buddy, each will increase by a factor of 4, while each will increase by a factor of 2. The value of the statistic will double to 2.88. Knowing this underlying mechanism, we should of course be counting pairs. In general, the mechanism, if not defensibly random, will not be known. The distribution to which the test statistic should be referred may, accordingly, be very different from chi-squared.
Binomial case
A binomial experiment is a sequence of independent trials in which the trials can result in one of two outcomes, success or failure. There are n trials each with probability of success, denoted by p. Provided that npi ≫ 1 for every i, then This has approximately a chi-squared distribution with k − 1 degrees of freedom. The fact that there are k − 1 degrees of freedom is a consequence of the restriction. We know there are k observed cell counts, however, once any k − 1 are known, the remaining one is uniquely determined. Basically, one can say, there are only k − 1 freely determined cell counts, thus k − 1 degrees of freedom.
''G''-test
are likelihood-ratio tests of statistical significance that are increasingly being used in situations where Pearson's chi-squared tests were previously recommended. The general formula for G is where and are the same as for the chi-squared test, denotes the natural logarithm, and the sum is taken over all non-empty cells. Furthermore, the total observed count should be equal to the total expected count:where is the total number of observations. G-tests have been recommended at least since the 1981 edition of the popular statistics textbook by Robert R. Sokal and F. James Rohlf.
