Qualitative variation

An index of qualitative variation is a measure of statistical dispersion in nominal distributions. There are a variety of these, but they have been relatively little-studied in the statistics literature. The simplest is the variation ratio, while more complex indices include the information entropy.

Properties

There are several types of indices used for the analysis of nominal data. Several are standard statistics that are used elsewhere - range, standard deviation, variance, mean deviation, coefficient of variation, median absolute deviation, interquartile range and quartile deviation.
In addition to these several statistics have been developed with nominal data in mind. A number have been summarized and devised by Wilcox,, who requires the following standardization properties to be satisfied:

Variation varies between 0 and 1.
Variation is 0 if and only if all cases belong to a single category.
Variation is 1 if and only if cases are evenly divided across all category.

In particular, the value of these standardized indices does not depend on the number of categories or number of samples.
For any index, the closer to uniform the distribution, the larger the variance, and the larger the differences in frequencies across categories, the smaller the variance.
Indices of qualitative variation are then analogous to information entropy, which is minimized when all cases belong to a single category and maximized in a uniform distribution. Indeed, information entropy can be used as an index of qualitative variation.
One characterization of a particular index of qualitative variation is as a ratio of observed differences to maximum differences.

Wilcox's indexes

Wilcox gives a number of formulae for various indices of QV, the first, which he designates DM for "Deviation from the Mode", is a standardized form of the variation ratio, and is analogous to variance as deviation from the mean.

ModVR

The formula for the variation around the mode is derived as follows:
where f_m is the modal frequency, K is the number of categories and f_i is the frequency of the i^th group.
This can be simplified to
where N is the total size of the sample.
Freeman's index is
This is related to M as follows:
The ModVR is defined as
where v is Freeman's index.
Low values of ModVR correspond to small amount of variation and high values to larger amounts of variation.
When K is large, ModVR is approximately equal to Freeman's index v.

RanVR

This is based on the range around the mode. It is defined to be
where f_m is the modal frequency and f_l is the lowest frequency.

AvDev

This is an analog of the mean deviation. It is defined as the arithmetic mean of the absolute differences of each value from the mean.

MNDif

This is an analog of the mean difference - the average of the differences of all the possible pairs of variate values, taken regardless of sign. The mean difference differs from the mean and standard deviation because it is dependent on the spread of the variate values among themselves and not on the deviations from some central value.
where f_i and f_j are the i^th and j^th frequencies respectively.
The MNDif is the Gini coefficient applied to qualitative data.

VarNC

This is an analog of the variance.
It is the same index as Mueller and Schussler's Index of Qualitative Variation and Gibbs' M2 index.
It is distributed as a chi square variable with K – 1 degrees of freedom.

StDev

Wilson has suggested two versions of this statistic.
The first is based on AvDev.
The second is based on MNDif

HRel

This index was originally developed by Claude Shannon for use in specifying the properties of communication channels.
where p_i = f_i / N.
This is equivalent to information entropy divided by the and is useful for comparing relative variation between frequency tables of multiple sizes.

B index

Wilcox adapted a proposal of Kaiser based on the geometric mean and created the B' index. The B index is defined as

R packages

Several of these indices have been implemented in the R language.

Gibb's indices and related formulae

proposed six indexes.

''M''1

The unstandardized index is
where K is the number of categories and is the proportion of observations that fall in a given category i.
M1 can be interpreted as one minus the likelihood that a random pair of samples will belong to the same category, so this formula for IQV is a standardized likelihood of a random pair falling in the same category. This index has also referred to as the index of differentiation, the index of sustenance differentiation and the geographical differentiation index depending on the context it has been used in.

''M''2

A second index is the M2 is:
where K is the number of categories and is the proportion of observations that fall in a given category i. The factor of is for standardization.
M1 and M2 can be interpreted in terms of variance of a multinomial distribution . M1 is the variance of the multinomial distribution and M2 is the ratio of the variance of the multinomial distribution to the variance of a binomial distribution.

''M''4

The M4 index is
where m is the mean.

''M''6

The formula for M6 is
where K is the number of categories, X_i is the number of data points in the i^th category, N is the total number of data points, || is the absolute value and
This formula can be simplified
where p_i is the proportion of the sample in the i^th category.
In practice M1 and M6 tend to be highly correlated which militates against their combined use.

Related indices

The sum
has also found application. This is known as the Simpson index in ecology and as the Herfindahl index or the Herfindahl-Hirschman index in economics. A variant of this is known as the Hunter–Gaston index in microbiology
In linguistics and cryptanalysis this sum is known as the repeat rate. The incidence of coincidence is an unbiased estimator of this statistic
where f_i is the count of the i^th grapheme in the text and n is the total number of graphemes in the text.
;M1
The M1 statistic defined above has been proposed several times in a number of different settings under a variety of names. These include Gini's index of mutability, Simpson's measure of diversity, Bachi's index of linguistic homogeneity, Mueller and Schuessler's index of qualitative variation, Gibbs and Martin's index of industry diversification, Lieberson's index. and Blau's index in sociology, psychology and management studies. The formulation of all these indices are identical.
Simpson's D is defined as
where n is the total sample size and n_i is the number of items in the i^th category.
For large n we have
Another statistic that has been proposed is the coefficient of unalikeability which ranges between 0 and 1.
where n is the sample size and c = 1 if x and y are alike and 0 otherwise.
For large n we have
where K is the number of categories.
Another related statistic is the quadratic entropy
which is itself related to the Gini index.
;M2
Greenberg's monolingual non weighted index of linguistic diversity is the M2 statistic defined above.
;M7
Another index – the M7 – was created based on the M4 index of
where
and
where K is the number of categories, L is the number of subtypes, O_ij and E_ij are the number observed and expected respectively of subtype j in the i^th category, n_i is the number in the i^th category and p_j is the proportion of subtype j in the complete sample.
Note: This index was designed to measure women's participation in the work place: the two subtypes it was developed for were male and female.

Other single sample indices

These indices are summary statistics of the variation within the sample.

Berger–Parker index

The Berger–Parker index equals the maximum value in the dataset, i.e. the proportional abundance of the most abundant type. This corresponds to the weighted generalized mean of the values when q approaches infinity, and hence equals the inverse of true diversity of order infinity.

Brillouin index of diversity

This index is strictly applicable only to entire populations rather than to finite samples. It is defined as
where N is total number of individuals in the population, n_i is the number of individuals in the i^th category and N! is the factorial of N.
Brillouin's index of evenness is defined as
where I_B is the maximum value of I_B.

Hill's diversity numbers

Hill suggested a family of diversity numbers
For given values of a several of the other indices can be computed

a = 0: N_a = species richness
a = 1: N_a = Shannon's index
a = 2: N_a = 1/Simpson's index
a = 3: N_a = 1/Berger–Parker index

Hill also suggested a family of evenness measures
where a > b.
Hill's E₄ is
Hill's E₅ is

Margalef's index

where S is the number of data types in the sample and N is the total size of the sample.

Menhinick's index

where S is the number of data types in the sample and N is the total size of the sample.
In linguistics this index is the identical with the Kuraszkiewicz index where S is the number of distinct words and N is the total number of words in the text being examined. This index can be derived as a special case of the Generalised Torquist function.

Q statistic

This is a statistic invented by Kempton and Taylor. and involves the quartiles of the sample. It is defined as
where R₁ and R₁ are the 25% and 75% quartiles respectively on the cumulative species curve, n_j is the number of species in the j_th category, n_Ri is the number of species in the class where R_i falls.

Shannon–Wiener index

This is taken from information theory
where N is the total number in the sample and p_i is the proportion in the i^th category.
In ecology where this index is commonly used, H usually lies between 1.5 and 3.5 and only rarely exceeds 4.0.
An approximate formula for the standard deviation of H is
where p_i is the proportion made up by the i^th category and N is the total in the sample.
A more accurate approximate value of the variance of H is given by
where N is the sample size and K is the number of categories.
A related index is the Pielou J defined as
One difficulty with this index is that S is unknown for a finite sample. In practice S is usually set to the maximum present in any category in the sample.

Rényi entropy

The Rényi entropy is a generalization of the Shannon entropy to other values of q than unity. It can be expressed:
which equals
This means that taking the logarithm of true diversity based on any value of q gives the Rényi entropy corresponding to the same value of q.
The value of is also known as the Hill number.

McIntosh's D and E

where N is the total sample size and n_i is the number in the i^th category.
where K is the number of categories.

Fisher's alpha

This was the first index to be derived for diversity.
where K is the number of categories and N is the number of data points in the sample. Fisher's α has to be estimated numerically from the data.
The expected number of individuals in the r^th category where the categories have been placed in increasing size is
where X is an empirical parameter lying between 0 and 1. While X is best estimated numerically an approximate value can be obtained by solving the following two equations
where K is the number of categories and N is the total sample size.
The variance of α is approximately

Strong's index

This index is the distance between the Lorenz curve of species distribution and the 45 degree line. It is closely related to the Gini coefficient.
In symbols it is
where max is the maximum value taken over the N data points, K is the number of categories in the data set and c_i is the cumulative total up and including the i_th category.

Simpson's E

This is related to Simpson's D and is defined as
where D is Simpson's D and K is the number of categories in the sample.

Smith & Wilson's indices

Smith and Wilson suggested a number of indices based on Simpson's D.
where D is Simpson's D and K is the number of categories.

Heip's index

where H is the Shannon entropy and K is the number of categories.
This index is closely related to Sheldon's index which is
where H is the Shannon entropy and K is the number of categories.

Camargo's index

This index was created by Camargo in 1993.
where K is the number of categories and p_i is the proportion in the i^th category.

Smith and Wilson's B

This index was proposed by Smith and Wilson in 1996.
where θ is the slope of the log-rank curve.

Nee, Harvey, and Cotgreave's index

This is the slope of the log-rank curve.

Bulla's E

There are two versions of this index - one for continuous distributions and the other for discrete.
where
is the Schoener–Czekanoski index, K is the number of categories and N is the sample size.

Horn's information theory index

This index is based on Shannon's entropy. It is defined as
where
In these equations x_ij and x_kj are the number of times the j^th data type appears in the i^th or k^th sample respectively.

Rarefaction index

In a rarefied sample a random subsample n in chosen from the total N items. In this sample some groups may be necessarily absent from this subsample. Let be the number of groups still present in the subsample of n items. is less than K the number of categories whenever at least one group is missing from this subsample.
The rarefaction curve, is defined as:
Note that 0 ≤ f ≤ K.
Furthermore,
Despite being defined at discrete values of n, these curves are most frequently displayed as continuous functions.
This index is discussed further in Rarefaction.

Caswell's V

This is a z type statistic based on Shannon's entropy.
where H is the Shannon entropy, E is the expected Shannon entropy for a neutral model of distribution and SD is the standard deviation of the entropy. The standard deviation is estimated from the formula derived by Pielou
where p_i is the proportion made up by the i^th category and N is the total in the sample.

Lloyd & Ghelardi's index

This is
where K is the number of categories and K' is the number of categories according to MacArthur's broken stick model yielding the observed diversity.

Average taxonomic distinctness index

This index is used to compare the relationship between hosts and their parasites. It incorporates information about the phylogenetic relationship amongst the host species.
where s is the number of host species used by a parasite and ω_ij is the taxonomic distinctness between host species i and j.

Index of qualitative variation

Several indices with this name have been proposed.
One of these is
where K is the number of categories and p_i is the proportion of the sample that lies in the i^th category.

Theil’s H

This index is also known as the multigroup entropy index or the information theory index. It was proposed by Theil in 1972.The index is a weighted average of the samples entropy.
Let
and
where p_i is the proportion of type i in the a^th sample, r is the total number of samples, n_i is the size of the i^th sample, N is the size of the population from which the samples were obtained and E is the entropy of the population.

Indices for comparison of two or more data types within a single sample

Several of these indexes have been developed to document the degree to which different data types of interest may coexist within a geographic area.

Index of dissimilarity

Let A and B be two types of data item. Then the index of dissimilarity is
where
A_i is the number of data type A at sample site i, B_i is the number of data type B at sample site i, K is the number of sites sampled and || is the absolute value.
This index is probably better known as the index of dissimilarity. It is closely related to the Gini index.
This index is biased as its expectation under a uniform distribution is > 0.
A modification of this index has been proposed by Gorard and Taylor. Their index is

Index of segregation

The index of segregation is
where
and K is the number of units, A_i and t_i is the number of data type A in unit i and the total number of all data types in unit i.

Hutchen's square root index

This index is defined as
where p_i is the proportion of the sample composed of the i^th variate.

Lieberson's isolation index

This index was invented by Lieberson in 1981.
where X_i and Y_i are the variables of interest at the i^th site, K is the number of sites examined and X_tot is the total number of variate of type X in the study.

Bell's index

This index is defined as
where p_x is the proportion of the sample made up of variates of type X and
where N_x is the total number of variates of type X in the study, K is the number of samples in the study and x_i and p_i are the number of variates and the proportion of variates of type X respectively in the i^th sample.

Index of isolation

The index of isolation is
where K is the number of units in the study, A_i and t_i is the number of units of type A and the number of all units in i_th sample.
A modified index of isolation has also been proposed
The MII lies between 0 and 1.

Gorard's index of segregation

This index is defined as
where
and A_i and t_i are the number of data items of type A and the total number of items in the i^th sample.

Index of exposure

This index is defined as
where
and A_i and B_i are the number of types A and B in the i^th category and t_i is the total number of data points in the i^th category.

Ochai index

This is a binary form of the cosine index. It is used to compare presence/absence data of two data types. It is defined as
where a is the number of sample units where both A and B are found, b is number of sample units where A but not B occurs and c is the number of sample units where type B is present but not type A.

Kulczyński's coefficient

This coefficient was invented by Stanisław Kulczyński in 1927 and is an index of association between two types. It varies in value between 0 and 1. It is defined as
where a is the number of sample units where type A and type B are present, b is the number of sample units where type A but not type B is present and c is the number of sample units where type B is present but not type A.

Yule's Q

This index was invented by Yule in 1900. It concerns the association of two different types. It is defined as
where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present. Q varies in value between -1 and +1. In the ordinal case Q is known as the Goodman-Kruskal γ.
Because the denominator potentially may be zero, Leinhert and Sporer have recommended adding +1 to a, b, c and d.

Yule's Y

This index is defined as
where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present.

Baroni–Urbani–Buser coefficient

This index was invented by Baroni-Urbani and Buser in 1976. It varies between 0 and 1 in value. It is defined as
where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present. N is the sample size.
When d = 0, this index is identical to the Jaccard index.

Hamman coefficient

This coefficient is defined as
where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present. N is the sample size.

Rogers–Tanimoto coefficient

Sokal–Sneath coefficient

Sokal's binary distance

Russel–Rao coefficient

Phi coefficient

Soergel's coefficient

This coefficient is defined as
where b is the number of samples where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present. N is the sample size.

Simpson's coefficient

This coefficient is defined as
where b is the number of samples where type A is present but not type B, c is the number of samples where type B is present but not type A.

Dennis' coefficient

Forbes' coefficient

This coefficient was proposed by Stephen Alfred Forbes in 1907. It is defined as
where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present. N is the sample size.
A modification of this coefficient has been proposed by Alroy

Simple match coefficient

Fossum's coefficient

Stile's coefficient

This coefficient is defined as
where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A, d is the sample count where neither type A nor type B are present, n equals a + b + c + d and || is the modulus of the difference.

Michael's coefficient

Peirce's coefficient

In 1884 Charles Peirce suggested the following coefficient
where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present.

Hawkin–Dotson coefficient

In 1975 Hawkin and Dotson proposed the following coefficient
where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present. N is the sample size.

Benini coefficient

In 1901 Benini proposed the following coefficient
where a is the number of samples where types A and B are both present, b is where type A is present but not type B and c is the number of samples where type B is present but not type A. Min is the minimum of b and c.

Gilbert coefficient

Gilbert proposed the following coefficient
where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the sample count where neither type A nor type B are present. N is the sample size.

Gini index

The Gini index is
where a is the number of samples where types A and B are both present, b is where type A is present but not type B and c is the number of samples where type B is present but not type A.

Modified Gini index

The modified Gini index is
where a is the number of samples where types A and B are both present, b is where type A is present but not type B and c is the number of samples where type B is present but not type A.

Kuhn's index

Kuhn proposed the following coefficient in 1965
where a is the number of samples where types A and B are both present, b is where type A is present but not type B and c is the number of samples where type B is present but not type A. K is a normalizing parameter. N is the sample size.
This index is also known as the coefficient of arithmetic means.

Eyraud index

Eyraud proposed the following coefficient in 1936
where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the number of samples where both A and B are not present.

Soergel distance

This is defined as
where a is the number of samples where types A and B are both present, b is where type A is present but not type B, c is the number of samples where type B is present but not type A and d is the number of samples where both A and B are not present. N is the sample size.

Tanimoto index

Piatetsky–Shapiro's index

Indices for comparison between two or more samples

Czekanowski's quantitative index

This is also known as the Bray–Curtis index, Schoener's index, least common percentage index, index of affinity or proportional similarity. It is related to the Sørensen similarity index.
where x_i and x_j are the number of species in sites i and j respectively and the minimum is taken over the number of species in common between the two sites.

Canberra metric

The Canberra distance is a weighted version of the L₁ metric. It was introduced by introduced in 1966 and refined in 1967 by G. N. Lance and W. T. Williams. It is used to define a distance between two vectors – here two sites with K categories within each site.
The Canberra distance d between vectors p and q in an K-dimensional real vector space is
where p_i and q_i are the values of the i^th category of the two vectors.

Sorensen's coefficient of community

This is used to measure similarities between communities.
where s₁ and s₂ are the number of species in community 1 and 2 respectively and c is the number of species common to both areas.

Jaccard's index

This is a measure of the similarity between two samples:
where A is the number of data points shared between the two samples and B and C are the data points found only in the first and second samples respectively.
This index was invented in 1902 by the Swiss botanist Paul Jaccard.
Under a random distribution the expected value of J is
The standard error of this index with the assumption of a random distribution is
where N is the total size of the sample.

Dice's index

Match coefficient

This is a measure of the similarity between two samples:
where N is the number of data points in the two samples and B and C are the data points found only in the first and second samples respectively.

Morisita's index

Morisita’s index of dispersion is the scaled probability that two points chosen at random from the whole population are in the same sample. Higher values indicate a more clumped distribution.
An alternative formulation is
where n is the total sample size, m is the sample mean and x are the individual values with the sum taken over the whole sample. It is also equal to
where IMC is Lloyd's index of crowding.
This index is relatively independent of the population density but is affected by the sample size.
Morisita showed that the statistic
is distributed as a chi-squared variable with n − 1 degrees of freedom.
An alternative significance test for this index has been developed for large samples.
where m is the overall sample mean, n is the number of sample units and z is the normal distribution abscissa. Significance is tested by comparing the value of z against the values of the normal distribution.

Morisita's overlap index

Morisita's overlap index is used to compare overlap among samples. The index is based on the assumption that increasing the size of the samples will increase the diversity because it will include different habitats
C_D = 0 if the two samples do not overlap in terms of species, and C_D = 1 if the species occur in the same proportions in both samples.
Horn's introduced a modification of the index

Standardised Morisita’s index

Smith-Gill developed a statistic based on Morisita’s index which is independent of both sample size and population density and bounded by −1 and +1. This statistic is calculated as follows
First determine Morisita's index in the usual fashion. Then let k be the number of units the population was sampled from. Calculate the two critical values
where χ² is the chi square value for n − 1 degrees of freedom at the 97.5% and 2.5% levels of confidence.
The standardised index is then calculated from one of the formulae below
When I_d ≥ M_c > 1
When M_c > I_d ≥ 1
When 1 > I_d ≥ M_u
When 1 > M_u > I_d
I_p ranges between +1 and −1 with 95% confidence intervals of ±0.5. I_p has the value of 0 if the pattern is random; if the pattern is uniform, I_p < 0 and if the pattern shows aggregation, I_p > 0.

Peet's evenness indices

These indices are a measure of evenness between samples.
where I is an index of diversity, I_max and I_min are the maximum and minimum values of I between the samples being compared.

Loevinger's coefficient

Loevinger has suggested a coefficient H defined as follows:
where p_max and p_min are the maximum and minimum proportions in the sample.

Tversky index

The Tversky index is an asymmetric measure that lies between 0 and 1.
For samples A and B the Tversky index is
The values of α and β are arbitrary. Setting both α and β to 0.5 gives Dice's coefficient. Setting both to 1 gives Tanimoto's coefficient.
A symmetrical variant of this index has also been proposed.
where
Several similar indices have been proposed.
Monostori et al. proposed the SymmetricSimilarity index
where d is some measure of derived from X.
Bernstein and Zobel have proposed the S2 and S3 indexes
S3 is simply twice the SymmetricSimilarity index. Both are related to Dice's coefficient

Metrics used

A number of metrics have been proposed.

Euclidean distance

While this is usually used in quantitative work it may also be used in qualitative work. This is defined as
where d_jk is the distance between x_ij and x_ik.

Gower's distance

This is defined as
where d_i is the distance between the i^th samples and w_i is the weighing give to the i^th distance.

Manhattan distance

While this is more commonly used in quantitative work it may also be used in qualitative work. This is defined as
where d_jk is the distance between x_ij and x_ik and || is the absolute value of the difference between x_ij and x_ik.
A modified version of the Manhattan distance can be used to find a zero of a polynomial of any degree using Lill's method.

Prevosti’s distance

This is related to the Manhattan distance. It was described by Prevosti et al. and was used to compare differences between chromosomes. Let P and Q be two collections of r finite probability distributions. Let these distributions have values that are divided into k categories. Then the distance D_PQ is
where r is the number of discrete probability distributions in each population, k_j is the number of categories in distributions P_j and Q_j and p_ji is the theoretical probability of category i in distribution P_j in population P.
Its statistical properties were examined by Sanchez et al. who recommended a bootstrap procedure to estimate confidence intervals when testing for differences between samples.

Other metrics

Let
where min is the lesser value of the pair x and y.
Then
is the Manhattan distance,
is the Bray−Curtis distance,
is the Jaccard distance and
is the Kulczynski distance.

Similarities between texts

HaCohen-Kerner et al. have proposed a variety of metrics for comparing two or more texts.

Ordinal data

If the categories are at least ordinal then a number of other indices may be computed.

Leik's D

Leik's measure of dispersion is one such index. Let there be K categories and let p_i be f_i/N where f_i is the number in the i^th category and let the categories be arranged in ascending order. Let
where a ≤ K. Let d_a = c_a if c_a ≤ 0.5 and 1 − c_a ≤ 0.5 otherwise. Then

Normalised Herfindahl measure

This is the square of the coefficient of variation divided by N − 1 where N is the sample size.
where m is the mean and s is the standard deviation.

Potential-for-conflict Index

The potential-for-conflict Index describes the ratio of scoring on either side of a rating scale’s centre point. This index requires at least ordinal data. This ratio is often displayed as a bubble graph.
The PCI uses an ordinal scale with an odd number of rating points centred at 0. It is calculated as follows
where Z = 2n, |·| is the absolute value, r₊ is the number of responses in the positive side of the scale, r₋ is the number of responses in the negative side of the scale, X₊ are the responses on the positive side of the scale, X₋ are the responses on the negative side of the scale and
Theoretical difficulties are known to exist with the PCI. The PCI can be computed only for scales with a neutral center point and an equal number of response options on either side of it. Also a uniform distribution of responses does not always yield the midpoint of the PCI statistic but rather varies with the number of possible responses or values in the scale. For example, five-, seven- and nine-point scales with a uniform distribution of responses give PCIs of 0.60, 0.57 and 0.50 respectively.
The first of these problems is relatively minor as most ordinal scales with an even number of response can be extended by a single value to give an odd number of possible responses. Scale can usually be recentred if this is required. The second problem is more difficult to resolve and may limit the PCI's applicability.
The PCI has been extended
where K is the number of categories, k_i is the number in the i^th category, d_ij is the distance between the i^th and i^th categories, and δ is the maximum distance on the scale multiplied by the number of times it can occur in the sample. For a sample with an even number of data points
and for a sample with an odd number of data points
where N is the number of data points in the sample and d_max is the maximum distance between points on the scale.
Vaske et al. suggest a number of possible distance measures for use with this index.
if the signs of r_i and r_j differ. If the signs are the same d_ij = 0.
where p is an arbitrary real number > 0.
if sign ≠ sign and p is a real number > 0. If the signs are the same then d_ij = 0. m is D₁, D₂ or D₃.
The difference between D₁ and D₂ is that the first does not include neutrals in the distance while the latter does. For example, respondents scoring −2 and +1 would have a distance of 2 under D₁ and 3 under D₂.
The use of a power in the distances allows for the rescaling of extreme responses. These differences can be highlighted with p > 1 or diminished with p < 1.
In simulations with a variates drawn from a uniform distribution the PCI₂ has a symmetric unimodal distribution. The tails of its distribution are larger than those of a normal distribution.
Vaske et al. suggest the use of a t test to compare the values of the PCI between samples if the PCIs are approximately normally distributed.

van der Eijk's A

This measure is a weighted average of the degree of agreement the frequency distribution. A ranges from −1 to +1. It is defined as
where U is the unimodality of the distribution, S the number of categories that have nonzero frequencies and K the total number of categories.
The value of U is 1 if the distribution has any of the three following characteristics:

all responses are in a single category
the responses are evenly distributed among all the categories
the responses are evenly distributed among two or more contiguous categories, with the other categories with zero responses

With distributions other than these the data must be divided into 'layers'. Within a layer the responses are either equal or zero. The categories do not have to be contiguous. A value for A for each layer is calculated and a weighted average for the distribution is determined. The weights for each layer are the number of responses in that layer. In symbols
A uniform distribution has A = 0: when all the responses fall into one category A = +1.
One theoretical problem with this index is that it assumes that the intervals are equally spaced. This may limit its applicability.

Related statistics

Birthday problem

If there are n units in the sample and they are randomly distributed into k categories, this can be considered a variant of the birthday problem. The probability of all the categories having only one unit is
If c is large and n is small compared with k^2/3 then to a good approximation
This approximation follows from the exact formula as follows:
;Sample size estimates
For p = 0.5 and p = 0.05 respectively the following estimates of n may be useful
This analysis can be extended to multiple categories. For p = 0.5 and p 0.05 we have respectively
where c_i is the size of the i^th category. This analysis assumes that the categories are independent.
If the data is ordered in some fashion then for at least one event occurring in two categories lying within j categories of each other than a probability of 0.5 or 0.05 requires a sample size respectively of
where k is the number of categories.

Birthday-death day problem

Whether or not there is a relation between birthdays and death days has been investigated with the statistic
where d is the number of days in the year between the birthday and the death day.

Rand index

The Rand index is used to test whether two or more classification systems agree on a data set.
Given a set of elements and two partitions of to compare,, a partition of S into r subsets, and, a partition of S into s subsets, define the following:

, the number of pairs of elements in that are in the same subset in and in the same subset in
, the number of pairs of elements in that are in different subsets in and in different subsets in
, the number of pairs of elements in that are in the same subset in and in different subsets in
, the number of pairs of elements in that are in different subsets in and in the same subset in

The Rand index - - is defined as

Intuitively, can be considered as the number of agreements between and and as the number of disagreements between and.

Adjusted Rand index

The adjusted Rand index is the corrected-for-chance version of the Rand index. Though the Rand Index may only yield a value between 0 and +1, the adjusted Rand index can yield negative values if the index is less than the expected index.

The contingency table

Given a set of elements, and two groupings or partitions of these points, namely and, the overlap between and can be summarized in a contingency table where each entry denotes the number of objects in common between and :.

Definition

The adjusted form of the Rand Index, the Adjusted Rand Index, is
more specifically
where are values from the contingency table.
Since the denominator is the total number of pairs, the Rand index represents the frequency of occurrence of agreements over the total pairs, or the probability that and will agree on a randomly chosen pair.

Evaluation of indices

Different indices give different values of variation, and may be used for different purposes: several are used and critiqued in the sociology literature especially.
If one wishes to simply make ordinal comparisons between samples, the choice of IQV is relatively less important, as they will often give the same ordering.
Where the data is ordinal a method that may be of use in comparing samples is ORDANOVA.
In some cases it is useful to not standardize an index to run from 0 to 1, regardless of number of categories or samples, but one generally so standardizes it.

Popular movies

The Hunger Games (film) - 2012 American dystopian action thriller science fiction-adventure film directed by Gary Ross and based on Suzanne Collins’s 2008 novel of the same name. It is the first insta...
untitled Captain Marvel sequel - part of Marvel Cinematic Universe....
Killers of the Flower Moon (film project) - Killers of the Flower Moon - film project in United States of America. It was presented as drama, detective fiction, thriller. The film project starred Leonardo Dicaprio, Robert De Niro. Director of...
Five Nights at Freddy's (film) - Five Nights at Freddy's - film published in 2017 in United States of America. Scenarist of the film - Scott Cawthon....

Popular books

Book of Revelation - The Book of Revelation is the final book of the New Testament, and consequently is also the final book of the Christian Bible. Its title is derived from the first word of the Koine Greek text: apok...
Book of Genesis - account of the creation of the world, the early history of humanity, Israel's ancestors and the origins...
Gospel of Matthew - The Gospel According to Matthew is the first book of the New Testament and one of the three synoptic gospels. It tells how Israel's Messiah, rejected and executed in Israel, pronounces judgement on ...
Michelin Guide - Michelin Guides are a series of guide books published by the French tyre company Michelin for more than a century. The term normally refers to the annually published Michelin Red Guide , the oldest...
Psalms - The Book of Psalms , commonly referred to simply as Psalms , the Psalter or "the Psalms", is the first book of the Ketuvim , the third section of the Hebrew Bible, and thus a book of th...
Ecclesiastes - Ecclesiastes is one of 24 books of the Tanakh , where it is classified as one of the Ketuvim . Originally written c. 450–200 BCE, it is also among the canonical Wisdom literature of the Old Tes...
The 48 Laws of Power - non-fiction book by American author Robert Greene. The book...

Popular television series

The Crown (TV series) - historical drama web television series about the reign of Queen Elizabeth II, created and principally written by Peter Morgan, and produced by Left Bank Pictures and Sony Pictures Tel...
Friends - American sitcom television series, created by David Crane and Marta Kauffman, which aired on NBC from September 22, 1994, to May 6, 2004, lasting ten seasons. With an ensemble cast sta...
Young Sheldon - spin-off prequel to The Big Bang Theory and begins with the character Sheldon...
Modern Family - American television mockumentary family sitcom created by Christopher Lloyd and Steven Levitan for the American Broadcasting Company. It ran for eleven seasons, from September 23...
Loki (TV series) - upcoming American web television miniseries created for Disney+ by Michael Waldron, based on the Marvel Comics character of the same name. It is set in the Marvel Cinematic Universe, shar...
Game of Thrones - American fantasy drama television series created by David Benioff and D. B. Weiss for HBO. It...
Shameless (American TV series) - American comedy-drama television series developed by John Wells which debuted on Showtime on January 9, 2011. It...