Kappa

Kappa as a Measure of Concordance in Categorical Sorting

Cohen's

Unweighted Kappa
Kappa with Linear Weighting
Kappa with Quadratic Weighting
Frequencies and Proportions of Agreement

		B			Total
		1	2	3	Total
A	1	44	5	1	50
	2	7	20	3	30
	3	9	5	6	20
Total		60	30	10	100
k = 3 N = 100

Kappa provides a measure of the degree to which two judges, A and B, concur in their respective sortings of N items into k mutually exclusive categories. A 'judge' in this context can be an individual human being, a set of individuals who sort the N items collectively, or some non-human agency, such as a computer program or diagnostic test, that performs a sorting on the basis of specified criteria. [Click here for an explanation of the conceptual and computational details of kappa.]

To begin, select the number of categories by clicking the appropriate button below; then enter your data into the appropriate cells of the data-entry matrix. After all data have been entered, click the «Calculate» button. To perform a new analysis, click the «Reset» button and start over. The analysis assumes that each entered value is an integer equal to or greater than zero._T

Note that measures of weighted kappa are meaningful only if the categories are ordinal and if the weightings ascribed to the categories faithfully reflect the reality of the situation. The weightings in this case are determined by the imputed relative distances between successive ordinal categories. By default, each of these distances is set at '1'. You are free to change any or all of these distances, though I recommend you do so only if you have good reason for it.

The author is grateful to C�sar Roberto de Souza for detecting an error in the original programming for this module and suggesting the appropriate correction.

Select the number of categories:	2 3 4 5 6 7 8
Number selected =

Basis for weighting: imputed relative
distances between ordinal categories_T

1~2	2~3	3~4	4~5	5~6	6~7	7~8	⇐ successive ordinal categories
							⇐ imputed relative distances

Data Entry

		B								Totals
		1	2	3	4	5	6	7	8	Totals
A	1
	2
	3
	4
	5
	6
	7
	8
Totals
Reset Calculate

The designation "nc" appearing in any of the following
cells means "this quantity cannot be calculated." This
will typically occur only when your data entries in the
above table include a substantial proportion of zeros.

Unweighted Kappa
Observed Kappa	Standard Error	.95 Confidence Interval
	Standard Error	Lower Limit	Upper Limit
Method 1
Method 2

		maximum possible unweighted kappa, given the observed marginal frequencies
		observed as proportion of maximum possible

Kappa with Linear Weighting
Observed Kappa	Standard Error	.95 Confidence Interval
		Lower Limit	Upper Limit

		maximum possible linear-weighted kappa, given the observed marginal frequencies
		observed as proportion of maximum possible

Kappa with Quadratic Weighting
Observed Kappa	Standard Error	.95 Confidence Interval
		Lower Limit	Upper Limit

		maximum possible quadratic-weighted kappa, given the observed marginal frequencies
		observed as proportion of maximum possible

Frequencies of Agreement
Category	Maximum Possible	Chance Expected	Observed
1
2
3
4
5
6
7
8
Total

Proportions of Agreement				.95 CI of Observed
Category	Maximum Possible	Chance Expected	Observed	Lower Limit	Upper Limit
1
2
3
4
5
6
7
8
Composite

Confidence intervals for proportions are calculated according
to the Wilson efficient-score method, corrected for continuity.

Home

Click this link only if you did not arrive here via the VassarStats main page.