Aim: To Calculate Sample Size to estimate Cohen’s Kappa with specified precision.
Formula Used
To calculate Pa or Pe or k, when other two are known
k = (Pa – Pe) / (1 – Pe)
Pa = k * (1 – Pe) + Pe
Pe = (Pa - k) / (1 - k)
k = Anticipated Cohen’s Kappa
Pa = Guessimate of Actual Observed Agreement between two raters (out of 1)
Pe = Expected (Chance) Agreement (out of 1)
ε = Precision / Allowable error (Out of 1) (e.g. 25% = 0.25)
Z1-α/2= the standard normal deviate corresponding the confidence level
Example:
You want to conduct a study to find out level of agreement for X-rays (Classified as normal or abnormal) between a radio-diagnosis PG student and her Professor. As per the previous reports, the gusstimate of observed agreement is 0.75 and Cohen’s kappa is 0.6. How many X-rays shall be included in the study, to find out Cohen’s kappa with a relative precision of 10%, at 95% confidence level?
Solution:
Here
Pa = 0.75, k = 0.6, Confidence level = 95%, precision = 10%
After putting these values, we get required sample size = 512.
What will happen after selecting this sample size, considering we get the results with anticipated Pa and k?
Pa = 0.75, k = 0.6
Pe = (Pa - k) / (1 – k) = 0.15/0.4 = 0.375
95 % CI of k = 0.6 ± 1.96 * 0.031 = 0.54 – 0.65 (Precision of ~ 0.6, i. e. 10 % of relative precision of k)
@ Sachin Mumbare