You are not logged in.
Pages: 1
All --
I'm trying to determine the best statistical tool to use in order to measure the level of agreement between inspectors. My goal is to group the inspectors into "tendency groups", where each group would consist of a number of inspectors that generally agree with other on a subjective measure.
I'll use a simple example to try to clarify. Let's say 20 teachers are asked to grade 10 different essays on a scale from 1 to 10. Once the grading is completed, the teachers would then be grouped together based on how closely they agree with each other's assesment. I'm looking for the statistical test to use in order to determine which teachers should be grouped together, how many logical groups they should be divided into and what is the confidence level of their agreement.
Ultimately, these inspectors would be grouped together going forward, having statistical confidence that they would make the same subjective decisions with some degree of probablity.
Here's a sample data set to play with:
Essay1 Essay2 Essay3 Essay4 Essay5 Essay6 Essay7 Essay8 Essay9 Essay10
Teacher1 2 5 5 9 5 4 6 8 5 1
Teacher2 9 2 7 10 1 10 4 1 8 1
Teacher3 10 10 1 2 10 8 3 8 1 1
Teacher4 10 8 5 8 10 10 4 3 6 10
Teacher5 8 5 3 1 10 6 3 1 2 4
Teacher6 6 8 7 1 2 3 9 5 8 7
Teacher7 5 8 10 1 6 3 10 4 6 10
Teacher8 3 5 5 8 3 2 5 10 5 3
Teacher9 4 4 9 3 10 4 7 8 2 8
Teacher10 3 10 8 10 2 3 7 4 3 3
Teacher11 6 9 5 9 10 1 3 10 1 1
Teacher12 8 4 2 7 1 6 7 10 10 6
Teacher13 7 3 3 8 4 9 1 6 6 2
Teacher14 7 9 3 9 3 9 5 2 5 2
Teacher15 9 4 6 6 9 6 10 5 8 7
Teacher16 1 9 2 4 9 10 8 4 6 8
Teacher17 10 6 10 1 8 10 1 7 3 4
Teacher18 9 2 10 8 7 4 10 3 10 9
Teacher19 4 3 1 3 8 10 2 1 5 10
Teacher20 5 9 9 7 5 9 2 2 10 10
In this example, Teacher1 and Teacher8 generally agree on their grading of the 10 different essays. How can I prove that statistically?
Any help would be greatly appreciated.
Thanks,
Vlad
Pages: 1