REFINING THE JUDGES’ ASSESSMENTS OF ITEMS IN PAIRED COMPARISON EXPERIMENTS

Abstract

REFINING THE JUDGES’ ASSESSMENTS OF ITEMS IN PAIRED COMPARISON EXPERIMENTS

Abstract

In the method of paired comparisons, items are compared on the basis of their qualitative characteristics assessed by judges through their sensory evaluations. Judges are offered items in pairs and are asked to pick the better one. The experiment is repeatedly executed to yield preference data based on binary digits – zeros and ones – allotted to the items by the judges. The preferred item is awarded rank one while the loser is assigned zero. As the binary digits fail to furnish the actual comparative worth of items and indistinguishably assign one to the preferred item and zero to the losing one, a methodology is proposed to measure the actual comparative worth of the competing items on a finer scale by assigning some refined rank on a finer scale to each of the two competing items. The assigned ranks are then converted to a refined paired comparison data-set in the form of a preference matrix to be used for ranking the items. For illustration, a real data-set on ice-cream brands is used to rank the brands using the renowned Bradley-Terry model for paired comparisons. 

Keywords

• Paired Comparisons,Bradley-Terry Model,Refined data-set,Maximum likelihood (ML) estimates,Iterative procedure,Test of Goodness of Fit,2000 AMS Classification: 62 J 15

References

1. Abbas N. and Aslam, M. Prioritizing the items through paired comparison models, a Bayesian approach, Pakistan Journal of Statistics 25(1), 59–69, 2009.
2. Abbas, N. and Aslam, M. Extending the Bradley-Terry model for paired comparisons to accommodate weights, Journal of Applied Statistics 38 (3), 571–580, 2011.
3. Bradley R. A. and Terry, M. E. Rank analysis of incomplete block designs: I. The method of paired comparisons, Biometrika 39, 324–345, 1952.
4. Bradley, R. A. Some statistical methods in taste testing and quality evaluation, Biometrics 9(1), 22–38, 1953.
5. Bradley, R. A. Paired comparisons: Some basic procedures and examples, In: Krishnaiah, P. R., Sen, P. K. (Eds.) (Handbook of Statistics 4, North-Holland, Amsterdam, 1984), 299– 326.
6. David, H. A. The Method of Paired Comparisons, 2nd Ed (Charles Griffin & Company Ltd., London, 1988).
7. Davidson, R. R. On extending the Bradley–Terry model to accommodate ties in paired com- parison experiments, Journal of American Statistical Association 65, 317–328, 1970.
8. Rao, P. V. and Kupper, L. L. Ties in paired comparison experiments: A generalization of the Bradley-Terry model, Journal of the American Statistical Association 62 (317), 194–204, 1967.