Empirical formula for creating error bars for the method of paired comparison

Abstract

The method of paired comparison based on Thurstone’s case V of his law of comparative judgments is often used as a psychophysical method to derive interval scales of perceptual qualities in imaging applications. However, methods for determining confi- dence intervals and critical distances for significant differences have been elusive, leading some to abandon the simple analysis provided by Thurstone’s formulation. Monte Carlo simulations of paired comparison experiments were performed in order to derive an empirical formula for determining error. The results show that the variation in the distribution of experimental results can be well predicted as a function of stimulus number and the number of observations. Using these results, confidence intervals and critical values for comparisons can be made using traditional statistical methods.