The problem of measurement bias in comparing selected subgroups

Jorge L. Mendoza, Seunghoo Lee, Dustin Fife

Research output: Contribution to journalArticle

Abstract

Estimates of subgroup differences are routinely used as part of a comprehensive validation system, and these estimates serve a critical role, including evaluating adverse impact. Unfortunately, under direct range restriction, a selected mean ((Formula presented.)) is a biased estimator of the population mean (Formula presented.) as well as the selected true score mean (Formula presented.). This is due partly to measurement bias. This bias, as we show, is a factor of the selection ratio, the reliability of the measure, and the variance of the distribution. This measurement bias renders a subgroup comparison questionable when the subgroups have different selection ratios. The selected subgroup comparison is further complicated by the fact that the subgroup variances will be unequal in most situations where the selection ratios are not equal. We address these problems and present a corrected estimate of the mean difference, as well as an estimate of Cohen’s d* that estimates the true score difference between two selected populations, (Formula presented.). In addition, we show that the measurement bias is not present under indirect range restriction. Thus, the observed selected mean (Formula presented.) is an unbiased estimator of selected true score mean (Formula presented.). However, it is not an unbiased estimator of the population mean (Formula presented.). These results have important implications for selection research, particularly when validating instruments.

Original languageEnglish (US)
JournalBritish Journal of Mathematical and Statistical Psychology
DOIs
StateAccepted/In press - 2020

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Arts and Humanities (miscellaneous)
  • Psychology(all)

Fingerprint Dive into the research topics of 'The problem of measurement bias in comparing selected subgroups'. Together they form a unique fingerprint.

  • Cite this