This study examines how variations in speaker order increase the potential for rater error in the performance based course. Seventy-six undergraduate raters were randomly assigned to one of eight treatment groups and asked to grade eight-week training course. Speaker order and presentation quality varied across groups and an ANOVA was used to examine significant differences across rater assessments, feedback quality and rater gender. Significant main effects were identified in each of the eight treatment groups suggesting that speaker order influenced rater scoring.
Turman, Paul D. and Barton, Matthew H.
"Bias in the Evaluation Process: Influences of Speaker Order, Speaker Quality, and Gender on Rater Error in the Performance Based Course,"
Basic Communication Course Annual: Vol. 16
, Article 6.
Available at: http://ecommons.udayton.edu/bcca/vol16/iss1/6