An arbitrator’s decision at Ryerson University released July 6, 2018 has ordered that student-based evaluations of teaching not  be used in tenure and promotion cases, effective immediately.

The decision stated that the evaluations “are imperfect at best and downright biased and unreliable at worst”.

With respect to the evidence provided Student Evaluations of Teaching (SET) at Ryerson, the arbitrator also wrote: “According to the evidence, which was largely uncontested, and which came in the form of expert testimony and peer reviewed publications, numerous factors, especially personal characteristics – and this is just a partial list – such as race, gender, accent, age and “attractiveness” skew SET results. It is almost impossible to adjust for bias and stereotypes. Student and faculty gender affects outcomes, as does grade expectation. Other systemic problems were identified by the experts, and in the literature. One example, and there were many, is the reliability of SETs completed on line versus those completed in class. There are differences between the results of absent students who complete SETs online and those who complete the forms in class. These differences need to be understood. Overall response rates also need to be considered: the lower the response rate the less reliable the results.  There is certainly no reason to believe that the views of responders can be extrapolated and applied to non-responders.”

Discussions on this and related matters has been going on since 2003, though this issue proved difficult. A grievance was filed in 2009, another in 2015, and a hearing held April 2018. The case is complex, and we encourage you to read the decision for yourself and share your views. We will be requesting copies of the submissions of the parties.