In typical classroom settings, students collaborate on tasks and submit group work for grading, often relying on peer evaluations to determine individual grades. We are concerned with the method of converting pairwise peer evaluations into individual final grades. The most common way to do it is as follows: every group member has a certain number of points to distribute among the rest of the group, and the final score of every student is the average number of points she receives from other group members. We call this the "pie-to-others method".
Assessment should be psychometrically valid and reliable. We argue that the "pie-to-others" method of evaluating individual contributions to group work is neither valid nor reliable. Moreover, by constructing a mathematical model of peer evaluation, we can measure exactly how much the "pie-to-others" (or, more generally, any assessment method of individual contribution to group work) deviates from being reliable. We will explain the worst-case scenario, i.e., derive the theoretical largest possible difference between the outcome of the "pie-to-others" and the fair grade a student deserves. By analyzing a large dataset (1201 students, 220 project groups, 6619 evaluations) collected in large undergraduate classes in an Asian university, we estimate that, in practice, about 1% of all students are misgraded by the "pie-to-others". Finally, we will present an easy fix to the pie-to-others method that makes it reliable.

Authors:
Fedor Duzhin, Nanyang Technological University, Singapore
Megan Zheng Chi Lee, Nanyang Technological University, Singapore

About the Presenter(s)
Dr Fedor Duzhin is a University Associate Professor/Senior Lecturer at Nanyang Technological University in Singapore

See this presentation on the full schedule – Sunday Schedule

Bookmark