Wednesday 26th June 2019, 12.30pm to 2.00pm
Speakers: Scott Slorach
Location: Room H/G/21, Heslington Hall, Campus West
“My friend did the same and got a better mark.” Can Adaptive Comparative Judgment achieve more consistency in assessment than traditional academic judgment?
Consistency and equity are two of the University’s core principles on which assessment should be based. Marker meetings and guides, double-marking, and internal and external moderation are designed to assure those principles. However, students still raise issues of consistency of marking standards, whilst the academic judgment underlying a mark cannot be questioned.
Adaptive comparative judgment (ACJ) is an alternative assessment method. Rather than individual (or double) marking, a larger group of academics is presented, through an online system, with pairs of student scripts. Individually, they exercise comparative judgment – simply, which script is the better – against an agreed assessment statement. The system pairs scripts for comparison, initially at random, then systematically, based on an emerging ranking order that progressively identifies more reliably, it is claimed, a student’s position within their cohort.
In this workshop we shall consider in more detail: the theory underpinning ACJ; its claimed reliability and ongoing consistency as against traditional marking; the intellectual and organisational processes involved; and the types of assessment potentially most suited to exercising ACJ. We shall will draw upon a comparative exercise undertaken at York Law School, where traditionally assessed formative assessments were re-marked using ACJ, raising interesting issues around both the results provided and the tutor experience. We shall also discuss student reaction to ACJ, and its additional potential for use by students.