Comparative Judgement

Wednesday 26th June 2019, 12.30pm to 2.00pm

Speakers: Scott Slorach
Location: Room H/G/21, Heslington Hall, Campus West

Workshop Summary

“My friend did the same and got a better mark.” Can Adaptive Comparative Judgment achieve more consistency in assessment than traditional academic judgment?

Consistency and equity are two of the University’s core principles on which assessment should be based. Marker meetings and guides, double-marking, and internal and external moderation are designed to assure those principles. However, students still raise issues of consistency of marking standards, whilst the academic judgment underlying a mark cannot be questioned.

Adaptive comparative judgment (ACJ) is an alternative assessment method. Rather than individual (or double) marking, a larger group of academics is presented, through an online system, with pairs of student scripts. Individually, they exercise comparative judgment – simply, which script is the better – against an agreed assessment statement. The system pairs scripts for comparison, initially at random, then systematically, based on an emerging ranking order that progressively identifies more reliably, it is claimed, a student’s position within their cohort.

In this workshop we shall consider in more detail: the theory underpinning ACJ; its claimed reliability and ongoing consistency as against traditional marking; the intellectual and organisational processes involved; and the types of assessment potentially most suited to exercising ACJ. We shall will draw upon a comparative exercise undertaken at York Law School, where traditionally assessed formative assessments were re-marked using ACJ, raising interesting issues around both the results provided and the tutor experience. We shall also discuss student reaction to ACJ, and its additional potential for use by students.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s