Review by: Dreier, Matthias (2004-08-18)

Usability of educational software is widely accepted to be crucial to the learning process. This publication reports on a comparative usability evaluation of two distance-learning environments. The authors tested the self-assessment component of IDLE and WebCT. IDLE was developed at the University of Patras, Greece. WebCT was developed at the University of British Columbia, Canada and is a commercial distance-learning platform.

The goal of the study was to explore the correlation of software usability and student performance. Participants of the usability study were 120 under-graduate electrical engineering students with no prior experience in using distance-learning environments. The self-assessment test to evaluate the student performance was a quiz consisting of multiple-choice and fill-the-blank questions. The usability was evaluated using a questionnaire consisting of ten questions. The usability evaluation revealed a significant better usability of WebCT. The students in the WebCT group performed significantly better than the IDLE group. Thus, the authors conclude that the usability of the distance-learning environment and the student performance correlates, indeed. The authors also compared the two groups with a paper-and-pencil group. The students of this group performed even better than the other two and required less time to complete the task.

This study is a valuable contribution to the ongoing discussion on the relation between usability and learnability. Even in a rather standard component of distance-learning environments – the multiple-choice test – the usability has a considerable effect on student performance. Obviously, novice users are very sensitive to usability flaws. The authors conducted a thorough evaluation and did their best to diminish the influence of other parameters. The methodological design of the study is exemplary. However, the article is rather lengthy. The discussion of the numerous usability heuristics and the context of the study is certainly not the most luminous part of the publication.