Review by: Dreier, Matthias (2004-10-01)

Parlangeli, Marchigiani and Bagnara evaluated the effects of usability on learning in a multimedia educational system at the University of Siena, Italy. Object of the evaluation was a CD-ROM-based multimedia learning environment for teaching mathematics to undergraduate students in Economics which is basically a textbook implemented as a hypertext.

To assess the usability of the learning environment two experts first conducted a heuristic evaluation. Then the authors conducted a usability test with ten participants using thinking-aloud as well as pre-test and post-test questionnaires. The results were discouraging. Both usability methods revealed over 50 violations of well-established usability principles. The potential end-users detected less usability flaws than the experts. However, the experts found most of the problems reported by users.

The second part of the study addressed the learning performance. The authors compared three groups of twelve students. One group got the CD-ROM, another group got the print-outs of the CD-ROM material, and the third group learned from a traditional textbook. The students were allowed to learn as long as they liked. The test revealed no significant differences in learning performance, but the CD-ROM group was less satisfied with the learning material. The authors expected well-designed multimedia learning environments to be superior to traditional learning materials. They conclude that bad usability prevents multimedia learning environments from living up to their promise.

The study of Parlangeli et al. shows that the general belief in the superiority of multimedia learning environments is sometimes misleading. It has been repeatedly argued that poorly designed hypertexts make users feel lost. The study supports these concerns. The article is also useful for its comparison of two usability methods. Unfortunately, the authors do no specify how long the three groups learned. Furthermore, it would be interesting to evaluate an improved version of the learning environment in a similar test.