Review by: Mazzola, Luca (2008-10-13)

The article focuses on research in supporting users in on-line learning activities: it analyses how the interaction cycle (the loop between learner's actions and system behaviour) could be improved using feedback structures.

The authors work in the Educational Technology Expertise Centre of the Open University Netherlands (OUNL): Glahn is a Ph.D. student, Specht is an associate professor and Koper is a full professor in educational technology.

The starting point for this paper arises from the awareness that a conceptual gap exists between the user actions in the learning environment and the responses provided by the system.One of the causes could be the representation of the system as a black-box, something that cannot be explored on the inside, but is portrayed only as its connections with the environment.The authors try to fill the gap opening the system with the analysis of the existing literature in the field of smart indicators. In the paper, they try to classify the existing literature (14 publications) about so-called “smart” indicators and analysed paper from three different fields: 6 from adaptive hypermedia, 4 from social awareness web and collaborative environment, 3 from recommendation system and 1 from another field.

Other than existing investigations on self-regulated learning and feedback, that normally focus on the users' side Glahn, Specht, and Koper focus on the system's side of the interaction cycle. They analyse the cycle from the user/environment input to the production of some useful indicator. An indicator is an object that draws our attention to ongoing relevant events only when it's really necessary: a charging/discharging/low-level battery light on a laptop is a typical example for an indicator. Learners naturally look for success indicators while learning (feedback), so their importance in non-formal and informal learning is stressed as element of support in the learning process. A system that uses smart indicators should be able to collect data from the user input and the learning environment (collect interaction footprints), transform them into information (produce indicators) and generate knowledge (present some “smart” indicators and, in this way, support the learner's judgement and reflection).

The authors adopt a 4-layers model (Zimmermann, Specht & Lorenz, 2005) as reference for the analysis of context-aware indicators: the layers are sensor, semantic, control and indicator.The division in layer classes is based on the role of objects: in the first phase sensor objects capture user information and the contextual state; after that, one or more semantic objects collect data from sensors and aggregate them into higher level information. Control objects interpret the information received using the contextual environment. Eventually, an indicator object represents relevant relations and actual values in a format that is understandable for humans.

In the analysed literature, the authors find 4 subtypes in which the sensor layer is differentiated: time related, social context based, user based, and environment grounded objects. The two categories of time and social context are widely used and play an important role.

Also, for semantics, the authors find a division: simple arithmetic, naïve statistic and network-based. The differences lie in the functions used to aggregate data from the previous level, varying from simple mathematical ones, like sum or equity, to statistic arithmetic means and to functions that are able to extract relationships between learners and information.

In the control layer they find 3 different strategies of increasing the level of abstraction: null (no further processing, only selection based on user preferences), clustering (grouping process based on learner profile and reorganization) and scoring with sorting (higher level approach, it involves a previous ordering process into the sensor measurement and a following clustering approach).

Finally, indicator objects are divided into 4 classes based on graphical aspect: embedded content, 1-D graphical, 2-D graphical, and 3-D graphical.

While we are still in an experimental phase for this kind of indicators, we need to better understand and measure their impact on the learning cycle. Previous works clearly stated that support of the interaction between learner and system is very important for the experience and advancement of the learner.

According to the paper, three main issues remain to be solved: what contextual information could better support the learners? How can a system collect and aggregate data to promote useful indicators? What's the effect of inserting different aggregators, strategies and indicators, alone or combined together, in the learning cycle?

The article addresses an emerging field in user modelling, adaptive hypermedia and information visualization and makes a summary in the current ongoing development. Even if the authors put together different existing approaches, some additional research is needed to focus on distinct layers of the model and to extend the number of analysed papers, in order to expose a more systematic view of the field.