Review by: Schönwald, Ingrid (2007-03-26)

This article discusses the role, influence and quality of trendwatching reports in the field of e-learning and presents a checklist for evaluating the quality of trend reports on e-learning.

Many HRD departments use trend reports as orientation guides for the development of their e-learning strategies. The high relevance of trend reports can be put down to the heterogeneity of supply on the e-learning market which makes the market intransparent for customers. In this environment there is a high need for predictive trend studies that give an overview of providers, target groups and possible e-leaning solutions to enable HRD decision makers to anticipate future developments and assist their organizations with the development of sound e-learning strategies and policies.

However, predictions on e-learning are characterised by methodological and technical constraints of disruptive technologies. For sound trendwatching research, different forecasting methods can be used, such as monitoring, opinion of experts, statistical trend analysis, modelling, and scenario construction.

The authors defined the concept of quality of trendwatching reports by using four dimensions:

1. Author and authority,
2. Accuracy of research and data collection,
3. Accuracy of the document, and
4. Objectivity of the presented content.

For each of theses four dimensions four to six criteria were constructed, which resulted in a 22-item checklist. Then four trend reports were selected that were published between 2000 and 2002 and which were frequently quoted by business press and academia. These four trend reports were evaluated by two assessors using the checklist.

The results of the evaluation process for the four reports were very similar. All reports had high scores in the 'author and authority' dimension. The dimension 'objectivity' was scored very low. While the reports used persuasive and quite enthusiastic language to predict e-learning’s bright future, they did not pay substantial attention to possible barriers that might have negative effects on the future of e-learning. Very low scores were observed for the dimension 'accuracy of research and data collection’. The studies were hardly reproducible by other researchers, as the conceptual framework, data used and the methodology were not made transparent.

The central conclusion of the authors is that the effect of the surveyed trend reports is rather the setting than the watching of trends. Thus they advise practitioners and researchers to be critical of the quality of trend studies in the domain of e-learning.

In spite of some limitations of this article, such as the limited number of tested trend studies and the intransparent derivation of quality criteria, the checklist presented in this article provides a useful conceptual framework to evaluate the quality of trend studies. Even if the e-learning trend reports of the recent years are much more sounded than in earlier years the constraints of predictions are still valid.

Review by: Hasanbegovic, Jasmina (2006-04-18)

In order to shed some light on the future diffusion and exploitation of eLearning, trend- watching studies are conducted which focus especially on the consequences of eLearning developments for future HRD practice, investments or policy. The results are cited by authors around the world almost uncritically. This article tries to look behind the scenes and discusses the role, influence and quality of trend-watching reports in the field of eLearning. A checklist was developed to evaluate the quality of current trend studies.

First, the authors present developments in the eLearning market which are very complex and non-transparent and therefore demand predictive trend studies that give an overview of providers, target groups and possible eLearning solutions.Second, the authors discuss the topic of sound trend watching research. They explain the methodical and technical constraints of disruptive technologies which complicate predictions and are exacerbated by market instabilities. Then they present several types of forecasting methods, their advantages and their choice of use. Third, the construction process of the checklist is described, the quality criteria themselves are presented and their application explained. Next, the results of the conducted evaluation of four selected trend studies are presented. The authors explain the very low scores for the dimension "objectivity" by the fact that the reports used persuasive and quite optimistic language to predict eLearning future and did not pay any attention to negative effects or possible barriers. Moreover, the mostly quasi- scientific reports showed very low scores for the dimension "accuracy of research and data collection". Finally, the authors point out, that most of the reports rather set trends than to watch them. They advise readers to be more critical of the quality of trend studies in the domain of eLearning and to demand sound methodology.

In fact, different organisations and institutions offer often very expensive trend reports without describing the author and authority, research methods and data collection, but which are then anticipated uncritically and cited by many authors. This reaction leads really to a trend setting. We need more investigations of independent organisations and universities, which are interested or engaged in the diffusion research of educational technology, which use new methods for this research (e. g. network analysis) and which especially unfold their methods, data collection and results. In summary, this clear and sound article opens the eLearning community's eyes to reflect on and question results of trend reports.