Review by: Hasanbegovic, Jasmina (2004-07-14)

The presented meta-analysis aims to address the main effects of problem-based learning (PBL) on the two outcome categories knowledge and skills.

For the analysis the authors selected 43 empirical studies on problem based learning in tertiary education. The characteristics of the learning environment had to fit their described core model of PBL. Hence, the dependent variables used in the studies had to be an operationalization of the knowledge and/ or skills of the students and to be conducted in a real- life classroom or programmatic setting.

After a thorough review of research literature a statistical meta- analysis is conducted including vote counts and the associated sign test on students knowledge and skills.

The results of this meta- analysis suggest that students in PBL are better in applying their knowledge as they suggest a robust positive effect from PBL on the skills of students.According to the moderator analysis, the research design of the study suggests that no significant variation in effect sizes for knowledge-related outcomes can be attributed to method-related influences. In contrast, the variation in effect sizes for skills outcomes was associated with the methodological factor ‘research design’. Another methodological moderating variable results from the implementation in environments varying in scope from one single course up to an entire curriculum, but no significant difference on achievement was recognized between a single course and a curriculum-wide implementation of PBL.

The differences of the effect sizes between the expertise level of the students results show that the differences arising in the first and second year between PBL students and traditional students disappear if the reproduction of knowledge is assessed when in a broader context all the students are asked to apply their knowledge. The effects of PBL on skills demonstrate a strong positive effect on the skills of the students on all expertise levels.

According to the retention period the analysis shows that students in PBL seem to remember more of the acquired knowledge as their knowledge has been elaborated more and consequently. For tests assessing skills, the study suggests that no significant variation in effect sizes can be attributed to the presence or absence of a retention period. The type of the used assessment method contributes to a significant variation in effect sizes as well as effects on knowledge and skills. This can be attributed to the specific operationalization of the dependent variable. The more an instrument is capable of evaluating the skills of the student, the larger the ascertained effect of PBL.

The benefit of this study can be seen in the structured and clear methodological proceeding of a meta- analysis that focuses on the main outcome categories knowledge and skills. All important instruments for a statistical meta- analysis are used and explained in great detail. The results underline theoretical approaches on PBL that emphasize the applied- oriented knowledge outcome and an effect on skills. Furthermore, the results of the study are compared with existing meta- analysis on PBL which also underline a robust positive effect of PBL on skills.

The limitations of this review are strongly related to its inclusion of field studies (quasi- experimental research) that supports a high ecological validity, but causes less internal validity relative to more controlled laboratory studies. Nevertheless, the authors argue correctly, when they say, that they try to bridge the gap between research and educational practice. However, differences between real-classroom and programmatic settings are not explicitly mentioned.