Shepard, K. (2009)
e is for exploration: Assessing hard-to-measure learning outcomes
British Journal of Educational Technology, Vol. 40, No. 2, pp. 386–398
Review by: Dommen, Jenny (2009-07-08)
In Higher Education, students are expected to acquire knowledge, abilities and attitudes before graduation, usually specified via intended learning outcomes. There exists a wide range of traditional and innovative approaches for assessing these outcomes. Affective outcomes, such as values or attitudes have consistently proved more difficult to assess by examinations, assignments or other traditional approaches. The author of this paper focuses on the more challenging aspect of assessing affective attributes in Higher Education. Besides this, his goal is to explore "how e-technologies are being used, or could be used, to meet these challenges" (p. 386).
The author takes a very broad perspective on this topic, starting to explore the different practices of assessment in Higher Education and after graduation. Whereas in Higher Education, it is still not very widespread to assess affective outcomes, after graduation, assessment often focuses on teamwork and networking skills, creativity and values which fit to the profession. Portfolios, interviews and peer or expert reviews are very common in appointment and promotion processes. Besides these developments in assessment practices, progress in information and communication technologies (ICT) provided new possibilities for assessment and selection processes, while adding flexibility, interactivity and connectivity (p. 387). In the subsequent parts of the article, the author explores the nature of assessment and its association with selection and evaluation. Alongside, observations on how ICT may support further development are discussed. Shephard builds his argument around different assessment paradigms:
- Norm-referenced assessment processes attempt to characterize individuals in relation to others, which forms the basis of selection processes. He argues with Biggs, that within Higher Education, this form of assessment is inappropriate, for it determines the characteristics of people at some fixed point in time (p. 390). In the work place though, the need to rank applicants for selection purposes is obvious. The author outlines the fact that portfolios are being used for selecting candidates for a job (e. g. teacher portfolios).
- Criterion-referenced assessment processes attempt to describe the characteristics of individuals in relations to some defined standards. Most progress to automate the process of assessment has been made within objective testing for cognitive skills and abilities. For affective learning, portfolios are the key innovation in supporting development and judgment.
- The author explores another form of assessment: construct-referenced assessment. In communities (e. g. a research community), there usually exists some socially constructed descriptions of practitioners, who describe conditions for new members to enter the community. Whereas this is still a form of criteria-based assessment, the criteria are usually very vague and often contain 'value sets' (p. 392). Within this construct-referenced form of assessment, the author sees the most potential for e-technologies to have a great impact, because the internet and other technologies empower participants to be involved in social construction of knowledge. Today, where the acceptance of a journal paper may depend on one or two peers, the wider online environment could extend the possibilities of peer rating processes.
- Ipsative assessment emphasizes the notion of self-assessment, self-reflection, personal-professional development and lifelong learning (p. 394). The author argues, that this form of assessment should be more widely acknowledged in Higher Education in order to foster the development of students' values and attitudes. Portfolios, e-portfolios and other technologies such as wikis could help to better support this development in the future.
In the last part of the article, the author explores the relationship between assessment and evaluation, two terms, which are used interchangeably by some, but not by others. The author argues that assessment could be managed as evaluation, if evaluation is understood in the sense of "negotiation between all stakeholders to determine the intrinsic value of what is being evaluated" (p. 395). In this sense, evaluation "recognizes that what is being evaluated has no foundational reality or objective truth and its value depends on the approaches and experiences of those engaged in the evaluative activities" (p. 396). The author thus comes to the conclusion, that portfolios, in attempting to assess (or rather report) affective attributes have more in common with evaluation (in the sense of negotiating) than with conventional assessment. He outlines that "evaluation can be achieved on a group-wide basis and that this avoids some of the really difficult issues in assessing values and related affective outcomes" (p. 396). This process involves the use of surveys, questionnaires and semantic differential techniques. Successful applications can as well be found in online settings.
In sum, the article reflects on the nature and practices of assessment in Higher Education and at post-graduate level, with a strong focus on the assessment of affective outcomes. The author argues for the increasing use of portfolios and other e-technologies in Higher Education. At several points in the article, the author reflects on the use of e-technologies for assessment purposes. But the promise of focusing on the added value of e-technologies for the assessment of affective attributes in contrast to traditional approaches remains at a very superficial level.