Educational Technology & Society 5 (3) 2002
ISSN 1436-4522

Evaluation of Learning Technologies in Higher Education

Grainne Conole
Professor, Research and Graduate School of Education
University of Southampton
Highfield, Southampton SO17 1BJ
United Kingdom

The 16 articles in this special issue cover some of the key research questions about the use and integration of learning technologies and a clearer understanding of how they can be used to support learning and teaching. There are five broad themes within which the papers sit:

  • Methodological issues around the use and impact of evaluation
  • Critiques on the broader socio-cultural issues
  • Evaluation of the use of communication technologies
  • Evaluation of the use of Virtual Learning environments
  • Multimedia and adaptive interactive environments

The three papers on methodological issues discuss some of the problems which arise out of evaluation studies and some strategies for addressing them.  The issues around the development of practitioner skills and motivation in a professional accreditation programme are discussed by Harvey et al. Williams gives an overview of the benefits of evaluation, adopting the participative evaluation philosophy which is used by many of the researchers in this issue. Finally Oliver and Harvey critique what is meant by impact in evaluation and, in particular, what does it mean in the context of evaluation of learning technologies? They argue that many projects are expected to demonstrate impact, but outline the complexities of impact in terms of student learning, changes in academic practice,  organisational and national change.

The two papers dealing with social-cultural issues focus on gender and culture in learning environments. Campbell describes a survey on attitudes towards learning technologies and their uses. She argues that there can be increased intimacy and democratization of online conversations. She highlights six key issues – necessity, coercion, relocating authority, ‘taken for granted’ assumptions, safety, and process as product, and concludes with a series of strategies for enhancing voice and power in CMC. Frank and Toland report on a study of the effect of culture in online communication. They focused on how students from different cultural backgrounds use email to communication with students and lecturers and their study shows that there are significant differences.

Several of the papers focus on the use of technologies to promote communication and collaboration through the use of email or computer-mediated communication (CMC) tools. Andrews and Schwatz have undertaken a detailed analysis of the use of cmc. In particular they explore the development of learning behaviours in VLEs and list three categories – social, operational and content. They align their work with Salmon’s five-step model of CMC. They identify a number of barriers to effective communication, linking levels of participation with student enjoyment, and conclude that different kinds of learning behaviour develops in different kinds of online learning environments. Conole et al report on an evaluation of an online course for medical practitioners, which again relates to Salmon’s five-step model. It concludes that the students benefited from a well structured and facilitated learning environment. The most important element of the course is cited as being the opportunity the course afforded for the sharing of expertise with colleagues across Europe. The section concludes with a paper by Creese and Kemelfield which explores the contrast in learning through a collaborative evaluation by practitioners and students. They also highlight the importance of social interaction in online environments and the dangers that badly constructed and facilitated environments may lead to a surface approach to learning. They conclude that online content cannot be developed in isolation and that the process of interaction that leads to learning must be considered. Repurposing commercial content does not necessarily save time.

The four papers in the Virtual Learning Environments (VLEs) section explore different aspects of the use of these systems to support learning. Deepwell reports on an institutional implementation of a VLE at Coventry. This is a timely paper, given the current high priority of these types of developments in most institutions in Higher Education.  The usability of Web-based learning tools is discussed in a paper by Storey, Phillips, Maczewski and Wang. They provide a comparison between two currently popular VLEs and discuss their usability from both staff and student perspectives. Thomas and Paine describe a study of students’ use of practical activities at a distance. They report significant differences between designers’ perceptions of student work and how students actually behave.

The final section covers multimedia and adaptive interactive environments. Handzic and Tolhurst report on the evaluation of an interactive learning environment in management education. They discuss the impact of interaction on knowledge and performance and their evaluation shows that interactions have a positive effect on individual learning.  Hanna reports on a study of the use of  assessments in a web-assisted pharmacology course and in particular described the development and evaluation of a problem-based learning environment used in pharmacology. An adaptive web-based engine for the evaluation of eLearning resources is outlined in a paper by Stergioulas et al.  It includes discussion of the evaluation of the design and development of adaptive data collection and presentational engines for evaluating elearning resources. The effectiveness of multimedia instructional modules is reported in a paper by Jensen et al. In particular they focus on the use of visualization modules to teach concepts. They identify the importance of additional contextual factors, teacher attitudes, and their impact on satisfaction and results. Finally, Moule describes a subject-based study which evaluated the use of a Basic Life Support CD-ROM. She reviewed its effectiveness as a learning tool and analyses user experiences. She identifies four themes of importance: student-centered resources, context of learning, skills development and technological support.


Copyright message

Copyright by the International Forum of Educational Technology & Society (IFETS). The authors and the forum jointly retain the copyright of the articles. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear the full citation on the first page. Copyrights for components of this work owned by others than IFETS must be honoured. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from the authors of the articles you wish to copy or