Steps toward computer-supported collaborative learning for large classes
Internet-based educational technologies have raised expectations that the quality of education and its delivery can be improved while costs are contained (Turoff, 1999). The past few years have been characterised by experimentation and exploration of opportunities, problems and solutions. Learning from this period, educators are now beginning to identify common factors associated with success. These include: pedagogical issues, such as the need to re-design courses for different modes of delivery (Duchastel, 1996-1997; Trentin, 2000); resourcing issues, including class size, student and teacher preparation for new modes of teaching and learning and for use of the associated technologies, provision of support services for teachers and students, and provision and maintenance of the technology; and institutional issues such as match between educational and technology strategy, methods for evaluation and extension of initiatives, and provision of incentives for teachers to adopt new methods (Fallows & Steven, 1999; Klobas & Renzi, 2000; Laurillard, 1993; Schrum, 2000).
These observations have been based largely on the accumulated results of studies of courses conducted at a distance or, in the case of support for on-campus education, of small studies conducted by individual innovators. We are now at a point where educators recognise the wider potential of the Internet and related technologies to improve the quality of on-campus education (Turoff, 1999). Computer-supported collaborative learning, CSCL (Koschmann, 1996) is of particular interest in university systems which have traditionally been based only on large classroom instruction. While, in the United States, large classes may be supplemented by smaller sections or tutorials, students in other countries often do not have the benefit of organized out-of-classroom activities. In these systems, the availability of CSCL provides an unprecedented opportunity to improve the quality of education and student experience.
CSCL provides opportunities for improved education, but also poses many practical questions. For example, the amount of interaction between students and teachers can increase dramatically, placing increased demands of time and technology access on all (Harasim, 1990). The differences in scale between a single course with small student numbers and a large classroom may be substantial. This article therefore focuses on the role of computer-supported collaborative learning (CSCL) in enhancing teaching and learning in on-campus courses that are taught in large classrooms (60 or more students). Pilot projects in CSCL at Bocconi University in Milan, Italy, during 1998/1999 were used to gauge the impact of such activities on teachers, students, and administrators in a university system which has traditionally been based on large classroom teaching and examination.
The educational environment
A short description of the Italian university system will help place the initiatives described in this article in context. Most Italian students begin university at 19 years old. They may select a 3 year course, leading to a Diploma Universitario or a 4-6 Corso di Laurea which requires a substantial research-based thesis and is equal to or more advanced than a research Masters in English-language systems. Italian universities emphasise transmission of academic knowledge rather than acquisition of skills and methods (MURST, 1999).
Individual teachers in several Italian universities have recently introduced initiatives in computer-supported learning, and interest is growing (CTU, 2000). Yet, there has been little systematic attention to the implications for teachers, students, and administrators. This situation is mirrored in many other countries.
The initiatives discussed in this article were taken by lecturers at Bocconi University in Milan, Italy during the 1998/1999 academic year. Bocconi is a private university of 12,000 students, offering Laureate courses in economics and commerce. In common with other Italian universities, most courses are taken in large classrooms without tutorials or other small group teaching.
Bocconi University encourages and supports its teachers to use new methods to improve the quality of teaching and learning. During 1997 and 1998, several teachers were taking steps toward incorporating experiential and small group learning activities into their courses (e.g., Bielli et al., 1999). They were supported by a sound and reliable information technology and communications infrastructure which provided email and WWW access for all staff and students, PC training for first year students, and access to trained, experienced and motivated educational computing support staff.
This article will focus on the computer-supported learning (CSCL) activities introduced during 1998/1999 into "ELAUT" (Elaborazione Automatica dei Dati per le Decisioni Economiche e Finanziarie, Computing for Data Analysis and Decision Making in Economics & Finance) an advanced unit in computing and statistics. ELAUT, which students complete during the first six months of their third or fourth academic year, consists of two parts: the first part addresses Internet technologies and their applications, and the second, statistical computing. In this article, we are concerned with the first part. The course in Internet technologies and their applications has evolved as the Internet has evolved. The educational goals have, however, remained the same: for students to learn to use the Internet effectively for communication and information gathering in their day-to-day life and in business.
ELAUT is an elective course, and the students who take it are highly motivated to learn about the Internet and its application in business. During the first part of the course, the history of the Internet, the underlying technology, and the organization, growth and evolution are treated as basic knowledge before students focus on four specialized themes. The themes address: tools and services, research for information, CMC (Computer Mediated Communication) basics, and web site development basics. The theme of CMC was introduced to the course for the first time in 1998/1999 with the collaboration of a visiting professor from Australia who continued to collaborate at a distance once she returned to her university.
Until 1998/1999, the course was taught as most courses in Italian Universities: with lectures and, to teach the use of Internet tools and the programming language, hands-on work in large classrooms equipped with PCs. (ELAUT was taught in a classroom with 100 PCs.) The teachers of ELAUT recognized, however, that additional learning activities could provide experiences through which students would gain a deeper understanding of the Internet and its current and future role. To provide opportunities for such experiential learning, the 1998/1999 version of ELAUT was enriched with an additional educational activity to be performed outside the planned lessons and on a voluntary basis.
Students could choose to work on a project chosen from within the four main themes of the course. The students responded enthusiastically and 95% of them asked to participate. Twenty students, working collaboratively in 4 groups of 5, were assigned to each of the four themes. They were set a theme-related task to complete within two months. A tutor was assigned to monitor the progress of the groups working within each theme.
The goals of incorporating CSCL in the course
The projects provided an opportunity to test the potential to introduce computer-supported collaborative learning (CSCL) into a course traditionally taught by lecture in a large classroom. In introducing a CSCL project, we had three sets of goals:
The CSCL projects
The groups working on the CMC theme were provided with a computer-supported collaborative work (CSCW) tool, "BSCW" (GMD, 1995-2000) to support their work. The students were therefore expected to learn about CMC through experience with software which supports group work. The course co-ordinator was tutor for this group, with support from the visiting professor.
Figure 1 shows the high level layout of the students' BSCW work space. The group project areas in BSCW were open (with student knowledge) to the tutor, who monitored the electronic discussions to ensure projects were on schedule and to identify signs of difficulty with the project or the technology.
Figure 1. ELAUT work space in BSCW: top level view
The project was based on a real task. Each group of students was asked to pilot test a proposed WWW site evaluation form by using it to evaluate 20 sites representing hotels in an assigned geographical area of Italy. They were also required to give a group presentation to class, in which they analysed their learning about CMC. To complete the project, students had to: plan how to complete the task, produce a list of hotels in the assigned area, assign hotels for evaluation to individual group members, submit the evaluations by the due date, and prepare and present a group presentation to class in the presence of the tutors for all themes. (A copy of the evaluation instrument, in Italian, is available from the authors. The instrument is part of work in progress described in Klobas et al., 2000). In addition to learning about CMC, therefore, participants were expected to learn about searching the Internet, WWW site evaluation and presentation.
Nine class meetings were scheduled during the project period. The first two meetings were dedicated to project goals and a BSCW workshop respectively. Attendance at the remaining meetings was voluntary (provided each group was represented by at least one student), but more than half the students (11, 55%) attended at least 7 of the 9 meetings.
There were no set topics for the voluntary meetings. Instead, these meetings dealt with issues raised by students in response to their experiences or by the tutors in response to observations about project progress. Activities included: checking work at the agreed deadlines, readjusting subtask execution, and resolving problems encountered using BSCW.
The students and teachers also used BSCW and email to raise questions and resolve issues that arose, and to celebrate achievements. Questions about use of the BSCW software were most frequently resolved by posting a response within a BSCW space available to all students.
Student response to the CSCL project was gauged using a variety of formal and informal methods which complemented one another to provide a consistent and reliable picture of students' activities and course outcomes. These methods included monitoring of progress during the project through review of electronic discussions and participation in progress meetings; a pre- and post-project survey of motivation, technology expectations and use, and learning outcomes; and review of students' experiences and learning as reported in a group reflection on the project. The survey questions (translated from Italian to English) are in Appendix A.
From the student surveys, we learnt that those who participated in the CSCL project had taken the ELAUT course because they wanted to improve their Internet or computing knowledge and skills. Several mentioned that they were attracted by the applied nature of the course. They volunteered for the CSCL project for three main reasons: interest in the field of tourism (8, 40%), interest in learning more about various aspects of the Internet including WWW searching (5, 20%), or because they found the project or the teaching team generally attractive (8, 40%). The students were aware that they were participating in an educational experiment.
Use of CSCL affected students, teachers, and administrators. In this section, we will describe the projects from the points of view of each group. The next section will highlight lessons learnt.
Student perspective on outcomes
We gauged students' response to the course in terms of their statements in electronic discussions, their use of the technology, their learning outcomes, and their own reflections on their experiences and learning. Each of these is discussed below.
The combination of monitoring messages on BSCW and resolving them through a general posting to BSCW or discussion in a project meeting proved very successful. All projects were completed by the agreed deadlines, and all potential problems were identified and resolved quickly and to teacher and student satisfaction. The students learnt from discussing their experiences face-to-face with students in other groups who had found other ways to solve apparent problems. Non-attending students learnt vicariously from reports of meeting outcomes or post-meeting solutions posted to discussion areas by fellow group members or the tutor.
Use of CSCW technology
The survey included three questions about use of the CSCW software during the project. The questions were asked twice, in slightly different forms: following completion of the ELAUT coursework but before the first project meeting, and after delivery of the project report. This method let us compare students' expectations about the technology with their actual experience. They were asked an open-ended question about the functions for which they expected to (and subsequently, did) use the CSCW software most often, how much time they expected to (and did) take to learn to use it, and how often they expected to (and did) use it.
The course co-ordinator classified responses to the open-ended question by CSCW function: communication, collaboration by sharing jointly authored files and other task-specific data, or co-ordination of task allocations and schedules. Their expected and actual use are summarised in Figure 2.
In stating their expectations, the students emphasised use for communication (14, 70%) and collaboration (9, 45%). They actually used the CSCW software more for collaboration (13, 65%), and in particular sharing of documents (11, 55%), than for general communication (9, 45%). Several respondents noted that they found email quicker and easier than the project-specific CSCW environment for communication that did not involve shared documents or URLs.
Learning outcomes were measured through the survey, from project tutor and independent assessment of the project presentations and reports, and from students' own reports of their learning. The survey and assessment results are discussed here, and the students' reports in a separate section below.
The survey included three questions about learning outcomes. Students were asked, both before and after project completion, to answer a question about requirements for software to support group work, and to rate their knowledge of PCs and the Internet. A comparison of before and after responses to the first question enabled us to identify ways in which project participation had increased students' knowledge, or changed their thinking, about groupware. A comparison of before and after ratings of knowledge showed how students' confidence about their knowledge had changed through participation.
Prior to commencing the project, students' knowledge of computer-supported collaborative work reflected their learning from the ELAUT class on CMC. They emphasised the need for software to support communication (18, 90%) and collaboration (11, 55%), but not co-ordination (1, 5%). The students also noted the need for integration with the Internet (6, 12%) and with office management software (4, 20%) and for the software to have the ability to handle graphics and data (4, 20%).
The students' project experience led to increased recognition of the need for CSCW software to support all aspects of collaboration (20, 100%), from exchange of documents (10, 50%) to socialization among group members (3, 15%). Understanding of necessary features for successful CSCW, particularly for appropriate security and access to controls also increased (from 3, 15% to 8, 40%). These differences are illustrated in Figure 3. The tutors considered these to be important learning outcomes, achieved through experiential learning.
Prior to beginning work on the project, half of the students described themselves as novice PC users (10, 50%), and only one (5%) as an expert. Following completion of the project, no students described themselves as novices, but still only one described himself as an expert. This is a good outcome. While novice users gained confidence, no student over-estimated their skill as the result of participation in the project.
All students who participated in this project were awarded the maximum number of marks available for project work. They demonstrated, through the final project presentations, that they had learnt a great deal about the Internet, WWW site searching and evaluation, and CSCW.
Student reflections on their experiences and learning
The students were asked, as part of their final group project report, to present a brief review of their experiences and learning under headings provided by the course co-ordinator. Students were asked, among other things, to draw conclusions about what they had learnt from their experiences in the course. Common elements in their reviews addressed difficulties in co-ordinating information and documents and problems with the WWW site evaluation task.
The need to address co-ordination problems resulted in much of the learning observed by students. They learnt, through trial and error, the value of sharing project data, links, files and documents through a central repository. They rapidly developed, and put into place, a preference for face-to-face meetings to deal with complex and sensitive issues, particularly those to do with task allocation and co-ordination. Although they appreciated the any time-any place characteristics of the Internet-enabled CSCW software, they only used it for sharing data or when they could not meet face-to-face because of holidays or travel commitments. Several theorists suggest media selection reflects the required richness of the communication (Daft & Lengel, 1990), but this pattern appeared to reflect a deeper understanding of the fit, not only between task and technology (Heeren & Lewis, 1997) but also with the context within which the various activities required to complete the task performed.
Overall, the students noted that, although the project took a lot of time and effort, it was worth it. They learnt a lot.
Validity of student responses
We have already mentioned that the range of methods of data collection, and the number of tutors involved, provided cross-checks for consistency (reliability). Given that no method of data collection was anonymous, could we be sure, however, that students were being honest with us? The candour of comments made in electronic discussions and in face-to-face meetings gave us some confidence that students had entered into the spirit of innovation in this course, and were being honest with us. We had further confirmation when we read their final reviews.
Teacher perspective on outcomes
The teachers were highly motivated and experienced in the technical aspects of the Internet and coped comfortably with all the experiential projects set for the ELAUT course. They had worked collaboratively themselves, all using email and some using (experimentally) BSCW, so were familiar with collaborative work and use of the Internet for this purpose. Nonetheless, only the visiting professor (extensively) and the course co-ordinator (once) had prior experience with CSCL, and the teachers were somewhat apprehensive, albeit enthusiastic.
Throughout the course, the course co-ordinator and the off-campus visiting professor exchanged reflective email, as well as email about practical aspects of the course to gauge teachers' response to the course. They also established a workplace within BSCW to record longer reflections and short comments by topic. The volume of text generated in this process suggested that they had learnt a great deal!
Several months after course completion, the teachers returned to these sources of reflection and analysed the contents by topics that had emerged both during the course and in subsequent discussion. Their analysis focused on 4 dimensions of computer-based learning: an educational dimension, a task dimension, an administration dimensions and the technology dimension. Their learning about each of these dimensions is discussed in detail elsewhere (Klobas & Renzi, 2000). Some key observations are discussed here.
From an educational point of view, two aspects of this course were new to the Italian teachers: the experiential learning and the collaborative (group) learning. In this regard, they faced one particular unexpected problem: How to assess group work. Because group work is rare in Italy, mechanisms for providing fair recognition of each student's contribution had not been developed. The availability of a visiting professor from a business school which used collaborative learning projects extensively helped to solve this problem.
The teachers had hoped to extend collaborative learning beyond learning within each group to learning among groups. The group meetings, and subsequent communications among students and from course co-ordinator to student, enabled successful collaborative learning across groups about how to use the software and co-ordinate work. They were less successful in sharing knowledge about working with CSCW. The final group presentation session did, however, successfully promote cross-group learning about working with CSCW. There was, for example, considerable discussion of the need for group leaders, with some groups arguing that a group must have a co-ordinator with a complete view of the project and others arguing that groups work better when leadership varies throughout the project according to task. Final written group reports included well-considered observations about different ways to structure computer-supported work groups, and different techniques for ensuring groups meet their goals; these observations had clearly been influenced by the presentation session discussions.
From the point of view of the task, the teachers found participation in this project exciting and they were very enthusiastic about how much they had learnt. On the other hand, they recognized that they had put in much longer hours than usual monitoring student contributions to the BSCW discussion databases and answering student queries on-line. It was obvious that it would not be possible to expect such an effort from teachers who were less enthusiastic about the field and computer-supported teaching and learning, nor would it be possible for the ELAUT tutors to put in such an effort in future years. From the more experienced teacher, tutors new to the on-line environment learnt some tips, including:
We have mentioned that the face-to-face meetings in class provided students with a mechanism for solving complex issues that were difficult to describe, or sensitive to discuss on-line. The teachers also enjoyed these meetings and appreciated the sense of connection with the students and their enthusiasm for their work.
It became clear, through discussion with the students, that the WWW site evaluation task was more time consuming to complete than the teachers had anticipated. The teachers were reminded of the need to make the substantive task very simple when students are learning how to work in a CSCL environment.
Overall, the teachers enjoyed the experience, but were tired at the end. They felt they had learnt a great deal about how to incorporate CSCL in their own courses, and how to develop learning experiences that were interesting and valuable for students. Although using technology with which they were familiar, they learnt a great deal about the need to combine task design with technology design to ensure projects are manageable. They were so satisfied with the experience that they begin to design the next (1999/2000) session of the course very soon after completing the 1998/1999 session.
Administrator perspective on outcomes
This project raised several important issues for university organization and infrastructure, in particular:
Overview of outcomes
The CSCL projects were a success. The students learnt from them, and enjoyed the experience. The teachers also learnt much and enjoyed the experience. The technology and room scheduling were within the capabilities of the university's administrative system, and did not create additional demands that could not be met. At this point, though, we should comment that room scheduling and technology support at Bocconi University is more flexible than we have experienced in Australian and English universities. Teaching and administrative staff, and students, all demonstrated flexibility in setting dates and times of meetings, and teachers were happy to let students choose whether to come to meetings or to learn from others' reports of those meetings. The university's attitude that learning is the student's responsibility, while the teachers and administrators are responsible for providing an environment for rich learning experiences, provided important cultural conditions for success.
The lessons that can be drawn from these outcomes include:
Although these lessons can be drawn from the project described in this article, we are also aware of how much more effort is required to move from a small course-specific initiative to a wider degree- or university-wide initiative.
Planning for a wider implementation for large classes, we recommend further investigations into the effects of differences in educational models across disciplines and levels of study. Others have already noted the probably value of providing different learning options for students with different attitudes and learning styles (Singer et al., 1996). We believe it is possible to build common profiles of differences in educational and technology design for differences in educational model and level of experience of teachers, students and support staff, and to use these profiles for planning to expand computer-supported education throughout a course, unit, or university. The time needed for diffusion of initiatives is likely to vary between profiles. Finally, we recommend further investigation into the impact of the pedagogical background (training and experience) of teachers involved, along each of these dimensions. As other authors have noted (Aggarwal & Bento, 2000), to move successfully into a computer-supported teaching and learning environment, university teachers may have to learn more about how to teach than about how to use technology.
CSCL project survey
1. What functions should be available in software to support group work?
2. Among these functions, which do you expect to use/did you use to support your group's work for this project?
For this project, you will use/have used software designed to support group work [BSCW].
3. How often do you expect to use/did you use this groupware?
4. How much time do you think you will need/did you need to learn to use this software?
5. How would you describe your knowledge of PCs: beginner, moderate, expert.
6. How would you describe your knowledge of the Internet: beginner, moderate, expert.
7. Why did you select this course (ELAUT)?
8. Why did you select this project?