Using BSCW in Learning & Teaching
Pat Jefferies and Ian Constable
It is generally acknowledged that over the past number of years there have been significant developments in education. For example, as noted by Vosniadou "recent approaches to learning emphasise the active, constructive nature of the knowledge acquisition process wherein the learner is not a passive recipient of information but an active and constructive interpreter of meanings" (Vosniadou, 1994). However, whilst the use of educational technology has, for a number of years, held out a great deal of promise for changing the learning environment in line with the approach outlined above, little evidence has so far been available to suggest that this has, as yet, met expectations.
Nevertheless, efforts continue to be made to utilise the technology successfully and, within the bounds of computer assisted learning, at least two main approaches can be identified. The first approach is in developing tutoring systems which are intended to replace the 'traditional' human teacher and the second is in developing open learning environments wherein the student may take control and determine his or her own learning pathways and goals.
To a certain extent it is with this latter intention that collaborative working tools are utilised within an educational environment. As Kanselaar & Erkens note, "the goal of many of these computer-based learning environments is typical for encouraging learning without specifying it directly (Levin & Waugh, 1988)." (Kanselaar & Erkens, 1994). However, as Kanselaar & Erkens further note, "the effectiveness of this approach depends on the validity of a specific learning principle, that of (eventually guided) discovery learning". (Kanselaar & Erkens, 1994) (Whether or not this particular principle is, in fact, valid will not be under discussion within this paper). However, it is worth noting that the advantages of cooperative learning in comparison to that undertaken purely on an individual basis have been emphasized by a number of recent cognitive-constructivistic models produced through research, (eg Bransford et al 1991, Collins, Brown and Newman, 1989). As noted by Mandl, Gruber & Renkl "the main goal of these models is to avoid decontextualized presentation of information that often leads to inert knowledge". (Mandl, Gruber & Renkl, 1994). Thus they quote the framework of Cognitive Apprenticehsip (Collins et al, 1989), wherein "the learner is encultured by authentic (learning) activities and social interaction" which then points to the importance of two kinds of collaboration:
Thus it may be perceived that the underlying philosophy for implementing collaborative group working supported by technology is based on a variety of beliefs and concepts. For example:
It can also be seen that each of these may, in turn, be based upon the belief that such objectives are desirable.
Whilst such beliefs may or may not be fully justified or proven, there are, however, a number of issues which need to be considered in utilising technology-based group working tools as opposed to facilitating group-working per se. Some of these might be based on technological, social, cultural, as well as pedagogical concerns and it is with the purpose of investigating some of these issues that the current experiment was undertaken.
BSCW was introduced as a shared resource tool for a module entitled "Innovative Trends in Information Systems". This module, undertaken by final year undergraduates on computing courses, was designed to provide students with the opportunity both to demonstrate their research skills and to follow up areas of specific personal interest in the areas of computer science and information systems.
The module had been run in previous years using either BSCW or FirstClass to facilitate the three types of interaction that, according to Moore & Kearsley might take place in any distance learning context - "learner-content interaction, learner-instructor interaction, and learner-learner interaction" (Moore & Kearsley, 1996).. The decision to use BSCW in the year in question was taken largely for pragmatic reasons. The FirstClass server is situated on another campus, the number of licences is limited, and ultimate control resides there. This would have meant that the creation of user accounts, etc., was carried out through a third person on another site, although local administration privileges would have been granted for the purposes of password resetting, access control, etc.. The BSCW software, on the other hand, is free, was installed locally with relative ease, and gives prompter responses and complete local control. In the event, students at the other campus studying the same module were also invited to participate in the use of the shared resource. The member of staff at the other campus did not, despite discussions before the start of the module, have the same sense of ownership of the medium as the instigating tutor on the base campus, and, as a possible consequence of this, usage by students on the distant campus was relatively low. (Figure 1).
The actual "taught" content of the module was a series of guest lectures delivered by colleagues, each choosing a topic based on their own particular interests or current research. These ranged from postmodernism in Information Systems to current research in computer graphics, from proposed developments in web mark-up languages to discussions on the Java virtual machine. Thus, the objective was not to teach content for future assessment, but to stimulate the students' minds and get them thinking about leading-edge research topics. The teaching and learning objectives for the module were not, however, to require original research from the undergraduate students. Rather, it was to allow them to demonstrate that they were capable of locating material relevant to their topic of interest, to read it intelligently, to absorb it and to reproduce a summary of the findings in a coherent, concise and readable manner. Specifically, the assignment rubrics stated the following:
You may assume your readers know little or nothing about the topic but are intelligent. After reading your assignment they should (still) be interested in the subject, have sufficient information to discuss it with those in the given field, and be clear about where you stand on the matter.
You may assume your readers know almost as much as you do about the general area under discussion. After reading your assignment, they will have gained by your synthesis of what is known, your analysis of what is happening, and your prognosis of what may be to come.
The need to use a shared (BSCW) resource was in effect dictated by the nature of the module. In the first place the central theme of the module was "innovative trends", and with the growing use of GroupWare products in business situations, the student would thus be presented with some approximation to what they might be required to use after graduation. However, previous experience in using various text-based "chat" or conference tools had shown that undergraduate students find the necessary suspension of disbelief difficult to maintain when they are, for the most part, seated next to each other in the same laboratory. For this reason, there was no attempt in this implementation of the module to use the various facilities BSCW offers to run real-time discussion fora.
The other main characteristic of the module, however, was the individual nature of the entire experience for the student; that is, the topics chosen for their assignments were individual ones, and the group activities (lectures) were for intellectual stimulation rather than content provision. Any attempt at the normal tutorial or seminar activity, where the objective is to concentrate on a specific topic, would therefore be less than effective as a feedback or discussion forum with staff. The students were encouraged to use, in the place of tutorials, a combination of BSCW and email. The BSCW tool was intended for asynchronous discussion purposes with other students; for example, by postings and answers to postings, (rather than in real time), for publishing their comments or reactions to documents or messages posted by others, and above all as a resource base where material found, which might not be of direct interest to the particular student, but which that student deemed might be interesting to his or her peers, could be placed. The use of email was encouraged to allow the students to communicate directly with the tutor about their individual problems or progress. In the event this proved to be at least as time-consuming for the staff concerned as the delivery of a "traditional" taught module. The tutor has the ability to respond immediately to requests for feedback on problems students face, but this can lead to an expectation on the part of the student of instant feedback, whether at a moment of crisis or not. The tutor also can be led to believe, almost certainly wrongly, that such instant feedback is necessary, as a mark of the caring professional, and necessarily beneficial. In many cases, the tutor used individual pleas for help or advice as a spur to provide timely and relevant information or help to all the students, by means of a posting on the BSCW site (with an email referring to the posting), or by an email alone. Emails were either sent via the normal University email client, where a group distribution facility was set up, or via the BSCW facility of selecting all users and emailing them. The latter was, in the end, considered less efficient, since it would lead to a split reference database of messages - those sent by the normal route and those sent by the browser interface when using BSCW.
What was on BSCW?
At the start of the module, the content consisted of some 300 documents, collected during previous years, in the general topic areas of CSCW/GroupWare, Teleworking, E-Commerce and Technology. These were, for the most part, either text documents prepared by the tutor, or localised URLs - i.e. web pages downloaded and set up on the BSCW server; the latest versions of BSCW offer this excellent facility, which avoids the bandwidth problem of many students accessing the same remote sites for large documents at approximately the same time. In addition, the assignments of previous students were also posted. This was in line with the ethos of the course, which was one of the synergy to be produced by the massed individual learning experiences - all the students, and the tutor, were to benefit mutually from what was learned by each individual. In certain cases, the feedback given during the assessment of previous students' work was posted in the form of an annotated version of the assignment; in these cases the assignment was rendered anonymous, of course.
There was a small proportion of teaching material, in the sense that one preliminary activity which all the students were invited to attempt was posted. This consisted of an invitation to study two documents and judge which was the better of the two in terms of the assessment criteria for the module that had been explained to them. The tutor was distressed to note how many of the students chose the "wrong" article as the better one, specifically the article (found in a peer-reviewed journal) written in excruciatingly poor and unproofread English and which used a great deal of pseudo-technical jargon to hide the fact that what was being said was little more than common sense, and, in certain parts, less than that.
This sense of gloom was further reinforced by the frequency with which the tutor was asked by students to differentiate between the "good" and the "bad" assignments of previous students. These requests were, of course, refused, though the students would be pointed towards methods of applying critical judgement of their own. One of the questions which deserves further consideration is whether the way in which the BSCW-based material for this module was selected was the most effective. The original (tutor-selected or written) material had the quality of being relevant, but the remainder echoed the general nature of material on the WWW as a whole, i.e. posted by interested parties, often uncritically, and with no moderation. One of the pleasures of tutoring this module was to see the enthusiasm of a certain (few) students in posting quantities of material relevant to their own interest, often stimulating responses from their fellow students. Moderation for quality, whether by the tutor or by peer-group committee, was considered but rejected as likely to dampen that enthusiasm. Perhaps in a module where the subject matter was more tightly constrained (e.g. "you will all research an aspect of X") it might be more acceptable to have moderation of posted material, in that students would see the advantage to themselves of high-quality focused material.
By the end of the module, the corpus had grown to some 440 items created by the tutor, and another 350 created by the students. These figures are necessarily approximations, and on average slight over-estimates, since the act of uploading a document to the BSCW server sometimes involves the user subsequently deleting it and recreating it - one only tends to spot errors after the event.
How BSCW was actually used
BSCW maintains an extremely effective historical record of every event which takes place. A reporting facility allows the administrator to schedule a regular report of exactly who has done what during the preceding period, and from this statistics can be derived on specific usage activities (read, create, delete, etc) by given users, broken down by date or time, etc.
Unfortunately, the scheduling in this particular case was inadvertently set up for weekdays only, and it is therefore impossible to confirm the amount of access to BSCW carried out by students from home at weekends via their own ISP connections. However, the impression gained from feedback from the students is that this was relatively low, due mainly to the costs involved. By contrast, it was clear that the majority of week-day access was made during those days when the students were necessarily most likely to be on campus because of their timetable requirements. Specifically, in this case, this was Mondays and Tuesdays, although significant access was also made at times when attendance was not required, such as on the day set aside for work on their final year project, Thursday.
Statistics for the usage by hour of the day also confirm, as one would expect, that the majority of accesses were made during the hours when the campus networked computers were available, although some students did, clearly, take advantage of the ability to access the system from off-campus, as is proved by the number of accesses between the hours of 9 pm and 8 am (4.3%).
Figure 4 demonstrates the distribution of read accesses throughout the course of the module. It will be noted that there was an apparent initial flood of enthusiasm in the first few days, though the BSCW statistics do not, of course, reveal the nature of the read access. That is to say, a read access event may be the record of a student clicking on a document and immediately closing it; it may be the record of an extended reading session. The tutor did in fact at one point suggest to students who were apparently making a large number of consecutive read accesses at intervals of a few seconds that they could not even skim-read at such a pace and that, while they were clearly taking to heart the suggestion in a briefing session that individual traffic on the BSCW site could be monitored and might, just might, be taken as a guide to their enthusiastic and constructive participation in the module, they might be unaware that the tutor/administrator could detect the speed with which they moved on from one item to the next. This was countered by righteous indignation and the explanation that the students concerned were actually accessing and printing out the documents for perusal at their leisure. This is in line, of course, with the commonly-noted preference for reading longer texts from paper rather than a screen. It will come as no surprise to note that the normal pattern of access, weighted towards the first days of the week, was reversed in the week where the Friday represented both the deadline for the first assignment and the deadline for confirmation of the title for the second.
However, further analysis of the BSCW reports revealed a strong correlation coefficient (0.73) between the number of Create and and the number of Read activities. The main reader (374 Read activities, 10% of the total) was also the main poster (53 Create activities, 19% of the total). The correlation would have been higher but for the classic case of a "lurker", one student who was the second highest "Reader" (7.3%) but created only once (0.4%). There was little significant correlation between the number of read activities and the eventual grade awarded for the module (0.22%); perhaps surprisingly, there was no significant correlation between the number of create events and the final grade (0.08%)
At the end of the experiment students were asked to complete a feedback questionnaire on their perceptions of using BSCW.
With regard to the actual usage that the students felt they made of BSCW (Figure 5), it is interesting to note that they reported that this was, in the main, evenly divided between using it as a resource for their own personal research and for 'lurking' rather than for actually contributing. This does not, however, accord with the actual statistics discussed earlier which demonstrated a very strong correlation co-efficient for 'Reads' and 'Creates'.
However, whilst all of the necessary "learner-content, learner-instructor and learner-learner" interactions might be facilitated within a computer conferencing environment it is the control of such interaction that is particularly important whatever the medium. Thus, as Ragan notes, "there is a need for frequent and meaningful interactions among the learners, with the instructional materials, and between the learner and the instructor", (Ragan, 1998). (How this might be facilitated and structured is discussed later).
As regards the advantages for gaining information through the use of BSCW, the students generally felt that it was the variety and range of resources available that were of most benefit to them and that these facilitated learning, (Figure 6). This does accord with other findings because, as Jensen notes, "research has documented, over and over, when participants make the learning their own, when they get to talk about it their way, without being manipulated and controlled, learning increases", (Jensen, 1996).
Again, in accord with previous findings with regard to use of the Internet for teaching and learning, (Jefferies & Hussain, 1998), students found that using BSCW raised issues of data overload and quality of data. Perhaps not surprisingly, however, unlike an evaluation of the Internet as a whole, the issues of slow speed and data security did not seem to feature highly as disadvantages. (Figure 7).
Nevertheless, as noted earlier, issues of quality were also of prime concern to the tutor and these findings seem to add further support for the need to control and moderate the resource as it is developed. Previous research does seem, in fact, to confirm this in finding that, much like face to face situations, "well-moderated student interactions structured by frameworks that ask good questions and allow for the establishment of certain ground rules create perhaps the most productive of online communities,” (Brown & Johnson-Shull, 2000). Beaudin also notes "the online instructor is key to organizing interaction and Hiltz suggests from her research that having a responsive moderator is key" but suggests "the instructor does not necessarily need to be the moderator and Driscoll suggests that participants can be assigned the task". (Beaudin, 1999). Obviously the tutor would have to weigh up the advantages and disadvantages or risks involved in nominating a participant as moderator.
As regards factors that encouraged the students to use BSCW, these seemed to stem largely from personal satisfaction at using the tool with tutor motivation and peer group involvement also playing a role in this. As White notes, it is the interpersonal environment where "there is a dialogue and interaction taking place in the online medium that really transform people and allow their personalities to come across the medium", (Young, 2000). The fact that it was a closed discussion environment did not appear to be too much of a factor in this particular experiment and previous experience of trying to encourage the use of external discussion groups with a different group of students had actually proven unsuccessful. Thus, as might be expected, students were more at ease using a 'closed' discussion environment for academic exercises than opening up their views to the whole world. (Figure 8). Again research has found that "one of the key factors in order to keep [the environment] interpersonal is limiting the number of students that are able to join it", (Young, 2000). It has also been reported that "if students feel they are part of a community of learners, they are more apt to be motivated to seek solutions to their problems and to succeed", (Ragan, 1998)
In contrast, factors that discouraged them from using BSCW were expressed as being a lack of experience using this particular type of medium and also a lack of confidence in making their opinions known to peers. (Figure 9) This latter would also support the assumption that opening up the discussion to a much wider audience may well further inhibit contribution as well as reinforcing the notion that "structural design in an online environment requires even greater sensitivity and attention than in the traditional classroom", (Brown & Johnson-Shull, 2000). Thus, "mindfully designed questions and guidelines must create the parameters of communities previously circumscribed by walls, teacher posture, and the physical proximity of peers", (Brown & Johnson-Shull, 2000).
Overall the conclusions were that using BSCW for the Innovative Trends in Information Systems module was probably an inappropriate choice and that choosing a more focused module would have probably encouraged the setting up of a much less anarchic resource which could have been more easily moderated for quality. This latter factor was also felt responsible for not allowing the scope for control or guidance that was required in using BSCW with undergraduates who seem to have less responsibility and less critical faculty than that which is required in using this particular tool for collaborative working for academic purposes. It was felt, therefore, that whilst BSCW might work very well with more mature, post graduate students, using it with undergraduate students required a great deal more focus and control on the part of the supervisor in order to optimise its value. It is felt, therefore, as Beaudin reports "that this exploratory study reinforced many of the principles and practices used in face-to-face classrooms to keep discussion on topic and should serve as a reminder that good instructional design is essential whether it is online or face-to-face", (Beaudin, 1999). Also, that the "structural design in the online environment requires even greater sensitivity and attention than in the traditional classroom" (Beaudin, 1999)
Nevertheless, use of BSCW did stimulate and facilitate a greater degree of collaborative working than may otherwise have been feasible and students reportedly got a good deal of personal satisfaction from the experience of using it. Thus, it may be concluded that valuable lessons have been learned from having conducted this initial research and that these will now lead to a much deeper consideration of developing strategies aimed at facilitating the greater degree of critical faculty, which may, in turn, lead to deeper conceptual insights, which are seemingly essential for effective implementation of collaborative tools such as BSCW as well as for higher education generally.