An evaluation of an online course for medical practitioners

Introduction

The growth in the use of the Internet in the past five years has been paralleled by a growth in the number of online distance learning courses. The perceived benefits of this form of learning include the opportunity to learn anytime, anywhere and to communicate and collaborate virtually across countries. For teachers online courses have been seen as beneficial as a way of reaching new student markets, offering more flexible learning opportunities for students, facilitating the tracking of student progress and activities, as well as providing an opportunity for creating new and innovative learning environments.

A variety of Virtual Learning Environments (VLEs) and associated tools have developed to support the developed of online courses and distance learning. There is now a wide range of VLE systems, both bespoke and off the shelf. The Joint Information Systems Committee (JISC) recently carried out a review of current activities in the development of VLEs and has commissioned a number of projects to develop Managed Learning Environments (MLEs), where the VLE is integrated into University systems such as student records, personnel data, finance and other administrative back end systems (JISC 2001).  A review of VLEs and their use in UK medical schools has recently been undertaken (Cook 2001). It presents data collected from 21 medical schools around the UK, including 6 in-depth case studies.

A considerable body of research has been carried out on the effectiveness (Leon 2002) of online courses. However many research questions still remain unanswered. How effectively do these kinds of systems support learning? Are they more effective than face-to-face learning? Which aspects of learning are best suited to online learning and which to face-to-face? How much time does the development and implementation of online courses take? What skills do teachers and students need in order to use online learning environments effectively? A sample of some of the different research strands and activities can be found in (Squires, Conole et al. 2000). In addition, Lockwood and Gooley have recently co-edited a collection of essays on successful developments in online and web-based learning (Lockwood and Gooley 2001). A number of recent books have focused specifically on networked learning, including two overviews of the foundations and theoretical perspectives (McConnell 2000; Steeples and Jones 2002).This paper aims to contribute to this debate by focusing in on the relevance of these questions in relation to CPD applied across a professional community distributed across Europe.

The online course in Neonatal Training in Europe

The project being evaluated was funded under the Leonardo da Vinci European programme and consisted of a lead site (Southampton University NHS Trust) working in conjunction with three partner sites across Europe (Denmark, Germany and Greece). The project aimed to develop an online course in Neonatal training, consisting of four learning modules. The course was to be delivered online to doctors across Europe, supported by a core of tutors from the project team.

The course was divided into four modules: Neonatal Sepsis, The Central Nervous System, Outcome Evaluation for Neonatal Conditions, and Neonatal Respiratory Module. Each module was designed in the same way, beginning with the module objectives, followed by four sequential sections: revision materials, curriculum topics, resources, and a section entitled ‘the future’. The intention was that modules would be designed from good, solid pedagogical principles, with an associated set of clear learning objectives and outcomes. The four modules ran consecutively, each module running for approximately one month. Each module required four learning hours a week, 16 in total. Overall the whole programme therefore represented 64 learning hours. The number of hours for each module related to the standard curriculum that ordinary medics complete.

The online resource was divided into two ‘zones’, a CD-ROM zone and a Website zone. All materials for the course were made available on CD-ROMs, which were distributed to participants by post at the start of each module. All materials were also mirrored on the Website. The hope was that this would reduce the amount of time students needed to be online and hence reduce costs. It was also hoped that this approach provided a more structured user-friendly learning environment for the participants.

The project used both asynchronous and synchronous communication. Asynchronous communication was divided into three main areas, i) group discussion conferencing rooms (for discussion of particular topics related to aspects of the learning materials), ii) a coffee room (open to everyone, for general communication and socialising) and iii) a staff room (closed for tutors to discuss administration and teaching issues). Participants also had access to a synchronous chat and paging facility. An action research based approach was adopted in the development and evaluation of the course. A similar approach was adopted by Allinson, who stated that

‘the purpose of applying action research to the development process was to direct the development in a structured ways, … to build up a body of local knowledge about using action research for the development of on-line courses”.(Allinson 2002)

Similarly, McConnell defines ‘action learning’ as

‘a problem or issue is posed and is diagnosed; this leads to a series of action steps being imagined which need to be taken in order to investigate the problem or issue;… the outcomes of this action are evaluated, and this in turn leads to a re-examination of the problem or issues in the light of experience and knowledge gained’ (McConnell 2000) 

Furthermore, the central philosophy throughout the course was a process of cooperative learning through real, authentic and problem-centred issues. Participants were divided into learning groups and worked cooperatively on addressing a series of posed problems around the curriculum content. The role of the tutor was to initiate and facilitate discussions and guide the cooperative learning process in addressing the problems set. Tutors also had an active role in terms of the iterative development and improvement of the course. 

The original aim of the project was to recruit 24 participants onto the programme, 6 from each of the partner sites, supported by 5 tutors. A total of 6 from Denmark, 1 from Germany, 5 from Greece and 7 from the UK were initially recruited to the programme. In the event, 4 from Denmark, 1 from Germany, 1 from Greece and 6 from the UK completed the programme. An additional 6 trainees from the UK were recruited for the last 2 modules. Therefore eighteen participants completed the programme, supported by 5 tutors and a project manager. The participants were originally grouped into 3 mixed-site groups. However, 2 of the groups were subsequently merged and a new fourth group started half way through the programme.  An Induction programme ran for 2 weeks prior to the start of the course. Each participant was required to post a mini-CV. The participants were introduced to the WebBoard and encouraged to use the conference areas to met their online tutor. Each group was assigned an online tutor (from each of the project sites), who facilitated the students’ learning process and experiences and provided help and assistance as required.

Evaluation methodology

A multi-faceted evaluation was carried out, following the philosophy of utilization-focused evaluation outlined by (Patton 1997). This consisted of the following components: i) an evaluation questionnaire distributed to all participants, ii) an analysis of data gathered via two online reflection fora, and iii) an analysis of the Website and associated CD ROM material.  Full details of the evaluation have been reported elsewhere (Conole 2001), this paper will concentrate on the analysis of the discussion boards. The course was monitored throughout the duration of the programme. The discussion boards associated with each of the four modules were analysed, along with data gathered about individual participants’ usage, logins and postings. The methodological difficulties of investigating online learning are well known and documented elsewhere (Crook 2000; McConnell 2000; Gunawardena 2001; Lally and de Laat 2002) and the limitations of the approach adopted in this study are acknowledged. It is hoped that the multi-faceted nature of the evaluation at least in part addresses this.

The scope of the external evaluation was agreed in conjunction with the project team. This included identification of project stakeholders and articulation of key evaluation issues. The evaluation methodology, associated instruments, and the focus of the research study were then developed.  A mapping of project stakeholders and respective concerns and interests was carried out; 8 key stakeholders were identified ranging from those involved directly in the project (project partners and participants) to the wider associated communities. From this a series of evaluation issues were derived which underpinned the approach adopted in the evaluation.

Data analysis

A total of 25 participants were initially enrolled on the programme, supported by 5 tutors and 1 project manager. Seven participants with very low logins (less than 20 logins) are deemed to have dropped out of the programme and their participation was excluded from the evaluation. A new cohort of 6 participants joined the programme half-way through and completed Modules 3 and 4 and were therefore included in the final evaluation and analysis. As might be expected usage of the Web site by the 5 tutors and the project manager was very high throughout the programme. Usage varied between participants but typical logins for active participants were in access of 100 logins during the programme. From the data gathered from the questionnaire, patterns of when participants accessed the site varied, showing that they did study at a range of times across the cohort. In general each participant tended to have their own preferred study time, some choosing to work afternoons, others in the evenings, some only during the week, whilst others also accessed the course at weekends. This flexibility of working patterns was evident in the responses from participants as well. For example one participant stated:

“…. I find doing this relatively easy as it fits round my work commitments and family” (P1)

This was reflected in her use of the Website, which was during the evening, Monday to Friday. Most participants logged into the site on a regular basis to keep up to date with new materials and messages being posted.

Participant activity

Not surprisingly, the highest number of logins was from the project manager, followed by the lead tutor with 478 logins. The remaining two tutors had a total of 202 and 105 logins respectively.  Eleven of the participants have high login records and evidence of active participation ranging from 294 to 96. One of the most active of these was a member of the (late starting) Group 4, with a total of 193 logins. The remaining 8 participants have logins of between 60 and 25, although many in this range were from Group 4. There were a total of 8 female and 10 male participants, supported by 5 male tutors and a female project manager. In terms of the total number of logins, the female participants logged in a total of 1181 times (59.1% of the participant logins) and the male participants logged in 817 times (40.9%). Inclusion of the number of logins from the tutors and the project manager gives total female logins of 2257 (56.3% of all logins) and 1752 male logins (43.7% of all logins). In terms of the total number of postings, the female participants posted a total of 254 messages (66.7% of the participant postings) and the male participants posted 127 messages (33.3%). Inclusion of the number of postings from the tutors and the project manager gives total female postings of 555 (47.1% of all logins) and 623 male postings  (52.9% of all logins).

The number of postings maps reasonably well to the ranking order for the total number of logins, with participants who login in frequently also being more active in terms of their total number of postings. Once again the project manager and the tutors contributed the greatest number of postings. Analysis of the discussion boards indicated that the majority of these postings were concerned with administrative details about the course, along with some degree of help/support on technical details. The project manager posted 301 messages and the five tutors posted 194, 154, 87, 48, 13  respectively, the order of activity mirroring the ranking in terms of total logins. One participant was particularly active, posting double the number of messages (163) of any of the other participants. A cluster of three postings followed this between 86 and 88, again by participants who had high login values. The postings then tailed from 65 down to 14, again mainly echoing the login ranking, with one exception. The high login of 193 from the Group 4 participant did not link to a high number of postings, which at 26 was towards the lower end of activity.

Discussion fora

Each module was supported by a set of learning materials and an online forum. As each module was completed the discussions were archived and the space cleared for the start of the new module. There were a total of 24 fora. Two online reflection fora (‘Reflections for trainees’ and ‘Reflections for supervisors’) have already been discussed.  Three fora (‘Module 1 archive’, ‘Module 2 archive’ and ‘Module 3 archive’) contain only one message each and are not discussed further. Likewise ‘Module 4 assessment conference’, ‘House rules’, ‘Web zone’ and ‘Link back to educational site’ are essentially specialised administrative or help and support fora dealing with particular topics and are not discussed further. The main fora fall into four categories: the group discussion fora  (which consists of the plenary forum and the 11 group discussion fora), the course development discussions (within the ‘supervisors forum’), a socialising forum (‘coffee room’) and a dissemination area (‘notice board’). The following discussion will concentrate on these 15 fora. Discussion activities within these 15 fora fall into five main types: Administration, Task setting, Socialising, Help and support, and Feedback and discussion. These categories were developed from analysis of the emergent themes arising from the questionnaires and the online reflection fora, along with analysis of the content within the discussion boards. Further results and findings will be discussed in more detail at the conference presentation.

Administrative messages included postings about where materials were located, deadlines for activities, start and finish of modules, location of archive material and reminders. Task setting postings were usually from the assigned group tutors outlining the details of each week’s task. Social postings included sharing of personal news (such as births, holidays, birthdays) and general banter between the participants. Help and support postings included requests for help with technical issues, queries about navigating the site, help on course procedures and protocols. Feedback and discussion included all postings, which involved discussion of the weekly tasks set, sharing of experiences and expertise, and the sharing of relevant literature. Each message was assigned to one category, when a message cut across categories it was assigned to the category that the majority of the message referred to. For example if a posting was primarily about feedback to the group on a task, interspersed with a small amount of socialising it was assigned to ‘Feedback and discussion’.

A total of 1154 messages were posted between the beginning of March 2001 and the end of August 2001. Feedback and discussion on the course content and materials and the activities set accounted for about half of the postings (49.5%), followed by administrative postings (19.8%), social postings (15.3%), help and support (10.1%) and finally task setting (5.2%).

Forms of activity - socialising

Socialising occurred across all of the boards but was most evident in the ‘coffee room’ (33.1%) and the ‘notice board’ (20.9%) areas. Most of the remainder of the social activities was spread across the module discussion fora, with the highest amount of socialising occurring in Group 1 during Modules 2 and 4. The ‘coffee room’ was actively used throughout the programme. Participants role-played in virtual space and, as they got to know each other better, shared personal news - such as holiday details and the arrival of new babies. The least amount of socialising occurred in the (late starting) Group 4, which for the two modules they completed recorded social postings of 4.0% and 0% respectively. The majority of the 4.0% consisted of participants introducing themselves at the start of the module. This difference in social levels may in part be accounted for by the increased induction time that the other groups had at the beginning of the course. This suggests that these groups were able to relax and feel more comfortable in the online environment, which relates well to Salmon’s model for best practice in e-moderating (Salmon, 2000). 

Forms of activity - administration

Administrative postings accounted for 19.8% of the total postings, most were from the project manager, although a significant number were posted by each of the tutors as well. These postings alerting participants to the start of new modules, gave information about where things could be found, offered reminders about course deadlines and provided general technical information about using the resources, as the following quote illustrates:

Hi everyone, Just a quick note to say that from tomorrow morning all the messages that relate to Module 1 will be archived and placed for future reference in the library. Your next module begins tomorrow (Wednesday) and will be in progress for 4 weeks.

Well done and enjoy your next assignment!  [T1, Group2 – Module 1]

The highest number of administrative messages (20.5%) occurred in the ‘supervisors’ forum’, with the tutors discussing administrative deadlines about the running of the modules, distribution of materials and task setting. There were also a reasonably high number of postings in the ‘coffee room’ and in the ‘notice board’ fora, (6.1% and 10.0% respectively), the majority of these were participants sharing information with each other. There was some variation in the number of administrative postings across the module group, whereas one might have expected these to be fairly consistent, numbers ranged from as low as 1.7% (for ‘Group 2/3 – module 4’ and ‘Group 4 – module 4’)  to a high of  9.9% (for ‘Group 1 – module 3’).

Forms of activity – help and support

Help and support accounted for 10.1 % of the total postings. As might be expected these were primarily at the start of the course, when participants were beginning to come to terms with the online environment and posted queries on where to find information or help with technical issues. There seemed to be some problems in the early stages with the distribution of the CD ROMs, with some participants posting to the forum that they had not received their CD ROM through the post, however these were dealt with quickly and there was little evidence of distribution problems later in the programme. In general the participants seemed to engage well with the online environment and had minimal problems using it. The low percentage of postings to the help and support forum corroborates this, as does data gathered from the questionnaires and the online reflections forum. In addition to technical and navigational queries, which were primarily addressed to and answered by the project team, the forum was also used to a limited extent by participants with queries about the course material or help with getting reference material. Participants appeared to have a good grasp of appropriate use of different fora and what each forum’s function was. Therefore as might be expected the majority of the help and support messages were posted in the ‘coffee room’ and on the  ‘notice board’ (18.8% and 7.7% respectively). One of the non-mainstream fora, ‘Webzone’, also had 13 postings (11.1%), however these were all around a particular help topic.  A second non-mainstream forum, ‘Link back to educational site’ accounted for an additional 6.0%. Help and support postings within the group fora ranged from 0% through to 12.0%.

As might be expected there were some problems with accessing and navigating the site in the initial stages of the course.

I have problems with the web-site & internet access– its taking me ages to get any info from it and I haven’t been able to see the Figures so I’m afraid my answers involve a bit of guess work: (Module 1, P2).

The combined Groups 2 and 3 seemed to experience a number of technical problems during Module 4, which accounts for the high number of help postings on this discussion forum. Problems seemed to be both technical and procedural as the following extracts illustrate.

Have completed some guidelines about waveforms but just realised that I don't know how to attach a file to WebBoard! any tips? (Module 4, P2)

Sorry to have been slow but my line went down at home - 3.5h trying to get thru to BT and then told line was fine - switched off modem and switched on again and all works beautifully!! Air was blue!! (Module 4, T2)

Has Mike gone mad? I thought all our group must be joining in when I saw the number of messages... Here is my attempt, I still can't quite get to grips with what this bit is for, and this would probably be best entitled loopy.. Legoland here I come! (Module 4, P1)

Some degree of technical help and support was evident in the early discussion boards, when participants were unsure of the protocols and ways in which they should be using the system:

What do we do with our replies? Do we post them back here? (Module 1, P2)

The response to this request was a very clear set of instructions on how each of the groups should be posting their replies to the list. Responses to activities were in two stages. An initial discussion of the issues within each group, followed by a group posting in the plenary session. This approach mirrors the traditional face-to-face seminar/plenary approach and would be a familiar format for doctors, both from their experience as students and through the use of this kind of technique at conferences.

Hi Kate,

The idea is that initially you feed your comments back to your fellow trainees in your group (I think that's Group 3 for you) where your supervisor is Gorm (who incidentally is away until tomorrow). Once you have all discussed and defined your Group response one trainee will post this response on behalf of everyone into the plenary session. By Tuesday there should be 3 responses in the plenary session (1 from each group) and these will form the basis for further discussion from everyone. (Module 1, SS)

Participants seemed to have few difficulties using the Website, as the following early posting illustrates:

Congratulations to Mike & Sue (and anyone else who is involved) Having spent a while browsing this afternoon I am very impressed with the whole WebBoard and Education site. It is superb. It is very easy to get sidetracked and be fascinated by all sorts of other facts irrelevant to the topic! (Module 1, P2)

There was also evidence of the participants coming up with alternative strategies for dealing with technical issues. In the following quote the participant suggests meeting up for a mini-discussion and gives alternative contact details for the tutors:

Hi Mike and Susan,

The chat rooms seem to be playing up. Is there any other way we can hold a mini-discussion, even if there are only the three of us? I am also finding it difficult to do other things on the site and have to make a detour to the page page to page one of you... (apologies about that sentence but you know what I mean...) You can't contact me at my work e-mail at the moment. My home one is XXX and I'm online there now. Also on ICQ number XXX. I'll try the chat rooms again now. Hope to speak to you soon. [MW, Group 1, Module 1]

Participants were evidently supportive of each other. There was evidence of encouragement between participants, with examples of mutual praise and a touch of humour, as some of the below examples demonstrate:

Thanks Lene and everyone for commenting on feeding policy. There are clearly many ways of "boiling an egg" to use an english colloquialism! (Module 4, T2)

Here's Group 2/3 answers; Sean you've done a superb job! (Module 4, P2)

There is good interaction between the tutors and the participants and evidence that the tutors were aware of the general rules of good practice for supporting online debates. In the following example the tutor provides praise for work done to date, coupled with a focus in on one issue which has emerged from the previous discussions.

Thanks for those postings - there is a lot of detail to talk through, perhaps we should look at themes & start with the IVH issue. (Module 4, T2)

Similarly with the next extract the tutor offers praise for a named individual and support for the group generally, again probing some of the discussion topic to keep the debate going.

Hi Narendra - thanks for your thoughtful response. I for one am not in favour of blanket screening tests for every case. I do think though that you do develop a "nose" for the baby who is behaving oddly (the antenatal history is not right, a couple of odd dysmorphic signs) and then you do have to do the lot really!! I really wanted to get you all to think about it when next faced with an encephalopathic child. (Module 4, T2)

As the course progressed the participants seemed to form bonds within their assigned group and turned to these colleagues first if they needed help, rather than asking the central team.

I can't open either of these documents. Any tricks or advice greatfully received (Module 3, P2)

Which received the following reply from the tutor of the group:

Hi - I can open both of the CLD documents but I cant open Ryan's - I'll ask him to repost it in .rtf format - its safer. PS - right click save to disc and open from your hard drive - its easier (Module 3, T2)

This sharing of expertise was seen as a key strength and benefit of the course and was echoed in comments by the participants on their motivations for being involved with the course as the following quotes demonstrate:

“Experience of working with colleagues whom I have never met with from all over Europe”

“I have refreshed my knowledge and also have got views of other colleagues participating in the project”

“Would like a forum to stay in touch … this is quite a knowledge base here”

Forms of activity – task setting and feedback

Task setting accounted for the lowest number of postings (5.2%) and was fairly consistent across the 11 group discussion fora and the plenary, with postings ranging from 5.0% through to 11.7%. Each module begins with an introduction from the supervisors, outlining what their role is and what the module will cover. Task messages were succinct and were used as a way of introducing new task activities, associated instructions and deadlines each week.

The most significant proportion of discussions fall into the feedback category, 49.5% of the total messages posted. Again as with the help and support discussions, participants were very clear as to where to post which types of messages. Thus only 8 messages on topic issues (1.4%) were posted in the coffee room, which was clearly differentiated as an area primarily for socialising. The remainder were evenly distributed across the group discussion fora, with values ranging from 3.9% through to 8.6%, with the highest levels being recorded for ‘Group 2&3 – module 2’. ‘Group 4 – UK trainees – module 3’ and ‘Group 1 – module 1 also showed high levels of debate (6.8% and 7.0% respectively). There was some variation in the length of postings and the degree of discussion around individual postings. Some groups posted long messages and engaged in significant discussions with each other, queried other participants’ postings, and supported or disagreed with each other’s claims. In other fora, postings were much shorter and primarily uni-directional, with respondents mainly just posting their replies to questions set by the tutor and not engaging in inter-participant discussions.

Inter-leaved with their responses to the tasks and activities, there was a significant amount of sharing of expertise and local practise, as the following quote illustrates:

We developed an interest in probiotics a few years ago and conducted a pilot study using lactobacilli. As far as I am aware, there is no human evidence that colonisation with resistant organisms is reduced by probiotic administration but I will check on this. [T3, Group 1 – Module 1]

Similarly,

I get the idea! The interesting thing is that I remember being told that one of the reaons we don't use cephalosporins here at the Hammersmith is because of the likelihood of generating resistance. However, our first line antibiotic, which we've used for ages, is cheap, doesn't need levels, is safe,tried,trusted and most importantly seems to remain effective is...Augmentin.[P3, Group 1 – Module 1]

Participants also reported that they found the Multiple Choice Questions useful, as a way of consolidating and reflecting on what they had covered in the module.

I find the MCQ's helpful. When you come up with a wrong answer you feel forced to go back and review the text - good for me.[P4, Group 1 – Module 1]

Postings in the ‘supervisors’ forum’ were also high (8.6%), with the distribution in this forum being essentially split equally between administrative topics and discussions around the materials, course content and task activities. Administrative discussion accounted for 40.5% of the total postings in the ‘supervisors’ forum’, whereas feedback discussions accounted for 42.2%.

One of the key strengths evident through the online discussions is the strong mix of linking the theory being transmitted in the course with the tutor and participants’ own personal experiences and practical knowledge.

The discussions became more animated and interactive as the course progressed. Initially some of the early postings appeared stilted and replies were often short and to the point. Many also concentrated on the subject content. In contrast later postings were a lot richer, with more discussion between participants, agreement or clarification of points made and, most importantly, an increase in the sharing of personal experiences to enrich the theory being discussed.

However when participants were asked about associated issues and concerns, many raised the issue of time commitment as a major barrier to undertaking courses of this kind. As the following quotes illustrate:

“Very well – the project has fulfilled my expectations and I have throughly enjoyed the experience although it has been a big commitment both in terms of time required and the skills I have had to acquire in a very short space of time!”

“The most limiting factor has been lack of time on my own behalf”

“Also difficulty with time to get online and participate sometimes.”

Analysis of the discussion boards confirmed the positive questionnaire responses about the liveliness of the debate on each of the four modules. The best degree of participation seemed to occur in the middle of the course, during Modules 2 and 3. There was a drop in participation and group discussions during Module 4.

The ‘supervisors’ forum’ was used by the tutors to discuss aspects of the course, such as design and development, as well as ongoing formative feedback as to how well the course was progressing. They shared strategies and ideas for motivating the students and encouraging participation in the online discussions. This forum was also used as an area to discuss the issue of decreasing participation from Groups 2 and 3. The forum was used to alert the group to the problem and the subsequent discussion resulted in a merging of the groups. As the tutors became more confident of the online enviroT2ent the need to use this forum decreased. It was evidently a valuable form of support at the beginning of the course.

In general the group discussions centred on the group tasks. The group’s rarely deviated from the main themes and it was evident that the groups worked on a reasonably equitable basis with a high degree of postings from most participants.

The ‘Webzone’ was little used as a discussion forum. It was primarily active at the beginning of the course and was used to deal with technical problems with the site or problems with the non-arrival of the CD material. Once the groups had become more familiar with the Website structure and were comfortable with the assigned groups they tended to ask questions within their group, rather than directly to the course team. Similarly, the ‘Link back to the education site’ had only 7 postings, which related to navigation problems in the early stages of the course. Ideally it would have been better to avoid having these smaller discussion fora like this, as they can confuse the users. Similarly the three fora – ‘Module 1 – archive’, etc each had only one posting. It would have been better to have a general discussion board space for postings on help and support along with a dedicated administration notice board.

The issue of confidentiality was raised at the start of the course and re-emerged in one of the discussion boards. One of the participants raised the follow issue:

And we've also had a difficult topical one. But before I give the details can someone clarify where we stand in terms of patient confidentiality and discussing genuine cases on this web-site? (Module 2, P2)

The tutor intervened with the following:

As you know there are "House Rules" for the Board which include stipulations concerning confidentiality. The Board is password protected and the contents will not be divulged to non-participants. Although it is possible that aspects of the evaluation could be published at some point we would make sure that no identifying details would be included. (Module 2, T3)

There did appear to be a significant gender bias in participation in the online discussions, with female participants posting and logging in twice as much as male participants. There were some cultural differences in terms of occasional misunderstandings and different usage of terminology and phraseology as well as levels of participation, however this was not significant and the cultural mix of the groups appeared to work well.

Conclusion

There were a number of important findings and recommendations from the evaluation. Overall the participants enjoyed the course, found it useful and unanimously said that they would use this form of learning again, however a key issue for all participants was a lack of time to dedicate to the programme. A consistent theme, which emerges from the evaluation, is the sense of the development of an online community and participants appeared to enjoy the course particularly because it gave them the opportunity to network and discuss views with colleagues. Participants felt that the schedule of the overall programme was too tight and that not enough time was available for discussing each of the modules. In addition, it would have been helpful if there had been a greater break in between modules. The value and importance of the online discussions came out consistently as one of the best aspects of the course. Furthermore the role of tutors in leading group discussions and guiding modules was critical to the success of the course. Participants stated a preference for those modules where the supervision was good and it was evident that there was a direct link between this, participant satisfaction and enjoyment of the course.

The central philosophy of the course, ie a process of cooperative learning through real, authentic and problem-centre issues appears to have been a successful strategy for enacting a course of this kind, which is CPD linked with professional networking. Dividing the participants into learning groups who then worked cooperatively on addressing a series of posed problems around the curriculum content, on the whole worked well, although there was a danger that different tutors leading these groups could significantly effect the dynamics of the group. Whether or not this model would be sustainable if the course increased in size is not clear. This issue is addressed in more detail in a comparison of tutor and student time online by Wilson and Whitelock (Wilson 1998). The role of the tutor was to initiate and facilitate discussions and guide the cooperative learning process in addressing the problems set. It is not clear however how much influence individual tutors had on particular group dynamics, what evidence can be gleaned from the data, suggests that tutors did have significant and different influences.

The pedagogical strategy of the course was designed to invoke active participation using the multiple resources available in the learning environments and the evaluation, particularly in terms of the active participation in the discussions fora, suggests that this was successful. In addition, it was designed so that the users are motivated to learn about a topic by searching for, evaluating and using authentic information. This learning experience mimics real life in targeting the learner as the routine information hunter and interpreter who constructs knowledge by problem solving with information tools.

Future research will need to attempt to address key questions around the development and use of these types of learning environments such as: What new pedagogical models are possible with innovative uses of new technologies and what is their impact? Will these approaches ultimately lead to transformations in practice and a real tangible improvement in the student experience?  How can these types of learning environments be designed and used to best effect for Continuing Professional Development and the development of international virtual networks of professional expertise?

References

• Allinson, A. (2002). An action research approach to the design and implementation of an on-line course in applied mechanics. Third International Conference on Networked Learning, University of Sheffield.
• Conole, G. (2001). External Evaluation of the Neonatal Training in Europe Project, Final Report for the Leonardo da Vinci Project, Bristol: University of Bristol,73.
• Cook, J. (2001). JTAP 623 - The role of VMLEs in UK Medical Education - JISC JTAP report, Bristol, University of Bristol,70.
• Crook, C. (2000). Motivation and the Ecology of Collaborative Learning. In K. L. R. Joiner, D. Faulkner and D. Miell (Eds.) Rethinking collaborative learning, London: Free Association Books.
• Gunawardena, C. (2001). Reflections on evaluating online learning and teaching. In E. B. a. M. Haughey (Ed.) Using Learning Technologies - International perspectives on practice, London and New York: RoutledgeFalmer.
• JISC (2001). MLE Information Pack.
• Lally, V., & de Laat, M. (2002). Deciphering individual learning processes in virtual professional development. Third International Conference on Networked Learning, Sheffield, University of Sheffield.
• Leon, P. (2002). Chameleons find quality colours. The Times Higher, 8-9.
• Lockwood, F., & Gooley, A. (2001). Innovation in open and distance learning - successful development of online and Web-based learning, London: Kogan Page.
• McConnell, D. (2000). Designing for CSCL. In D. McConnell (Ed.) Implementing computer supported cooperative learning, London: Kogan Page,109-150.
• McConnell, D. (2000). Implementing Computer Supported Cooperative Learning, London: Kogan Page.
• Patton, M. Q. (1997). Utilization-focused evaluation - the new century text, USA: Sage.
• Squires, D., Conole, G. (2000). The Changing Face of Learning Technology - Selected Papers from ALT-J, Cardiff: University of Wales Press.
• Steeples, C., & Jones, C. (2002). Networked learning: perspectives and issues, London: Springer.
• Wilson, T., &. Whitlock, D. (1998). Is it sustainable? A comparison of student and tutor online time across three distance-learning courses. Alt-J, 6 (1), 25-31.