Educational Technology & Society 5 (3) 2002
ISSN 1436-4522

Preparing students for the virtual organisation: an evaluation of learning with virtual learning technologies

Trish Andrews
Learning Resources Development Unit, TEDI
The University of Queensland, Ipswich Qld 4305
Australia
Tel: +61 7 3381 1258
Fax: +61 7 3381 1252
t.andrews@mailbox.uq.edu.au

Gavin Schwarz
Department of Management
University of Wollongong, NSW 2522
Australia
Tel: +61 2 4221 5939
Fax: +61 2 4221 4289
gschwarz@uow.edu.au

 

ABSTRACT

The paper explores the development of learning behaviours in a virtual management course and the factors that impacted on this development. Data suggest that most teams experienced three kinds of learning behaviours – social, operational and content learning. We propose that the need for technical expertise and team participation will vary during these different stages of learning. Addressing the characteristics of these stages, we comment on the development of a ‘completion phase’ of team development. We argue that the extent to which teams demonstrate different learning stages has a significant impact on the development of on-line learning behaviours. Discussing these results, we suggest why different teams develop distinct learning behaviours, with accordant emphasis on teaching as a moderating and co ordinating role, despite current virtual team pedagogical expectations.

Keywords: On-line learning, Learning behaviours, Team learning, Virtual learning


Introduction

The combined effects of globalisation, technological change, workforce demographic modification and mutable economic conditions are creating new working and learning environments. This scenario places heavy demands on the ways that people work, learn, communicate and interact (Cascio, 1999; Mowshowitz, 1999). Yet our understanding of how people behave in such environments is still evolving. In this paper, we examine a tangent of this perspective by exploring the learning and teaching challenges of working in virtual teams in a virtual classroom situation.

Virtual teams are groups of geographically and organisationally dispersed co-workers that come together using a combination of information and communication technologies (ICTs) to accomplish an organisational task (Townsend, DeMarie and Henderickson, 1998). In response to increasingly competitive marketplaces and facing a number of communication, workplace and membership challenges in the changing working climate, these collaborative teams are offered to organisations as an organising and social solution (e.g., Duarte and Tenant Snyder, 1999). Virtual teams are portrayed as promising the flexibility, responsiveness, cost effectiveness and improved resource utilisation necessary to met the demands of changing work environments (Lipnack and Stamps, 2000). Similarly, educational organisations are facing many of the same challenges in integrating ICTs as they move into more flexible teaching and learning methods and integrate on-line approaches to pedagogy.

From an educational perspective virtual scholarship is seen as a viable way of providing learning to people without the restrictions and requirements of attending on-campus sessions (King et al., 2000). The virtual team learning environment is often presented as supporting social constructivist learning activities that lead to deeper and more effective learning outcomes (Hiltz et al., 2000). Group or team learning, a central tenet of social constructivist approaches, can be facilitated by the same on-line interactions that characterise virtual team interdependence. These interactions can encourage the development of a collaborative learning environment whereby students can complete learning activities in ways that best suit their needs. Similarly, students can draw on each other’s knowledge and experience and apply it to the new knowledge or information they are presented with (see Doolittle, 2000 for comment). Another anticipated outcome of adopting on-line learning and group teaching and learning activities is the opportunity that this provides for the acquisition of essential working skills. This process includes becoming more proficient in intra- and inter-personal communication, and developing ICT literacy skills, all of which are considered essential attributes for the contemporary workplace (Graham, 2000; King et al., 2000).

While the expectations relating to virtual learning environments are high, there is, however, still limited understanding of what these working environments truly deliver in terms of learning outcomes. This paper reports the findings from an exploratory, real-time case study on virtual team interaction. Reviewing the methods and practices used to teach undergraduate students about working together in a virtual team environment, we ask, how appropriate is the virtual team as a teaching tool? Moreover, what are the learning consequences of relying on technology-based teaching solutions? The purpose of this research was thus to foster a better understanding of the kinds of learning behaviours students develop when using ICTs and working in virtual teams. The next section of the paper details the research basis and rationale for the study. This is followed by a discussion of the research project and some preliminary findings. The paper concludes with a discussion of the effectiveness of the learning behaviours in meeting desired outcomes.

 

Research foundation

Current research on technological innovations in teaching and learning

While much has been written about knowledge acquisition, learning, learning styles, learning conduct and the role of the teacher/agent in the contemporary electronic classroom, substantively little is known about the development of learning behaviours in ICT dominated environments, whether they be in the education (Graham, 2001) or the organisational field (e.g., Lipnack and Stamps, 2000). A recent study by Salmon (2000) has, however, addressed this deficiency by discussing the development of learning behaviours in an asynchronous, voluntary on-line learning environment, referred to as computer mediated conferencing (CMC). Salmon contends that students follow particular stages in developing on-line learning behaviours and that:

given appropriate technical support, e-moderation and a purpose for taking part in CMC, nearly all participants will progress through these stages of use in CMC (2000:26).

Her five-step model of learning behaviours – generated from a review of 3000 voluntary bulletin board postings over two years of a Master of Business Administration course at the Open University – identified five sequential phases of learning in online environments: (i) access and motivation, (ii) online socialisation, (iii) information exchange, (iv) knowledge construction, and (v) development. In the first phase of access and motivation students are involved in setting up and getting started with on-line learning. In the socialisation stage the focus is on getting to know one another and creating a productive learning environment. The information sharing stage is concerned with students finding their way through the messages and making sense of them. In the fourth stage, that of knowledge construction, students interact with each other in “more exposed and participatory ways” (Salmon, 2000:32). In the final or development stage of on-line learning students demonstrate critical thinking and other advanced learning behaviours. While Salmon’s work has resulted in a useful model it provides but one view of learning behaviours for a particular on-line (i.e., voluntary bulletin board) environment.

Nonetheless, Salmon’s finding shave provided the stimulus for further research in the field, while her five-step model has generated a framework to incorporate such research. We suggest that critical, insightful research in evaluating ICT-driven educational innovations is fundamental to acquiring more knowledge about the use of technological innovations in a variety of higher education environments (supporting Graham, 2001) and it’s perceived impacts on student learning (supporting Freeman, 1999). Seizing on this challenge, several research projects have explored some of the challenges associated with evaluating technological innovations in tertiary teaching and learning environments. For example, the BP Evaluation of Learning Technologies Project (ELT) in the UK (e.g., Oliver and Conole, 1998) has a stated objective:

to develop a transferable research tool which will enable practitioners to carry out evaluations of their use of educational technology (http://www.unl.ac.uk/tltc/elt/index.htm#public)

Similarly, the Flashlight project in the USA (Ehrmann, 1999) aims to:

help institutions study and improve educational uses of technology (http://www.tltgroup.org/programs/flashlight.html)

Both these projects, and others like them, are exploring a wide range of issues relating to the evaluation of learning technologies. These issues include the development of toolkits for evaluation to guide the researcher through the pitfalls of evaluation (e.g., Oliver and Conole, 1998), frameworks for evaluation of technological innovations (e.g., Oliver 1997), and a wide range of data collection instruments especially developed for the evaluation of technological innovations (e.g., Erhmann, 1999).

Current research into evaluations of technological innovations has also highlighted the complexity of the subject matter. Such a perspective illustrates the importance of identifying a clearly defined purpose or focus when undertaking these teaching activities. (Joyes, 2000). For instance, Jones and Scanlon (1999:10) note that:

It is important to determine the aims or rationale for the use of the technology in order to decide the appropriate focus for the evaluation.

Supporting this sentiment, Draper et al. (1996) suggest that there are five purposes of evaluating learning technologies. First, formative evaluation has an important role to play in obtaining rapid feedback on technological innovations and so enable development teams to problem solve or make adjustments to the design if necessary. Second, summative evaluation is most commonly used to indicate how a resource might best be utilized. Third, illuminative evaluations focus on the participants and the factors that impact on them in relation to their use of technological innovations for learning. Fourth, integrative evaluation explores ways in which technological innovations can be best integrated into the overall learning environment. Finally, evaluations for quality assurance aim to demonstrate that quality is being monitored and improvements made where required (see Oliver, 1997 for discussion). Defined purpose then can be considered a critical element in learning more about the impact of technological innovations in education.

Seizing on the excitement generated from the advent of new technology and accompanying new organising forms, some tertiary institutions have moved rapidly to employ ICTs as an educational tool. The aforementioned observations on the altered learning environment, however, suggest the need for caution if we are to learn how to effectively teach and learn with these facilities. In particular, we should heed the recent findings from studies on organisational learning pointing to the complexity of the ways that team members operate when they employ virtual teams as communities of practice (e.g., O’Leary, Orlikowski, and Yates, 2001; Robey, Min Khoo and Powers, 2000). Together, these different perspectives overlap and offer oblique support for Orlikowski’s (2000) assertion that being aware of how people use and interact with technology in their on-going practices enables a deeper understanding of roles of social practices within the workplace, and by extrapolation, learning in the classroom. In this paper, it is the latter issue that we focus on in attempting to stimulate more critical debate on the subject.

 

Research focus

With the dual purpose of illuminative and formative evaluation, this project set out to better understand the kinds of learning behaviours and processes that developed in a tertiary virtual learning environment. We recognise that certain kinds of learning behaviours emerge in particular ICT-mediated learning environments (i.e., Salmon, 2000). Nonetheless, we sought to identify what these behaviours might be in a team-centred application of ICTs. We were also interested in exploring the factors that might impact on the development of these learning behaviours, including:

  • Student level of familiarity with working in virtual environments
  • Student level of familiarity with ICT
  • Student attitudes to working in teams
  • Student skills in working in teams

While this project (and the commentary that follows) is an exploratory lens on the topic, by concentrating on these behaviours, we sought to identify any issues that might be impacting negatively on student learning and make adjustments where necessary. This approach recognises that teaching in a virtual environment becomes a collaborative process, with faculty reliant on the development of student’s skills and competency. In line with Joyes (2000) we followed an integrated approach to data collection and analysis.

 

Research project and method

The research project focuses on the experiences of 114 students in an elective course at the Ipswich campus of the University of Queensland, Australia. It explores the ways that students develop on-line learning behaviours, what these behaviours might be and the role of teams and students attitudes to working and learning in teams in establishing on-line behaviours and an on-line learning culture. The majority of subjects taught at this campus employ flexible learning methods of teaching, reliant on a variety of ICTs. Part of a three-year bachelor of business communication and electronic commerce program, the course focused on the key issues and themes of managing the virtual organisation. In order to capitalise on the range of technology choices on offer, the students used a series of ICTs to organise and complete several learning tasks. The process involved students forming small teams.

 

Virtual team processes

Students were allocated the task of working through 10 learning modules over a 13-week semester. Furthermore, students were required to complete several team-assessed exercises, and based on the results from these exercises, write a strategic report co-operatively with their team. There were only three face-to-face class meetings scheduled throughout the semester, and all other interactions took place using a purposely developed interactive web-site. Matching many of the functions of similar software systems, the site employed realtime chat (ICQ), bulletin board postings, email functions, and faculty notice boards. The students used the site to discuss weekly modules, raise learning problems or issues, plan assignments and post or exchange files. Faculty monitored realtime chat sessions and had access to an electronic discussion list. This list was where most interactions with students took place, although students did have the option of participating in voluntary meetings with faculty if they wished to do so.

The course focused on exponential, experiential learning. In order to complete assessment materials, students were required to use a series of different forms of communication, while independently working through a weekly reading and exercise module. The key to this process was that it was sequential, with each module building on the previous week’s work. The rationale for employing this process was to force the student body to consider whether the virtual team was the optimal way of operationalising everyday situations. Moreover, it was meant to encourage students to reflect on the effectiveness of new communications environments. Rather than situate learning about being virtual in the seminar room, the course was crafted in a way that gave students the experience to answer these questions for themselves. To understand the theoretical frameworks taught to them, they needed to ‘go virtual’. For this reason, the course required students to communicate, function and perform using a variety of ICTs as the primary means of learning. In order to track this process retrospectively, students were required to keep a learning journal. The journal was part of course assessment, with students required to write up their experiences using the journal as the basis of this reflective account. Each week they were encouraged to reflect upon the issues raised with their virtual team. Students needed to record what had occurred to both their team and themselves, and then outline what they had learned from this experience. The case study method was chosen to enable us to capture the social contexts and dynamics of student’s virtual work experiences.

 

Evaluation design

The methodology adopted in this study draws on a well-accepted, interpretative research paradigm. Analytical, iterative induction is promoted here as a process of constant learning to guide the case studies (Van Maanan, 1988). Induction was used in the study as the stimulus to capture the most accurate picture of the set of learning variables that arise. This point of view is an acknowledgement that reality is socially constructed. As such, knowledge is an emergent phenomenon, reliant on an individual’s subjective interpretation of a situation (Denzin and Lincoln, 1994; Patton, 1990). Without contextualising this interpretation, a researcher may well be incapable of understanding or making sense out of this enacted, self-reflexive reality. The value of the interpretative research philosophy adopted is that it enables the underlying connections between and among the different parts of this social reality to be illuminated. This is done through an examination of the rules and shared meanings that make social practices possible. Geertz’s (1973) interpretivist anthropology was used as a guiding analytical force, in attempting to utilise a methodology that would yield rich, socially constructed and contextually bound insights,. While this approach celebrates the prominence of a first person, subjective experience, it is always grounded and local (rather than speculative and abstract), making generalisation difficult. Nonetheless, this interpretivist stance provides the basis of the thick description that is a virtue of the constructed reality of naturalistic inquiry (Lincoln and Guba, 1985).

The interpretivist methodology is also appropriate for the epistemological basis of the study, which is grounded in social constructivism approaches to teaching and learning. Social constructivism takes the view that learning is a matter of constructing knowledge from experience (Hendry,1996). From this view, learning can be seen as having both active and interactive aspects (Hiltz 1998).  Social constructivism also contends that “new knowledge builds on existing knowledge” (Peters, 2000:167). Thus, social constructivist learning environments value activities that involve generating ideas, reflective practice, active engagement and relevance to students (Sherry, Billing and Tavalin, 2000), as well as knowledge-building conversation and joint task execution among collaborative groups of students (Hamada and Scott, 2000). The interpretivist approach has previously been used in exploring the outcomes of online learning (e.g., Salmon, 2000).

 

Data collection and analysis

Qualitative and quantitative data were collected throughout the duration of the course. The primary method of data collection revolved around three separate surveys distributed in weeks one, six and thirteen. These questionnaires formed the basis of our comparative analysis. Three measures were used to ascertain the changes in learning, vocational habits, and the importance of the team. The preliminary survey aimed at ascertaining student’s initial thoughts about learning and working in a virtual team and a virtual teaching environment. Using a modified version of Brown et al.’s (1986) ten-item measure and Mael and Ashforth’s (1992) six-item measure of social identification, we intended to uncover attitudes towards working in teams and perceptions of how learning would take shape. This first survey was particularly focused on how students perceived themselves, teamwork and different forms of learning. Follow-up surveys built on the first set of findings, asking more specific questions about learning experiences and on the notion of teamwork. Specifically, Haslam et al’s (1999) ‘three-things’ manipulation was employed in identifying the nature of team versus individual learning, and how individual team members felt about working virtually.

Other methods of data collection that took place both during and after semester included evaluating bulletin board and notice board postings, reviewing chatgroup interactions, and examining email contact with faculty. Additionally, once the course was completed unstructured interviews with faculty were undertaken. The focus of the interviews was on the staff member’s teaching experience wherein they were encouraged to reflect on the appropriateness of choices made (e.g., teaching, technology, assessment, and communication), learning outcomes, student reaction and what they had learned in the process. In addition, student’s reflective journals were coded for content. These journals generally took the form of an informal expression of learning processes, team interactions and any thoughts about the course. Like bulletin board postings, email and chat, these journals were coded for content until thematic saturation was achieved (following Yin, 1989).

The process of exploratory inquiry is not just to produce findings, and as such it is not an end in itself. Rather, one has to do something with this data collected. Accordingly, there were two aspects to the way that we reviewed the data collected. First, we used the descriptive statistics collected to inform initial patterns of results. The SPSS statistics analysis package was used to analyse these data. Second, qualitative data analysis employed Wolcott’s (1994) three-step approach of rendering qualitative data to tell a story. The first step consisted of constructing data out of the descriptive account given by respondents. Like Patton (1990) this meant focusing on the descriptive data collected and identifying and categorising the primary patterns in the data. The second step in data management was to transform these data into a more consistent, organised scheme through labelling information gathered. We were able to impose some order on the data through thematic- and pattern matching (Yin, 1989). In the third step, having identified standards, sorted and ordered the data collected, we reviewed the patterns that we had found. Constant comparison of qualitative findings with quantitative data was undertaken throughout this three-step process. By doing this, we attempted to maximise the validity of our results and illustrate the robustness of our findings. For this paper, we concentrate our review of learning behaviours on the themes identified within the qualitative data collected, and support these themes with quantitative data where appropriate.

 

Preliminary results and discussion

The findings of this study provide significant insights into the development of learning behaviours in a team based on-line learning environment. Findings also identify some of the factors that influence this development. The study shows that learning behaviours are closely related to team performance, and that positive team management contribute to the development of on-line learning behaviours. Poor team management skills and difficulties with technology were major elements in limited development of on-line learning behaviours.

84 students out of the 114 enrolled responded to the first survey (a response rate of 74%.) These surveys were not identified by any demographic data. 80 students, 46 males and 34 females (response rate 70%), responded to the second survey. Only 32 students (response rate 28%) responded to the third survey and again no demographics are reported. The much lower response rate for survey 3 is a result of the survey being of a voluntary nature and administered in the final week of semester. In response to the question on survey 2 relating to ethnic background 33 reported Australian, 23 reported Chinese, 7 reported other, and 17 didn’t respond.

The analysis of the bulletin board postings, journal data, and questionnaire answers identified several patterns relating to the development of learning behaviours. Observations from lecturing staff also provided commentary and insights into these developments. The patterns showed that teams and individuals demonstrated wide diversity in the kinds of on-line learning behaviours demonstrated. Learning behaviours identified included those identified by Salmon (2000). Additionally, other kinds of learning behaviours included those we have termed 'operational' and ‘content’ learning. By operational learning we meaning learning that supports effective functioning in the new learning environment. Content learning refers to learning that relates to the subject matter of the course. We propose that the need for expertise and participation will vary during different stages of learning. During the initiation phase of a project, teams spent a far longer time in social learning or getting to know each team member than was anticipated. This process was facilitated by the informality of on-line communications.

Student technical knowledge also forced faculty to re-evaluate team learning objectives early in the course. All students had extensive knowledge of basic computing packages, however, few had used ICQ or web-based chat facilities. This limitation required a focus on operational learning. The lack of technical knowledge hampered student interaction in the first phase of the course, with students needing to quickly learn how best to use non-verbal communication tools. A common complaint early in proceedings was the limitations of students without requisite skills not being able to communicate as effectively as they wanted to with their peers (eg, ICQ becoming a one line, short answer advice session, rather than consolidated information sharing session). Additionally, the technical inexperience led to a large percentage of students relying less on their teams to complete tasks, and spending more time doing project tasks by themselves, in an effort to complete these tasks more fully than they could on-line (ie, content learning). Teaching staff therefore had to intervene to guide students in ways to manage technology. These on-line sessions included advice on how to intersect individual project work with teamwork in a way that the two overlapped better. Having defined and consolidated team procedures, teams therefore created clear role differentiation, project task diffusion and workload capacity decisions. Moreover, towards the end of the course, an extensive period of group and individual reflection on the task and on team dynamics took place. This generated what we have termed a completion phase of learning.

The bulletin board postings and journal entries indicate that the development of these behaviours appears to be heavily influenced by the nature of the teams. Three types of teams were identified - high performing teams, middle performing teams and low performing teams. The teams were ranked according to the kinds of learning behaviours they exhibited. There were high levels of consistency in the behaviours of teams across the different performance levels. The performance level of teams also correlated closely with the overall marks they received for the course. Thus, high performing teams were those who received marks above 80%, middle performing teams were those who scored marks between 60-80% and low performing teams were those who scored marks below 60%.

In the survey item “barriers to learning” in survey 1, students identified communication issues as the most significant barrier. Technical difficulties, attitude and issues related to information are also identified as barriers. The data for the survey items tabled below were provided in the form of descriptive words. 'Descriptions' refers to the different responses describing the issues identified in the data. The results are summarized in the table below.

 

Technical skills (lack of)

Technical problems (breakdown, access)

Attitude (including motivation and traditional views about learning

Information (ownership of information, lack of information)

Communication (lack of face-to-face contact, senses, interaction)

17 descriptions

11 descriptions

10 descriptions

10 descriptions

31 descriptions

Examples of Descriptions:

technology incompetence; must be computer oriented

Examples of Descriptions:

technological breakdowns and difficulties; lack of infrastructure

Examples of Descriptions:

not willing to learn with technology; unreliability

Examples of Descriptions:

if information is power, people will keep it for themselves; little customer service

Examples of Descriptions:

no face-to-face communication; lack of interpersonal interaction

Table 1: Barriers to learning in virtual team sample

Themes that emerged-in response to the survey item measuring student perceptions of the need to involve all team members in their learning experience suggests the link between levels of participation in learning (particularly social learning) and student enjoyment of the virtual team process. Team organisation – and the way that this aided or hindered operational learning – also emerged as a major factor in the bulletin board postings.

 

Team participation/involvement/communication

Co-ordination/organisation/negotiation/deadlines/tasks/practices

Enjoyment (making team participation fun and interesting)

Coercion (using force to complete team tasks)

31 descriptions

27 descriptions

37 descriptions

19 descriptions

Examples of Descriptions:

leading by example; creating the appropriate environment and providing motivation/guidance

Examples of Descriptions:

assigning specific tasks; regular contacts and monitoring work progress

Examples of Descriptions:

trying to make it interesting; being happy

Examples of Descriptions:

peer pressure; making them

Table 2: I can make people participate by…

In survey 2 the item “things that your team does well” identified communication as a significant factor. Other factors included team management and learning. This survey item was repeated in survey 3 and the same issues were identified. Some unproductive behaviours were also described as things the team did well, such as time wasting. These same observations were also evident in the bulletin board postings and journal entries. Members of high performing teams demonstrated good communication and excellent team management skills in their postings, while members of low performing teams demonstrated none or very limited team management behaviours in their on-line interactions.

 

 

Communication/collaboration/relating to one another/trust/Co operation

Arguing/Disagreeing

Tasks/Delegation/completion

Thinking/brainstorming

Time wasting/procrastinating

Survey 2

102 descriptions

1 description

73 descriptions

11 descriptions

5 descriptions

Survey 3

37 descriptions

1 description

15 descriptions

2 descriptions

1 description

 

Examples of descriptions:

trust one another;encourage

Examples of descriptions:

disagree

Examples of descriptions:

delegate tasks; punctual with tasks

Examples of descriptions:

think collectively; brainstorm

Examples of descriptions:

procrastinate

Table 3: Team Proficiency

In the item “things you’re team does badly”, communication, including using technology and management were identified as significant issues. Participating in learning activities was also highlighted. These issues were also identified in both surveys 2 and 3. Poor performing teams showed little understanding of team management skills in their bulletin board postings and there was little or no evidence of on-line learning of content.

 

 

Communicating/expressing feelings/chat technology

Manage/co-ordinate focus

Thinking/understanding/specific assessment requirements

Survey 2

26 descriptions

87 descriptions

16 descriptions

Survey 3

37 descriptions

1 description

15 descriptions

 

Examples of descriptions:

sometimes drift to personal stuff too much; mange chat sessions

Examples of descriptions:

time; leaving things too late, meetings go for a long time

Examples of descriptions:

understanding the requirements of modules

Table 4: Team Difficulties

Responses to the item “how would you approach the course differently” indicated that students would place a greater emphasis on better communication and team management. These findings were consistent across survey 2 and survey 3.

 

 

Social (choosing team/communicate more regularly

Operational Management/organisation/co-ordination/tech use and skills

Knowledge (ideas, understanding)-

Other (not do course, not change anything)-

Survey 2

22 descriptions

24 descriptions

9 descriptions

16 descriptions

Survey 3

7 descriptions

15 descriptions

4 descriptions

10 descriptions

 

Examples of descriptions:

get to know my team before starting module one; meet more regularly

Examples of descriptions:

have more structure - definitive roles; be more organised and discuss topics systematically

Examples of descriptions:

realise the importance of seeing from each other's point of view; be more open to ideas

Examples of descriptions:

not do the course; wouldn't do it

Table 5: Perceptions of the need to change team behaviour

While there was no gender difference identified in responses relating to how students felt about teams, there was a significant ethnic difference. Students who identified as “Chinese” responded more positively to the following questions than those that identified as “Australian”. The seven students who identified themselves as belonging to the “other” group were not included in the analysis.

  • I feel uneasy with my team-mates( Xchinese=2.14, XAustralian=1.50) (F(1,32)= 5.26, p<.05)
  • The organisation of the course was beneficial to my learning (Xchinese=3.47, XAustralian=2.43) (F (1,34)= 8.70, p<.01)

Knowing all the people in a team before the course commenced significantly influenced how positively participants responded to several questions asked, compared to knowing none or only some of the team members. If participants knew all the members in their team before hand they responded more positively to the following:

  • I am a person who was glad to belong to my virtual team (Xknew all =4.06, Xknew none or some = 3.27) (F(1,47)=7.615, p<.01).
  • I feel strong ties with my virtual team ( Xknew all=4.13, Xknew none or some=3.21) (F(1,47)=8.929, p<.01).
  • I feel a strong sense of trust amongst team members ( Xknew all=4.13, Xknew none or some=3.39) (F(1,47)=5.68, p<.05).
  • On the whole I am satisfied with my team. ( Xknew all=4.13, Xknew none or some=3.30) (F(1,47)=7.32, p<.01).
  • Learning in teams is an effective way to learn( Xknew all=3.75, Xknew none or some=3.03) (F(1,47)=4.23, p<.05).

 

Discussion of findings

The development of learning behaviours in a team based on-line learning environment where students are required to use ICTs for a central part of the learning activities contrasts with Salmon’s (2000) model of on-line learning which identified five stages of learning behaviours. As outlined earlier, Salmon considered that given the appropriate conditions, nearly all participants will progress through these stages (2000:26). In the current study, few teams demonstrated all five of Salmon’s stages. Low performing teams demonstrated little on-line interaction, had few bulletin board postings, and had very limited social interaction. Comments from the survey on students’ perceptions of what teams did badly (Table 4), identifying poor team management as a significant factor, supports the lack of bulletin board and other on-line activity demonstrated by these teams and limited or no reference to organisational learning or content learning. Indeed, it could be assumed that these teams largely avoided the use of on-line technologies for learning. This finding agrees with the observations of lecturing staff who noted that during the initiation phase of the project, several teams spent a far longer time than was anticipated in social learning and getting to know each team member. The length of time some students spent familiarising themselves with team members lead to several problems in the first few weeks of the course. These problems included falling behind on course content and not having clearly defined team goals or processes.

High performing teams, on the other hand, evinced high levels of organisational learning, knowledge learning and a completion. These teams seemed to skip over the socialisation and information sharing phases described in Salmon’s model (2000). Again, in contrast to Salmon, the high performing teams (and also to some extent middle performing teams) demonstrated another learning phase, that of operational learning. In this phase, teams outlined their procedures for working in teams, allocated task and responsibilities and demonstrated considerable understanding of successful group processes. This phase may be a result of team based learning rather than individual learning. It could also be equated to the joint task execution mentioned by Hamada and Scott (2000). Accordingly, the bulletin board postings of high performing teams were characterised by a high level of organisation, meeting agendas, and attachments of drafts. There appeared to be a high level of group cohesion (evidenced by a lack of conflict, and supportive and encouraging comments made between members). There were limited personal or social comments and considerable focus on course content and the sharing of ideas. These findings are supported by survey results (Table 3), suggesting that team organisation and good communication were critical factors in things that teams do well. In the completion phase, students brought closure to the tasks and reflected on the outcomes of their activities. Their discussion postings demonstrated a high level of understanding about what worked well and what might have been done differently.

Bulletin board postings indicate that middle performing teams confirmed most closely to the stages of Salmon’s model. These teams moved rapidly through Salmon’s access and motivation, and the socialisation phases. The information sharing stage was not evident in bulletin board postings. These teams also demonstrated the operational and completion phases demonstrated by the high performing teams, although usually at a lower and less sophisticated level.

The survey data relating to Table 4 (“things your team did badly”) underscores that students provided the same responses to the issue of learning difficulties in survey 2 as they did in survey 3. This finding may suggest that while individuals recognised what team difficulties were, bulletin board postings suggested they were not able to make significant differences to the ways they were operating. Interestingly the focus of responses to the survey item “how would you approach the course differently” indicated that socialisation and operational factors were of most importance (Table 5). Importantly, the data are not coded in such a way as to ascertain any correlation between team types and responses. Although a number of responses indicate that some students would not do such a course again, it is not known if there is any relationship between this attitude and low performing teams.

The finding indicating that where students knew all members of their teams they had positive feelings about teams and team relationships is also of interest. While the data were not coded in such a way as to support this supposition, it is worth speculating that teams where members all knew each other were amongst the high performing teams. Research relating to other virtual learning environments (such as video-conferencing environments), strongly indicate that students feel much more comfortable with the learning environment and have more successful learning experiences if they know other students and lecturing staff before the commencement of the course (Klease, Andrews and Druskovich, 1996).This observation would indicate that including or developing social learning processes in on-line learning environments may be critical to promoting effective learning behaviours, particularly for low and middle performing teams. Again the role of facilitation in ensuring these processes could be considered a key teaching role. Additionally, the ethnic differences identified how students viewed working in teams is another notable finding. This result provides a limited insight into possible cultural differences as to how students develop on-line learning behaviours which may be worthy of further exploration. Being a preliminary investigation into virtual team learning patterns, further research is needed to test and confirm these findings.

 

Conclusion

From a review of learning behaviours in a virtual management course and the factors that impacted on this development it appears that different kinds of on-line behaviours develop in different kinds of on-line learning environments. In our virtual learning sample, different teams exhibited different learning behaviours and only middle performing teams appeared to develop learning behaviours in line with Salmon’s (2000) model. We surmise that good understandings of how to use technology, as well as the ability to develop team relationships, are both critical elements in supporting and encouraging the development of an effective on-line learning environment. This observation suggests the need for greater attention to the critical role lecturing staff play in moderating and facilitating on-line learning behaviour. As a caveat, we acknowledge that our findings are limited by our speculations on learning patterns. In particular, we recognise that while it is a reasonable assumption that there is a relationship between positive feelings about teams and successful team outcomes, our surveys were not initially framed to ascertain this feature of learning. Similarly, from our results, we assume a relationship characterising the attributes of a well-performing team and their learning behaviours, despite our study not initially focusing on this feature of virtual teams. Future research therefore needs to more purposely test these aspects of the virtual learning environment. With more institutions adopting sophisticated technologies for learning purposes, further research programmes need to address the transferability of these conclusions, and in more classical terms, assess their external validity.

 

References

  • Brown, R. J., Condor, S., Mathews, A., Wade, G., & Williams, J. (1986). Explaining intergroup differentiation in an industrial organisation. Journal of Occupational Psychology, 59, 273-286.
  • Cascio, W. F. (1999). Virtual Workplaces: Implications for organisational Behaviour. In C. L. Cooper & D. M. Rousseau (Eds.) The virtual organisation, Chichester: John Wiley, 1-14.
  • Comeaux, P. (1995). The impact of an interactive distance learning network on classroom communication. Communication Education, 44, 353-361.
  • Denzin, N., & Lincoln, Y. (1994). Handbook of qualitative research,Thousand Oaks, CA.: Sage Publications.
  • Draper, S. W., Brown, M. I., Henderson, F. P., & McAteer, E. (1995). Integrative evaluation: An emerging role for classroom studies of CAL, http://staff.psy.gla.ac.uk/~steve/howtoref.html.
  • Doolittle, P. (1999). Constructivisim and on-line Education, Virginia Polytechnic Institute and State University, http://www.tandl.vt.edu/doolittle/research/tohe1999/online.html.
  • Duarte, D. L., & Snyder, N. T. (1999). Mastering virtual teams: Strategies, tools and techniques that succeed, San Francisco: Jossey Bass.
  • Erhmann, S. C. (2000). Studying and improving the use of technology to support collaborative learning: An illustration of flashlight methods and tools, http://www.tltreoup.org/resources/F_illustrative_1.htm.
  • Freeman, M. (1999). Educational innovation: Hype, heresies and hopes. Flexible learning: Issues in student access and equity, http://www.bus.uts.edu.au/finrecon/staff/mark/aaanz.html.
  • Geertz, C. (1973). The interpretation of cultures: selected essays, New York: Basic Books.
  • Graham, C. (2001). Keynote address ODLAA Forum, Sydney.
  • Graham, C. (2000). Dot.com or not.com? Implications for career and enterprise education in the digital economy. Paper presented at the Open Learning 2000 “Generating Opportunities” 4th International Conference, December 6-8, 2000, Brisbane, Australia.
  • Hamada, T. & Scott, K. (2000). A collaborative learning model. The Journal Of Electronic Publishing, 6 (1),http://www.press.umich.edu/jep/06-01/hamada.html.
  • Haslam, S. A., Oakes, P. J., Turner, J. C., & McGarty, C. (1999). Social identity salience and the emergence of stereotype consensus. Personality and Social Psychology Bulletin, 25, 809-818.
  • Hendry, D. (1996). Constructivism and Educational Practice. Australian Journal of Education, 40 (1), 19-45.
  • Hiltz, S. R. (2000). Measuring the Importance of Collaborative learning for the effectiveness of ALN: A multi-measure, multi-method approach. Journal of Asynchronous Learning Environments, 4 (2),  http://www.aln.org/alnweb/journal/Vol4_issue2/le/hiltz/le-hiltz.htm.
  • Hiltz, R. (1998). Collaborative learning in asynchronous learning networks: Building learning communities,  Invited address at WEB98, Orlando, Florida.
  • Jarvenpaa, S. L., & Leidner, D. E. (1999). Communication and trust in global virtual teams. Organisation Science, 10, 791-815.
  • Jones, A., & Scanlon, E. (1999). Evaluating Learning and Teaching Technologies in Further Education. In Oliver, M. (Ed.) Conference Proceedings of the evaluation of learning technologies, University of North London, http://www.unl.ac.uk/tltc/elt/elt99.pdf.
  • Joyes, G. (2000). An educational model for supporting higher education lecturers in the integration of new learning technologies. Educational Technology and Society, 3 (4), 56-68.
  • King, B., McCuasland, H., & Nunan, T. (2000). Converting to on-line Course and Program Delivery: The University of South Australia Case Study. International Review of Open and Distance Learning, 1 (2), 1-21.
  • Klease, G., Andrews, T., & Druskovitch, D. (1996). Blurring the boundaries – Developing effective teaching and learning models for Multi-site Videoconferencing. Proceedings, Open Learning Conference Windows to the Future, December, Brisbane.
  • Lincoln, Y., & Guba, E. (1985). Naturalistic Inquiry, London: Sage.
  • Lipnack, J., & Stamps, J. (2000). Virtual teams: People working across boundaries with technology, New York: John Wiley.
  • Mael, F. A., & Ashforth, B. E. (1992). Alumni and their alma mater: A partial test of the reformulated model of organisational identification. Journal of Organisational Behaviour, 13, 103-123.
  • Mowshowitz, A. (1997). Virtual organisation. Communications of the ACM, 40 (9), 30-37.
  • O’Leary, M., Orlikowski, W. J., & Yates, Y. (2001). Managing by canoeing around and other strategies: Lessons from the Hudson’s Bay Company. Knowledge Directions, Spring/Summer, 26-37.
  • Oliver, M., & Conole, G. (1999). Evaluation of learning technologies, University of North London, http://www.unl.ac.uk/tltc/elt/index.htm#public.
  • Oliver, M., & Conole, G. (1998). Evaluating Communication and Information Technologies: a toolkit for practitioners. BP ELT report no. 3, University of North London.
  • Oliver, M. (1997). A framework for evaluating the use of educational technology. BP ELT report no. 1, University of North London.
  • Orlikowski, W. J. (2000). Using technology and constituting structures: A practice lens for studying technology in organizations. Organization Science, 11, 404-428.
  • Paulsen, M. (1998). Teaching Techniques for Computer-Mediated Communications, http://home.nettskolen.nki.no/~morten/.
  • Patton, M. Q. (1990). Qualitative evaluation and research methods (2nd Ed.), Newbury Park: Sage.
  • Pedhazur, E. J., & Schmelkin, L. (1991). Measurement Design and Analysis: An integrated approach, Hillsdale, N.J.: Lawrence Erlbaum Associates.
  • Peters, M. (2000). Does Constructivist Epistemology Have a Place in Nurse Education. Journal of Nursing Education, 39 (4), 166-170.
  • Robey, D., Min Khoo, H., & Powers, C. (2000). Situated learning in cross-functional virtual teams. IEEE Transactions on Professional Communication, February/March, 51-66.
  • Salmon, G. (2000). E-moderating: The key to teaching and learning on-line, London: Kogan Page.
  • Sherry, L., Billing, S. H., & Tavalin, F. (2000). Good on-line conversation: Building on research to inform practice. Journal of Interactive Learning Research, 11 (1), 85–92.
  • Townsend, A. M., DeMarie, S. M., & Hendrickson, A. R. (1998). Virtual teams: technology and the workplace of the future. Academy of Management Executive, 12, 17-29.
  • van Maanan, J. (1988). Tales of the field: On writing ethnography, Chicago: University of Chicago Press.
  • Wolcott, H. (1994). Transforming qualitative data: Description, analysis, and interpretation, Thousand Oaks, CA: Sage.
  • Yin, R. K. (1989). Case study research (2nd Ed.), Newbury Park: Sage.
  • Young, J. R. (1997). Rethinking the role of the professor in an age of high-tech tools. Chronicle of Higher Education. October 3, A26-28.

decoration


Copyright message

Copyright by the International Forum of Educational Technology & Society (IFETS). The authors and the forum jointly retain the copyright of the articles. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear the full citation on the first page. Copyrights for components of this work owned by others than IFETS must be honoured. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from the authors of the articles you wish to copy or kinshuk@massey.ac.nz.