Supporting knowledge elicitation for learning in virtual teams
The growing complexity and dynamics of professional work increasingly requires knowledge intensive collaboration in teams. Work performance more and more becomes a knowledge- and collaboration-based activity. Solving problems, designing products and services require intensive communication of personal knowledge for the construction of collective solutions. Advances in Information and Communication Technology (ICT) enable distributed teamwork. Continuous learning while working will be obligatory to meet the performance requirements of the workplace. Social and economic developments, such as the globalisation of the economy and the shift towards the so-called information economy, put heavy pressure on education. On-the-job flexibility requires just-in-time application of necessary know-how from employees. Knowledge is seen as a competitive advantage. Virtual work environments become common as effective settings for knowledge sharing across business units (Nonaka & Takeuchi, 1995; Von Krogh et al., 2000).
Looking at traditional education, a change can be observed. The transfer value of education to work settings has been rather low. New alternative forms of education, like blended learning and cooperative education have to meet these new demands of the workplace. The integration of working and learning implies a return of education to real life settings. Many educators now consider both social context and social processes as an integral part of learning activities. Theories of constructivism and situated cognition suggest that for learning to be useful, the learner needs to be actively involved in constructing new knowledge within meaningful contexts, not merely absorbing it (Duffy et al., 1993). This is in line with competency-based education.
In both business and education similar developments can be observed. Learning and working in virtual teams increases in importance in both contexts (Bastiaens & Martens, 2000). The convergence of work, learning and the virtualisation of teamwork require new educational frameworks (Westera et al., 2000). The aforementioned developments lead to new educational design questions on how to design and support collaborative learning and working.
An example of a new learn-work environment in which education and business merge, is the Virtual Business (VB) environment (Westera et al., 2000). Designed at the Open Universiteit Nederland, it offers a virtual learning framework for graduate students and an organisational learning environment for on-the-job learning of professionals. Learning in a VB team occurs in a virtual enterprise setting embedded into the professional’s daily workspace. The VB environment primarily focuses on learning embedded in real work tasks and activities. Learning in a VB is team oriented, competency and problem based, embedded in an organisational context. It includes support of personal and collective competency growth. Concurrently, VB learning encourages explicitly a pro-active knowledge sharing and knowledge management attitude of its members.
The creation of shared knowledge is of crucial importance to the success of collaborative learning processes. Research into these processes uses multiple methods and perspectives from social sciences, psychology, organisation sciences and human computer interaction. (Dillenbourg 1999; Kraut, Galegher & Egido 1988; Von Krogh et al., 2000). One of the major topics of interest is knowledge elicitation (Chi et al., 1994; Dillenbourg, 1999; Plötzner et al., 1999).
Construction of shared understanding and solutions requires more than simple exchange of explicit information. Elicitation of unarticulated ideas of participants lie at the basis of negotiated agreement upon common goals and collective solutions. This implicit knowledge is often referred to as tacit knowledge (Leonard & Sensiper, 1998; Polanyi 1967). There is no clear understanding regarding optimal prompting of elicitation and sharing of tacit knowledge. Researchers emphasise the fact that knowledge elicitation doesn’t arise spontaneously (Chi et al., 1994). Ideas for stimulation range from elicitation via external knowledge representations, structured dialogues, argumentation elicitation and the use of artefacts to community formation. (Chi et al., 1994; Plötzner et al., 1999; Stahl, 2000). Many support tools are task-oriented. Some offer very dedicated support in close relation to the task at hand. Others offer more generic support for information exchange. Empirical studies (Kraut et al., 1988) indicate that social interaction probably is so critical to successful knowledge creation that neglecting this aspect will limit collaborative knowledge construction.
Thus it is pivotal to consider the design of optimal support for learning and task performance processes occurring in asynchronous communication of learners and professionals collaborating virtually (Bastiaens & Martens, 2000; Dillenbourg, 1999). This article focuses on the design of tools supporting elicitation of implicit knowledge in function of the construction of shared understanding. Commonly encountered is the view of embedding support tools in a single mode. These tools are either generic or very specific, dedicated to the common characteristic of a group. Existing literature, however, shows that several critical variables regarding the use of knowledge elicitation support have not yet been explored. One such variable is the need for flexibility of elicitation support. Aspects crucial for effective activation of elicitation of not yet articulated knowledge seem to be related to characteristics of individual team members and the context of learning and working. Experts and novices typically prefer different kinds of support. As a result of differences in individual learning and working habits, we wonder which dimensions need to be addressed for optimal support. To which extent do task variables, context variables, the individual’s profile and team characteristics have to be triggered?
Expectations derived from the research literature point in different directions with respect to the nature of knowledge elicitation itself and the interdependence of critical factors influencing elicitation support of co-working professionals. Hence further exploration was needed to clarify this. Therefore we executed an electronic Delphi study to elicit the perceptions of a group of representative experts regarding knowledge elicitation processes and its support.
Delphi study: exploration of critical success factors for knowledge elicitation support
An electronic Delphi study - ACE - was chosen as an effective and efficient method to elicit insights into augmentation of knowledge elicitation. Delphi studies often have been used for forecasting and policy making (Linstone & Turoff, 1975; Kenis 1995). We compared interview and survey research to the Delphi method. On the dimension of specificity and subjectivity, all these methods are strong fitting neatly to a particular research question. The choice of the Delphi method was based on the following advantages. The Delphi method (Kenis 1995; Linstone & Turoff, 1975) offered us the possibility to address a group of experts more efficiently than via interviews or surveys. In Delphi sessions experts are able to amend and act in response to all ideas posted by their peers. Consequently each expert has the opportunity to take all factors brought forward into account and integrate them into their final judgment. Due to sequencing and pre-structuring needed for interviews and questionnaires relevant insights could be missed, while these insights are easily expressed in the open format of a Delphi study. Apart from its strength on eliciting all kinds of important information, a Delphi also allows the study to fit easily into the work habits, time pressure and agenda constraints of the expert population.
We divided our Delphi study in two sessions. The first session centred on critical success factors for elicitation support of collaborative teamwork in virtual learn-work environments. The leading question was what experts consider decisive for knowledge elicitation support. In the second session we focussed our attention on the required functionality and customisation of knowledge elicitation enablers. In this session we tried to identify which methods of prompting show potential.
Our type of Delphi study, sometimes referred to as Group Delphi (Kenis 1995; Webler 1991), can be described as a technique aiming at obtaining the most reliable converging or contrasting expert opinions. Experts are questioned, in consecutive debate sessions. Feedback from former sessions is mirrored until a stable state of opinions is reached. Typical for the ACE Delphi study was a heterogeneous response group. Experts reacted under their own name. The Delphi was in Dutch. The electronic Delphi study took place asynchronously during a 4-week period. There was no face-to-face interaction. Each participant had time for thought and equal opportunity to contribute. The experts’ opinions were elicited in two debate sessions wherein experts could change and specify their opinion. Feedback to the group presented individual and collective views. Conclusions were reached taking each other’s contributions into account.
Special care has been given to the selection of group members. We pursued a broad range of expertise since the objective was to explore the variety of views. Experts were selected from the national community based on their reputation, either derived from their publications or their professional track record. Special attention was given to find representative experts of equal importance in the professional community. Of the sixteen Dutch experts participating in this study, nine had a scientific background as researcher or designer in higher education, seven participants were practitioners working as staff member or executive in commercial enterprises.
The Delphi study started with an introduction into the electronic environment, a survey of the expert’s background and personal statements with respect to collaborative knowledge elicitation. The first session focussed on generic critical success factors for collaborative knowledge elicitation in asynchronous distributed working teams. Special attention was given to three dimensions of knowledge elicitation, namely elicitation in function of personal competency growth (column A in Table 1), organisational knowledge productivity (column B in Table 1) and co-construction of shared understanding, a collective memory (column C in Table 1).
The second session focussed on specific needs and functionalities of knowledge elicitation support. Special attention was given to instrumentation of knowledge elicitation prompts (column D in Table 1) and aspects to study (column E in Table 1).
In both sessions facilitation was low profile, restricted to the process, keeping the debate open enough to enable new perspectives to be shared. After each session the facilitator provided a synopsis. The participants commented and agreed with the synopsis as an adequate representation of the debate. Participants expressed their personal perspectives on issues brought forward via electronic forms indicating their perception of importance and consent with the statements as posed. Afterwards the results were aggregated in a concluding report.
Table 1. Matrix ACE Delphi study with numbers of expressions
The Delphi method is both appreciated and criticized for the richness of qualitative insights it generates. Thereby leaving the researcher with the challenge to adequately structure and evaluate the data. The latter was achieved by identifying semantic expressions, defined as discernible expressions of a participant’s thoughts. Three reviewers categorised them into a matrix in which rows depict the main issues raised by the participants and columns A to E, described previously, depict the issues raised as input for both the Delphi sessions. Results are presented in Table 1.
In Table 1, a total of 142 semantic expressions are discerned, 84 of them occurred in the first debate and 58 in the second debate. Statements in the first debate cluster primarily around team awareness and team dynamics for organisational knowledge productivity (cell B7). Experts stress the importance of complementary assets (like personal characteristics and knowledge) of individual team members as a basis for trust, shared conventions and shared ambitions within the team (cell B3). A productive knowledge attitude of team members (cell B1, A1, C1) and knowledge awareness stimuli in the virtual team environment (cell B1, A1, C1) are perceived as especially important aspects of distributed teamwork.
Recruiting personnel with a knowledge productive attitude is one way of accommodating, facilitating this attitude another (cell D1). Affording productive knowledge creation in teams is perceived as a quite complex process. It requires a mixture of facilitation, concurrently addressing multiple dimensions of collaboration and knowledge interaction (cell B13, A13, C13, D13). Experts mentioned the significance of a challenging environment in which creative unrest is combined with basic stability. This in turn requires articulation and co-ordination of team expectations and conventions (cells B11, A12, B12, D12).
In general, the experts state that knowledge elicitation prompting only will sort lasting effects if the particularity of the context is taken into account and explicitly reflected in the design of elicitation support, (cell B6, A6, D6). There was relatively little attention for generic instrumentation. Experts rather looked for metastrategies to develop customisable support (cell D12).
The experts converged as to the importance of clear and explicit articulation of the added value of the team setting (cell B10, A10). Opinions diverged with respect to the question in how far knowledge processes within a virtual team differ from face-to-face settings.
Analysing the clustering of expressions in column B, it appears that experts are aware of the significance of the organisational aspects of learning. Learning for individual competency growth is discerned as relevant for knowledge elicitation support though learning related to knowledge productivity received more attention. Expressions explicitly addressing knowledge elicitation processes in function of the creation of a collective memory however attracted less distinct attention. Experts perceived it as subcategory belonging to the wider context of teamwork. Several cells, (B5, B11, A5, C5, D 5) show the importance of explicit articulation of shared goals and explication of meaning.
The results of the second session show a clustering of expert attention on support for knowledge awareness (cell D3) and affordances for a knowledge productive attitude (cell D1, E1). Once more the problem of contextuallity of team support and community formation was stressed (cell D6, D8).
Experts found facilitation of a knowledge constructive attitude of team members of the greatest importance. A team should be equipped with knowledge finding tools and a stimulating environment for community formation. These are assumed to trigger team members to constructively express their ideas in function of the problem solution process. Neither generic support with respect to representation of arguments and dialogue enhancers nor dedicated tools for the task at hand were perceived as factors of primary importance.
Discussion and conclusions
The current study provides insights into knowledge elicitation within virtual learn-work activities embedded in an organisational setting. Blending support to multiple aspects of a specific situation is needed. Interdependency of the variables influencing collaborative knowledge elicitation changes over time. Current research concentrates on task specific support of team members. The results of the Delphi study point towards the need for support of social and community related aspects of knowledge processes occurring in professional teamwork.
Clearly the qualitative character of a Delphi study has its limitations. Much is left to the expressiveness of the experts and the reviewers’ skillful interpretation. Therefore, in the near future this study will be followed by experimental research in which we will try to manipulate key variables that resulted from this study. Results point to the importance of customising multiple aspects to a specific situation. Each context requires a mixture of team and knowledge awareness and task related prompts. Based upon generic know-how with respect to enabling virtual team dynamics, and community formation social and task related knowledge prompts should be designed dedicated to the constraints and dynamics of the organisational context. Knowledge elicitation has to be prompted during the lifecycle of a team or community accommodating to its evolution and changes over time, primarily focussing on knowledge awareness, stimulating knowledge constructive interactions.