Dynamic Goal-Based Role-Play Simulation on the Web: A Case Study
Som Naidu, PhD
The work described in this paper has been partly supported by "The Teaching and Learning Multimedia Educational Technologies Committee" at the University of Melbourne. This project is situated in a subject (World Politics in Transition) in the Political Science Department at the University of Melbourne. Prior to 1997 this subject relied on the conventional approach to teaching and learning which comprised lectures, tutorials and individual research carried out by students. Students were assessed on the basis of a 3,000-word essay and a two-hour exam at the end of the subject. The new approach (for those students who chose to select this option), sought to achieve the same learning outcomes with the help of role-play simulations and collaboration, and the communications and researching capabilities of the Internet. This new "goal-based role-play simulation" on the Web provides a virtual space for students to examine the political processes and theories that are introduced in the lectures.
Our pedagogy is based on the belief that learning is most efficient and effective when it is situated in realistic settings where learners are clear about, not only the reasons for learning but the context or the ecology of their learning environment. This view contrasts with the notion that subject matter can be represented in schemas, stored in memory, and retrieved when needed. Gestalt psychologists, such as Wertheimer, Kohler, and Koffka, argued in favour of the role of insight, perception and reflection in the learning process as opposed to association based primarily on past experience, such as that proposed by Thorndike, Skinner and Pavlov (Bower & Hilgard, 1981).We wanted to create situations that were not only motivating and challenging, but that necessitated the learning of facts, principles and procedures, as well as the cultivation of insight, perception and reflection. To achieve this outcome, we used and improved upon the simulations first developed in the early 1990s by Vincent and Sheppard (Vincent & Sheppard, 1998). This approach can be also described in Schank's terminology as a Goal-Based Scenarios (Schank, 1997) or more accurately as a Dynamic Goal Based Scenario (Linser & Naidu, 1999). A Goal-Based Scenario (GBS) is essentially a simulation in which learners assume a main role, which has associated with it a mission. Their "goal" is to accomplish this mission or task associated with their role(s) in the scenario. In order to achieve this goal the learner needs to acquire particular skills and knowledge. This is where and when the learning takes place. Goals in this context refer to the successful pursuit of the task at hand. A GBS therefore serves both, to motivate learners and also give them the opportunity to "learn by doing". As long as a goal is of inherent interest to learners, and the skills needed to accomplish those goals are the targeted learning outcomes, we have a match and a workable GBS.
One of the most harmful misconceptions people have about learning is that being able and smart comes from knowing a lot of rules. Behind this notion is the sense that reading a lot of textbooks, articles and resources, and absorbing what these say will lead one to acquire the knowledge and understanding. While it makes sense to say that learning comes from content knowledge, most of that knowledge in practice looks quite a bit different than what you find in textbooks. The architecture of this learning environment follows from the premise that if we are to prepare better students for the challenges of the contemporary workplace, we must shift our focus from a content-centered to a problem-based approach. Problem-based learning is based on the principles of a situated cognitive model of learning (Schank, 1997; Schank and Cleary, 1995). The primary propositions of the situated cognitive model of learning are: a) that understanding is gained through interactions with authentic cases and in situ; and b) cognitive conflict is the stimulus for learning, which also determines the organization and nature of what is learned (Savery & Duffy, 1995).
Brown and Palincsar (1989) assert that change is more likely when individuals are required to explain, elaborate or defend their positions to others as well as to themselves, which gives rise to cognitive conflict in an individual. These authors argue that cognitive conflict arises when the learner is exposed to disagreement between existing knowledge and new anomalous information. Chin and Brewer (1993) examined the role of cognitive conflict in promoting conceptual change, and attribute these changes to the following four cognitive attributes: status of the anomalous data in the perception of the students; characteristics of prior knowledge; learner’s perception of the credibility and validity of the new information; and processing strategies. Whether cognitive conflict is experienced by the learner, and if it will result in change is dependent upon those four cognitive aspects ascribed by Chin and Brewer (1993). While the design of role-play simulation does not explicitly solicit cognitive conflict, the interactions among the roles challenges the theoretical basis of various actions that students take in the simulation. When the students are engaged in the role-play, a level of cognitive conflict is evident, for example, in defending a situation or a position in the pursuit of goals as required.
If the skills which we would like students to learn have genuine utility in particular contexts, then one way to approach curriculum is to identify such contexts, and allow students to explore situations which they find most engaging and in which these skills naturally come into play. Dewey’s (1938) observations on the essential role of activity in learning are an early expression of an idea later elaborated with respect to the design of learning environments. Providing a means for a student to directly experience problem solving episodes takes advantage of the way people naturally learn (Schank & Cleary, 1995), and can expose the student to a variety of cases, ie., whatever range of phenomena the environment is capable of recreating. In particular, people learn by experiencing failure (Schank, 1982). This is because, in seeking to explain an anomalous event, mental machinery is set into motion which can dynamically alter memory organization. In other words, people learn from having to explain (Schank, 1986). Since the need to explain arises only when something unexpected happens, a good learning-by-doing environment would put students into situations where they can possibly encounter failure and encounter these failures in a safe and protected environment. The role-play simulation described in this paper offers students exactly that opportunity, that is the benefit of making mistakes without serious consequences. The requirement at the end of the simulation for the preparation of "role summaries" also forces students to engage in explanation and internalization of the learning activities.
The activities in an environment must of course be coordinated with the environment itself, or put another way, the activity should take place within an authentic setting (Brown, Collins & Duguid, 1989). Learning in situated settings provides for richer sources of knowledge than does learning from didactic descriptions. This notion is a familiar one in domains where apprentices acquire their skills from master practitioners, and can be adapted for instruction in cognitive domains(Collins, Brown & Newman, 1989). Anchored Instruction (Bransford, Sherwood, Hasselbring, Kinzer, & Williams, 1990), for example, is a model which calls for creating an authentic task environment in which learners can appreciate the utility of the skills and knowledge they are acquiring. Of relevance here is that concepts, activity, and context are inseparable components of learning. The notion of situated activity therefore implicates, not only situatedness, but activity, or put another way the pursuit of goals. The situatedness and the explicit articulation of individual goals are two important aspects, which have been encapsulated in the role-play simulation generator software that has been used to generate the simulation described in this paper.
Dynamic Goal-Based Scenarios
The idea that learning must begin with a goal arises from observations of how people learn naturally. People learn something because it helps them achieve some goal (Schank, 1997). Situated approaches to learning environments (e.g., Anchored Instruction) share a common emphasis on how learning should take place (namely, via the exercise of skills and knowledge within an authentic context). The role of the student in anchored instruction activities, for example, includes observing some events (e.g., watching a video or reading a newspaper report), verifying the accuracy of some information, looking for clues, and applying those clues to solving a problem (CTGV, 1991). The importance of goal pursuit to learning though, suggests that students should be given opportunities to put into practice the skills being taught. This approach has two benefits. First, practice facilitates skill acquisition (Anderson, 1983), second, applying skills toward achieving a specific goal provides a context in which those skills are useful (Bransford, Sherwood, Hasselbring, Kinzer, & Williams, 1990; Collins, Brown & Newman, 1989). The first steps in constructing a learning episode then, are articulating the target skills, and selecting a goal for the student to pursue which is appropriate to those skills. The learning environment must be able to support the pursuit of such articulated goals and skills in an authentic context.
The learning environment must then be designed to include a meaningful context for pursuing that goal, in which the student’s activities are both engaging and plausible with respect to the goal the student is pursuing. Goal-Based Scenarios (Schank, 1997) provide an explicit account of instructional environments in which the learner is engaged in pursuing a goal, within a simulated environment, in order to master a set of target skills. The student is an active participant in such a scenario, assuming a role in which resources provided by the program are available to help the student progress toward completing the task. The key organizing principle behind a Goal-Based Scenario (GBS) is that the instructional goals are distinct from the task goals (called the mission) which are set within some activity (called the context). One approach has been to provide resources on demand, which requires a sophisticated ontology to classify different cases and resources into a structure so that the software can pick up the appropriate case or resource for presentation at the appropriate time.
The instructional goals of GBS comprise the development of skills as well as knowledge of facts and cases. In order to gain an understanding of processes, relations and theories as well as knowledge of facts and cases a GBS needs to be reconceptualized so that the instructional goals of knowledge and understanding can be included within the activity undertaken by participants. One way is to give the task goals a pedagogical value so that the learning activity itself becomes a reflective process in which reevaluation of action takes place. In such an environment both the scenario and the task goals are neither fixed nor given but are rather altered by the very actions of those engaged in learning. In other words the scenario becomes a "dynamic" scenario and this requires participants to understand both processes and relations as a byproduct of the necessity of evaluating the effectiveness of their own actions.
In Dynamic Goal-Based Scenarios the task goals are not invented for the student. Rather, it is the student who constructs these goals in accordance with the role as well as the strategies by which these goals are to be achieved. They also comprise the attainment of facts, principles and cases, as well as the understanding of the issues and theories which help to explain these cases. As a consequence, the very activity of constructing task goals requires participants to research facts. In pursuing strategies to fulfill these goals they learn to understand the relations and processes that they themselves create. Thus the instructional goals of knowledge and understanding are achieved in the course of the participants' activity in constructing and pursuing the task goals. This is implemented in the form of formal writings of role profiles at the beginning of the simulation in which the students are asked to articulate the goals of the role/character they were playing. These role profiles were made available to other role players so that a dynamic mechanism could be set into action during the simulation as each role evaluated the stipulated orientation of all the other characters in the simulation to keep on adjusting theirs as the simulation progressed.
Our role play simulation extended the concept of "resource" from pre-packaged multimedia content, which are presented to the learners on-demand, to an integrated communication environment designed to provide the "situatedness" for authentic learning. Communication in this environment is made possible by using the Web. The web enables learning at any place and any time -- a feature particularly appreciated by adult learners whose experiences are explicitly leveraged in the simulation. The simulated world, which provides both the context and the arena in which students interact, resembles the real world in both its political history and the characters (roles) occupying it. An initial scenario developed by the lecturer adds to the scenario a number of fictional events or facts. From that moment on an alternative world (to the one we call the real world) emerges from the responses and activities of the roles - thus new events created by the students provide the context for learning. As new events are created, students are confronted with new problems to evaluate and act upon. Their responses then need re-valuation once they become aware of the effects (both desired and undesired, and intended and unintended), through the actions of others. Mistakes become part of this learning experience. They are not so much to be avoided, as learned from, and the lessons applied to newly emerging situations and contexts.
Learning Design Architecture
Following from this then, the four essential ingredients of this learning and teaching architecture are:
First, goal-based learning is acknowledged as a strong motivator of learning. Typically, goal-based learning comprises a scenario or context, which includes a trigger or a precipitating event. This event may be presented as a critical event and usually requires an immediate response from students.
The second critical ingredient of this learning architecture is role-play, both in the sense of playing a role, playing with possibilities and alternative worlds, and playing to "have fun". The strategy of learning through playing is significant, not the least because 'having fun' in the process of learning is an extremely useful motivator. More importantly, it gives students a personal stake in theproceedings. A distinction is drawn, sometimes between a "simulation" and a "game". A game will have a sense of "winning" or "losing". The work described here is a "simulation" in that at the end of the activities, there is no "game to win or lose". Students in this web-based simulation are organized into teams playing particular roles. Students play out their roles within the context of the given crises or situation. In order to play out their roles effectively they need to do research. Data for this research is available via a large number of links on the role-play website but it is also necessary for students to do traditional library research as well as attend lectures and tutorials. The provision of resources by this mechanism serves to simplify the simulation generator software in that no elaborated schema is necessary to classify the resources to provide "resources on-demand".
This simulation is designed to create a safe and authentic environment to situate student learning in the area of political science. It has sufficient richness in it to reflect the complexity and authenticity of the "real world". The "authenticity" in the simulation is necessary in order to ensure that there is a "personal stake" in the decisions taken in the simulation. However, it is particularly important to recognize that some students could suffer intense psychological stress during the simulation exercise because of the roles they play. Students ought to know that they are able to "escape" from this artificial world and return to the "real world". The simulation generator used for this simulation makes this possible. It provides a clear separation of the simulated from the real world. This is considered to be an important contribution of this simulation generator in comparison with the use of generic email or text-based conferencing systems. This escape from, and re-entry into the simulated world is an important element for situating learning by providing distinctly different environments for experiential learning and reflective thinking.
The third critical ingredient of this learning architecture is the Web. The Web houses the virtual space for the role-play, enables communication and collaboration among students, and between the students and the lecturers. The Web also enables access to "just-in-time" resources by making available to students resources (such as up-to-date news from electronic newspapers and web-sites etc.), from all over the world as and when they need them. Without this capability the content of the role-play would be significantly weaker.
The fourth critical ingredient is the traditional face-to-face lectures and tutorials. Many experienced on-line educators (Price, 1998; Hedberg & Harper, 1998; Brown, 1998; Durham, 1998) have emphasized the importance of including face to face interaction in teaching with the aid of computer mediated communication and online teaching. The importance of incorporation of these techniques into the learning architecture is critical to the presentation of facts, cases and theories. They also provide communicative events that stimulate reflection about actions undertaken and strategies pursued by comparing real world events with the simulated ones.
Technological architecture and modeling of the simulation
A "Role-Play Simulation Generator" generates the role-play described in this paper. The role-play simulation generator (RPSG) provides a structured learning environment to support particular areas of study such as "World Politics in Transition". Indeed, it has applicability in all areas of learning where simulation of roles, problem solving, communication and interaction among participants is involved. The role-play simulation is based on the following foundations of learning (see Figure 1).
These four foundations of sound learning define the critical attributes of the RPSG (i.e., the Role-Play Simulation Engine encapsulated in the rectangle in Figure 1), and the design of the Role-Play simulation. They also provide the creator of a simulation a sound pedagogical framework for building a role-play simulation. This comprises defining the learning tasks (scaffolding), identifying the relevant information (resources), defining the sim-conferences (interactions), and specifying the roles of participants and their rights to the conferences (social structure).
The role play simulation generator enables the creator of the simulation to specify the roles which are central to the operation and success of the role-play simulation. The generator also enables the simulation creator to define tasks, create sim-conferences, and assigned rights to participants in these conferences, as well as provide specific information and scaffolds to support the simulation. For example, a simulation creator can send sim-mail at the appropriate time to support a learner, or set tasks to guide the learners towards achieving specific learning goals. This is an example of scaffolding. Interactions between the learners are communicative events.
This simulation is driven by a variety of “tasks”. The composer can set “tasks” for specific roles. These tasks can have time limits. When a role acts on a task, the output of that action may become a task for other roles. These tasks can serve as scaffolding for the students guiding them progressively towards the final overall goal of the learning experience. When necessary, these tasks can also be used for assessment purposes.
Roles and participants
In the Role-Play Simulation Generator, the lecturer who creates the simulation is denoted as "simulation composer" (composer for short), in the simulation. The composer creates World(s) and Role(s). All Worlds of this simulation have the same role-structure. The term "World" refers to the grouping of participants into smaller groups instead of running a simulation with many more roles. The lecturer, acting as composer, sets up a participant list, groups the participants into different Worlds and then assigns the participants different roles. Participants play the simulation via the assigned roles. Freeman and Capper (1999) have argued for the need for anonymity in simulations. Accordingly, the real identity of individual participants in the simulation is not revealed. If there is collaboration outside the simulation (e.g., having more than one participant playing the same role within a World, such as is the case in "World Politics in Transition" simulation described here), the simulation does not distinguish between different participants playing the same role. Private communication between participants playing the same role can take place via the "NotePad".
A "simulation conductor" and a "simulation controller" support the composer in this work (conductor and controller in short, respectively). The simulation controller is a tutor or a lecturer who is able to modify the information presented for the roles, monitor the messages and jump in to support particular roles, if necessary. The simulation conductor, on the other hand, is a hidden role in certain World(s). The conductor can monitor only selected activities in the assigned Worlds. Different roles have different rights in the conferences. The conductor's role, for instance has the ability to read all messages within the World where the conductor lives. However, the conductor does not exist in the message recipient list and is therefore "invisible" to all the other roles. The participation of the conductor in conferences is subjected to rights set by the composer.
The Role-Play Simulation Generator is based on the abstraction that human interactions are communicative events requiring information exchange. By providing a safe, controlled and authentic simulated communication environment, students can play different roles in a complex social situation. There are four types of information interacting in the simulation:
Figure 2 shows the welcome screen of the "World Politics in Transition" simulation. Information provided by the composer is shown in the main frame with all the available conferences on the left-hand frame, and student's tools below the main frame. The copyright notification is at the bottom frame.
Interaction types and rights
Roles in the simulation are played via a messaging system that utilizes e-mail communication between participants, as well as general-purpose messaging to roles within the same World and/or conferences. The general-purpose message is very similar to email. However, since it is not a "real" email system, roles can only communicate with other roles by selecting role names in the recipient list. The real name of individual participants is not recognized by the mailing system, although the conductor can monitor all messages passing through the general-purpose message system. The simulation mail system operates within the context of the role-play simulation. Unlike other generic email-based simulations, the simulator removes the risk of confusing the role with real life, which is an important aspect of this simulation. The anonymity of participants is enforced by the simulation mail system automatically. The conference is implemented to reflect the various kinds of forums found in politics, commercial and other environments.
The conferences in the simulation have a special feature that is not found in most conferencing software, and this incorporates the notion of different document types. For instance, it is possible to set up a particular "News Agency" as a conference in which there are three types of documents: draft, submitted and news. Every role in a World may have "read" rights to the document type labeled as news. In addition, there may be several roles called "Reporter A", "Reporter B" etc. who will have "read", "write" and "create" rights for the document type called draft. Reporters also have the right to convert drafts into submitted form. "Reporters" can work on their drafts, discuss such drafts among the reporters without any other roles looking at the draft. When satisfied and/or agreed upon among the reporters, the draft may be converted to submitted form. The role of "Editor" may have "read" and "write" access (but not "create" rights) to submitted form. Hence, the Editor can only work on the submitted document. The Editor may also have the right to convert the document type submitted to news effectively broadcasting the news to the rest of the World.
We have created many conferences in this simulation including two simulated news services: BBC and CNN (see Figure 3). It shows the two document types within the BBC conference area. To exchange information, the role players used the document type called "Memo". All roles have read rights to the document type called "Article".
Using RPSG for simulation generation
The task of setting up of Web-based simulations has proven to be beyond the interest and technical ability of many, if not most teaching academics. There are many reasons for this. Some of the reasons for this are lack of sufficient technical skills, and the time it takes to acquire the necessary skills. The Role-Play Simulation Generator enables educators to design and implement a web-based simulation as easily as navigating through a Website. Figure 4 is a screen shot of the simulation generator showing how to set up the welcome screen's graphical information, and general setup and copyright message in the simulation generator.
The Role-Play Simulation Generator is built on a sound pedagogical foundation. It enables the simulation generator to design learning tasks that are in line with their intended learning goals. The use of the Simulation Generator empowers innovative teachers to experiment with creating meaningful role-play simulations and getting their students to interact, collaborate, discuss, lobby and practice the skills and theories demanded by their field of study. It empowers educators by reducing their dependence on the technical elements of employing Web-based role-play simulations.
In the following section of this paper, summative evaluation data is presented from a case study application of this role-play simulation. An end-of term questionnaire was administered to the students in the World Politics in Transition Simulation to seek feedback on their learning experience from using this role-play simulation. The students in this subject participated in this simulation in lieu of a formal examination. The assessment weighting for work on the simulation comprised: 25% on the writing of the role profile, 25% on the degree of participation, 25% on the quality of participation, and 25% on the final role summary. A selection of the questions asked about their experience, and their responses to these questions are presented in the following.
The perceived effectiveness of the simulation in enabling students to achieve the specified learning outcomes was found to be generally high by the students. Table 1 shows the questions asked, and the average of the responses on a scale of 1 to 5 where 1 is "very useful" and 5 is "useless".
Table 1. Student responses towards the achievement of learning outcomes
On this scale, the average rating for questions relating to the understanding and identifying of facts, issues, problems and factors, was between 1.34 to 1.92 which is in the "very useful" to "useful" region.
It is interesting to note that on another question that the simulation added "active and dynamic dimensions to classroom learning processes", the average was an overwhelming 1.08 indicating that the students welcomed the new innovative learning environment.
Table 2 shows how the simulation was perceived by the students as an instrument for learning on a 4-point scale, where 1 is "strongly disagree" and 4 is "strongly agree".
Table 2. The simulation as perceived by the students
On the whole, students reported finding the simulation very useful in various ways. The issue about the writing of "the position papers" which is rated 2.95, is surprising as the initial position papers were written before the start of the simulation.
Time spent on the simulation
Most students suggested having spent a lot of time on the simulation. This is reflected in Figure 5.
The difference between time spent by those with Internet access at home and those without was subjected to a t-test which revealed a t value of 2.1447 (df=14) suggesting significant difference between those with Internet access at home and those without (p<0.05). Those with Internet access at home spent significantly more time on the simulation than those without. Due to the way the questionnaire was administered, it was not possible to link the time spent on the simulation with their overall performance on the subject.
Students were given the opportunity to comment more freely, among other things, on the usefulness of web-based simulation in supporting learning. Some of the responses to the critical implications of web-based collaborative learning are presented in the following.
Question. Would the use of a web-based simulation such as this influence your decision to take a course, if so why? If not why?
In this paper we have articulated the theoretical basis of a dynamic goal-based role-play simulation utilizing the collaborative learning capability of the Web. The subject of this simulation was World Politics in Transition. Results of the summative evaluation process have shown that the move from the traditional lectures (including seminars, tutorials, paper-based examinations, essay writing and reliance on printed books and articles) to this simulation significantly transformed the learning and teaching processes in this course in a number of ways. Although the students were not explicitly aware of the pedagogical design behind the simulation, the overall experience has been very positive. Firstly, it has brought students to the center of the learning process rather than putting them in passive and receptive roles. Secondly, it has transformed the way students and teachers carry out teaching and learning by emphasizing communication and collaboration rather than individual activity. Thirdly it has allowed for flexibility in the delivery of material in terms of the number of participants, the timing and spatial location of the teaching and learning process. And fourthly, it has taught everyone new skills and competencies, not only about teaching and learning but communication and collaboration. For the teaching academic, the simulation generator has transformed the previously tedious, and technically complicated process of creating a simulation in order to transform learning into a goal-directed and fun activity.
This project has been supported financially by "The Teaching, Learning and Multimedia Educational Technologies Committee" of The University of Melbourne, Australia.