Developing on-line learning materials for higher education: An overview of current issues
Director, Educational Media Systems
College of Engineering and Architecture
Washington State University, Pullman, Washington, USA
Tel: (1) 509 305 4053
Fax: (1) 509 335 9608
Nils S. Peterson
Director, Educational Technology Unit
College of Education
Washington State University, Pullman, Washington, USA
Emily C. Johnson
Assistant Professor, Department of Kinesiology and Leisure Studies
College of Education
Washington State University, Pullman, Washington, USA
* Manuscript received Jan. 01, 1999; revised Mar. 11, 1999
The combined effects of increased faculty workloads, changing learner needs, and improved instructional technologies have resulted in increased demands for on-line learning materials. Numerous approaches have been taken to develop these materials, but with limited success in fulfilling the need for on-line learning materials (Carswell & Murphy, 1994).
Initially, the use of computer-assisted instructional materials (CAI) to enhance traditional teaching was a novel concept. However, increasing pressures at all levels of education perpetuated a need for time-efficient, effective teaching modalities that maintained the quality of teaching. CAI were considered to be a viable solution to these problems (Grieves, 1992; McDonough, et al. 1994a,b; Marshall, et al. 1995; Misner, et al. 1992; Nahata, 1992; http://www.ncteam.ac.uk/tltp/tltsn/cases.html.) so adventuresome instructors invested many hours in developing CAI materials (Carew, et al. 1985; Dori & Yochim, 1994; Grieve 1992; Lilienfield & Broering, 1994; Marshall, et al. 1995; McDonough et al. 1994(a, b); Misner et al. 1992; Nahata,1992; Peterson & Campbell 1985; Teyler & Voneida, 1992; Walaszek & Doull, 1985; http://www.ncteam.ac.uk/tltp/tltsn/cases.html).
Offering CAI over the World Wide Web (WWW) has become the most recent option in computer-assisted instruction. With combined emphases on learner-centered and distance education, development of multimedia-rich CAI that can be accessed over the WWW has become not only an attractive and creative option for university professors, it is now a suggested or even required course of action. CAI have the potential to serve a dual purpose by enhancing the learning experience for resident students, while opening the educational experience up to distance students. As Ellen Julian of the International Data Corporation Independent Research Organization says – "There is an "explosion" of interest from colleges for WBT (web-based training)".
The suggestion or request to develop computer-based training materials may originate from a state's Board of Regents, the State Legislature, a university's administrative community, or others. One specific plan at Washington State University (WSU) in Pullman, Washington, United States, has had university funding withheld pending successful achievement in targeted areas that emphasize the use of instructional technologies.
The technology-related performance measures at WSU include increasing the number of:
Part of the strategy for selecting an increased use of educational technologies as one of the accountability areas is its potential to also improve faculty and student performance. Similar situations are occurring at many institutions.
Yet, an abundance of high-quality CAI, designed in a modular fashion to promote widespread reuse and coordination of the existing materials does not currently exist. One need only compare the volume of CAI materials available to choose from to the volume of printed text materials for the same topic to make this point very clear. Therefore, instructors interested in using CAI may either wait until a satisfactory selection of CAI materials become available, or engage in developing such materials.
This paper is organized around issues that are relevant to the increased interest in developing on-line learning materials. The goals of this paper are to:
Goal 1: Present a discussion of the resources required to produce effective on-line learning materials at universities
One of the largest obstacles to producing multimedia-rich learning materials is the cost (Blanchaer, 1985; Carew 1985; Carswell & Murphy, 1994; Golas, 1993; Hutchings, et al. 1994; Marshall, Samson & Dugard, 1994; Marshall, Samson & Dugard, 1995; McDonough, et al. 1994; Siviter, Linecar, & Siviter, 1994; Templeton, 1985). These materials are very resource-intensive to develop (Golas, 1993; Marshall, Samson & Dugard, 1994; Marshall, Samson & Dugard, 1995; McCormack & Jones, 1998; Porter, 1997) because they require time, personnel, and equipment. Figure 1 is a compilation of many different researchers' attempts to quantify the time needed to develop a single hour of CAI (see Figure 1). The instructional goal of the learning material may affect development time as well (Golas, 1993). Development time increases as the learning goals for the materials ascend Bloom's learning taxonomy (http://www.wested.org/tie/dlrn/blooms.html) from knowledge to skill to attitude and as the technical complexity of the computer work increases from basic, to intermediate, to high (Table 1) (Golas 1993). Obviously, many more hours would be required to develop an entire course.
Figure 1. For each series, the legend lists the total number of hours of CAI developed by each investigator and the investigator's name (Marshall et al., 1994).
Table 2 provides a more applied version of the information contained in Table 1. Verbs associated with the individual learning goals are provided, as are examples of computer activities that may be included at the intersection of each learning goal and level of technical complexity. Computer-mediated communication tools, (such as group collaboration tools, discussion groups, and chat rooms) and computer-based assessment tools may fit at all levels of depending on the instructional design of the learning activity.
Many variables affect the development process. The amount of experience the developers have, the total number of hours of instructional materials developed, the complexity of the learning goals for the materials, as well as the technical complexity of the electronic materials, all affect development time. Throughout many reports, several trends are very clear:
Considering the time estimates above, increasing the number of technology-based courses and courses available to off-campus students presents two major problems: 1) providing the time, people, and equipment to complete a huge volume of work; and 2) finding the money to cover the costs of these resources. Both of these issues represent immediate concerns to a university. Resources that are redirected toward CAI represent a reduction of support in other areas, such as science research laboratories or resident teaching laboratories, for a couple of examples. Therefore, maximizing the effective use of resources and minimizing the cost of developing CAI, while still achieving instructional objectives, is crucial. Allocating resources to developing on-line learning materials may also raise political issues within and between academic units as some may view CAI as valuable, while others may not.
Goal 2: List the shortcomings associated with faculty attempting to develop CAI
Many universities have tried to increase usage of educational technologies by providing workshops and technical consultation to faculty to assist them in reengineering their own courses to use technology. However, this approach has had limited success (Carswell & Murphy, 1994). Expecting faculty to convert their own teaching materials into a media-rich, computer-based format is unrealistic because:
The result of this approach at best is an inferior product at an escalated price, or more frequently than not, it results in no product at all. Because the institution cannot give faculty sufficient release time to develop computer-based learning materials, even faculty with advanced technical skills find it difficult to develop CAI (Blanchaer, 1985; Carew, 1985; Carswell & Murphy, 1994; Hutchings, et al. 1994; Marshall, Samson & Dugard, 1995; McDonough, et al. 1994; Templeton, 1985).
Goal 3: Describe the changing role for faculty who are implementing virtual teaching materials
CAI may augment university teaching for both resident and distant students alike. In either case, implementing computer-based training materials into curricula can alter the role of faculty considerably (Brooks, 1997; Carswell & Murphy, 1994; McCormack & Jones, 1998), especially if the courseware materials completely replace the traditional classroom laboratory or lecture experiences. However, continued faculty involvement in the course delivery and preparation is crucial. Regardless of the delivery mode, updating teaching materials should be a continuous and ongoing process for faculty. Faculty content experts should continue to edit and revise the CAI even after they have used them to teach a course, in the same manner they should revise and edit their lecture notes in a traditional teaching scenario (Porter, 1997). Because on-line materials are never really finalized the way printed materials are they allow a maximal plasticity that is ideally suited to ongoing updates and revisions. This is another advantage of on-line materials compared to texts, videos, compact discs, or other hardcopy forms of courseware materials.
If an instructor, who uses structured CAI within a course, continues to hold traditional meeting sessions with students, the classroom time can mature into an enriched experience (Brooks 1997; Kommers, 1996; McCormack & Jones, 1998) which benefits students more by allowing more interaction between faculty and peers than in the traditional classroom. In traditional classroom settings, students and faculty often do not interact, rather they sit (as if separated by miles), and write down words describing a linear presentation of materials. In the computer-assisted teaching model, students could access computer-based materials during their course preparatory time, and class time could be reserved for conceptual discussions, peer interactions and mentoring. Rather than spending time making linear presentations of lecture material, instructors could implement creative teaching strategies in the classroom (Brooks, 1997; Kommers, 1996; Porter, 1997).
Time liberated by replacing the most didactic lectures with computer-delivered lectures may benefit faculty, students, and universities in several ways (Brooks, 1997; Kommers, 1996; Porter, 1997). For example, it may allow faculty time to facilitate more learner-centered activities by promoting interaction between peers and between students and faculty. The virtual instructor can promote peer to peer interaction by involving students in scholarly debates in the discussion group and chat room (Porter, 1997), or by designing group assignments to be completed by students in the virtual space. Peer to peer interaction gives students a chance to discuss their opinions with others. Discussions in the chat rooms often move students beyond just trying to formulate a correct response into higher order thinking as they discuss why they think one answer is more correct than another, and defend their ideas with other. In the virtual environment, a large number of students can be "intellectually linked", without having to be "geographically linked". Furthermore, students that may not be inclined to participate in class often participate in the on-line environment. The time saved by using CAI may also be used by faculty to update the course materials, complete research, write grants, mentor students, or pursue other critical scholarly activities, all of which ultimately benefit the university.
The importance of maintaining the "human interaction" component in an on-line teaching and learning environment cannot be overemphasized. Interactions between instructors and students, as well as peer interactions between students, are requisite to facilitating critical thinking and promoting enriched learning.
Goal 4: Provide partial solutions to these problems based on empirical findings from one model (a specific university computer-based courseware development laboratory at WSU)
Given the quest for computer-based learning materials and the general lack of universally acceptable, computer-assisted materials, some universities are seeking alternative courses of action for developing CAI. A pragmatic method is needed that capitalizes on faculty expertise, is realistic for the instructors and universities, and which balances cost against yield (Carswell & Murphy, 1994).
As one example of an alternative method, WSU's College of Engineering and Architecture hosts a student-staffed courseware conversion laboratory (Educational Media Systems, EMS) that is successfully developing educationally stimulating, on-line learning materials, without requiring faculty to enter data into the computer. In this model, instructors acquire an understanding of the capabilities of instructional technologies so they may assist the laboratory development team in determining how to effectively present course content via technology, (Carswell & Murphy, 1994), provide detailed content for the courseware conversion projects (Carswell & Murphy, 1994) and confer with the laboratory director in determining the instructional design for the project. The development team decide which technologies best suit the application. The laboratory director supervises the undergraduate student workers as they complete the technical component of converting a course from the traditional format to a virtual, web-based format. The instructors, laboratory director, and student technicians confer on a regular basis to assure that content integrity is preserved throughout the CAI development process.
Development of CAI typically includes several stages (Siviter, Linecar, & Siviter, 1994). Most development schemes share the general sequence of analysis, design, production and installation, with evaluations at various stages (Bostock 1996). A brief outline of the EMS development team's CAI development stages is provided below. In this model, course instructors provide subject matter expertise; the EMS lab Director is the pedagogy expert, and the EMS lab Director and student technicians serve jointly as technical experts.
Stage 1: Courseware Specification
Stage 2: Scope of Project (to avoid digression of original focus and feature creep)
Stage 3: Resource Summary (specification of all activities that consume resources)
Stage 4: Content Development
Stage 5: Storyboard Templates
Stage 6: Rapid Prototype
Stage 7: Courseware Integration
Stage 8: Electronic Storyboard Edit
Stage 9: Rapid Prototype Test
Stage 10: Adjustments and Mass Production
Stage 11: Full-scale implementation
Stage 12: Maintenance
There are many advantages to forming a multimedia courseware development laboratory. Namely, faculty may contribute their expertise as content developers (Carswell & Murphy, 1994), and capitalization costs for all computer and multimedia equipment are spread out over many projects, thus reducing the cost per project (Porter, 1997). Also, a team of individuals working together develops an extensive collection of skills and abilities that exceed those of an individual or a small group of individuals completing one or two projects in isolation. This collective skill base enables the development team to create educationally stimulating learning materials. Another distinct advantage is that the laboratory's collection of reusable software tools expands over time, resulting in saving time on subsequent projects. Furthermore, ideas and methodologies associated with one CAI may stimulate ideas for other projects.
Using student employees to implement the courseware conversion projects is a mutually beneficial situation. It benefits the development effort because many students possess the technical skills required to create educationally stimulating learning materials, and because students are paid at a lower hourly rate compared to faculty, staff, or professional courseware developers. It benefits the student employees because they gain diverse experiences and cultivate an interdisciplinary perspective via involvement in the technical, educational, and humanitarian components of the project. They also benefit from working in a "team" environment. The combination of their academic and work experiences enhances their future employability in positions that highlight their unique set of skills.
One key component to the success of the EMS lab's student-employee model is its location in the College of Engineering and Architecture. Most of the student employees are majoring in Computer Science, Electrical Engineering, or Management Information Systems. As such, they complete coursework that directly benefits their work in EMS, such as programming, graphic design and interface design classes. Student employees are generally familiar with more than one programming language based on their coursework, then get practical programming experience in the EMS lab. They also add numerous valuable skills to their repertoire during their work time in EMS, such as streaming media, animating, system administrating, etc. Work in the EMS lab constitutes "working in the major" for these students, and enhances their placement in industry after graduation.
The student employees are the "technical experts" for EMS' streaming media, system administration, animation, Java, and active server page (ASP), and all other projects. The development, administration, support and maintenance of the CAI is completed by undergraduate students, with management by the lab director.
Another important benefit of employing students from Computer Science, Electrical Engineering, or Management Information Systems majors is their awareness of the cutting edge products in computer technology. The EMS lab is always scouting out the newest technology and Beta (pre-release) versions of software programs to expand the functionality and pedagogical capabilities of the courseware materials. A downside to employing the newest technology is that developing with cutting edge technologies may produce courseware materials that require more powerful client machines than many of the courseware users will have at the time of delivery.
Another advantage of employing student workers is their high energy levels and quest to succeed. The student employees are usually keen to tackle new tasks, which is good because the requests for CAI are high. In many classes, the lab is developing materials only several weeks ahead of when the actual class is accessing them. Student employees accept challenges like these enthusiastically, which keeps the lab maximally productive and moving full speed ahead.
The preceding EMS model is presented as a partial solution to resolving the need for CAI in higher education. Many crucial components and services must also be provided by institutions to succeed as distance education providers. Instructors, technicians, and university administrators must form a team to promote distance education. The development team must determine the hardware and software to be used to best represent a particular course, and the instructional methods to be employed. University administrators must support the distance courses by allocating annual line items to the appropriate budgets (Porter, 1997).
In spite of its many advantages, there are significant drawbacks associated with the student model compared to a traditional, permanent staff model. Because the EMS lab actually serves the courses it develops, the student employees are in a position with high expectations and responsibility. Their performance in EMS determines whether or not several thousand of their peers will have access to university course materials. This can be difficult because the student technicians carry full class loads, and engage in social activities. Therefore they must juggle their schedules between class time, study time, work time, and social time. Students work a maximum of twenty hours per week, but during weeks when they have heavy exam or project schedules, or family or friends are visiting, they diminish their work hours to accommodate the academic and social demands.
In a non-student-staffed distance education development laboratory, school vacations would traditionally represent a time to "catch up" on projects and "make headway" while students are away. In the student model, most of the lab employees leave during the vacation times as well. This scenario can have its greatest impact between semesters. For example, Christmas break represents a time when the lab needs to be very productive getting new classes online, debugging new applications that are loaded between semesters, invalidating student accounts from the previous semester, archiving old discussion groups and setting up new ones, completing revisions to class materials, etc. But, time slip student employees are not obligated to work holidays, and may return to campus just in time for classes to begin. Therefore, there is inevitably a hectic period trying to get prepared for each new semester.
In practical application, many of the EMS student employees have a sense of responsibility for their work in EMS, and end up working holidays, and often overextend themselves during the week to keep up with school work and EMS work. In fact, many of the students complete homework assignments in the lab so they are available if needed, and most are available by email or phone at home. Remember, each student is the technical expert for at least one function in the lab. Therefore, the employee is not backed up by a staff person with similar technical expertise. As may be surmised, this has its down side. Sometimes the heavy responsibility of their EMS work becomes too great for students to cope with in addition to their academic and social demands, resulting in the lab not being able to satisfy every demand placed on it.
Another large cost of employing students is the loss of skills and experience the lab suffers every time a student graduates. Because the EMS lab is predominately staffed by student employees, the workforce is in a constant state of flux. This situation requires some special management strategies to avoid losing skills and expertise in the development team as a whole when one student graduates and leaves employment of the EMS lab. It is important to build a tier of employee's skills. For example, at any one time there is one student who is the primary JAVA expert, and another students who is devoting part of their paid work time under studying the Java programmer. If the timing is optimal, the understudy is a competent Java programmer when the senior Java programmer leaves.
Another management strategy to assure a smooth the transition between employees is to ask student employees to file detailed procedures reports of their EMS work activities and provide documentation for their projects so a new student employee may take over the project. Comprehensive documentation is a good practice in any lab, regardless of the staffing model, so this is not a unique cost for the student staff model. However, permanent staff skills and experience carry over for extended periods of time, which is a distinct advantage. Fortunately, most of the students, who work in EMS, continue working there until they graduate, which usually amounts to more than two years of EMS service.
Goal 5: Extend these empirical bases to include some strategies.
One important strategy for economizing the development of computer-based training materials is to select appropriate courses to convert. Choices include courses that:
Specific selection criteria may vary between institutions, and may include more than just course selection criteria. In addition to selecting the appropriate courses to convert, there are other strategies that may help defray development costs. For example, the following strategies may need to be pursued:
A discussion of each follows:
An instructor taking leave from their position for a year usually is paid less than their full salary. The balance of their salary may range in value from $10,000 to $60,000 depending on annual rate of pay and leave time rate of pay. A visiting instructor usually will be hired to replace the regular instructor in the interim. The salary savings by not hiring a visiting instructor combined with the differential between normal salary and leave salary of the regular instructor may be sufficient to cover the cost of developing CAI for the course(s). This process will assure continuity of course content during the sabbatical year because the regular instructor's materials will serve as content to generate the computer-based teaching materials.
If computer-based teaching materials may be used to temporarily satisfy a portion of an instructor's responsibilities for a semester or two, the next question may be, "If the computer-based learning materials can replace faculty for a semester, can they replace them permanently?" (Carswell & Murphy, 1994). The answer is "No". Updating teaching materials is a continuous and ongoing process for faculty, regardless of the delivery mode. Failure to maintain faculty involvement would critically undermine the academic integrity of the teaching materials. Furthermore, only faculty can provide mentoring, respond to student questions on an individualized basis, and conduct scholarly research.
Students enrolled at universities not offering certain courses may elect to complete those courses if the courses are offered by another institution over the world wide web (WWW). In the on-line learning environment, virtual instructors can enroll students located anywhere with Internet access. Students can meet with their professor and fellow class members in on-line classrooms, and access multimedia-rich on-line materials over the WWW. The institution that offers the class will benefit from the additional full time equivalencies (FTE) generated from enrollment in the virtual class.
There are numerous multimedia classes being taught across the United States. A beneficial class project may consist of a courseware conversion. The goals of such a class may be for the students to acquire multimedia skills and to begin the courseware conversion process for a single class. Materials produced by the multimedia class students could be embellished by more experienced technicians (such as the students working in the courseware conversion factory), and applied toward making a complete courseware conversion.
Another way to stretch development monies is to have many projects completed by the same development team, who are already skilled at working together. This situation maximizes the ability to coordinate reuse of items already developed. These items include interactive figures and graphs which dynamically change one variable while a student manipulates the value of another variable, and other scripts or codes written by the lab, as well as complete modules in some cases. If course materials are designed to maximize modularity, all reusable portions can be extracted for use in other projects. For example, consider how many different courses could benefit from a module covering the First Law of Thermodynamics, (physics, biochemistry, electrical and mechanical engineering, to mention only a few disciplines).
There are several methods described in the literature that delineate the step by step procedures for developing multimedia projects. Following one of them lessens the likelihood of omitting crucial and time saving steps and reduces the probability of a developer repeating courseware development mistakes made by others.
Multimedia courseware development is a time-, personnel-, and resource-intensive process. The more multimedia-rich a project becomes, the more resource intensive it also becomes. For these reasons, developing highly interactive, multimedia materials may not be the option chosen by some individuals developing CAI. Less multimedia rich materials may be a viable option and are less resource-intensive to produce.
Commercial developers have gained interest in the courseware conversion process and commercial courseware products and courseware conversion services are becoming available. Any attempt to produce materials that represent university curricula will still necessarily involve university faculty in order to maintain the integrity of the learning materials. Issues concerning ownership of materials produced will need to be resolved if commercial companies are to develop on-line learning materials for universities.
Commercial companies may be able to produce the materials at a lower cost compared to universities based on a higher volume business. In the case of an American company, Real Education (http://www.realeducation.com/), the advertised development cost is quite low, but costs are re-couped by ongoing administrative fees charged per student on a credit-hour basis. Additionally, the offering by Real Education is designed to capture the intellectual property rights for the course materials, allowing the company to re-market the course or its derivatives to other institutions for additional student fees. The implication of Real Education's model of assessing student fees per course hour is that tuition presently paid by students to the university will be transferred to Real Education. Therefore, either tuition must be raised by the amount of Real Education's fee, or a portion of students' tuition, which has formerly gone to supporting the institution's operations will be lost for that purpose.
Still other strategies to support the development of CAI need to be explored. One approach is to obtain grant money for the development of electronic learning materials. Faculty content experts may collaborate with educational technology experts in preparing grant proposals. The investigators may produce a prototypic CAI module to submit with the proposal, demonstrating to the funding agency that the investigators already have the facilities in place to produce CAI.
Another strategy is for universities to move to a new structure for dispersing central university funds away from the usual disciplinary approach, and toward an interdisciplinary approach that rewards all units required to successfully complete courseware conversion projects. Funds may need to be allocated to a content expert in one unit, a courseware conversion laboratory in another unit, and possibly other central units such as extended degree programs and information technology units to help manage the overall infrastructure for delivering distance education courses.
Universities have demonstrated great interest in developing on-line learning materials. University faculty members are the logical choice to provide subject matter expertise and instructional design. However, university faculty should not be expected to complete the technical work of building the courseware because they must meet numerous other performance measures, and may lack critical technical skills.
Developing on-line learning materials is very costly. It may be advantageous for universities to form courseware development laboratories because they consolidate hardware and software resource requirements, technical skills and, reusable features. Furthermore, employing students as computer technicians in the courseware conversion laboratory benefits the students by enhancing future employability after completion of their degree.
Implementation of multimedia-rich computer-assisted instructional materials into curricula can enhance the learning experience. Well-developed, high-quality multimedia materials can stimulate different learners based on their preferred learning styles, as well as expand distance learning opportunities. One important point is that the computer-based learning materials are not designed to replace faculty, or even reduce faculty numbers. On-line learning materials require revision and updating every time they are delivered. The faculty's new role with increased technology will be to keep the information updated and current in the computer-based materials, and to enhance their mentoring and research commitments with the time liberated by replacing stand up lectures.
A special expression of appreciation is extended to two key people for having the insight, vision, and energy to create the Educational Media Systems Lab at WSU.