The Effects of a Computer-Based Instructional Management System on Student Communications in a Distance Learning Environment
#8 Monroe Hall, Southwest Missouri State University
901 South National, Springfield
Missouri 65804 USA
Kenneth C. Holloway Ed.D.
Southwest Missouri State University, USA
Sandra K. Wegner Ed.D.
Professor and Associate Dean for the College of Education
Southwest Missouri State University, USA
* Manuscript received February 2, 1999; revised June 6, 1999
With major universities offering distance learning courses in over 75% of the states (Charp, 1998), and estimates that by the year 2007 almost 50% of all learners in post-secondary education will be enrolled in some type of distance learning (Kascus, 1997), it is safe to state that distance learning and especially Internet-based distance learning has "arrived". Internet-based delivery of higher education coursework has undergone unbelievable growth in the past half-decade and it appears that it will enjoy continued growth as well as a measure of longevity. Once a simple communication tool for the military and selected universities, the Internet has expanded its influence to all levels of the educational hierarchy and as Plotnik (1996) points out, will only continue to maintain its general upsweep.
The growth and current stature of distance learning have not come without some concerns. Sherry (1996) noted that technology has often been pursued more for its novelty than for the more compelling reason of enhancing educational experiences. Equal access, appropriate use of multimedia, changing roles of teachers and students and effects of technology on content, skills and attitudes are often lost in return for a "forward looking" institutional image.
Specific concerns regarding distance learning opportunities have been addressed throughout the literature. Lack of technological expertise on the part of both teacher and student (Wegner, Holloway and Crader, 1997), resistance to change on the part of faculty (Parrott, 1995), student passivity (Filipczak, 1995), hardware limitations (Kerka, 1996) and learner isolation (Kubala, 1998) are but a few of the perceived drawbacks to distance learning. While these are all legitimate areas of concern, for the most part, they relate to training and technology issues that have had fairly obvious remedies. Some concerns, however, still linger.
Many practitioners decry the lack of research concerning the potential effects of distance learning on the educational opportunities experienced by students (Neal, Ramsay and Preece, 1997). While several early studies (Nixon, 1992; Sercy et al., 1993) examined telecommunication-based courses versus traditional formats (finding no significant differences in student grade point averages in either format), there, unfortunately, has been very little data generated concerning the quantity or quality of student learning opportunities.
Central to the concern of quality learning is the assurance of ample opportunity for communication. Communication is fundamental to effective instruction (Barnes and Lowery, 1998). Effective communication and student-instructor interaction has been found to enhance the learning process (Kozma, Belle and Williams, 1978) and improve problem-solving abilities in adults (Lee, 1990). Communication is indeed basic to the instructional process.
Distance learning systems, especially Internet-based systems, obviously cannot compete with face to face communication opportunities. Therefore, for instructional integrity to be maintained, alternatives must be found which adequately address the instructional requirements for communication and feedback. Communication options available to students engaged in Internet-based instruction are plentiful and increasing at a fast pace. E-mail, discussion pages, list-servs, IRCs (Internet Relay Chat), the Internet, ftp, electronic white boards and Internet-based video and audio conferencing software are but a few of most popular communication options (Tennant, 1996). However, even with access to the many different communication tools, students are often reluctant or incapable of utilizing them. Studentsí lack of technical expertise is the most typical reason for the under-utilization of technology, though inconvenience of accessing and understanding multiple communication software packages is also a serious problem.
To assist students with communication technology, more and more institutions are turning to computer-based Instructional Management Systems (IMSs) for the delivery of content and facilitation of communication. Typically, IMSs have integrated communication packages that allow students communication opportunities without the use of multiple specialized software. Additionally, these systems do not require the student to reconfigure their computer for usage. Commercial versions of such Instructional Management Systems have seen a dramatic increase in the past year with as many as twenty such systems now available. Due to this relative short period of utilization, there has been little research on the effects of IMSs on student learning or communication.
The dilemma facing these researchers was typical of the issues which confront most university instructors utilizing distance learning. Internet-based distance learning is becoming a common method of delivering coursework; becoming an expectation, if not a mandate, on many university campuses. Internet-based distance learning presents several challenges to instructional effectiveness, especially in the area of communication. Past Internet delivery of coursework by these researchers, which relied on separate, "stand alone" communication technologies, produced disappointing results in the number of student interactions and communication opportunities. Would an IMS, utilizing an integrated communication package, produce significant difference in the number of student communications? Would there be significant difference between the number of student uses of e-mail, the Internet or traditional technologies such as the telephone with an IMS as opposed to a traditional Internet-based approach? What additional types of communication behaviors would students utilizing the IMS engage in?
The population selected for this study was comprised of graduate students enrolled in an advanced level, curriculum design and evaluation course over a two-year period of time. The control group was comprised students enrolled in the 1997 Spring and Fall semesters and the experimental group was comprised of students in the 1998 Spring and Fall semesters. Students in the control group (N=14) utilized communication technologies including telephone, e-mail, fax and research web sites. Students enrolled in the experimental group (N=22) utilized a commercially-developed instructional management system called TopClass. The instructional management system included e-mail, connections to Internet research sites and all coursework. An asynchronous discussion area, class announcements page and test-taking site were also features of the IMS. Students in the experimental group were instructed that they could also use the telephone and fax, which were not included in the TopClass software, for communication purposes.
A problem-based model of instruction was utilized by the instructor in both control and experimental groups. Variables of course content, syllabus expectations, course products, text, instructional support, course problems and grading rubrics were held constant for the duration of the study. Control group materials were made available on a departmental web page, while the experimental group course material was part of the TopClass web site.
Problem-based learning. The problem-based learning model of instruction was selected due to its adaptability to the Internet as well as its reliance on student-centered inquiry. Problem-based models have been in place in the medical field for decades and it is speculated that the model is used in well over 80% of all medical schools in the United States (Bridges and Hallinger, 1993). For the purposes of this study problem-based learning was defined as:
"...learning that results from the process of working toward the understanding or resolution of a problem. The problem is encountered first in the learning process and serves as the focus for application of problem solving or reasoning skills, as well as the search for or study of information or knowledge needed to understand the mechanisms responsible for the problem and how it might be resolved." (Barrows and Tamblyn, 1980, p.18)
In generating the problem-based materials for the course, the authors followed recommendations of the research in order to maximize exposure to pre-defined knowledge bases as well as support opportunities for student interaction and use of problem-solving skills. Maximum student autonomy was desired as were authentic experiences resulting in the practical application of theory to real-world problems. A synthesis of the research revealed that for effective problem-based learning to be realized:
Instructional Strategies. A total of three problems was generated for use by both the control and experimental groups. Each problem included a list of focusing questions, product specifications providing a general description of the product to be generated, key terms and concepts related to the problem, a list of resources, and a scoring guide for assessing the final product.
Students in both the control and experimental groups were divided into small groups of two to three for the study, discussion and solution of the problems. Small group selections were based primarily on geographic proximity, though groups with members over thirty miles away from each other were common. The assigned problem-solving groups formed the primary work units for the class though students were encouraged to communicate with all members of the class. At the end of the semester, groups were required to present a discussion of their problem solution process as well as the curriculum products they generated. A required examination comprised of objective, short answer and essay questions was administered at the conclusion of the course.
The range of technological expertise differed greatly within each class but did not appear to present impediments to the instructional process. All students had access to all technologies used in the course and all students had a basic familiarity with the communication software required for the course. To assure a basic technological competency, members of both the control and experimental groups received training in the use of their respective communication tools.
Data concerning student use of communication technologies were collected for both the control and experimental groups throughout the term of the course. Both groups also completed exit surveys. The number of e-mails sent by students to the instructor and telephone calls made to the instructor were collected as the course was conducted. The numbers of student accesses to Internet research sites, as well as general student comments, were obtained by the instructor from post-course evaluation instruments.
Due to the nature of technologies utilized in the experiment, data concerning the number of e-mails between students in each group required separate collection methods. In the IMS, utilized by the experimental group, the instructor had direct entry to student e-mail accounts enrolled and therefore could access the actual number of e-mails between students. Since the control group utilized private e-mail systems, the instructor did not have access to student accounts. Therefore, data concerning communication between students in the control group were collected through survey.
To assess student communication in both the control and experimental group, a tabulation of the number of student utilizations for each communication technology was made. Table 1 reports the descriptive statistics for each of the communication technologies used by the control group. Table 2 reports the same descriptive statistics for the experimental group.
Table 1. Descriptive Statistics for Control Group Communication (N=14)
Table 2. Descriptive Statistics for Experimental Group Communication (N=22)
Analysis of the data in Tables 1 and 2 shows an observable difference in the number of student utilizations of each of the communication technologies which were Internet related. Mean scores for e-mails made to the Instructor were 11.09 for the experimental compared to 2.0 for the control group while e-mail sent between students was 33.0 for the experimental group compared to 3.92 for the control group. Student use of Internet research sites also indicated an appreciable difference. Students in the experimental group reported an average of 11.0 accesses of the Internet while the experimental group reported an average of 28.04.
Traditional technologies had less difference in usage between the two groups. Telephone usage in the control group (mean =2.42) was slightly higher than that of the experimental group (mean =1.45). In the utilization of fax technology, the means of the two groups were almost identical. The control group made minimal use of the fax (mean =0.28) as did the experimental group (mean =0.27). No student in either the control group or experimental group used the fax technology more than once.
E-mail Usage between Students and the Instructor
One of the critical questions researchers wished to address was whether the use of an instructional management system would have any effect on the communication patterns typically used in distance learning. It was the belief of those involved in the study that the ease of use and integrated nature of the IMS would lead to more e-mail communication than had previously been experienced with "stand alone" systems. Initial analysis of the frequencies regarding student-instructor e-mail communication seemed to support this belief. While there were still occurrences where students sent no e-mail to the instructor, the number of contacts through e-mail appeared to be much higher in the experimental group than in the control group.
To determine whether there was any statistical significance between the overall usage of e-mail for student-instructor communication, a t-Test for Independent Samples was administered to the frequency of e-mail of the control and experimental groups. Table 3 reports the results of that test.
Table 3. t-Test for Independent Samples/ Student-Instructor E-mail
The t- value of 3.073 and the probability factor of 0.0021 were sufficient to show a significant difference between the student-instructor email usage of the control and experimental groups at the 0.05 level.
E-mail Usage Between Students
As important to the researchers as communication patterns between students and the instructor were communication patterns between students. E-mail is the most basic and popular of Internet communication technologies. Researchers, as in the case of student-instructor email, speculated that an IMS would have a positive effect on e-mail usage by students communicating with other students. Frequency counts, again, appeared to support this notion. As noted earlier, due to the nature of the e-mail technologies used, data were collected differently for each of the groups. Data for the control group were collected through a survey administered at the end of the course. Data for the experimental group were collected by direct access to student e-mail accounts. While the nature of data collection was a source of concern, these researchers believed that the method of data collection had only minimal effects on the ultimate results.
To determine whether there was statistical significance between the overall usage of e-mail for student-to-student communication, a t-Test for Independent Samples was administered to the frequency of e-mail of the control and experimental groups. Table 4 reports the results of that test.
Table 4. t-Test for Independent Samples/E-mail Between Students
The t- value of 3.7013 and the probability factor of 0.0004 were sufficient to show a significant difference between the student-to-student email usage of the control group and the experimental group at the 0.05 level.
An important part of effective problem solving is accessing data relevant to the solution of the problem. Both the control and experimental groups had access to research sites from a course "Resources" page posted on the Internet. The control groupís resource page was located on a departmental web site, separate from their e-mail accounts and other technologies. The experimental groupís "Resource" page was integrated into the IMS, conveniently located near all the studentís communication technologies. It was the belief of the researchers that convenience and increased student confidence in the use of technology brought about by ease of use, would result in higher Internet usage. Frequency counts appeared to confirm this speculation, as the control group reported an average of 11 Internet accesses compared to an average of 28.05 Internet accesses in the experimental group.
To determine whether there was statistical significance between the overall usage of the Internet for research purposes, a t-Test for Independent Samples was administered to the number of Internet accesses reported by the control and experimental groups. Table 5 reports the results of that test.
Table 5. t-Test for Independent Samples/Internet "Accesses"
The t-value of 2.6975 and the probability factor of 0.0054 were sufficient to show a significant difference between the Internet research usage of the control group and the experimental group at the 0.05 level.
The telephone is the most frequently used non-Internet technology. Researchers did not believe that there would be an increase in overall telephone usage due to the expense of long-distance phone calls. Researchers did, however, speculate that the control group would utilize this technology more than the experimental group. Tabulations of instructor-student telephone calls showed that the average number of calls for the control group (mean =2.43) were higher than that of the experimental group (mean =1.45). It appeared that the control group made greater use of this more traditional technology than did the experimental group.
To determine whether there was any statistical significance between the overall usage of telephone communication between the instructor and students, a t-Test for Independent Samples was administered to the number of instructor-student telephone calls made in the control and experimental groups. Table 6 reports the results of that test.
Table 6. t-Test for Independent Samples/Telephone
Despite the belief that telephone usage would be significantly higher in the control group, the t-value of -1.2937 and the probability factor of 0.1022 were insufficient to show a significant difference at the 0.05 level.
Another traditional technology utilized by students was facsimile transmission or "Faxing". Faxes were used primarily as a means of sending larger documents between students and the instructor. The majority of faxes were sent because there was a breakdown in Internet or IMS technologies or the student did not trust the effectiveness of the other technologies. As in the case of telephone usage, researchers felt that fax usage would be higher for the control group than the experimental group. Analysis on initial data did not confirm these speculations. The means and standard deviations were nearly identical for both the control and experimental groups.
To determine whether there was any statistical significance between the overall usage of facsimile transmissions between the instructor and students, a t-Test for Independent Samples was administered to the number of instructor-student faxes in the control and experimental groups. Table 7 reports the results of that test.
Table 7. t-Test for Independent Samples/Fax
Again, despite the belief that the use of facsimile transmissions, a traditional technology, would be significantly higher in the control group, the t-value of Ė0.0824 and the probability factor of .4674 were insufficient to show a significant difference at the 0.05 level.
Despite relatively small sample sizes and a reliance on student-reported data in some instances, the study yielded several interesting conclusions. First, there were significant differences in communication behaviors between the control and experimental groups. The use of e-mail for communication between instructor and student as well as between the students themselves was significantly higher as was the use of the Internet for research purposes. These technologies were integrated features of the IMS used by the experimental group. The control group had access to the same technologies but had to utilize separate software packages. The convenience and familiarity of a single software package, containing all applicable communication technologies needed by students, had a positive effect on student communication patterns. As educational institutions become increasingly involved in distance learning, it would be in their best interest to provide integrated instructional management systems as a platform for the delivery of course content and instructional communication.
Another conclusion related to the availability of more traditional technologies to support Internet-based distance learning. While there was a slight decline in the use of non-Internet-based communication technologies by the experimental group in this study, the difference was not significant. Traditional technologies, such as the telephone and facsimile transmissions, appear to appropriately augment the Internet learning environment. The researchers observed that the telephone, especially, appeared to effectively support student learning. As the Internet is still a visually oriented media, there is a great need to make available auditory support mechanisms for students. The IMS employed in this study did not have provision for oral communication. A search of available IMSs, at the time of the initial selection process for this study, revealed none that could provide this feature. In the two years that have passed during this study, several IMSs have appeared on the market that profess to have audio conferencing as an option. Given the results of this study, it would seem that in selecting an IMS, special consideration should be given to those systems that can provide this important communication tool.
Similar conclusions can be made concerning facsimile transmissions, though integrating fax capabilities into an IMS is probably not the answer. Faxes were used by students to deal with printed material which was already in existence and typically not available in electronic form. Conducting a distance learning course without facsimile transmission capabilities would ultimately require students to utilize either personal conveyance or the postal service for delivery of certain documents. This situation would result in a barrier to timely communication. Though used sparingly by the students in both the control and experimental groups in this study, it was the opinion of the researchers that fax machines were invaluable tools for the smooth delivery of distance instruction.
In summary, despite the small sample size and certain methodological limitations, it can be stated that the use of an instructional management system for the delivery of distance learning classes appeared to have a positive effect on the number of Internet-based communications of students. This information is valuable given the proliferation of distance learning opportunities that is taking place. Communication is critical to the instructional process. Providing opportunities for communication in a distance learning environment provides an interesting and difficult challenge; challenges which are significantly different from those in a face to face environment. Our students deserve, and in many cases demand, the best learning environment we can provide. Any technology or system, which enhances communication opportunities in a distance learning situation, should be pursued aggressively. We owe this to our students and as well as our profession.
Finally, it should be noted that the assurance of purposeful learning in an Internet-based environment requires the same, if not more, attention to instructional design, diligence of the instructor and opportunities for meaningful communication than traditional in-class models. Technology, and especially distance learning, does not guarantee academic success. In the case of effective communication, technology can actually be a barrier. However, if the strengths of technology are maximized, while the weaknesses (especially barriers to communication opportunities) are adequately addressed, student learning need not suffer. Indeed, if approached in this manner, Internet-based delivery of distance learning may just indeed be the effective educational alternative it is purported to be.