Internet Instructional Method: Effects on Studentsí Performance
Universiti Utara Malaysia (UUM) offers a degree course in IT, whereby students are able to specialize in four areas: Information System, Networking, Multimedia and Artificial Intelligence. Specialization area is decided in the forth semester. When specializing in Multimedia, students have to take a compulsory course coded as TV3014, Multimedia Design.
This course deals with the client centered multimedia project cycle and the art of interactive writing, designing and managing multimedia project. Students are required to produce coursework and sit for final examination. Assessment percentages are 60 and 40 respectively. Marks for coursework are calculated from individual as well as group projects. The projects must have the potential to appeal to all five human senses through visual images, animation, sound and text (Ismail, 1997).
Measuring the performance of the students in a design course is a hard task since multimedia design is an act of collaboration process. It is rarely a personal and individual process (Lawson, 1997). It requires an integrative and multi-disciplinary input from various people such as graphic artist, audio specialist, animator, users, clients and scriptwriter. Multimedia design is not just about choosing the obvious medium for particular communication requirement (Alty, 1998). With respect to the above issue, the students are assigned individual and group projects. This allows individual team member to be independently creative, but able to work coherently with the other members (Bakar, 1998).
The final examination questions are divided into three sections:
The first group (Group A) attended a four-credit-hour per week lecture. Tools used to deliver lectures were printed textbooks, lecture notes, and reference books together with regular discussions. Lecture notes were detailed and contained important information. To sum up, this group was very much spoon-fed. Contact hours with the lecturer were regular with almost all members of the groups meeting the lecturer at least one hour per week outside teaching hour. The other group (Group B) was more independent and attended only a two-hour per week discussion sessions without any lecture. No paper textbooks or reference were used. All books were replaced with an on-line (web interface) electronic book. The electronic book contained the same contents from the regular printed textbooks that were used in the previous group. Students were also asked to find more information related to multimedia design via the Internet and presented the information they found in a two-hour sessions per week (total contact hours is still the same as previous group, i.e. four hours per week). In order words, the students made full use of the advantages of on-line information. They also managed to get on-line help from existing multimedia developers either locally or internationally.
Students are from different classes over a period of two semesters. In order to eliminate different evaluation standards from different lecturers, the samples are all students of the same lecturer (the author). Project and final examination questions over the period were of the same standard and difficulty. Group A was taught with conventional method, while group B with Internet electronic book. Table 1 shows the number of samples involved in the experiment.
Table 1. Frequency table of classes
The above samples were used to prove the following hypothesis:
H : Internet instructional method does not have any significant effect on student performance
In order to prove hypothesis H, a further three sub-hypotheses were constructed:
H1 : Using Internet instructional method has no significant effect on coursework marks.
H2 : Using Internet instructional method has no significant effect on final examination marks.
H3 : Using Internet instructional method has no significant effect on overall course marks.
Hypothesis H is not rejected only if all H1, H2 and H3 are not rejected.
Data Analysis Method
Deciding which of the groups had done better was not an easy task. Slight errors can contribute to different results. To cope with this, the following method of t-test calculation was adopted in order to calculate the significance different between two means (Mcintosh, 1976).
(1) The difference between means is found, say M A† - M B
(2) The quantity of the standard error, SE, is calculated from:
T1 = [((STD A)2 / n A)
+ ((STD B)2 / n B)]
(c) The ratio of difference and SE is found and denoted
by t c
(d) Whenever n > 50, the theoretical value of t, denoted by t t can be considered as 2.00
(e) Where t c > t t or t c = t t , the difference between means are regarded as significant.
††††† Where t c < t t , there is no significant difference between the two means.
The results are tabulated into three categories: Coursework marks (CW), Final exam marks (FE) and Overall marks (OVL) as shown in Table 2.
Table 2. Summary of means and standard deviations
The mean values in Table 2 seem to suggest that group B obtained better results than group A. In order to find out whether these are the indeed the case, tests of significant means difference were performed and the results are tabulated in Table 3.
Table 3. Summary of significant difference between two means
All t calculated values are greater than the theoretical value of t, thus making the differences significant.
How reliable is this finding? There were no ways of controlling external factors such as students from both groups using other tools apart from tools provided to them. Group A might be using the Internet outside lecture hours and group B might be utilizing paper books in their spare time. It is in the authorís belief that she had managed to limit the influence of external factors by:
By asking questions that relate to the content of her textbooks forced students to read and use her books, thus minimizing the effect of answers given were as a result of reading from other published textbooks. Students were required to answer in Malay and use only Malay multimedia jargons that can only be found in her textbooks (no other Multimedia Design book written in Malay was available at the time of the experiment), thus again forcing them to read her books. It can be said from these measures that the final examination results and the ability of the students to answer depended on them using either the printed or on-line electronic books. Group B which used the on-line electronic book had the advantages of hyperlinks to other multimedia design web sites proved to get better examination results (there is significant difference between means of the final examination marks).
Coursework marks were based on individual and group projects development. Higher marks were given to projects that exhibited professional design themes and follow the standard human-computer interface design guidelines. Having been involved in multimedia development since 1994, it was quite easy for the author to identify which project was better in quality and hence deserved good marks. Producing good quality design can only be achieved through experience and trying or evaluating various available products. Having creative minds and talent will also contribute to the success of multimedia design.
The more products the students try or experience the better design quality they produce. This statement can be supported from the result of the mean coursework marks in both groups. Mean coursework mark group A is 72.13 while group B is 77.54. Is it possible to say that the difference between these means can be contributed to Internet usage? As shown on page 3, there is indeed a significant difference between these means. This is due to the fact that before developing their projects, students in group B tried using more products and prototypes which are easily found and available on the Internet. They were able to incorporate design ideas accumulated from experiencing existing free products. Majority of members in group A on the other hand, when asked did not try any products other than the CD-ROM titles proposed by the author. In addition, since no forcing measure was introduced to group A to use the Internet, their level of Internet usage was also found to be much lower than group B.
Since all three sub-hypotheses, H1, H2 and H3 are rejected, it is possible to state that Internet instructional method does have significant effect on studentsí performance. By saying this, it is believed that incorporating Internet environment into conventional teaching method and forcing the Internet as one of the primary resource centers, are ways of increasing the educational value, promoting learning and providing students with good experiences. These activities make use of existing Internet sources to construct instructional strategies that require learners to construct learning experiences in alignment with a prescribed curriculum (Betz, 2000).
In preparing the on-line electronic book, the author used about the same amount of time when she was preparing the paper version. No extremely preparation hours were involved (this might be due to the reason that the author is experienced in electronic publishing). Preparing on-line materials should take no longer than its paper version. Identifying a usable design limit is important in minimizing authoring process time. Those interested in electronic authoring, should learn to be satisfied once the material is usable. Producing a usable material is not difficult once the standard usability guidelines are taken into consideration (Shneiderman, 1998; Dix et al., 1998; Nielsen, 1997, 1999). It is also important to note that when designing on-line materials, designers tend to spend more time in producing fancy and flashy interface. Fancy interface with heavy media usage does not guarantee usable material. More time should be spent on finding and providing hyperlinks that are relevant to the course. This will ensure the advantages of students getting more information from other Internet based sources.
This experiment was a first attempt on utilizing Internet as instructional strategy. Not much effort was put in focusing on the educational use of the Internet other than as web courses or supplements to courses. The main purpose was to study the possibility of deploying the Internet as one of the teaching strategies. As it is found that Internet effects performance positively, there is now a concrete reason for the author to integrate Internet into her teaching strategy.