Educational Technology & Society 4 (3) 2001
ISSN 1436-4522

Internet Instructional Method: Effects on Studentsí Performance

Norshuhada Shiratuddin
School of Information Technology
Universiti Utara Malaysia, Malaysia
shuhada@uum.edu.my
shuhada@dis.strath.ac.uk

 

ABSTRACT

In determining the effect of Internet instructional method on studentsí performance, two groups of unrelated students over a period of two semesters in Multimedia Design (TV3014) course offered at the School of IT, Universiti Utara Malaysia were used as samples. One group was taught with conventional teaching method and the other with Internet based electronic book. As this was a first attempt in deploying Internet in teaching environment, it was primarily employed as educational means of web courses and supplements to courses. This paper reports a personal experience and a case study of implementing Internet based electronic book and the effect it has on studentsí performance in the course. Through hypotheses testing, it is possible to conclude that employing Internet in educational settings proves to have significant effect on studentsí performance.

Keywords: On-line electronic book, Internet Instructional strategy, Measuring performance


Introduction

Universiti Utara Malaysia (UUM) offers a degree course in IT, whereby students are able to specialize in four areas: Information System, Networking, Multimedia and Artificial Intelligence. Specialization area is decided in the forth semester. When specializing in Multimedia, students have to take a compulsory course coded as TV3014, Multimedia Design.

This course deals with the client centered multimedia project cycle and the art of interactive writing, designing and managing multimedia project. Students are required to produce coursework and sit for final examination. Assessment percentages are 60 and 40 respectively. Marks for coursework are calculated from individual as well as group projects. The projects must have the potential to appeal to all five human senses through visual images, animation, sound and text (Ismail, 1997).

Measuring the performance of the students in a design course is a hard task since multimedia design is an act of collaboration process. It is rarely a personal and individual process (Lawson, 1997). It requires an integrative and multi-disciplinary input from various people such as graphic artist, audio specialist, animator, users, clients and scriptwriter. Multimedia design is not just about choosing the obvious medium for particular communication requirement (Alty, 1998). With respect to the above issue, the students are assigned individual and group projects. This allows individual team member to be independently creative, but able to work coherently with the other members (Bakar, 1998).

The final examination questions are divided into three sections:

  • Structured questions on designing and managing theories.
  • Structured questions on hands on lab experience.
  • Essays on both theory and practical knowledge.

The first group (Group A) attended a four-credit-hour per week lecture. Tools used to deliver lectures were printed textbooks, lecture notes, and reference books together with regular discussions. Lecture notes were detailed and contained important information. To sum up, this group was very much spoon-fed. Contact hours with the lecturer were regular with almost all members of the groups meeting the lecturer at least one hour per week outside teaching hour. The other group (Group B) was more independent and attended only a two-hour per week discussion sessions without any lecture. No paper textbooks or reference were used. All books were replaced with an on-line (web interface) electronic book. The electronic book contained the same contents from the regular printed textbooks that were used in the previous group. Students were also asked to find more information related to multimedia design via the Internet and presented the information they found in a two-hour sessions per week (total contact hours is still the same as previous group, i.e. four hours per week). In order words, the students made full use of the advantages of on-line information. They also managed to get on-line help from existing multimedia developers either locally or internationally.

 

The Samples

Students are from different classes over a period of two semesters. In order to eliminate different evaluation standards from different lecturers, the samples are all students of the same lecturer (the author). Project and final examination questions over the period were of the same standard and difficulty. Group A was taught with conventional method, while group B with Internet electronic book. Table 1 shows the number of samples involved in the experiment.

Group

Females

Males

Total

A

47

34

81

B

45

43

88

Total

92

77

169

Table 1. Frequency table of classes

 

The above samples were used to prove the following hypothesis:

H : Internet instructional method does not have any significant effect on student performance

 

In order to prove hypothesis H, a further three sub-hypotheses were constructed:

H1 : Using Internet instructional method has no significant effect on coursework marks.

H2 : Using Internet instructional method has no significant effect on final examination marks.

H3 : Using Internet instructional method has no significant effect on overall course marks.

 

Hypothesis H is not rejected only if all H1, H2 and H3 are not rejected.

 

Data Analysis Method

Deciding which of the groups had done better was not an easy task. Slight errors can contribute to different results. To cope with this, the following method of t-test calculation was adopted in order to calculate the significance different between two means (Mcintosh, 1976).

(1) The difference between means is found, say M A† - M B

(2) The quantity of the standard error, SE, is calculated from:

 

T1 = [((STD A)2 / n A) + ((STD B)2 / n B)]
Text Box: SE = ÷ (T1)

Where :
n A= number of students group A
n B= number of students group B
STD A= standard deviation of group A
STD B = standard deviation of group B
And:
(n A and n B are > 50)

 

(c) The ratio of difference and SE is found and denoted by t c
Text Box: t c = (M A - M B) / SE

(d) Whenever n > 50, the theoretical value of t, denoted by t t can be considered as 2.00

(e) Where t c > t t or t c = t t , the difference between means are regarded as significant.

††††† Where t c < t t , there is no significant difference between the two means.

 

Findings

The results are tabulated into three categories: Coursework marks (CW), Final exam marks (FE) and Overall marks (OVL) as shown in Table 2.

 

GROUP

 

N

Mean

Std Dev

A

CW

81

72.13

7.34

 

FE

81

55.48

15.05

 

OVL

81

66.51

10.85

B

CW

88

77.54

8.93

 

FE

88

67.25

11.15

 

OVL

88

69.69

7.72

Table 2. Summary of means and standard deviations

 

The mean values in Table 2 seem to suggest that group B obtained better results than group A. In order to find out whether these are the indeed the case, tests of significant means difference were performed and the results are tabulated in Table 3.

 

 

CW

FE

OVL

t c

4.28

5.74

2.18

t t

2.00

2.00

2.00

Result

SD

SD

SD

Table 3. Summary of significant difference between two means

 

NOTE :
SD ††††††††† = Significant difference
NSD †††††† = No significant difference

All t calculated values are greater than the theoretical value of t, thus making the differences significant.

 

Discussion

How reliable is this finding? There were no ways of controlling external factors such as students from both groups using other tools apart from tools provided to them. Group A might be using the Internet outside lecture hours and group B might be utilizing paper books in their spare time. It is in the authorís belief that she had managed to limit the influence of external factors by:

  • using printed books and electronic books that she authored herself
  • uploading the electronic version only when group B was being taught
  • asking examination questions that relate to the content in her books only
  • requiring students to use only Malay multimedia jargons when answering questions

By asking questions that relate to the content of her textbooks forced students to read and use her books, thus minimizing the effect of answers given were as a result of reading from other published textbooks. Students were required to answer in Malay and use only Malay multimedia jargons that can only be found in her textbooks (no other Multimedia Design book written in Malay was available at the time of the experiment), thus again forcing them to read her books. It can be said from these measures that the final examination results and the ability of the students to answer depended on them using either the printed or on-line electronic books. Group B which used the on-line electronic book had the advantages of hyperlinks to other multimedia design web sites proved to get better examination results (there is significant difference between means of the final examination marks).

Coursework marks were based on individual and group projects development. Higher marks were given to projects that exhibited professional design themes and follow the standard human-computer interface design guidelines. Having been involved in multimedia development since 1994, it was quite easy for the author to identify which project was better in quality and hence deserved good marks. Producing good quality design can only be achieved through experience and trying or evaluating various available products. Having creative minds and talent will also contribute to the success of multimedia design.

The more products the students try or experience the better design quality they produce. This statement can be supported from the result of the mean coursework marks in both groups. Mean coursework mark group A is 72.13 while group B is 77.54. Is it possible to say that the difference between these means can be contributed to Internet usage? As shown on page 3, there is indeed a significant difference between these means. This is due to the fact that before developing their projects, students in group B tried using more products and prototypes which are easily found and available on the Internet. They were able to incorporate design ideas accumulated from experiencing existing free products. Majority of members in group A on the other hand, when asked did not try any products other than the CD-ROM titles proposed by the author. In addition, since no forcing measure was introduced to group A to use the Internet, their level of Internet usage was also found to be much lower than group B.

 

Conclusion

Since all three sub-hypotheses, H1, H2 and H3 are rejected, it is possible to state that Internet instructional method does have significant effect on studentsí performance. By saying this, it is believed that incorporating Internet environment into conventional teaching method and forcing the Internet as one of the primary resource centers, are ways of increasing the educational value, promoting learning and providing students with good experiences. These activities make use of existing Internet sources to construct instructional strategies that require learners to construct learning experiences in alignment with a prescribed curriculum (Betz, 2000).

In preparing the on-line electronic book, the author used about the same amount of time when she was preparing the paper version. No extremely preparation hours were involved (this might be due to the reason that the author is experienced in electronic publishing). Preparing on-line materials should take no longer than its paper version. Identifying a usable design limit is important in minimizing authoring process time. Those interested in electronic authoring, should learn to be satisfied once the material is usable. Producing a usable material is not difficult once the standard usability guidelines are taken into consideration (Shneiderman, 1998; Dix et al., 1998; Nielsen, 1997, 1999). It is also important to note that when designing on-line materials, designers tend to spend more time in producing fancy and flashy interface. Fancy interface with heavy media usage does not guarantee usable material. More time should be spent on finding and providing hyperlinks that are relevant to the course. This will ensure the advantages of students getting more information from other Internet based sources.

This experiment was a first attempt on utilizing Internet as instructional strategy. Not much effort was put in focusing on the educational use of the Internet other than as web courses or supplements to courses. The main purpose was to study the possibility of deploying the Internet as one of the teaching strategies. As it is found that Internet effects performance positively, there is now a concrete reason for the author to integrate Internet into her teaching strategy.

 

References

  • Alty, J. (1998) Multimedia Interfaces: How can we Measure their Effectiveness? Paper presented at the International Conference on IT & Multimedia (ICIMU 98), 28-30 Sept 1998, Universiti Tenaga, Malaysia.
  • Bakar, S. A. (1998). Teaching Multimedia Design Through Computer Supported Collaborative Work (CSCW). Paper presented at the International Conference on IT & Multimedia (ICIMU 98), 28-30 Sept 1998, Universiti Tenaga, Malaysia.
  • Betz, M. (2000). Curriculum, Instruction, and the Internet. Educational Technology & Society, 3 (2),
    http://ifets.ieee.org/periodical/vol_2_2000/discuss_summary_0100.html.
  • Dix, A. J., Finlay, J. E., Abowd, G. D. & Beale, R. (1998). Human-Computer Interaction, New York: Prentice Hall.
  • Ismail, A. H. (1997). A Directed Knowledge Networks: A possible approach to teaching and learning at Multimedia University. Research Report, Malaysian Telecom University (UNITELE).
  • Lawson, B. (1997). How Designers Think: The Design Process Demystified, 3rd Ed., Oxford: Architectural Press.
  • Mcintosh, D. M. (1976). Statistics for the Teacher, 2nd Ed., Oxford: Pergamon Press.
  • Nielsen, J. (1997). Changes in Web Usability since 1994,
    http://www.useit.com/alertbox/9712a.html.
  • Nielsen, J. (1999). Ten Good Deeds in Web Design,
    http://www.useit.com/alertbox/991003.html.
  • Shneiderman, B. (1998). Designing the User Interface, 3rd Ed., New York: Addison-Wesley.

decoration