Educational Technology & Society 5 (4) 2002
ISSN 1436-4522

Multimedia as a Cognitive Tool

Fawzi Albalooshi
Department of Computer Science
College of Science, University of Bahrain
P. O. Box 32038, Isa Town, Bahrain

Eshaa M. Alkhalifa
Department of Computer Science
College of Science, University of Bahrain
P. O. Box 32038, Isa Town, Bahrain



Two of the modalities used to present information to students, namely, animation and verbal representation are in a constant competition in effectiveness, without any persistent winner, except when it comes to conceptual versus procedural knowledge.  Here, we present an architecture that combines the two into a multimedia tutoring system.  This system is tested and results indicate that combining the two media leads to a cognitive interaction that promotes student learning with no less than 40% from their post classical-classroom session levels.  A test for individual differences indicates that this group is almost equally divided between those described as “spatially oriented” and those described as “verbally oriented”.  Learning across the two types of learners does not show any significant differences, except with respect to one question.  This implies that perhaps, the two media may have ambiguous internal factors that support each other.  Additionally, individual learning styles does not seem to be a clear-cut division, and is instead a “preference” of one modality as a primary source of learning, not an only one.

Keywords: Multimedia, Cognition, Modalities, Animations, Verbal representations, tutoring systems

1. Introduction

Intelligent Tutoring Systems (ITS), offers a great deal of flexibility in control, making them highly adaptable to individual student progress.  This makes them excellent candidates to play the role of “Cognitive Tools” (van Jooligen, 1999).  These tools are capable of supporting learners by explicitly representing information.  They allow learners to see the structure of the cognitive process by externalizing it and freeing memory for the more important learning task at hand.  The simplest form of a tool is a pen and paper, where students can write notes to remind them of the numbers involved when performing addition.  Therefore it should not be surprising that computer based educational systems impose themselves at the top of the list of Cognitive Tools.

However, the sudden growth of multimedia computer systems necessitated the need for a deeper understanding of the characteristics of each of the different media.  Norman (1988) indicates that each media has “affordances” and “constraints” that would be either beneficial or counter-active to educational goals.  Complexity grows exponentially when the aim of the selection is to include these media into a shell representing an adaptable ITS system.  The shell itself would be flexible to individual student needs.  Therefore, it should not be surprising to see research start off in highly controlled specific cases.

For instance, Sharples and duBoulay (1988) argue that learning medical concepts is normally acquired through induction.  This is done by showing students several scenarios and allowing them to generalize their own models over the possible cases.  The problem with this approach is that it leads students to over generalization because they are not always exposed to the extreme possibilities.  However, when students are exposed to a controlled set of images through a computer-based tutor then highlighting the extreme cases becomes possible and the problem is alleviated. 

Another experiment tested if individual differences have any effect on solving syllogistic reasoning problems.  These are usually in the format, A is related to B with a premise, B and C are related with a second premise, then the subject’s task is to say what, if anything, follows from the given information.  These problems can be solved either through drawing a diagram, Euler’s circles or through natural deduction using symbols.  Monaghan and Stenning (1998) categorized subjects according to their performance in the paper-folding test (PFT) as designed by French, Ekstrom & Price (1963).  The test requires subject to visualize the array of holes that result from a simple process.  A paper is folded a certain number of folds, a hole is made through the folds then the paper is unfolded.  Students are asked to select the image of the unfolded paper that shows the resulting arrangement and results are discriminated along a median split as high versus low visualization abilities.  Students of both groups where then split into two groups and taught how to solve syllogisms either through Euler’s circles, which is graphical, or through natural deduction, which is serial.

Following this, students were given a test with 8 syllogisms each, selected to cover a range of difficulties.  They were instructed to solve them in one of the two ways according to the way they were taught.  Results showed that those who scored high on the PFT test made fewer errors when taught to solve them using Euler’s circles than their serialist counterparts, who scored low on the PFT test when given the same teaching method.  Oddly enough, this influence only seemed to take place in the final stage or in the translation of the results from the graphical modality into sentential form.  The most important result is that most subjects would either perform better when taught verbally or when taught through diagrams according to their abilities or preferences.  Additionally, those with visualization abilities seem to need a stage of “translation” from one modality to the other.  In order to draw the student closer to the cognitive operation, the medium that forms the cognitive environment should not be intimidating.  Additionally, the amount of freedom designated to the student plays the main role of the entire process.  The only problem appears when the system somehow hinders the learning ability, and the results show a channeling of the student to think through one of the two methods.  This leads to one main conclusion.  The ideal way of describing a process to students must allow for possible individual differences.


2. Verbal/Pictorial vs. Animated

Now that both methods of representation seem necessary, a question arises as to whether one can subsume the other and present itself as the “ideal” method of teaching the behavior of processes.  In short, is animation the “ideal” way? Well, evidently from research, there seems to be a serious difficulty in getting clear-cut results to say that animation is more effective than verbal/pictorial representation or vice versa.

Pane, Corbett, and John (1996) ran a detailed study to assess the effects of dynamics representation in a computer-based system that teaches developmental biology.  They compared animation to a textual description that is enriched with carefully selected still images.  They found no difference in student performance when declarative questions are given.   Another study (Lawrence, Badre & Stasko,1994) showed that with respect to teaching algorithms, “active laboratory” sessions seemed to result in better student performances.  During these sessions students created their own algorithms and saw them animated.  They performed better in “procedural questions” as compared to students who were exposed to animations of previously selected examples.

Two other experiments showed that animations might aid students in procedural knowledge by allowing them to “predict” the next step in an algorithm’s behavior.  However, similar results were found when students were asked to predict algorithm behavior from static diagrams (Byrne, Catrambone & Stasko, 1999).  Then what role does animation play?

“When the perceptual system cannot directly perceive change over time, it will seek out implicit evidence of change.” (Freyd, 1987)

Perhaps these findings are not as surprising as they may seem at first sight, if we presume dynamic mental representations.  Freyd (1987) showed through several experiments the existence of a memory distortion that represents a shift forward to the next expected state when even one image is shown.  One of her experiments involved two static images of a man jumping off a wall.  A subject is shown one image first.  The subject would then be shown another image and asked whether they are the same.  For example, if in the first image, the man is in the air, then subjects would readily identify that the image of the man standing on the wall is not the same.  On the other hand, they would take longer to identify the difference if the order of the images was reversed. 

This implies that subjects “expected” the second image to follow the first one temporally.  This order was maintained in the experiments described above.  In fact, the first study on developmental biology (1996) replaces an animation with a sequence of four images that show different screen shots of stages in the animation.  These images, according to dynamic representation are no different from exposure to the animation itself because a dynamic cognitive representation would fill in the gaps.

If the two representations appear similar, then where lay the difference?  The clearest difference is that when images are presented as a cognitive tool, the externalised representation carries less information.  Cognition has to account for the “expected” stages to recreate the complete animation, which is cognitively taxing.  A more interesting difference though, seems to lie in the predictive ability of animation in showing students the “direction” of thought when images can frequently be unordered.  When a sequence is shown, and then an image, a student may be more readily prepared to “predict” as is the usual requirement in procedural type questions.

It is not a purpose to observe the interplay between animation and verbal presentation, but its use on helping student in the learning process. Meanwhile, both ways can occur separate or in the same time and the focus will remain on designing an adaptable system, capable to combine what students need from both methods. This internal mixture of the two methods will emphasise individual progress related to preferences or abilities, forming an ideal presentation directly related to the person.


3. More Choices with Multimedia?

So far, work has shown that there is a strong reason to believe the existence of individual differences.  Therefore, an educational system that provides the two representations that are associated with these differences is unlikely to do worse, than those that include either one or the other.  The idea is to cover for individual differences in preferring one representation to the other as well as to provide them both in parallel.  Only through this, can any interaction between the two be assessed.  If there is no interaction, then one would render the other redundant and the total impact will be no better than that obtained in the positive experiments described.  If on the other hand, an interaction does exist and is a negative one, then each of the representations would negatively affect the other.  Performance would worsen following a classroom lecture as “confusion” may result.  However, if the interaction is a form of fortification then an improvement that exceeds expectations will result.  This would imply the existence of a positive interaction between the two modalities.


4. Multimedia Data Structures Tutoring System

The selected subject was Data Structures including the concepts of Stacks, Queues, Lists and Trees.  The system is a precursor for a Multimedia Intelligent Tutoring System, which is currently under development.  These topics were presented to students in both media simultaneously; animation and verbal description.  The screen was therefore divided into two windows; one containing a carefully written description of the concept and the other an animation that the student can start, stop, and partially control.

The module itself is represented as a Java Applet with the aim of placing the system on the Internet.  It has several sections each concerned with one of the topics listed above and each in turn has several screens associated with it representing Terminology, Operations, Examples and Quiz.  The Terminology page explains the basic terminology students need to learn for that data structure and is purely verbal.  The Operations page shows and explains through text and animation the basic operations that can be performed.  Examples include preset examples represented again through both animation and textual descriptions.  The Quiz page is a student self-assessment exercise.

Students are given the full navigational freedom to go to any page they wish and repeat the animations included as many times as they wish within the specified time allotted for the experiment.  They also had the ability to control the speed of the animation by selecting a number from 1 to 6.  This was included because students complained about the slow speed in the experiments run by Pane, Corbett, and John (1996).  The loading time of the applet was somewhat slow but the running time was appropriate since the subject matter covered only the basic essentials of each topic.  Students were urged to think of new possible cases with the basics they were shown.


5. Experiment 1: Evaluation of the Module

The multimedia module was tested through an experiment that compared its effects on student performance to standard classroom lectures.  Additionally, the experiment tested its effects on students who already attended the classroom lecture.  This set up was based on an analysis of current evaluation techniques that are used (Albalooshi & Alkhalifa, 2002), and a proposal of this technique as a possible method of testing that would highlight the system’s benefits.

The predictions that were made are that it will not result in a lower level of performance than the classroom lecture, while it will be able to result in a highly significant improvement in student performance from their post-lecture test levels.

Students were distributed into three groups based on a quiz they were given earlier in the course to ensure that all groups are comparable tested over two consecutive days.  Group 1 attended the lecture only while group 2 attended the lecture and the following day used the module.  Group 3 on the other hand, did not attend the lecture and just used the module on the second day.  Both groups 1 and 2 took a test at the end of the classroom lecture.  All groups took the second test on the second day, which was highly similar to the first test with a difference in the order and wording of the questions.


5.1. Subjects

45 students from the University of Bahrain volunteered to participate in this experiment in exchange for class credit.  They were distributed evenly into three groups of 15 students each.


5.2. Materials

Materials included in this experiment concentrated on Stacks as a data structure.  They included one classical lecture given to groups 1 and 2.  Additionally, students in groups 2 and 3 used the multimedia module for that particular data structure.  Then the tests included 7 questions, which tested comprehension of the various parts of the presentation as well as the ability to recreate or imagine new uses or applications of stacks.


5.3. Results

As shown in figure 1, group 2 showed a highly significant improvement in test results following using the system when compared to their post-classroom lecture levels.  An ANOVA test showed F= 9.19 with p< .005.  No significant differences were found between group 1 who attended the classroom lecture only and group 3 which used the system only.  In this case, F=.598 with p<.446 which shows that they are extremely comparable.


Figure 1. Study results


Another point of view is to examine the scores by using the total average, which is 10.639, therefore approximating the border-line becomes 10.5 and the rest of the scores will be divided around this line.  It is to be noticed that the scores of the third group were not so high, but most of them were over the average and comparing with the second group, shows that the results are close even if the third group took only the module option while group two had both lecture and module learning.  It also underlines how much the second group improved their test results after taking the CBI and in the same time showing that the first group had not improved much only with the lecture learning. 

When a comparison is made in individual questions, an interesting phenomenon seems to take place.  Students who attended the lecture only, group 1, were better at the question “Using an example, explain briefly the stack concept and its possible uses?” than students of group 3 who used the system only.  This was with a significance of p< .03.

Additional information was found by comparing student performance in similar questions in the pretest before using the system and the post-test after using the system.  An ANOVA test compared the grades of the same students in both situations on a per question basis. Most of the difference or improvement came from the questions: “Using an example, explain the stack concept and its possible use?”, “How could we implement a stack in a program?” and “List the data variables and operations associated with the stack?”.  The significance was p< .000 in all three cases.


6. Discussion

Results seem to indicate a strong and positive interaction between animation and verbal representation with effects in such a solid improvement in student levels.  This provides strong support to the predictions made here in that having the two modalities in parallel may have better results than having each on its own.  The question at which classroom only students did better was interested in showing how representation may imply a “limiting” effect to imagination.  When students are presented with examples that take some form, it becomes more difficult for them to break out of the boundaries of that example and find another.  Students who used the system seemed to be directed towards how a stack functions rather than application areas and showed this in their responses. 


7. Experiment 2: Individual Differences between Subjects

Results clearly show that the multimedia system is capable of resulting in a significant degree of improvement in student levels with respect to the concept of Data Structures.  However, the assumptions of the interplay between the two media, is so far only based on the degree of improvement, which is around 40%.  There is no direct link in the results to fortify the claim that this system is capable of improving student performance for students with individual differences.  Surely, if there is an interplay between the two media, then it should be just as effective when used by students with more serialist bias as when used by students with a more holist bias.  To further support our claims that this system is capable of accounting for both types of students the following experiment was conducted to identify students who belong to each of the above groups.


7.1. Subjects

15 students from the University of Bahrain, who are the members of group 2 in the first experiment.


7.2. Materials

The materials used here were the same as those used by Monaghan and Stenning (1998) represented in the paper-folding test (PFT) as designed by French, Ekstrom & Price (1963).  It is composed of two parts each with a total of ten questions and students are given exactly 3 minutes per part.


7.3. Results

Each student’s score is added up in the two parts and then students are divided along a median split into two groups.  Those who score high in this test are considered as ‘holists’ and have a higher ability to visualize objects while those who score low in this test are considered as ‘serialists’.  This is exactly the same categorization procedure followed by Monaghan and Stenning (1998).

Following this procedure student results in the pre and post-tests were grouped according to their spatial ability test results only amended to have an equal number of students in each group.  These results were then compared to each other with respect to the percentage of improvement as is shown in table 1.



Q1 plus Q8 mapped to Q1

Q3 mapped to Q2

Q4 mapped to Q3

Q6 mapped to Q6

Holists Group





T-Test results





Serialists Group





T-Test results





Table 1. Comparison of results with respect to the percentage of improvement


Results indicate that although the group was indeed composed of students with different learning preferences, they all achieved comparable overall improvements in learning.  Notice though the difference in the degree in learning in Q4 four.  The question is: List and explain the data variables that are associated with the stack and needed to operate on it?  This particular question is clearly closer to heart to the Serialists Group than to the Holists group, therefore it should not be surprising that they find it much easier to learn how to describe the data variables that would students who likes to see the stack in operation.  Another point to ponder on is that the Holists group made a bigger improvement in the Q1+Q8 group, which is the question: Using an example, explain the stack concept and its possible use? Which clearly is closer to a holist’s heart than it would be to a serialist.

The results shown above further support the claims made by Mohaghan and Stenning (1998) that learner differences do indeed affect learning performance of students as is shown in the comparison between improvement levels above.  Yet these results also show that the system presented here, does indeed account for these differences.


8. Conclusion

It seems that the system presented was able to utilize multimedia as a cognitive tool capable of resulting in a significant improvement in learning with both holists and serialists.  If we ponder the causes of these results we find that the system presents the educational material through animation and textual representation simultaneously.  Holists are customarily described as students with a strong visual ability (Monaghan and Stenning, 1998).  This implies that this group learns more readily from a graphical or animated representation.  On the other hand of the scope, serialists are reported to learn best with verbal representation.  Therefore, the combination of both media allowed students with either of the above strength to learn from the same system.

So what is the role of the second media that is available? Does it hinder learning, stand idle, or fortify learning?  Clearly an improvement of 40% eliminates the first possibility because learning achieved in uni-media settings did not achieve this percentage of improvement.  But if we wish to assume that it is capable of it, we can go on to note that the animation part did not include full names and descriptions of the objects shown.  Yet, students were capable of verbalizing their answers to show that they benefited from the verbal descriptions in the system.  This would not happen if the verbal representations were not read and understood by the holist group.

This discussion seems to point at one main conclusion; which is that the provision of the second media results in a positive fortification of learning especially within the domain of Data Structures and this effect did not interact with student preferences as both groups exhibited the same effect.  This makes multimedia an ideal candidate as a cognitive tool.


7. Future Directions

The tested system was implemented as a Java Applet, which did not take long to show its limitations with respect to flexibility, functionality and speed when placed on the Internet.  A follow up system was designed and is being implemented as Java Servlets.  These are server side processes that are responsible for filling up the different parts of a frame-based page.  These areas are divided into three main areas, one for the verbal description, the second for the animated Flash file and the third for interaction with students.  In short, Servlets offer the ability to call any one of the verbal descriptions and the matching animation according to student progress ensuring adaptability.  It would be interesting to find out if students would prefer a particular type of animation for a particular section and prefer verbal descriptions for another.  Additional tests could be done to students similar to those for individual differences to test if any interaction occurs with the multi-modal representation.



The second author would like to thank god, first and foremost for insight, and for giving her a guiding star.  Both authors wish to thank Prof. WaheebAlNaser for his support.  This research is supported by a grant from the Deanship of Scientific Research, University of Bahrain.



AlBalooshi, F., & Alkhalifa, E. M. (2002). Evaluating Multimedia Educational Software: The DAST Experience, Paper presented at the 2002 Information Resources Management Association International Conference (IRMA 2002), May 19-22, 2002, Seattle, Washington.

Alkhalifa, E. M. (2001). Why Re-Invent the Wheel? Insights from Cognitive Research on Continued Education in Logic. Paper presented at the A Millennium Dawn in Training and Continuing Education conference, April 24-26, 2001, Isa Town, Bahrain.

Byrne, M. D., Catrambone, R., & Stasko, J. T. (1999). Evaluating animation as student aids in learning computer algorithms. Computers and Education, 33 (4), 253-278.

French, J. W., Ekstrom, R. B., & Price, L. A. (1963). Kit of reference tests for cognitive factors, Princeton, NJ: Educational Testing Services.

Freyd, J. (1987). Dynamic Mental Representations. Psychological Review, 94 (4), 427-438.

van Jooligen, W. R. (1999). Cognitive tools for discovery learning. International Journal for Artificial Intelligence in Education, 10, 385-397.

Lawrence, A. W., Badre, A. N., & Stasko, J. T. (1994).Empirically Evaluating the Use of Animation to Teach Algorithms.Technical Report GIT-GVU-94-07, Georgia Institute of Technology, Atlanta.

Monaghan, P., & Stenning, K. (1998). Effects of representation modality and thinking style on learning to solve reasoning problems, Paper presented at the 20thAnnual Meeting of the Cognitive Science Society,August 1-4, 1998, Madison, Wisconsin, USA.

Norman, D. (1988). The design of everyday things, New York: Currency/ Doubleday.

Pane. J. F., Corbett, A. T., & John, B. E. (1996). Assessing Dynamics in Computer-Based Instruction. In Tauber, M. J. (Ed.) Proceedings of the 1996 ACM SIGCHI Conference on Human Factors in Computing Systems, New York: ACM Press, 197-204.

Sharples, M., & duBoulay, B. (1988).Knowledge representation, teaching strategy and simplifying assumptions for a concept tutoring system.Paper presented at the European Conference on Artificial Intelligence, August 23-28, 1998, Brighton, UK.


Copyright message

Copyright by the International Forum of Educational Technology & Society (IFETS). The authors and the forum jointly retain the copyright of the articles. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear the full citation on the first page. Copyrights for components of this work owned by others than IFETS must be honoured. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from the authors of the articles you wish to copy or