Case study on technology and distance in education at the Harvard Business School
Brian J. DeLacey
Dorothy A. Leonard
In 2000, Harvard Business School (HBS) created a wholly-owned, non-profit subsidiary — Harvard Business School Interactive, or HBSi — with a mission of driving organizational excellence through traditional and online learning.
The introduction of online learning poses a challenge for HBS, which has a long tradition of face-to-face, discussion-oriented education. This article covers the educational approach at HBS, the findings from our research into the use of technology in learning at HBS, and two caselets (small case studies) on hybrid online/in-class courses at HBS.
HBS was founded in 1908 with 15 faculty members. One professor who championed the creation of a business school called it a great but delicate experiment in management training (Cruikshank, 1987). Harvard University President Charles Eliot saw a need for this kind of education as far back as 1895, when he read a story arguing that railroad management be acknowledged as a science. As a school for professional managers, HBS dealt from the beginning with workplace issues. A Harvard economics professor at the time noted:
There are two types of Executive Education programs. Open enrollment draws students from many different companies into programs ranging from three-day special discipline or topic seminars to 10-week long programs. These cover many of the same general management topics that second-year MBA students study. In contrast, executive education custom programs focus on the specific needs of individual companies, and are designed to meet the needs of executives of those companies. In all of these programs, the primary (but by no means exclusive) pedagogy is case-based, in-class, faculty-directed discussion.
The essential premise underlying HBS pedagogy is that management and leadership skills require more than the acquisition of facts: expertise is built through practice and experience. Mailick et al. (1998) noted benefits of this kind of experiential learning. The historical challenge to instructors has been how to deliver experiential learning in a limited amount of time in a group setting and (usually) in geographically concentrated space.
Experiential learning at HBS: MBA and Executive Education
Professor Arch Shaw brought the case method of teaching, in use at the Harvard Law School since 1870, to the business school in 1911. At first, the cases were real-world dilemmas posed by managers. Professor Shaw invited fifteen business managers to visit classes. As described by Cruikshank (1987):
Eventually these “walking cases” evolved into written cases designed to convey the experience of senior business executives. Cases are still problem-based explorations of real-world experiences intended to simulate the issues that graduates will face as managers.
HBS faculty, selected for their ability to engage a group and facilitate discussion, favor cases because they stimulate debate and enhance active learning. Case-based instruction places heavy demands upon both students and instructors as described by Garvin (1991):
As we will discuss later, this art of discussion teaching (which HBS professors learn through lengthy apprenticeships) is challenged when many of the assumptions about time, place and technical support are changed—as they are in distance learning.
The question arises, how can technology help? The authors began a research project to learn what we at HBS already knew about learning and the use of technology in learning, looking closely at how this might apply in dispersed settings. We were interested in how faculty members were incorporating technology into their pedagogy.
Technology-mediated Instruction at HBS
In 1995 - a time when not all faculty used a computer in their office - HBS’s newly appointed Dean, Kim Clark, set the goal of becoming world-class in the use of information technology. HBS has since made huge strides in the use of information technology. One alumnus recently noted the remarkable changes that have taken place since she graduated 13 years ago. She reported upon her return to campus, when she asked students about the role of technology in their HBS experience (Singleton, 2001):
Today, technology on campus is in widespread use and many faculty members have experimented with its use in teaching. A few experiments have directly involved dispersed learning and we highlight two of those in the caselets presented below. However, much of the technology in use in the classroom, including simulations, video cases, multi-media cases (with text, audio, video, and graphic materials delivered via the intranet), computerized exercises and polls, could be adapted to distance learning. Therefore, faculty learning about the uses of such technology in the HBS setting is likely to be very helpful in understanding some of the challenges and opportunities for distance learning.
The authors conducted hour-long interviews with 23 HBS faculty members between January and March 2001, including those teaching in MBA and Executive Education programs. Table 1 indicates the core questions involved. Because we selected faculty members who are early adopters, knowing they had used technology in their pedagogy, we do not present perspectives of non-users. Our objective was to learn about what was already known, in the heads and experience of faculty members, but had never been organized or documented about how technology can enable dispersed learning.
Table 1. Sample interview questions
Findings from Faculty Interviews
In the following sections, we first present faculty comments about what was necessary for the preparation and delivery of technology-enabled, online and dispersed learning sessions in MBA and executive programs. Some of these requirements are particular to the HBS institution because of its history and culture, but others are undoubtedly generic, and are mentioned by other writers. We then note some opportunities seen by our faculty for technology-enhanced education, and challenges to such programs. Table 2 provides an overview of what we see to be Requirements, Opportunities, and Challenges for technology and distance in education. We next present two caselets exemplifying both the opportunities and the challenges. One focuses on a 2nd year MBA Course, Management in Perspective; the other is Executive Education’s Program for Global Leadership (PGL).
Table 2. Summary of requirements, opportunities and challenges
Requirements: Preparation and Delivery
Meet Face-to-face first
All experiments conducted so far have begun in the classroom so that students get to know each other and the professor. Peters (1994) notes the unique aspects of the face-to-face teaching environment where “[s]ubjectively experienced forms of action and their traditions and rites play an important role” (p. 154). Wenger (1998) describes such environments as where “interpersonal relations involving imitation and modeling, and … cognitive processes by which observation can become a source of learning” (p. 280). Certainly, some situations require hands-on application of learning in team settings (see Edmondson et. al., 2000).
No one knows how long a group takes to build a level of trust and understanding. One senior HBS professor believes that “an intense four- to five-day period” may be enough to enable members “to know how to interpret what is said by email.” Citing the importance of this to subsequent virtual meetings, he noted: “I will never be responsible for a distance-learning experience where I can’t get people face-to-face first.”
Managers who work with dispersed teams have also noted the importance of initial face-to-face gatherings. Leonard et al. (1997), Moreland et al. (2000) and Dede (1996) describe the need to balance “virtual and direct interaction” to maintain effective communications.
Bransford et al. (2000) advance a fundamental principle: “people learn by using what they know to construct new understandings”(p. 68). This suggests the value in sharing content prior to the distance-learning project. Establishing a content base shared by all participants is critical for dispersed classes, even more than for a co-located class. The depth of that base is dictated by the work to be done by students while they are dispersed. Professors teaching in the PGL program established core elements of the content base on new topics in several special meetings prior to the participants’ returning to their (dispersed) home organizations.
The procedures for working together while physically dispersed have to be set up ahead of time (Leonard et. al., 1998). This may seem obvious, but the extent to which groups need to think of all the roles and process rules necessary for smooth communication surprised some professors. Professors in PGL found they needed to help their teams identify critical roles (e.g., the person responsible for all presentations and the scheduler) and discuss future communication processes.
Research on the existence of “transactive memory” (Rulke & Rau, 2000) implies there is additional value in sharing information about participant backgrounds prior to the distance project. Additional research suggests this can be done equally well from a distance (Moreland & Myaskovsky, 2000), but in our examples the distance-learning teams all had a chance to work together and those that had the most effective face-to-face time seemed to be most successful in completing their projects.
Although HBS has provided much administrative and technical support in creating the virtual meeting space, professors still bear more responsibility in the virtual classroom than in the physical one for anticipating logistical and technological needs. A professor whose field is information technology and who is far more experienced than the average instructor says, “the technology must be invisible; when people are running around pushing buttons…the whole image of what you are trying to do falls apart…and that’s why I keep worrying about it” (personal communication).
Participants in online programs also have to become so comfortable with technology that their use of it is routine. Many executives require training in sending messages and transferring files. As our adult learning programs tap into younger, more technology-savvy generations, some of this need will disappear.
Provide Supporting Materials
“Both Piaget and Gagné stress that thinking involves operations on content…[and]…the two complementary elements of skill and content both can be learned through systematic exercise” according to de Sanchez (1987, p. 414). In order to exercise skills independently dispersed students need access to pre-screened content, i.e., resource materials, even more than they would in the classroom setting, where it is easier to learn from each other, borrow materials, browse in the library, etc. Library staff, the professors themselves, and support staff put together extensive packets for participants to use while working in dispersed locations.
Because both technology and pedagogy in distance learning are still in their infancy, a number of professors stressed the importance of setting realistic expectations. Even with the best planning, the technology is subject to unanticipated glitches. Students, they found, are relatively tolerant of experiments if they know that the process is not yet totally proven. A good example of this is found in our MBA caselet, Management in Perspective.
When the distance learning is asynchronous, as in the case of the PGL projects, the professors had to assume two responsibilities: 1) periodically supply enough new material—electronically—to keep students engaged; and, 2) demand periodic deliverables from students for review and comment.
In synchronous online discussions, keeping students engaged is critical and requires a different approach than in the classroom. According to one professor: “I have to poll relentlessly. The goal is to get the whole audience involved.” This faculty member presents options and choices to students in order to keep them attentive as well as involved in discussion. He solicits student comments based upon their polled responses. In a classroom setting, debate maintains student focus; there are more potential distractions to students participating online or from a distance.
Physical distance does not eliminate the need for coaching by the professor—not only about progress on deliverables but also, on occasion, about interaction among virtual team members. This coaching was done through various media but several professors used telephone conferencing.
As with classroom exercises, it is always possible to set up peer-to-peer reviews of materials and shift some of the responsibility for coaching to the students. In a first year MBA class, the professor assigned students to teams of 4 to build a web site for a fictitious pizza company as part of week-long team exercise. Throughout the project, students were asked to provide feedback and do peer reviews. The instructors found that the student teams who sought peer feedback on their prototypes during the process created a superior final product.
Several exercises used technology to help students generate data. One exercise, which employed the computer as a “Shadow Partner,” involved discovering company and industry information on the web. Another used video and threaded discussions to communicate with alumni on-line at first, and subsequently in on-campus interviews. In each of these projects the participants generated a significant portion of the content. Moreover, they did so through guided exploration such as gathering questionnaire data on their own. All the exercises required enormous preparation and logistical support, but conveyed the benefit of being obviously and immediately relevant to the participants.
Achieve Closure on Content
Students need an opportunity to synthesize what they have been learning in their dispersed groups. Especially in the executive programs in which participants studied in dispersed teams while they were also working at their regular jobs in their home organizations, their thought and attention have been fragmented. The opportunity to focus on what they have learned is critical to their sense of satisfaction with the dispersed learning experience. In all the HBS experiments, this intellectual synthesis of content materials, and sometimes convergence on practical action items, was achieved through face-to-face meetings. Veterans of teaching in PGL discovered that they needed to spend more time on this issue than they had originally anticipated.
Take Learning out of the Classroom
HBS “long” Executive Education programs (such as the Program for Management Development and the Advanced Management Program) have been steadily shortened over the past decade from 12 weeksto their present length of 10 weeks. The school started two new programs (PGL and The General Manager Program) as the first major new programs in many years, at least partly in response to the need for shorter programs. These newer six-week programs were created in “3 and 3 mode”—three weeks face-to-face, an intervening period of a month or more when participants go back at work, and a final three-week session for face-to-face completion of the program.
Professor Chris Bartlett, who directs the PGL program notes: “What we teach over two years in the MBA program we smash into three months in our Executive Education program. The market is telling us not 12 weeks--10 weeks; not 10 weeks--8 weeks; not 8 weeks--6 weeks. At some stage, there is an elastic limit to this. So we have to find ways in which we can [shorten the on-campus time].” His conclusion: “We have to take the learning out of the classroom. … I’m very clear [that] distance learning is not distance teaching. What I want to do is engage [students] to learn from each other in a network and from the broad resources they have out on the Internet. [My job is] to frame the issues so they can learn.”
Another professor who leads one of the custom executive education programs, set up a distance-learning project with company executives to include a module he felt participants really needed exposure to. “We always wanted to do something on working capital management but we didn’t have time in the program to do it.” His solution was to conduct online sessions with 20 to 33 students from 20 different countries, crossing many different time zones.
A number of professors find that a degree of online preparation ahead of time enhances the classroom experience. One professor explains:
A number of the professors send students to the web not only for data about a particular company but also to web sites and demonstrations of products and services as a kind of living case. Whereas, with a written case, professors are sometimes blind-sided because students have “cracked the case” by finding through the Internet the outcome to the dilemma posed in the case. Cases dynamically constructed out of current materials never go out of date. The responsibility for creating these cases falls heavily on the participants—but instructors still must carefully select subject companies and ensure that the right materials are available to support the pedagogical purpose of the class discussion.
A number of the professors have demonstrated that a hybrid of online with in-class learning can deliver very high-quality experiential learning — especially when the on-line portion is tied closely to work experience. Students in one course learned directly, but online, from exchanges with alumni about moving from individual contributor into the role of general manager and constructed from those interactions guidelines for their own career transitions. Raelin (2000) supports the value of integrating theory and knowledge with work practice.
Case discussions in the HBS classroom are usually focused at least partially on decision points—alternative courses of action. Voting on these alternatives: 1) clarifies issues, as students debate the alternatives; 2) encourages active consideration as the students have to commit publicly and be prepared to defend their decision; 3) provides energy to the discussion. The same benefits accrue in distance learning. Technology enables more information gathering than does a show of hands in the classroom. Online, every student votes for or against a decision and the resulting distribution of comments can be fed back to the class, either in real time for a virtual or co-located gathering or over a period of time to a class conducted asynchronously.
Accommodate Less Verbal Students
Asynchronous, dispersed classes have advantages for several classes of students. First, some students have a strong preference in their in-born thinking styles (as measured by such diagnostics as the Myers-Briggs Thinking Indicator) that leads them to contribute more if they have time to prepare their thoughts. Second, some students find it very difficult to speak out in a large class gathering, either because of inherent shyness or because English is at least a second language and they are not bi-lingual. (See Sadler-Smith & Riding, 1999, for recent research on this topic.) Third, given the large numbers of students and limited class time, finding “air time” to speak can be a challenge.
The HBS MBA program tends to select and reward students who are able to debate and discuss easily even on the basis of very incomplete information. Notes one professor: “The online classes offer a much less threatening environment to many students. Most students who won’t speak in class do so readily from the apparent security of their homes during the online classes.” This professor found that 33% of students felt the online sessions were more interactive than face-to-face sessions.
The (now) old joke about the dog at the keyboard, “On the Internet, no one knows you are a dog” has a point. While most of the technologies employed enable the professor to identify and grade contributions by students, students themselves experience very different group dynamics when they are not physically co-located. Experiments with negotiating situations (by a doctoral student at HBS) illustrate the kind of unexpected bias that anonymity can remove: MBA students conducting negotiation exercises were tougher when they knew they were negotiating against a woman than if they knew their opponent was a man. Such biases disappear, the doctoral student found, if one does not know the identity of the opponent.
There is a potential benefit to asynchronous classes. Professors who teach in Executive Education at HBS note that the more compressed schedule in these programs allows less time for self-discovery and reflection; instructors tend to provide more analysis and even conclusions for the students. Research on creativity demonstrates that time constraints inhibit creative thought (Mueller et al., 2000). Time for incubation is critical to creative thought processes (Leonard et al., 1999). Therefore, providing some time for thought between sessions or between required responses may aid creative problem-solving.
Although most of the video at HBS is used in the classroom, the very fact that the images are converted to digital form means that they can be transmitted for online programs. Proponents of video are clear about its many advantages: 1) video commands more interest than does text; 2) video conveys subtleties about personalities that text does not capture(e.g., in one multimedia case, the manager as seen on video gives quite a different impression of his attitude towards his workers than he does through the written description); and, 3) video introduces students to places and activities that they could not easily visualize (e.g., one video case takes viewers on a Harley-Davidson motorcycle ride in California). In short, video is closer to real experience than is written text. One senior professor says simply, “video helps make learning more accessible.”
However, with current technology, images projected on the computer screen are so small that the video lacks the power it has in the classroom on a large screen. One professor compares it to looking “through binoculars.” Another professor with experience in television production decries the quality of online video. Nevertheless, students are still very enthusiastic about video in cases.
Expand the Reach: Communities of Practice
An opportunity that is as yet under-exploited is the ability to expand the learning opportunity not only in space but in time. New ideas include ways to “time-shift” the learning and intersperse it with other activities—in, as it were, the “white spaces” when students are driving or resting or relaxing.
Students also yearn for contact with fellow learners beyond the formal programs. Especially in those programs that had an on-line project, the virtual teams wanted to keep in contact with each other and with faculty about their further progress on their “action learning” projects.
This creation of a learning community beyond the particular educational program differs from the personal networks spawned in the classroom. Professors saw this as a tremendous, unanticipated benefit to the project participants. One professor commented on the emails that participants continued to send to each other after the program finished and on the importance of their “reinforcing each other's careers moving forward. That to me is the most important thing I want to do in this project.” A great deal of research has presented the benefits of communities of practice (Wenger, 1998), and knowledge sharing across communities (Lesser et al., 2000).
Participants in the executive programs also request help in forming learning communities within their own organizations. During the programs, executives identify opportunities to transform work around them. However, they recognize that such opportunities are obvious to them only because of the educational experience they have undergone. Given that not all of their colleagues back home could, or would, go through the same experience that they have, how do they transmit what they have learned? HBS is currently experimenting with various ways to leverage in-class sessions so that much of the content can be delivered remotely.
Executive Education Caselet: Program for Global Leadership
PGL for senior executives consists of two 3-week residential sessions plus a distance-learning project. In part, the program provides participants with:
At the end of the 2nd week of the program, students select one of 4 distance-learning projects. One project that we studied was Knowledge Management & Organizational Learning (KMOL):
Prior to departing from the first session, students met in their teams (with five or six members) to discuss the assignment and receive the following:
The distance-learning module was divided into three phases:
Participants each had a goal to construct a concrete action plan for changes in their own companies. In addition, each team worked on a presentation to deliver to their PGL classmates.
The professor maintained contact with all the groups throughout their distance-learning project via email and telephone. As soon as they arrived on the HBS campus for the final part of the PGL program, students met with their KMOL team members — face-to-face — to finalize a team presentation.
All participants attended the same preparation, yet two groups began on different footings:
Students benefited immediately by applying their learning about knowledge management to their own business. One participant even arranged an offsite on the topic with his corporate colleagues during the distance-learning project.
Students made extensive use of a number of technologies: email, file-sharing (posting interim documents to their team website) and teleconferencing. One group used videoconferencing successfully in the early stages of the distance module, but asynchronous methods proved easier to manage across disparate time zones. Teleconferences were easy to arrange but widely differing time zones, hectic work schedules, and busy international phone connections similarly limited their use. Access to online resources from anywhere made it easy for participants to share updates (such as interim working documents) even when they were traveling.
There was little to no use of on-line threaded discussions (“too much extra effort required to go to a website and find the discussion”) or on-line meeting video and audio (“slow connections and corporate IT restrictions.”) Many distance-learning programs utilize text-based discussions with threads (Bonk & King, 1998), but if there is a choice “higher bandwidth” media (e.g. video) has been reported as being preferred over something purely text-based (Katz, 2000) systems.
All three dispersed teams delivered very impressive presentations upon their return to HBS and received critiques and comments from their classmates as well as two KMOL experts recruited to hear the presentations. Representative comments gathered after completion of this distance-learning project follow.
A CEO from Brazil described the significant benefits he saw:
A Director of Human Resources from a company in the United States shared this view:
A Country Manager from Singapore described the importance of establishing the working team:
Another participant observed simply:
At the close of the program, students wanted to keep their document and file-sharing systems online. They committed to continue sharing “electronic real estate” and “to keep in email contact on status and progress classmates were making about their company projects.”
MBA Caselet: Management in Perspective
Management in Perspective, a second year MBA elective conducted by Professor Quinn Mills, focuses on topics such as Leadership Styles and Managing in Changing Environments. A full class of 102 students participated in this 14-week class, which featured company founders and leaders attending class to discuss their own “live” business settings with students. This pedagogy harkens back to the 1911 beginnings of the case method of instruction, when Professor Arch Shaw first introduced “walking cases.”
Desiring to maximize the use of class time for live interactions with executive visits, the professor designed additional learning outside the classroom. He set up an online learning component using technology to aid synchronous - but dispersed - learning.
Each student was required to attend at least one online learning session. Professor Mills divided students into 6 groups for these 50-minute sessions. After the online session was completed, students could replay the sessions from digital recordings if they missed a session or if they wanted to replay all or part of the class. Students attended these online sessions from their homes, using a telephone conferencing system for spoken discussion and a computer connected to the Internet to view slides and graphical materials.
The professor prepared his own slides and content but received essential assistance with the online class from a research associate and information technology engineer who typed in questions and helped students establish and maintain network connections.
The professor set student expectations at the beginning of each online learning session: “This is the first time we have done this; something is bound to go wrong. Do not be upset. If you connect in, and you are cut off …I will talk to you the next day about it. Don't worry about it.” Even though this online format was new, students had no significant problems. A major challenge remains making all online experiences work this smoothly.
The professor then discussed his online presentation, following each slide with a student discussion. Students and faculty engaged in “online conversation” via several methods. Students “raised their hand” to participate in discussion with a simple mouse-click. When Professor Mills noticed (from his computer screen) students wishing to contribute, he chose the person to speak by calling on them over the phone.
The professor and his research associate polled students prior to class on topics related to the next executive visit. In addition, Professor Mills used the polling mechanism during online class to post questions synchronously. He prepared some typewritten, online questions in advance to go with specific slides in his presentation. In addition, he created and typed in “on-the-fly” questions when students took the conversation in new directions. Professor Mills reviewed totals, averages, and specific student responses with a simple click of his mouse and incorporated this live information into his cyber-classroom pedagogy, making these results visible to students could help move the discussion in new directions or encourage reflection and summarization.
The professor noted that a small percentage of students opted to attend multiple online sessions and several were noticeably more interactive in the online sessions than in the classroom setting.
The technology supporting this learning innovation worked well: it was not flawless, but it did have excellent error recovery. In one instance, the system automatically (and transparently) redialed students when their telephone lines were unexpectedly disconnected.
Professor Mills reported (Mills et al., 2001) that the online cyber-classroom of approximately 20 students created greater interaction and a more intimate conversation than the typical face-to-face amphitheater classroom of 102. At the close of the program, he noted that “using synchronous learning as a supplement to the traditional classroom … provides an effective tool for intensifying the quality of the learning experience.” Students reported similar satisfaction when they were polled at the end of the course. Results included the following highlights:
Professor Mills provided his perspective on the success of the effort: “Adding the online classes to the course significantly improved students’ grasp of the course’s subject matter as compared to the previous year, in which online classes were not held. Student ratings of the course also improved year over year.”
As Mitchell et al. (2001) observed, “despite recent advances, moving into technologically mediated instruction and course delivery remains akin to exploring uncharted territory” (p. 106). Here we point out just a few of the obvious challenges we see going forward.
There is always a huge gap between knowing what can and perhaps should be done, and implementing those ideas. The professors who have experimented with distance learning and, for that matter, with new technology, have done so because they were interested in new pedagogy—not because they were likely to be rewarded for their efforts. For example, only recently has HBS given multimedia cases any weight in promotion review packages, although they require much more work than do written cases. For junior faculty, any extra effort expended on teaching unrelated to their research, takes away from the work on which their chance of promotion is based. Not surprisingly, then, the junior faculty who have conducted experiments have tended to be those who are researching subject areas related to distance learning, such as e‑business.
No faculty members receive recognition of the extra time required to plan and carry out distance-learning projects. For example, if professors and participants alike are working during off-campus breaks, such programs are in effect much longer than the weeks of face-to-face instruction. In addition, distance-learning conferences may take place at midnight in the United States in order for it to be mid-day in the participants’ countries. Yet our current system of allocation for teaching credits does not acknowledge the extra work.
In a face-to-face class faculty members have numerous pedagogical tools at their disposal which are not available online. An obvious element is the classroom itself and the physical and geographic presence of students. An experienced professor contrasted teaching in class with conducting sessions online, including the problem of allocating credit for participation (which usually constitutes 50% of the student’s grade):
In a class of 80 people that is 80-minutes long, I don’t have any problem remembering important comments made in class. The location of the person who made the comment helps a lot in this regard. … It’s amazing how much you use your eyes to “hear” in some sense.
Not all of our students are technically proficient, yet we are asking them to use technology in exercises and projects so we run the risk that the technical experts in any group effort will dominate. In the online web-building exercise set up for first-year MBAs in the Technology and Operations Management course, the professor leading the exercise said that:
Of course, this kind of domination by one “expert” is common to any group exercise, including face-to-face ones, but when using technology, instructors have to be alert to disenfranchising large numbers of students if the technology overrides the learning objectives.
With the exception of a few specially qualified part-time instructors, fulltime members of the faculty deliver the educational programs at HBS. As noted above, distance-learning programs will—at least initially and perhaps always—require more, rather than less preparation time. The most significant constraint on any educational initiative is faculty time. Moreover, although we have argued above that a combination of classroom with online instruction can deliver the kind of experiential learning for which HBS is known, not all professors will have the desire and skill to design an on line component to their courses—especially if that component is to be delivered repeatedly to large groups of students whom they have not personally met. Therefore, it is likely that some other facilitators will need to be involved in course and program delivery. We do not have enough alternative models for such facilitation at this time to judge what will work.
One faculty member said he had to learn a whole new way of thinking about teaching the material and “it took a while to ‘get it’.” He went on to comment that he had learned a lot in working with professional instructional designers. Even if the content of the course was the same as that delivered in the classroom, “Online interactive learning is different.”
The stakeholders in educational programs (those who purchase the program, those who deliver the program, participants, participants’ colleagues, and supervisors) are far from unanimous in agreement as to what constitutes a successful outcome. When the client is not the individual learner alone, but various individuals in the organization paying for the education, those individuals are responding to different incentives.
The one desirable outcome upon which we all might agree is learning by student participants. Yet learning is difficult to measure when the subject is management. First, there are no standard tests for management perception and understanding. Second, some learning is valued only in retrospect or after some years of experience. Alumni returning to business school rate the utility of courses differently than do current students. Third, learning may be judged by changes in attitude or behavior—but who is the best evaluator of such changes? The expectations of managers in client companies regarding the outcomes of educational programs vary from having more enlightened employees to fomenting organizational revolution. Finally, we tend to measure what we can most easily observe. In our current classroom programs, we ask participants to rate sessions soon after they have experienced them; this immediacy ensures that details are fresh in memory. However, it also heightens the likelihood that more entertaining sessions are more highly rated—regardless of their educational content.
The difficulty of educational measurement is by no means confined to distance learning and is especially challenging when it comes to “deep learning” – knowledge of multiple models, viewpoints, relations, reasoning processes, problem solving abilities, and first principles (Sandberg & Barnard, 1997.) One has only to listen to the national debate about education to realize how difficult it is to measure and evaluate learning experiences, just as it is to measure scholarship. For example, consider the March 19, 2001 cover story of Business Week: “Critics warn that annual testing would create test mania and divert kids from broader learning. But there's ample evidence that performance can rise with well-designed accountability systems that use tests in addition to other measures.” From our vantage point, this challenge spans “not only the product or content of learning but also the process by which the [person]… gets or fails to get mastery of materials, for only in that way can the efficacy of pedagogy be examined” (Bruner, 1966, p. 164).
It is still unclear just what kinds of learning will be amplified and what kinds dampened by more dispersed modes of delivery. However, there seems to be, in general, a heightened emphasis on job and task-relevant education. It will not be surprising, therefore, if we find increasing pressure to measure enhanced job performance as an indicator of learning. We agree with Marshall (2000) when she says, “[W]e need systematically applied, carefully designed assessment to provide us with data. By working with the data we can begin to evaluate the design and delivery plans currently in distance-learning situations and begin building models…that lend themselves to rigorous evaluation” (p. 7). Currently, we can rely only upon student ratings and the professors’ subjective opinions of the courses.
One professor viewed his distance-learning experiment in Executive Education, with Working Capital Management, a success: “I must say I enjoy it a lot. It’s a lot of fun to do. It’s not as good as being there — from my own standpoint, it’s about 75% to 80% of what they would get out of it if they were face to face — but for the students in the program, they got what they needed to learn.”
Professor Bartlett, director of the PGL program, told participants that they should be able to return to their companies within one year, benefits from the educational experience that match or exceed the cost of the program. That is a challenge that few other educators would care to issue.
A critical success factor in extended learning activities, whether in a university or corporate setting, is a supportive environment (Rosenberg, 2001.) Organizational support is crucial to the success of enriching learning with technology. As described by Berge (2001):
Key to the success of sustaining initiatives in technology-enhanced learning and distance education is the commitment and support of the organizations’ top [leaders]…. These leaders will need to exhibit enthusiasm for, champion, and allocate resources to these programs while encouraging and rewarding instructor cooperation. Such leaders can build credibility for distance education, maintain currency in the field, and gather support and partners inside and outside the organization … Both top-down and bottom-up support are needed for successful, sustained distance training and education at the high stages of organizational capability (p. 351).
It is clear that the learning environment has changed — and it is important for organizations to recognize this and act on it in order to “create an atmosphere for continuous renewal” (Leonard, 1998).
Blending tradition with technology may be HBS’s best approach for continuous renewal to develop its core capabilities around learning. In the words of Singleton (2001), an HBS graduate from the class of 1988:
The primary purpose of HBS has not changed over the years: it remains “to educate leaders who make a difference in the world.” However, as HBS Dean Kim Clark suggests on the school’s website (http://www.hbs.edu) there are many changes in the wind:
In this time of extraordinary change throughout the world, our clarity of purpose gives the School unmatched strength and influence. Our commitment to general management education focuses on building a deep understanding of business, teaching with skill and passion, and communicating ideas that have power in practice. Our dedication to field-based, problem-focused research, and to the case method of instruction, remains as meaningful today as it was at the beginning of this century.
But much about the School is changing as we embrace a rapidly changing world. We are educating students and building knowledge for a global community that is increasingly entrepreneurial and ever more reliant on technology -- and therefore more dependent on its shifts. These times demand creative leadership.
The formation of HBSi virtually ensures that the next “delicate experiment” for HBS will be to expand the use of technology and involve faculty in more distance learning.
This research has focused largely on the in-depth experience and viewpoints of faculty members involved in technology mediated learning. Future research will address more extensively the viewpoints of students. We also need much more understanding of whether and under what circumstances we can build on-line communities for life-long learning. Many website “fields of dreams” have been built, but so far their use is limited.