Escaping from Flatland:
Multimedia Authoring

Lawrence M. Hinman

hinman@sandiego.edu

The world is complex, dynamic, multidimensional; the paper is static, flat. How are we to represent the rich visual world of experience and measurement on mere flatland?

· Edward R.. Tufte, Envisioning Information. (Cheshire, Connecticut: Graphics Press, 1990), p. 9

Introduction

The paradigm for authoring is changing, and the changes give a new twist to Tufte’s question. No longer are we faced with the challenge of expressing a multidimensional world in the one-dimensional and static world of flat paper. Now the challenge is to express this dynamic, multidimensional world in a virtual, multidimensional, and dynamic medium. Many of us now inhabit that new virtual world, but we still act and write as though we were permanent residents of flatland.

From Flatland to the e-Document

The key to this change is a shift in the basic document with which we work. The basic document is no longer the sheet of paper, the paradigm of flatland. Increasingly for many of us, the basic document is a web page or, less frequently, a CD-ROM—what I shall call an e-document. Most of us have skills, sometimes quite finely honed, that allowed us to communicate very effectively when the basic document was a piece of paper. Like old-time railroad firemen working on new diesel locomotives that have no need for our skills, many of us continue to write as though we were writing to paper, not an e-document. If we are lucky, we simply fail to exploit the possibilities of this new medium; if we are less fortunate, our old skills actually impede effective communication with our readers.

This shift is not the result of the introduction of computers per se. Computers were around for decades without any significant change in the nature of the basic document, because essentially computers were text-based computing machines. Nor was the introduction of a graphical user interface (GUI) the turning point, for initially that just allowed users to manipulate text in easier ways. Rather, the crucial transition occurred with a shift in input and output in computing. Even under a GUI such as Windows, computing remained resolutely text-based as long as the principal input device was a keyboard and the principal output device was a printer.

During the last five years, input possibilities have changed dramatically. Scanners, increasingly inexpensive in the last two years, have allowed ordinary users to scan in photos, diagrams, and drawings easily and cheaply. Microphones allow the easy input of voice recording. Drawing tablets are still comparatively rare, which is perhaps indicative of how slow the shift to the new paradigm is; they are neither expensive nor difficult to use. Audio and video capture devices are becoming increasingly common, and now the principal barriers to capturing audio and video are legal ones relating to copyright and royalties. Home video and digital cameras have certainly made it painless to import images and video that individuals shoot on their own. CD-ROMs also provide users with access to images, sounds, and video clips that could never be compressed onto floppies. Finally, and most significantly, access to the Web has allowed ordinary users to grab images and clips and drag them into their own machines and programs. Taken together, these developments have transformed the average computer from a text-based machine to a genuine multimedia machine, capable of importing and working with text, sound, images, and motion.

None of this, of course, would be very helpful is there wasn’t a way of getting multimedia out of the computer. Again, a series of developments has transformed the situation during the last five years. Black and white printers gave way to inexpensive, high quality color printers—still flatland, but at least now in color. Writable CD-ROMs provided the disk space the hold large, graphics-intensive products, as well as interactive multimedia programs written in programs such as Director. High quality projectors—still, unfortunately, quite expensive—now allow graphically-based presentations for a large audience, even if in fact that are still usually used only to project text in PowerPoint with graphics that are decorative at best, distracting and confusing at worst. Most importantly, the World Wide Web has opened up the possibility of dynamic, interactive, multimedia web pages that truly mark the escape from flatland—until, of course, we reach the age of holographic projection. The basic HTML specification introduced the capability of graphics, Dynamic HTML and Microsoft’s new specification, HTML Plus Time, offer the possibility of genuinely dynamic documents. RealVideo plug-ins from RealNetworks and Shockwave plug-ins from Macromedia further enhance the multimedia capabilities of the Web. The new web page— multimedia, dynamic, and interactive—provides a new paradigm for authoring.

This new type of document, the e-document, demands new skills of us in at least three principal areas. Now projector-based presentations, web pages and CD-ROMs can be dynamic, interactive, and rich in multimedia. Each of these three dimensions demands new skills of us. Let’s look more closely at each of these three areas.

Multimedia

I grew up in a black-and-white world—not just a racially divided world, which it was, but also a world of black letters on white paper. The closest I came to multimedia was a typewriter ribbon with black on top and red on the bottom. Of course, the exciting red print never showed through on the carbons!

This world of black letters on white paper—or at least a white screen—is still home to me. When I want to add color to a document, I make things red and bold. While not color blind, I count myself among the chromatically challenged—I have few good instincts about what colors I should choose. I search for pre-defined palettes of colors that go together, since I lack confidence in my own judgments to choose sets of colors that complement one another.

The disturbing thing about this is that it means that I no longer write as well as I used to. The medium has changed, and now I realize there are areas of writing in which I am less skilled.

Multimedia is not just about colored text, of course. It is about graphics and sound. These too require new skills of authorship—and of acquisition. Increasingly, as I write, I find I need images. Ten years ago, when I was writing about Kant, all I needed were words. Now I need pictures—at the very least, pictures of Kant, perhaps also images that help us to understand better what Kant was saying.

I also now need music and sound and voice. Previously, even if spoken words, sounds, and music came to my mind as I was writing about someone, there would have been no way to include them. Now, if I talk about Dr. Martin Luther King, Jr., for example, I will typically include photos and audio and viceo clips from his speeches, perhaps even with “We shall overcome” in the background. Without sounds, images, and clips, we are as lost today as we would have been without a pencil a generation ago. We not only need the ability to use these things skillfully, we also need the things themselves.

All of this is because the medium has changed from the paper document to the electronic one. This new document type demands both new skills and new resources. At the same time, it presents new opportunities to us, opportunities for a richness of presentation that even five years ago would have been incalculably beyond our reach.

Others do this far better than I. The work done here at Carnegie-Mellon by Robert Cavalier and others is a good example of this. The series of ethics CD-ROMs, including The Case of Dax and Abortion in America, are pioneering accomplishments that have set the standard for us for years to come. The Pluralism Project at Harvard University provides another excellent example with its CD-ROM, On Common Ground (http://www.fas.harvard.edu/~pluralsm/).

Authoring in this new multimedia environment thus requires both new skills and access to multimedia materials—graphics files, sounds, audio and video clips—that would have had little or no place in the paper document.

Nor is the act of writing the same—indeed, it is now better called “authoring” than just writing. We used to put words on a piece of paper, and that was it—that was the final product. Now we create flow lines or scenes in a move. I often write in Authorware these days, and this is what a typical screen looks like.

One drags objects onto a flow line—text, graphics, sound, video, animations, and calculations—one sets up interactions.

Other programs, such as Director, use a score. There are scenes, cast members, and sprites that flit across the stage.

Notice how different this form of authoring is from merely putting words on paper.

One of the key differences underlying these factors is the changing nature of the temporality inherent in new virtual documents. Paper documents were basically static—a point that Tufte saw clearly. Although readers could turn pages successively, the document itself did not have a duration. This is changing with multimedia authoring in two ways. First, multimedia works often contain branching and loop-back possibilities that introduce a unique kind of temporality. Second, multimedia documents—and this includes presentation documents developed in programs such as PowerPoint--often contain temporally-paced sequences, animations. Let’s briefly consider the possibilities and challenges that animation offers.

Dynamic Animation

If prior to multimedia it had been difficult to convey the richness of image and sound, life in flatland also suffered from another defect: it was very hard to capture in words the dynamic quality of what you were discussing.

I first realized this in teaching. When I teach, I often find myself diagramming arguments on the board. What I want students to see is that arguments are dynamic, that is, that they move. So I draw arrows all over the board. One thing leads to another, challenges swoop in from the side, and we move through the steps of the argument. This has always been easiest for me to capture when I was at a blackboard, talking, writing, and drawing.

It is this dynamic quality of arguments that animation promises to let us capture. For example, when I talked about the slippery slope fallacy in logic, I used to draw the slope on the board and then move down the slope, showing how each step leads to the next. Now I use a Flash animation that combines narration, movement, and interaction with the reader, who must click to move from one stage to the next. There is no way I can put this animation on a piece of paper—at best, I could print out a series of key frames that display the major highlights of the animation.

The dynamic quality of presentations is already contained in presentation programs, which by their very nature span a length of time. Although this is not prominent in the interface that PowerPoint uses, the slides created in PowerPoint are implicitly stretched out over a timeline. This reason that this is not more explicit is that PowerPoint presumes the actual pacing of the show will be done on the spot by the speaker, but in fact each slide can be set to show for a specific duration.

The basic metaphor for this kind of writing is significant. It is the script or score, something already familiar to those who write music or score dance productions. Consider the interface that Flash, a popular animation program, uses. Here’s the score for the slippery slope animation:

Nor are animations limited to logic. In the first Critique, Kant refers to his project as bringing about a Copernican Revolution. This, too, lends itself to animation. I found myself doing an animation that showed the traditional sense of the Copernican revolution—namely, the earth revolved around the sun instead of the reverse. Then I illustrated Kant’s point that this insight depended on realizing that something that seems to be in the object (i.e. the motion that seems to be in the sun) is actually coming from the subject (i.e., the motion of the earth). Thus Kant’s point about the transcendental turn in philosophy: we must look for ways in which things that seem to belong to the objects actually come from us as subjects. To further illustrate this, I turned to the paintings of Seurat, whose theory of optics came from Helmhotz, who in turn got it directly from Kant. Seurat painted little dots on color on the canvas—the first bitmap graphics, by the way—so that the colors could then be mixed in the mixed, creating a vibrancy that was otherwise beyond reach. Something that seems to be in the object—the mixed colors—actually comes from the mind and is projected onto the object.

Increasingly, I find myself turning to animations to capture the dynamic quality of something that otherwise I would have reduce to the static world of flatland. Even my opening remarks about the changing nature of computing, particularly in regard to I/O, are much more effectively presented through an animation, which is now available on-line at http://ethics.sandiego.edu/Flash/computers.html.

Interactivity

Texts, or at least good texts, have always been interactive in some minimal sense. An author expects a reader to react while reading a text. However, this was largely implicit. Now the medium has changed, and one of its possibilities is that the author can now draw the reader/viewer into a dialogue and shape responses based on the reader’s answers.

When we write traditional expository prose, we are often anxious to insure that the reader is following our line of argument or explanation. Yet once we finished writing and publishing the passage, we had to let go of it and let it go on its own, just as an archer can no longer guide the arrow once it has left her bow. Well, just as we have developed smart weapons that track their targets after being fired, so now the interactivity of web-based and CD-ROM-based projects allows us to continue to interact with the reader and to adjust the program in light of the reader’s reactions.

I have found this capability initially most powerful as a teaching tool in dealing with material that I have taught often. When we teach something a number of times—in my case, introductory logic—we come to know the mistakes that students typically make and also to know what they need to review in order to avoid the continuation of those mistakes. I find myself developing on-line logic exercises, when students make some of these typical mistakes, I am able to send them to, say, specific slides in a PowerPoint presentation to review key concepts that they have misunderstood.

Conclusion

A new generation is growing up with this new medium. My daughter, now seven, was able to log onto her computer and enter her user name when she was four. CD-ROMs have become one of her basic sources of knowledge, both at home and at school. Last year, she got her first program that integrates voice recognition. She talks to her computer and her computer responds. She is not, in this regard, all that unusual. Moreover, she and all her classmates work on computers regularly in her first grade classroom at our local public school. In ten years, these are the students we will be teaching, students who have grown up in a world where the basic document is more likely to be electronic than paper.

And it is this generation that we must learn to write for. In order to do this, we must not only retain our traditional skills in English grammar and spelling and style, but we must develop new skills to master the new e-document. These are essentially the skills of the graphics artist (for images), the musician (for sound), the director (for video) and the choreographer (for the dynamic), which must be added to those of the seasoned teacher (for the interactive).

For many of us, this is a daunting challenge, and we should not be surprised to find that our students are often better at some of this than we are. Yet it is also a liberation, for it offers us ways of expressing much more fully insights and ideas that previously we had compressed to fit in flatland. Finally, it is also an invitation to collaboration, since few of us can realistically hope to master the wide range of skills necessary for authoring in this new medium.