A Visual Agent for Performance Graphics


Visual Agents are software entities which assist people in performing graphical tasks. One useful and interesting graphical task is to make a text-and-graphic record of a group meeting. When the recording is done live in view of the group on a large display, such as on a whiteboard, it is a performing art which can be called ``Group Graphics.'' During group graphics, collaboration is enhanced because the group members can see the record as it is being made, offering suggestions and corrections. And if graphic recording of this kind takes place on a large computer display, then visual agents can be brought into play.

Presented in this paper is a visual agent which acts as a whiteboard assistant for group graphics, helping a person to graphically record the conversation and concepts of a working group on a large display.


Figure 1. Visual agents in vmacs have complete access to user actions and the state of the visual world in the midst of text-graphic performance.


Visual agents operate in the context of the vmacs system for performance text-graphics.

Previous work has been done on constructing an interface for improvising graphics in the service of facilitating cognition for both groups and individuals. Initially much of the effort simply went into making the human-machine interface fast and rich enough to support this kind of activity [Lakin80a, 86a, 86b, 89]. The resulting medium is an extremely agile graphics editor with no pop-up menus to distract the audience; it is called vmacs.

However, from the very beginning of the effort, it was clear that group graphics was an extremely demanding task. Manipulating text-and-graphics at the speed of group thought is hard. Although gifted human performers can and do manage it on recalcitrant media such as large sheets of paper, it was clear that most performers could use some assistance. So in parallel with the interface design, effort also went into development of computational engines for processing text-graphic activity, both the static images resulting from the activity as well as the actual moment-to-moment dynamics of the activity itself [Lakin80a, 80c, 83, 87, 90, 92].

The visual agents described in this paper are the result of that research.


Context: Performance Text-Graphics on the Big Screen

A Visual Agent to Assist Group Graphics
Graphic Dialog with a Visual Agent
Figure 1. Visual Agents in vmacs have complete access to user actions and the state of the visual world in the midst of text-graphic performance
Figure 2. Dave's response when the user draws two connecting lines

Visual Agent Architecture
Figure 3. Visual Agent architecture showing relation between Controller and Visual Parser
Figure 4. Visual Agents in vmacs communicate with each other through multi-media postings to a visual blackboard
Implementation of Dave the Visual Agent
Figure 5. Machine-readable Visual Grammar Notation for the context-free grammar used by Dave in parsing user generated text-graphic patterns
Historical Summary

Related Work

Current Status

Future Work


This paper was presented at the 1994 AAAI Spring Symposium on Software Agents at Stanford University in March 1994, and published in the working notes, and republished in the book SOFTWARE AGENTS, Oren Etzioni, Program Chair, Technical Report SS-94-03, ISBN 0-929280-59-8, The AAAI Press. The paper was also accepted and published in the Believable Agents Symposium at the same conference.

It is also available on the Web in PostScript (sans figures temporarily), or on paper by request from PGC.

Return to papers list

(C) Copyright 1994, 2009 PGC