Visual grammars can be useful in the spatial parsing of formal visual languages. Spatial parsing allows a general purpose graphics editor to be used as a visual language interface. This provides the user with the freedom to use different visual languages in the same image, blackboard-style. He or she can first simply create some text and graphics, and then later have the system process those objects under a particular system of interpretation. The task of spatial parsing can be simplified for the interface designer/programmer through the use of visual grammars. For each of the formal visual languages described in this paper, there is a specifiable set of spatial arrangements of elements for well-formed visual expressions in that language. Visual Grammar Notation is a way to describe the spatial critieria (or rules) which distiniguish those sets of spatial arrangements and the associated underlying structures; the context-free grammars expressed in this notation are not only visual, but also machine-readable, useable directly to guide the parsing. Once a visual grammar has been written for a formal visual language, parsing can be accomplished. And once parsed, expressions can then be processed semantically and appropriate action taken. Visual grammars and semantic processing for four formal visual languages have been presented.
Understanding informal conversational graphics taking place in a general purpose graphics editor is the broader theoretical context for this work. Enroute to the overall goal of computer participation in conversational graphics (such as blackboard activity), we began with the parsing of special purpose visual languages. Not only are they simpler, but since they are often embedded in (and may have `grown out of') general purpose graphics activity, lessons learned there will likely be applicable to the more difficult problem.
The development of spatial parsing within vmacs has profited from contributions by Harlyn Baker, John Bear, Pascal Fua, Scott Kim, Larry Leifer, Mike Lowry, Paul Martin, Rob Myers, Alex Pentland, Fernando Pereira, Warren Robinett, Ted Selker, Stuart Shieber, Josh Singer, Richard Steele, Hans Uszkoreit, Mabry Tyson, and Machiel Van der Loos.