4. Spatial Parsing for Visual Languages

The previous section introduced the notion of spatial parsing for visual languages and explained some advantages. This section will discuss such parsing in more detail as a user interface technique. The next section will then present visual grammars as a way to accomplish spatial parsing.

4.1 Definitions

The purpose of spatial parsing is to aid in the processing of visual languages. As an operational definition of visual language, we say: A visual language is a set of spatial arrangements of text-graphic symbols with a semantic interpretation that is used in carrying out communicative actions in the world.* Spatial parsing deals with the spatial arrangement of the text-graphic symbols in a visual phrase from a visual language: Spatial parsing is the process of recovering the underlying syntactic structure of a visual communication object from its spatial arrangement **.

4.2 Examples of Visual Languages

Examples of communication objects (or visual phrases) from five different visual languages are shown in Figure 1 (images were constructed in the vmacs(TM) graphics editor, Section 8). The first four communication objects are from formal visual languages; the fifth object is a piece of informal conversational graphics to remind us of the theoretical context of this work. An expression in a simple Bar chart language is in the left corner of Figure 1. Feature Structures are a notation employing brackets of differing sizes to encode information for a natural (textual) language expression. The Visual Grammar Notation uses text and graphics to represent a context-free grammar for visual communication objects. SIBTRAN is a fixed set of graphic devices which organize textual sentence fragments and provide additional semantic information. And the title block in the lower left corner is from an Informal graphic conversation discussed in Section 7.



Figure 1. Visual communication objects from 5 different systems.



* Note that if we substitute ``strings of textual symbols'' for ``spatial arrangements of text-graphic symbols'' we have something strikingly similar to a characterization of written natural language. Interestingly, the text-graphic definition includes the textual one. A paragraph of text is one kind of arrangement of text-graphic symbols.

** Again the definition is parallel to one for textual parsing: textual parsing is the (rule-governed) process of recovering the underlying (in the sense that it was used to generate the linear form) syntactic structure from the linear form of a sentence.