4.3 Parsing and Interpretation of Visual Phrases in the User Interface

The text-graphic objects representing parses for four of the visual communication objects are shown in Figure 2 (the `spiderwebs' are a concise way of diagramming tree structure recovered by parsing; the more traditional notation will be shown later). The four parses were all performed in the same image space by the same function: parse-appropriately tries spatial-parse with each grammar from a list until the parse succeeds (if no parse succeeds, then ``unrecognized visual expression'' is signaled). The claim is that parse-appropriately represents progress is building user interfaces for formal visual languages: we now have one general purpose function with a list of `known' formal visual languages for which it can handle visual expressions. The elements for the expressions were created in a general purpose graphics editor, and then the spatial syntactic context of the elements was utilized in parsing them.



Figure 2. Appropriate parses for different visual communication objects residing in the same image.

After parsing a visual expression, the structure thus recovered is then used directly by a specialized semantic processor in taking appropriate communicative actions. Interpretations based on the parses in Figure 2 are shown in Figure 3. Again, the four interpretations were all performed in the same image space by the same interpretation function, interpret-appropriately. interpret-appropriately calls parse-appropriately, and selects the proper semantic processor based on the result of the parse.



Figure 3. Appropriate interpretations for different visual communication objects residing in the same image.