vizliterals: using graphics in and as code
 
Fred Lakin, The Performing Graphics Company, © 2010 2011


 

Rationale:

"Literals or constants are the values we write in a conventional form whose value is obvious. In contrast to variables, literals (123, 4.3, "hi") do not change in value." Dennie Van Tassel, "Literals in Programming Languages," [1]


 

Visual Programming Language design has run into some problems. Most VPL systems are special purpose; as yet there are few if any good general purpose notations. This is in part because designing a visual notation is very difficult, let alone implementing it. Therefore, appropriate tools are needed to make design and experimentation easier. Visual literals can help. They are a notation convention that allows visual objects to be mentioned directly in code which lives in a graphical programming enviroment. Very useful if you are trying to write parsers and interpreters for visual languages (both human computer and human human).

Visual literals are also handy for GUI's and other graphic programming tasks.

For example, here is one way to set the value of a variable using a visual literal (your syntax may vary):

 



 
Discretionary Eval in a ManDiMan Interface

But first some necessary foreplay. Visual literals      using visual objects directly in code statements      can't be done in just any old programing enviroment. The hospitable programming environment must have two properties: manual display manipulation and discretionary evaluation.

Manual display manipulation, or "mandiman," is when all control of objects on the screen is explicitly managed by the user. New objects appear at a place designated by the human, and are then dragged, scaled, evaluated, rotated and edited by her or him until such time as they are erased (also by her or him). Nothing scrolls automatically.

Notice how I slipped in "evaluate" to the list of manipulations above? Discretionary evaluation of visual objects (as instructions for a computer within the framework of a programming language [2]) is one of the things a user can do in a mandiman environment. Mandiman and discretionary eval work together to create a programming environment which supports visual literals. Instead of having two windows, one for code chunks and then a scrolling shell where results are displayed (and statements can be typed in for evaluation), we have just one graphical space. Text and graphic objects are created and simply sit there until evaluated, at which point the result is returned at the cursor. This (no scrolling, remember?) could result in the screen very rapidly filling up with stuff; good thing the mandiman interface is so darn agile that the user can quickly and easily manage the screen manually by dragging, scaling and erasing new and old objects as necessary (just a flick of the finger, to think it is to find it already done).

So, visual literals have been used in the vmacs mandiman environment for quite a while [3]. It's like this: there are code chunks lying around on the screen, with text and also quoted visual objects inline      "visual literals"      and then discretionary eval of any chunk if and only if you explicitly "pop" it, at which point the result is returned at the cursor. Obviously this kind of activity most logically takes place in a graphic editor that is also a programming environment. And a key feature of such an editor is Manual Display Management: nothing ever scrolls unless explicitly scrolled by the user. Yup, back to that.

Here's the ManDiMan spiel:

<rant>
What to do with new output? Display managers that automatically put new stuff at the bottom suck, especially when they make room by moving the old stuff at the top up and out of view. Scrolling is just a hack because we can't make text and graphics do what we want really really fast. So we relinguished display management to robots in order to avoid the responsibility (like in The Day the Earth Stood Still).

Well I for one do not welcome our robo scrolling overlords!
</rant>

Below are various examples from the vmacs mandiman enviroment with discretionary eval. Because anything on the screen can be a form to be evaluated, or be the result of evaluating such a form, in the static images below we have used as a meta visual element (which does not actually appear on the screen) to point out results which are returned at the cursor (which is also shown in the images, looking like , and which does appear on screen).


Examples


 The claim is that the basic visual literal notation technique will work with any language, although langs with interactive interpreters are obviously more fun.


 First example, Lisp version:

 
 
The spider webs represent underlying tree structure of the graphics objects on the screen. They are simply an overlay which can be turned off or on. Their pupose is to show structure without compromising the spatial arrangement of composite visual objects; they allow spatial structure to be completely separate from tree structure. The advantage of this capability will become clearer in comparison when we try to make the Python example work (next section).

They are called "trees" even though they look more like roots. The fine lines show the grouping structure of text graphic objects without violating their visual integrity. They show the objects to be members of ordered, recursive lists (during manual manipulation of objects, the order is often ignored and the lists are just used as simple groupings, except of course when constructing visual objects which are to be evaluated).

Note: all Lisp examples computed and returned on the screen as shown in running vmacs system (and visual programming environment).

Also note: is an example of cross cultural eleetspeak, or l33tspeak or even . In this case the Chinese character tien is used for "E".



 Python version (artist's conception):

 
 
Note it was necessary to cheat a bit for this example because Python's left to right character orientation for mentioning more than one literal makes things a little tricky. The example above mentions the whole pattern of seven parts as a group. But, it turns out that the obvious way to get that composite visual object into the vizpython environment from its parts:

 
 
... doesn't have exactly the result for which we might have been hoping:

 
Plus as the spider webs indicate, the exact underlying structure for the text graphic object submitted to the vizpython interpreter could get messy. In the assignment statement which sets the value of , is each char after the a separate visual object (not as shown) or does the parser have to break up the monolithic string itself (as shown) and then figure out the proper tree? (ugh!) A visually and logically clean syntax for vizpython is left as an exercise for the reader.


More vizlisp examples:
 
 

 
 





 

 
 

 

 
 

(this last form starts a subprocess for animating the drag at 24 frames per second, resulting in the streak)


References

[1]   "Literals in Programming Languages," Dennie Van Tassel, 2004
http://www.gavilan.edu/csis/languages/literals.html

[2]   "Computing with Text Graphic Forms," Fred Lakin, 1980
http://www.pgc.com/cwtgf

[3]   "The vmacs System for Text Graphic Performing," Fred Lakin, 1996
http://www.pgc.com/vmacs

[4]   Visual Grammars for Visual Languages, Fred Lakin, 2010
http://visualgrammarsforvisuallanguages.com