The primary way to store information on computers and digital media today is in text form and the reasons for this are traditional, it's easy to search and store text and it takes up little space. Stepping outside the traditional and limited sign systems of the digital era, allowing users to introduce their own signs, drawings and diagrams, seamlessly with typed text, will emulate the properties of traditional paper based media and add semantic richness to digital content.

Today we are seeing new technologies emerge, technologies that allow us to use computers more or less the way we have used other media, for example the traditional notepad. This blog entry is about the notepad, its advantages and easy of use, and its communicative power in allow us to both write and draw to convey concepts. For others to understand, as in the communal notepad used in education and meetings, or for ourselves such as when takes notes during a lecture.

The notepad has many nice properties that designers of new digital replacements have attempted to copy, most prominently its size, weight and accessibility. Today on the web we still rely on typing for getting our message through and use ugly ASCII drawings to illustrate relationshops between concepts and to create diagrams. With the advent of handhelds with touchscreen interfaces that allow you to use your stylus much like you use your pen, and laptop computers with similar interfaces, there are now no technical reasons why we can't sketch, draw and type on the screen a lot like we sketch, draw and write in our notepad.

The text-based approach that has dominated the digital world for so long does have benefits however. Text stored on a computer has meta properties. It isn't just a series of signs, it is abstract and allows advanced manipulation such as copying and transfer, a feature regular writing lacks. When you jot something down you use signs that have conventionalized value, people who understand your handwriting will at least be able to recognize the letters you used, and if they know the language they will probably understand what you have written. When you type on the computer your are restricted to a set of predefined symbols, characters and letters, which all have a predefined meaning. Since it's all standardized and the symbols are abstract, you can display the same text (same combination of symbols) in several formats, with different layout, font face or color. You can also process the content, classify and analyze it statistically, and store it for quick and easy retrieval.

The notepad allows you to be inventive, you can create your own signs, you aren't limited to what your keyboard and the computer's sign system afford. You can invent your own system of signs, you can write in code that consists of signs that no else knows.

Below is an example, this is an old very famous code, something I learnt as a kid being a member of the Swedish scout organization. Writing in code is one of the things scouts do, I would assume it is related to the originally military traditions of the movement, ideals that were later discarded for a more pacifist agenda (in Sweden anyway).

The symbols above were drawn in Fireworks using my Wacom tablet but I hope they're clear enough, and enough well-defined to make sense of. The key can be found at the end of this page. 

Another example where drawing is handy is when you need to draw a map for someone else to follow:

If you were trying to convey the same message over email you'd have to make an ASCII drawing, like this:

                       |
                       |
                       |        ^  my
            old        |       [ ] house
              windmill +----------
            \  /       |
           __\/__      | 200 m
< stockhom  /|\        |
           /_|_\       |
-----------------------+----------------------
        3 km           |
                       |
                       |
                       |
                       |                      

Now wouldn't be great if we could use text and any arbitrary symbols we wanted, intermixed, and it wouldn't require any special hardware of software? Imagine if you could make a drawing, with your mouse or stylus, right on the screen in the textfield of your blog or on a forum and have the computer treat it as a drawing. Giving you the same freedom expression as that afforded by the notepad while at the same time allowing you to scale and edit it by storing it as vector graphics.

Taking it a step further, both typing and writing could be supported and drawings could have abstract meaning, semantic value, that would "make sense" to computers as well. We cannot use bitmaps for this purpose as they are semantically dead, and have non information regarding their meaning or their relationship with other content. A way this could work would be linking two typed words with a drawn line to establish a concept, doing it using either a toolbox of set signs and symbols, like in Microsoft Visio, or by letting the computer interpret your drawing as a formalized diagram. The possibility of using an intuitive and human input format and still maintain the benefits of the digital medium and its abstraction and meta information is there.

Can we do this today? I believe so. We have standards for storing and exchanging text (XML), we have standards for presenting it (XSL, XSLT and CSS) and a standard for semantically rich vector graphics (SVG), all it takes is combining it. We need a rich media control for all major operating systems that allow you not only to type and format your text in terms of font face, color and layout, in terms of its semantic type (headers and paragraphs) but also allows you to draw on the screen, introducing novel signs and figures, with a relationship to the text in order to illustrate or explain a concept. We need a unified format for storing this sort of information, on the web and elsewhere, making it seamless and effortless to store and retrieve without apparent separation between content types, separation introduced due to technical constraints rather than conceptual difference.

The key to the code above.

Trackback URL for this post: