Aesthetic Computing: Toward a Restructuring of Mathematics and Computer Science

Aesthetic Computing

Paul Fishwick, University of Florida


Introduction

As our culture gives way to increasingly affordable methods for creating physical and virtual objects, with rapid prototyping machines and 3D hardware, we are forced to reconsider how we create representations for mathematics, science and engineering. Older languages began with clay tokens, and progressed toward flatter typographically-oriented forms for reasons of economy. It was cheaper to mix inks on papyrus or press a stylus to clay than to use sculptured tokens. The gradual shift toward flatter and more stylized, standard representations has tended to place less importance on the artistic product in favor of efficiency. As a collective culture, we have defined progress as technical efficiency, and this has, indeed, led to great discoveries and more productive lives. Ironically, the very technology that has gradually eroded the role of art in technics is the basis for a rebirth in formal representation. With increased efficiency for creating both virtual and physical customized products, we have an opportunity to re-phrase, re-present, and re-mediate formal representations found in arithmetic, algebra, and software.

The first step in this process is to recognize that there already are aesthetics in mathematics and computing. Mathematicians talk of beautiful proofs, physicists talk of symmetry in their group structures, and computer scientists talk of well crafted programs and algorithms. Donald Knuth, for example, has led the way with his "literate programming" in surfacing beauty, structure, and aesthetics in algorithmic structure. Whether you define aesthetics in terms of beauty or in a more disinterested, Kantian fashion, it remains an important part of formal structure development. Our mathematical culture, and its progeny in computer science, are very much attuned to traditional text-based structure. Additionally, researchers in the area of visual programming and information visualization have begun to pave a way to restructure formal representation.

The only problems with today's computer science aesthetics is that they are limiting given the aesthetic landscape found in art, with its plethora of styles, periods, and genres. If we accept the hypothesis that the limitation of accepting some of these styles is primarily founded on the economy of sign production (i.e., making the formal representations), then our representations should theoretically evolve as the signs become less expensive to produce. In short, if one can create a castle or anthropomorphic agent quickly, then why shouldn't these objects be considered over ink impressions such as "X" and "("? Perhaps new graph and pattern grammars can be enlisted to help us formalize such structures, while maintaining a rational, but personalized, structure?

How will mathematics and computing change as a result of efforts in aesthetic computing? The first change is in the improved psychological metrics such as memory, comprehension, and motivation, assuming that tests for these confirm our expectations. This, in turn, leads to more people understanding and working with formal structures, thus democratizing the field. Education will be the first beneficiary, but like the emergence of the desktop metaphor for Operating Systems, the benefit will gradually move from neophytes to experts, who also ultimately want a more engaging interface to formal structure despite the legacy factor that discourages change. Will reframing or remediating lead to new discoveries in science? This seems likely given the history of diagramming and visual modeling.

A news group has been set up to analyze some of these issues. Please feel free to join us at the "aestheticcomputing" group at http://www.yahoogroups.com

CAPTION: This sculpture represents the dynamics of a pond ecosystem, built from plexiglass sheets, hardwood, and wire. The representation is a Finite State Machine (FSM) reflecting the simple 3-state dynamics of an ecosystem. Artist: Prajakta Ugrankar, University of Florida.

Class Experiences

I recently finished teaching a class on aesthetic computing, with the express purpose of exploring the use of artistic methods and processes within common representations found in computing. Computer Science employs a wide variety of modeling types for portraying information and algorithms. For example, finite state machines (FSM) represent discrete phases for a natural or artificial process, and entity-relationship (ER) diagrams represent how information is connected via entities, attributes, and relations among these. There are many motivations behind aesthetic computing, with the primary one being the ever-increasing trend toward personalization in all products, including human-computer interfaces. Mass customization is affecting not only the array of individualized products afforded by rapid prototyping fabrication and other advances in manufacturing, but also re-presentations of media. Since representations for computing are forms of media, there is ample opportunity in investigating how personalized interfaces and model structures can be used to build alternative views of phenomena and software. If the economy of labor and production permits us to construct both virtual and augmented constructs as easily as typographically-oriented, flat media, then we are on the brink of a revolution in how we think about models for computing, and ultimately, representation in mathematics. We did not choose flat artifacts such as paper because they served as ideal repositories for conveying knowledge. Instead, they were chosen out of purely economic reasons, and with the introduction of the computer, the economies have shifted significantly to permit us to return to ancient forms of representation (albeit in virtual or augmented forms), as well as to generate new ones.

The aesthetic computing class had 10 students, all of whom are enrolled in the relatively new Digital Arts and Sciences (DAS) curricula. DAS students can be enrolled either in the College of Fine Arts or the College of Engineering. While my students happen to come from Engineering, all students take a common pool of courses from both colleges, and in their Junior and Senior years, they take Digital World Production Studio classes to facilitate work in teams. A good deal of what transpired in the 15-week class is stored on the web. I lectured for several weeks, and students each gave talks. Invited lecturers from Art, Music, English (new media), and Information Technology provided needed injections of fresh ideas. Students were required to take specific computing models and translate them into aesthetic, personalized, expressions. The two primary end products of their labor were physical and virtual models. The physical model was, at it sounds, a multimedia sculpture or architecturally oriented piece, and the virtual model was built in software using tools such as VRML, 3D Studio MAX, and Flash. The physical model serves the following purposes: 1) an artwork capturing the semantics of a computing model, 2) a tutorial device for teaching modeling, 3) an architectural model for future actualization, and 4) a prototype for a future tangible user interface with physical components being used to construct the computing model, which is sensed, identified, and automatically input to a computer. The virtual model had similar purposes to the physical, but emphasized attributes hard to achieve with physical materials: interaction, dynamics, and world navigation.

With such a new subject, the students and I discovered new ideas and incrementally forged a methodology for aesthetic computing. Since the students are hybrid artists/engineers, I found it easy to talk about the philosophy and principles underlying semiotics, analogy and metaphor; however, we ran into a snag in the application of the metaphor. The use of metaphor plays a crucial role in taking existing computing model representations and extending these into a more exploratory, aesthetic medium. Ideally, some time in the future, one might imagine building computing models, directly from scratch, using artificial buildings, landscapes, people, and off-the-web-shelf objects. But, we must begin with what we know and extend ourselves into the new domain. Thus, metaphor was carefully applied with clearly enunciated source and target structures to allow students to begin with known model representations in order to generate target aesthetic ones. For example, one might begin with a flowchart (a type of source computing model, representing control flow), invent a target metaphor style (say, landscape architecture), and then specify the formal mapping from source to target. The snag was one where many students applied the metaphor but without any rules. An example of this would be mapping a specific software flowchart to a particular scene from Shakespeare’s Hamlet. While this is a valid mapping, and fascinating as a project, there are no rules that allow for a more general engineering framework. An example of a rule might be “Decision Blocks in the flowchart shall be mapped to stage locations in a Play”. This rule allows people to more easily understand the semantics of the flowchart in the target theatre domain without explicitly knowing Hamlet. It may be that these rules assist in communication and facilitate faster modeling, whereas a dearth of rules promotes a more secure, encrypted, model. So, both mapping approaches may see their uses.

Overall, the class was a great success and a lot of material was presented and absorbed on both sides of the pedagogical fence. There are a number of issues remaining, and many valid, mind-probing questions such as 1) How do we tackle the cultural dilemma of going from flat typographical representations to more aesthetic ones?, 2) Will communication suffer as we promote a personalized Tower of Babel?, and 3) When will the tools catch up to where it will be practical to use these sorts of aesthetically-inspired models over today’s Platonically oriented, aesthetically-challenged representations?

Further Information

There is a News Group on aesthetic computing, and some related papers. There is also an upcoming Dagstuhl Seminar. The following graduate class will be taught in Spring 2002.