Richard Stoakley, Matthew J. Conway, Randy Pausch
The University of Virginia
Department of Computer Science
Charlottesville, VA 22903
{stoakley | conway | pausch}@uvacs.cs.virginia.edu
(804) 982-2200
virtual reality, three-dimensional interaction, two-handed interaction, information visualization
This paper explores a user interface technique which augments an
immersive head tracked display with a hand-held miniature copy of the
virtual environment. We call this interface technique the Worlds in Miniature (WIM) metaphor. In addition to the first-person perspective offered by a virtual reality system, a World in Miniature
offers a second dynamic viewport onto the virtual environment. Objects
may be directly manipulated either through the immersive viewport or
through the three-dimensional viewport offered by the WIM.
In addition to describing object manipulation, this paper explores ways in which Worlds in Miniature
can act as a single unifying metaphor for such application independent
interaction techniques as object selection, navigation, path planning,
and visualization. The WIM metaphor offers multiple points of view and
multiple scales at which the user can operate, without requiring
explicit modes or commands.
Informal user observation indicates that users adapt to the Worlds in Miniature metaphor quickly and that physical props are helpful in manipulating the WIM and other objects in the environment.
Many benefits have been claimed formally and informally for using
immersive three-dimensional displays. While virtual reality technology
has the potential to give the user a better understanding of the space
he or she inhabits, and can improve performance in some tasks [17], it
can easily present a virtual world to the user that is just as
confusing, limiting and ambiguous as the real world. We have grown
accustomed to these real world constraints: things we cannot reach,
things hidden from view, things beyond our sight and behind us, and
things which appear close to each other because they line up along our
current line of sight. Our virtual environments should address these
constraints and with respect to these issues be "better" than the real
world.
In particular, we notice that many implementations of virtual
environments only give the user one point of view (an all-encompassing,
immersive view from within the head mounted display) and a single scale
(1:1) at which to operate. A single point of view prohibits the user
from gaining a larger context of the environment, and the 1:1 scale in
which the user operates puts most of the world out of the user's
immediate reach.
FIGURE 1
To address these two concerns, we provide the user with a hand-held
miniature representation of the life-size world (figure 1). This
representation serves as an additional viewport onto the virtual
environment and we refer to it as a World in Miniature
(WIM). The user can now interact with the environment by direct
manipulation through both the WIM and the life-size world. Moving an
object on the model moves the corresponding life-size representation of
that object and vice versa.
Previous work has explored the use of miniature three-dimensional
models for visualization and navigation. Our contribution is in
extending this work and integrating it with object selection and
manipulation under the unifying Worlds in Miniature
metaphor. As an adjunct to the WIM, we have explored the advantages and
disadvantages of grounding the user's perception of the model with a
physical prop; in this case, a clipboard.
The rest of this paper discusses related previous work, a description
of our WIM implementation, the basic interaction techniques we have
used to demonstrate the effectiveness of the WIM concept, and the
importance of asymmetric two-handed interaction for this metaphor. We
conclude with results from informal user observation of the WIM
interface and a discussion of future work.
Many researchers have dealt with the open questions of
three-dimensional object manipulation and navigation in virtual
environments. The World in Miniature
metaphor draws on these previous experiences, and attempts to
synthesize an intuitive, coherent model to help address these
questions. Previous work falls into three categories: object
manipulation, navigation in virtual environments and object selection.
We define "navigation" to cover two related tasks: movement through a
3D space and determining orientation relative to the surrounding
environment.
Ware's Bat [22] interface demonstrates the use of a 6 degree-of-freedom
(DOF) input device (a position and orientation tracker) to grab and
place objects in a virtual environment. In this work, Ware used the bat
to pick up and manipulate the virtual objects themselves, not
miniature, proxy objects. Ware found that users easily understood the
1:1 mapping between translations and rotations on the input device and
the object being manipulated. This study was a unimanual task and did
not place the user's hands in the same physical space as the graphics.
In Sachs's 3-Draw [19], we see two hands used asymmetrically in a
three-dimensional drawing and designing task. In addition to this,
Sachs used props for each of the user's hands and found that relative
motion between hands was better than a fixed single object and one free
mover. 3-Draw was not implemented in an immersive, head-tracked
environment and the system did not provide multiple, simultaneous
views. The input props controlled the point of view by rotating the
object's base plane.
Hinckley's [13] work with props exploited the asymmetric use of hands,
which follows from work by Guiard [12]. This work showed how a prop in
the non-dominant hand can be used to specify a coordinate system with
gross orientation, while the user's preferred hand can be used for fine
grain positioning relative to that coordinate system. This work is also
three-dimensional but non-immersive and directly manipulates an object
at 1:1 scale in a "fishtank VR" paradigm.
3DM [3] was an immersive three-dimensional drawing package, but
provided only one point of view at a time and required the user to
change scale or fly explicitly to manipulate objects which were
currently out of arm's reach. Butterworth states that users sometimes
found the scaling disorienting.
Schmandt's [20] early explorations of Augmented Reality (AR) used a
half-silvered mirror over a stationary drafting tablet in order to
specify both a base plane and a slicing plane in computer generated
VLSI models. He found this surface invaluable in constraining the
user's input to a plane. The system was not immersive and presented a
single scale at any given time.
Aspects of Bier's Toolglass and Magic Lenses
[2] work, closely resemble a two-dimensional analog to the WIM. For
example, when using a magnification lens, objects can easily be
manipulated at a convenient scale, while maintaining much of the
surrounding context.
Darken's [7] discussion of navigating virtual environments enumerates
many important techniques and compares their relative strengths and
weaknesses. Several of the navigation techniques presented were
WIM-like maps, but were primarily two-dimensional in nature. Through
the WIM interface, some of these techniques have been extended into the
third dimension.
Ware [23] explored the possibilities of holding the three-dimensional scene in hand for the purpose of quickly navigating the space. He found this scene in hand
metaphor particularly good for quickly viewing the bounding-cube edges
of a scene. The scene in hand task was a unimanual operation which
employed ratcheting to perform large rotations.
The work most closely resembling the WIM interface was Fisher's map cube
in virtual reality [9]. The NASA VIEW system used a three-dimensional
miniature map of the immersive world to help navigate. In addition, the
VIEW system used multiple two-dimensional viewports to jump from one
place in the virtual environment to another. A user's manipulation of
the "map cube" was unimanual. A similar map-cube concept was referred
to as the God's-eye-view in the super cockpit project [11].
Many researchers have explored methods for selecting objects in a
virtual world. Common approaches include raycasting [10] [22] and
selection cones [14]. Both of these techniques suffer from object
occlusion and therefore need to be tied closely with some mechanism
that can quickly establish different points of view.
Put-That-There [4] used selection via a combination of pointing and
naming (or description). Pointing in this two-dimensional application
is analogous to raycasting in virtual environments.
To explore the benefits and limitations of the WIM metaphor, we built a
simple three-dimensional modeling package that could be used as a
design tool for a traditional architecture design project called a Kit of Parts.
We provide the user with an immersive point of view through a
headmounted display (HMD) tracked by a Polhemus 6 degree of freedom
tracker. To control the position and orientation of the WIM, we
outfitted the user's non-dominant hand with a clipboard attached to
another Polhemus position sensor. The surface of the clipboard
represents the floor of the WIM. By looking into the clipboard, the
user gets a aerial view of the entire scene.
In his or her other hand, the user holds a tennis ball modeled after
the UNC cueball, in which we have installed two buttons and another
Polhemus sensor. This buttonball was used as the selection and
manipulation tool for all of our user observation and WIM development.
By pressing the first button on the buttonball, users can select
objects; the second button was left open for application-specific
actions (figure 2).
FIGURE 2
The WIM graphics attached to the clipboard are nothing more than a
miniature copy of all the surrounding graphics in the immersive
environment. Each of the objects in the WIM copy are tied to their
counterparts in the immersive environment through pointers and vice
versa at the point of WIM creation. In this way, when an object
responds to a method call, the object has enough information to ensure
that the same method gets called on its "shadow" object. Thus the user
can manipulate the objects in the WIM and the objects in the world will
follow (video figure 1 - The WIM Interface). The environment itself (in
miniature) becomes its own widget for manipulating objects in the environment [26].
The Kit of Parts modeler was implemented using the Alice Rapid
Prototyping System [6] running the simulation on a Sun Microsystems
Sparc 10\xaa and rendering on a Silicon Graphics Onyx Reality Engine
2\xaa . Typical rendering rates were about 25 frames per second (FPS),
while simulation rates were typically 6 FPS. A Virtual Research Flight
Helmet\xaa was used for the display and was tracked with a Polhemus
Isotrak\xaa magnetic tracker. The buttonball and clipboard each carried
a Polhemus tracker sensor for position and orientation information.
In this section, we discuss basic application independent WIM-based interaction techniques.
Being able to see objects from many different angles allows us to
quickly remove or reduce occlusion and improves the sense of the
three-dimensional space it occupies [21]. Because the WIM is a
hand-held model, the user can quickly establish different points of
view by rotating the WIM in both hands. Note that this form of "WIM
fly-by" can often give the user all the information that he or she
needs without destroying the point of view established in the larger,
immersive environment. We believe that this interaction technique can
establish a new viewpoint more quickly and with less cognitive burden
than a technique that requires an explicit "flight" command and
management of the flight path.
If the virtual, immersive environment is sufficiently large, there will
be objects that are out of physical arm's reach. If the user must touch
an object to select it, the user would have to employ a separate flying
mechanism, which means moving the camera; a sometimes disorienting or
otherwise inappropriate approach. Armed with a World in Miniature,
the user now has the choice of selecting objects either by pointing to
the object itself (as before) or by pointing to its proxy on the WIM.
By turning the model in his or her hands, the user can even view and
pick objects that are obscured by his or her current line of sight from
the immersive camera viewpoint. The WIM provides a second (often
"bird's eye") point of view from which to examine the scene.
Once objects are selected, the WIM allows us to manipulate those
objects at either the scale offered by the WIM or the one-to-one scale
offered by the immersive environment. If the scale of the WIM is
smaller than that of the immersive world, manipulating objects on the
WIM necessarily gives the user far-reaching coarse-grained control of
objects.
The WIM can also display objects at a greater than one-to-one scale,
implementing a three-dimensional magnifying glass of sorts. This gives
the user very fine grain control of objects through the WIM at the
expense of range. Though we have not implemented zooming in our current
system, we clearly see the need for allowing the user to get more
detail on the WIM or to zoom out to view more context. We are currently
pursuing this avenue of research.
We speculate that because the WIM is clearly a model attached to the
user's hand, it is seen as something separate from the rest of the
immersive environment. The WIM therefore naturally offers two different
scales to the user without requiring explicit modes or commands.
Putting these ideas together, we can consider an example task: hanging
a picture on a wall. This task is typical of a broad class of
two-person tasks in which the proximity required to manipulate an
object interferes with the desire to see those effects in a larger
context. With a WIM, a single user can stand at a comfortable distance
to view the picture in context, while at the same time reaching into
the WIM to manipulate it.
Of course, the user could choose to use the WIM the other way around:
fly close to the wall to stand next to the picture, then use the WIM to
view the entire room in miniature to determine if the picture is
straight. Examining relative strengths and weaknesses of each of these
approaches is an area for further study.
There are other common ways to approach the manipulation of objects at
a distance. For example, a ray can be used to select and manipulate an
object. Unlike the WIM, ray-casting suffers from an occlusion problem
and a lever arm problem, as mentioned before.
Viewing, selection, and manipulation are independent operations.
Because the WIM gives the user another scale at which to operate, the
user can choose the most appropriate scale for any given subtask, and
even switch scales in the middle to suit the requirements of the task.
For example: the user can reach into the WIM to select a distant object (taking advantage of the greater than 1:1 scale of the WIM), and then reach out to the immersive world to move the WIM-selected object at a distance in 1:1 scale [22] [14] all the while viewing the scene in the WIM.
Our current implementation allows users to rotate objects, through ratcheting
(repeated grabbing, rotating and releasing) [24] and is therefore more
awkward than a rotation done with just the fingers [14]. Interestingly,
some users found it just as effective to grab the object and to
counter-rotate the entire WIM.
In our current implementation, rotation is gridded to 30 degree
increments, primarily to assist in aligning rectilinear objects [14].
We found that if the rotation grid is too course (greater than about 45
degrees), some people assume that they cannot rotate at all and if set
to 15 degrees or less, users report that aligning rectilinear objects
is very difficult.
To make the view travel through the immersive environment, the most
common user interface technique in virtual environments is probably
"flying." If the WIM includes some representation of the user as an
object in the scene, the user can simply reach into the WIM and "pick
himself up" to change his location in the environment. This raises the
question of when to update the immersive world as objects in the WIM
are manipulated. We enumerate three possibilities.
When changes are made on the WIM, we usually move the real object and the proxy object simultaneously, something we refer to as immediate update.
Under some conditions, immediate update is either not desirable (due to
visual clutter or occlusion) or impossible (the complexity of the
object prevents changes to it from being updated in real time). In
these situations, we use post-facto update, where the immersive environment updates only after the user is finished with the WIM interaction and has released the proxy.
A good special case of post-facto update is the case of moving the
user's viewpoint. We find that immediate update of the camera while the
user is manipulating the camera proxy is highly disorienting, so
instead we wait until the user has stopped moving the camera, and then
use a smooth slow in / slow out animation [15] to move the camera to
its new position. This animated movement helps maintain visual
continuity [14].
Another useful form of update delay is batch update. Here, the user makes several
changes to the WIM and then issues an explicit command (e.g. pressing
the second button on the buttonball) to cause the immersive environment
to commit to the current layout of the WIM. This is useful for two
reasons. First, before the user commits his or her changes, the user
has two independent views of the environment (the "as-is" picture in
the immersive world and the "proposed" picture in the WIM). Secondly,
it might be the case that moving one object at a time might leave the
simulation in an inconsistent state, and so "batching" the changes like
this gives the user a transaction-like commit operation on the changes to objects in the scene (with the possibility of supporting rollback or revert operations if the changes seem undesirable halfway through the operation).
The Worlds in Miniature
metaphor supports several kinds of displays and interaction techniques
that fall loosely under the heading of visualization. These techniques
exploit the WIM's ability to provide a different view of the immersive
data with improved context. It would seem that the WIM is good for
visualization for all the same reasons that a map is good for
visualization:
Spatially locating and orienting the user: the WIM can
provide an indicator showing where the user is and which way he or she
is facing relative to the rest of the environment.
Path planning: with a WIM we can easily plan a future
camera path in three-dimensions to prepare for an object fly-by. The
user can even preview the camera motion before committing him or
herself to the change in the larger, immersive viewpoint.
History: if the user leaves a trail behind as he or she
travels from place to place, the WIM can be used like a regular 2D map
to see the trail in its entirety. Dropping a trail of crumbs is not as
useful if you cannot see the trail in context [7].
Measuring distances: the WIM can be configured to display
distances between distant (or very closely spaced) points that are
difficult to reach at the immersive one-to-one scale. The WIM also
provides a convenient tool for measuring areas and volumes.
Viewing alternate representations: the immersive
environment may be dense with spatially co-located data (i.e.
topological contours, ore deposits, fault lines). The user can display
data like this on the WIM, showing an alternate view of the immersive
space in context. The improved context can also facilitate the
observation of more implicit relationships in a space (e.g. symmetry,
primary axis) and can display data not shown in the immersive scene
(e.g. wiring, circulation patterns, plumbing paths). Here, the WIM
might act as a filter; similar to a three-dimensional version of Bier's
"magic lenses" [2] or one of Fitzmaurice's "active maps" [10].
Three-Dimensional Design: the WIM serves many of the same
functions that architectural models have traditionally served (e.g.
massing model, parti diagram).
Until now, we have considered only a single instantiation of a WIM in a
virtual environment, but clearly there might be a reason to have more
than one such miniature active at a time. Multiple WIMs could be used
to display:
- widely separated regions of the same environment
- several completely different environments
- worlds at different scales
- the same world with different filters applied to the representation
- the same world displayed at different points in time
This last option allows the user to do a side by side comparison of
several design ideas. A logical extension of this notion is that these
snapshots can act as jump points
to different spaces or times, much the same way hypertext systems
sometimes have thumbnail pictures of previously visited documents [1].
Selecting a WIM would cause the immersive environment to change to that
particular world [9] (video figure 2 -- Multiple WIMs).
Multiple WIMs enable users to multiplex their attention much the same
way Window Managers allow this in 2D. These multiple views into the
virtual world, allow the user to visually compare different scales
and/or different locations [8].
Through the exploration of the previous interfaces, several issues
arose concerning the interface between the human and the WIM tool.
One of our early implementations of the WIM work did not use physical
props; the user grasped at the WIM graphics as he or she would any
other graphical object in the scene. As long the user continued the
grasping gesture, the WIM followed the position and orientation of the
user's hand and when released, it would remain hovering in space
wherever it was dropped. While this was sufficient for many tasks, we
found that rotating the WIM without the benefit of haptic feedback was
extremely difficult. Invariably, users would contort themselves into
uncomfortable positions rather than let go of the WIM to grab it again
by another, more comfortable corner.
After Sachs [19], we decided to use physical props to assist the user's
manipulation of the WIM itself. To represent the WIM, we chose an
ordinary clipboard to which we attached a Polhemus 6 DOF tracker for
the user's non-dominant hand. For the user's preferred hand, we used a
tennis ball with a Polhemus tracker and two buttons.
This prop allows the user to rotate the WIM using a two-handed
technique that passes the clipboard quickly from one hand to the other
and back when the rotation of the WIM is greater than can be done
comfortably with one hand. Interestingly, some users hold the clipboard
from underneath, rotating the clipboard deftly with one hand. Both of
these techniques are hard to imagine doing in the absence of the haptic
feedback provided by a physical prop.
Before we settled on the buttonball as our primary pointing device, we
experimented with a pen interface to the WIM. This technique is most
appropriate for manipulation of objects when they are constrained to a
plane [20] (the base plane being the default). When manipulation of
objects in three-dimensions is called for, a pen on the surface of the
clipboard does not appear to be expressive enough to capture object
rotation well.
Our implementation of the WIM metaphor takes advantage of several
previously published results in the field of motor behavior that have
not been fully exploited in a head tracked virtual environment [2]
[12]. The most important of these results state that a human's dominant
(preferred) hand makes its motions relative to the coordinate system
specified by the non-dominant hand, and the preferred hand's motion is
generally at a finer grain [12]. In our case, the non-dominant hand
establishes a coordinate system with the clipboard and the dominant
hand performs fine grained picking and manipulation operations.
While the dominant hand may be occupied with a pointing device of some
kind, it is still sufficiently free to help the other hand spin the WIM
quickly when necessary.
Like all real world artifacts, the shape of the props and the users'
experience suggest things about the usage of the props [16]. For
example, the shape of the clipboard says something to users about its
preferred orientation. The cursor's physical prop is spherical,
indicating that it has no preferred orientation, and in fact it does
not matter how the cursor is wielded since rotation is relative to the
plane specified with the non-dominant hand, which holds the clipboard.
The clipboard also provides a surface that the user can bear down on
when necessary. This is similar to the way an artist might rest his or
her hand on a paint palette or a guitarist might rest a finger on the
guitar body.
The use of the clipboard as a prop presents some problems of its own.
Holding a physical clipboard, even a relatively light one, can cause
users to fatigue rather quickly. To overcome this problem, we created a
simple clutching mechanism that allows the user to alternately attach
and detach the WIM from the physical prop with the press of a button.
When detached, the WIM "floats" in the air, permitting the user to set
the prop down (video figure 3 -- Prop Clutching). This clutching
mechanism extended well to multiple WIMs: when the user toggles the
clutch, the closest WIM snaps to the user's clipboard. Toggling the
clutch again disengages the current WIM and allows the user to pick up
another WIM.
Another technique for relieving arm stress is to have the user sit at a
physical table on which the clipboard could be set. Users can also rest
their arms on the table while manipulating the model. The presence of
the table clearly presents a mobility problem for the user because it
prevents the user from moving or walking in the virtual environment and
so may not be ideal for all applications.
In our experience, one of the first things a user of the WIM is likely
to try is to hold the WIM close to his or her face in order to get a
closer, more dynamic look at the world. Users quickly discover that
this is an easy, efficient way to establish many different points of
view from inside the miniature. Unfortunately, many times the physical
prop itself gets in the way, preventing the user from putting the
tracker in the appropriate position to get a useful viewpoint.
Fortunately, the ability to disengage the WIM, leaving it in space
without the clipboard helps alleviate this problem.
We observed ten people using the WIM. Some had previous experience with
virtually reality, and some had three-dimensional design experience
(e.g. architecture students). Users were given a simple architectural
modeler and asked to design an office space. We proceeded with a rapid
"observe, evaluate, revise" methodology to learn about the Worlds in Miniature interface.
The user was given a miniature copy of the room he or she was standing
in, and a series of common furniture pieces on a shelf at the side of
the WIM (figure 3). The user was then asked to design an office space
by positioning the furniture in the room.
FIGURE 3
In many ways, this design task replicates the traditional architectural design project known as a Kit of Parts.
The furniture pieces (half-height walls, shelves, tables and table
corner pieces) represent the kit of manipulable objects. Moving the
traditional kit of parts project into virtual reality was particularly
appealing for several reasons:
- It constrains the space in which the user can work.
- Since users are designing a life-size space, clearly seeing that space at 1:1 scale is helpful.
- The WIM succinctly replaces the plans, elevations and models typically associated with this type of design project.
The WIM that we used was a 1/4" scale version of the immersive world,
with a snap spacing of 1/8" (0.5 scale feet). In addition to the
translation snap, rotation was constrained to be about the Z axis in
increments of 30 degrees.
We observed users in order to see how viable a solution the WIM
interface was to several types of tasks. While it was not our intention
for the study to produce concrete numbers, we were after what Brooks
refers to as interesting "Observations" [5]. We hoped to gain some
sense of:
1 ) How quickly do users take to the WIM metaphor?
2 ) How do users like the weight and maneuverability of
the physical clipboard?
3 ) Do users like clutching? Do they take to it easily?
4 ) How do users feel about moving themselves via the camera proxy?
None of the users expressed problems establishing the mapping between
objects on the WIM and objects in the life-size virtual world. Several
users were even able to grab and manipulate objects on the WIM without
explicitly looking at the WIM itself. This was the strongest evidence
that people had developed an internal model of the three-dimensional
space established by the WIM in their hands.
Arm fatigue was not a major problem. Subjects developed a pattern of
raising the WIM, making the manipulations and then lowering the WIM out
of view to look at the life-size world. Also, subjects were sitting,
which allowed them to rest their arms on their legs and adjust their
gaze to look down at the WIM. The weight of the head-mount was the only
fatigue complaint we received, although most users were not immersed
for more than fifteen minutes.
The participants generally understood and liked the clutching
mechanism. They seemed to have no problem associating the second button
on the buttonball with the clutching of the clipboard.
Users found that rotating the camera with immediate update of the user
view was very disorienting. Not surprisingly, we also found that the
animation of the user's view from a starting point to it's ending
point, needed to be as smooth as possible. Problems arose when
rendering frame rates were insufficient due to environment complexity
and when users tried to move great distances in a short period of time,
causing large translation deltas for any given rendering frame.
Intentional misregistration: Interestingly, none of our users noticed that the position of the WIM is about a ten centimeters above
the physical location of the clipboard. This was done to allow users to
place the center of the buttonball on the floor of the WIM. Without
this offset, the closest a user could get to the floor is the radius of
the buttonball. When asked whether the miniature world was in
registration with the clipboard, all the users said yes, even though
they often moved the cursor through the base plane of the virtual model
during placement tasks. This of course should be impossible because the
clipboard would prevent the physical buttonball from penetrating
its plane. We believe this to be a good example of the perceptual
phenomena known as "visual capture", (roughly: the visual perception
system dominating the proprioceptive system) [18].
Selection by trolling: Pressing the buttonball's selection button and trolling
through objects in order to select them seemed a much more effective
selection technique than the traditional "position-then-select"
technique. Because the world was sparsely populated with objects, users
had few problems accidentally passing through (hence selecting) the
wrong object. Few users discovered the trolling technique on their own.
We found it extremely useful to take advantage of the well known backface culling
option on the SGI hardware. By modeling rooms such that the polygons
making up interior walls face into the room, the viewer could look at
the WIM from any angle without the exterior walls occluding the
interior while leaving the walls on the opposite side of the WIM
visible (video figure 4 - Backface Culling). We later decided that this
backface culling greatly improved the usefulness of the WIM for
many tasks, and had the hardware not performed this, we would have
implemented it ourselves. Interestingly, none of our users in our
informal studies noticed this effect until it was pointed out to them.
Another issue related to our software implementation was that the
lighting effects on the model do not represent the lighting effects
seen on corresponding objects in the virtual world. This may not be a
problem for many applications, but will probably be an issue for
classically trained architects who traditionally use models to test
lighting.
Because of the encouraging experience we have had with the current WIM
implementation, we intend to continue exploring some of the issues
behind WIM-based interaction.
Value of the Prop: While anecdotal evidence suggests that the WIM
interface with the clipboard prop is faster than manipulating the WIM
with a virtual hand, an informal study is planned to determine to what
extent the presence of the prop enhances the WIM interface.
Scrolling: At some point, scaling issues cause a straightforward WIM
interface to break down. For example: if the WIM's scale is very small,
selection and manipulation become to difficult. One solution is to
maintain a reasonable scale, and allow the user to scroll the world
into the working volume of the WIM.
Clipping: Clipping a large WIM in X and Y seems intuitive, (clip to the
comfortable limits of the user's reach) but the clipping in Z remains
an issue yet to be explored.
Zooming: When the immersive world is large, we need to adjusting the
scale of the WIM to keep the working volume within the user's physical
reach. We intend to explore the best ways of extending this
functionality to the WIM.
Multiple Participants: The WIM may serve as a useful tool for multiple
user interaction. Any number of users could manipulate their own
personal WIM and all participants could watch the effects in the
life-size environment and on their own WIM's.
Three-Dimensional Design: We would like to give the WIM to more
architecture students to determine whether it is a more effective
design medium for some tasks than traditional pencil, paper and foam
core modeling board.
Two-Dimensional Widgets in VR: Even thought the
environment is three-dimensional, it seems clear that there is still a
place for some of the 2D widgets to which we have become accustomed
(e.g. buttons). We have speculated that the clipboard, and possibly a
table would be viable surfaces on which to project standard 2D GUI
widgets, whenever those controls would make sense. We see this as an
interesting opportunity to explore ways of "recovering" decades of 2D
GUI results in the relatively new arena of virtual environment
interfaces and head tracked displays [25].
The WIM interface gives the user of an immersive three-dimensional
environment a chance to operate at several different scales through
several different views at once without engaging explicit modes. From
our informal user observations, we believe some of the most important
features of this technique to be:
- intuitive use of the WIM metaphor
- multiple, simultaneous points of view
- multiple scales for selecting and manipulating objects
- easy manipulation and selection of objects at a distance
- improved visual context
- removal of occlusion and line of sight problems
- general applicability across a wide range of application domains
The most encouraging result from our informal observations was that
users fluidly used multiple capabilities of the WIM. For example, users
often manipulated objects in the World in Miniature
while simultaneously changing their point of view. This level of
integration implied to us that users were able to focus on the task
without being distracted by the interface technology.
We would like to thank Rich Gossweiler, Ken Hinkley, Tommy Burnette,
Jim Durbin, Ari Rapkin, and Shuichi Koga for their help in the
construction of the WIM interface, for the creation of the supporting
video tape, and for their many invaluable ideas. We would also like to
thank the SIGCHI reviewers for their many thoughtful suggestions and
improvements.
[1] Apple Computer, Inc.. Hyperscript Language Guide: The Hypertalk Language. Addison-Wesley 1988.
[2] Eric A. Bier, Maureen C. Stone, Ken Pier, William Buxton, Tony D. DeRose, Toolglass and Magic Lenses: The See-Through Interface. SIGGRAPH '93, pp. 73-80.
[3] Jeff Butterworth, Andrew Davidson, Stephen Hench, Marc Olano, 3DM: A Three Dimensional Modeler Using a Head-Mounted Display. Proceedings 1992 Symposium on Interactive 3D Graphics, pp. 135-138.
[4] Richard A. Bolt. Put-That-There. SIGGRAPH '80, pp. 262-270.
[5] Frederick P. Brooks, Grasping Reality Through Illusion: Interactive Graphics Serving Science. SIGCHI '88, pp. 1-11.
[6] Robert DeLine, Master's Thesis. Alice: A Rapid Prototyping System for Three-Dimensional Interactive Graphical Environments. University of Virginia, May, 1993.
[7] Rudy Darken, John Sibert, A Toolset for Navigation in Virtual Environments. UIST '93, pp. 157-165.
[8] Steven Feiner, Clifford Beshers, Visualizing n-Dimensional Virtual Worlds with n-vision. Proceedings 1990 Symposium on Interactive 3D Graphics, pp. 37-38.
[9] Scott Fisher, The AMES Virtual Environment Workstation (VIEW). SIGGRAPH '89 Course #29 Notes.
[10] George Fitzmaurice, Situated Information Spaces and Spatially Aware. Communications of the ACM, 36 (7), 1993, pp. 39-49.
[11] Dr. Thomas A. Furness, III, The Super Cockpit and Human Factors Challenges. Human Interface Technology (HIT) Laboratory of the Washington Technology Center, September 1986
[12] Yves Guiard, Asymmetric Division of Labor in Human Skilled Bimanual Action: The Kinematic Chain as a Model. The Journal of Motor Behavior, pp. 486-517, 1987.
[13] Ken Hinckley, Randy Pausch, John C. Goble, Neal F. Kassell, Passive Real-World Interface Props for Neurosurgical Visualization. SIGCHI '94, pp. 452-458.
[14] Jiandong Liang, Mark Green, JDCAD: A Highly Interactive 3D Modeling System. 3rd International Conference on CAD, pp. 217-222, August 1993.
[15] John Lassiter, Principles of Traditional Animation Applied to 3D Computer Animation. SIGGRAPH '87, pp. 35-44.
[16] Donald Norman, The Design of Everyday Things. Doubleday. 1988.
[17] Randy Pausch, M. Anne Shackelford, Dennis Proffitt, A User Study Comparing Head-Mounted and Stationary Displays. Proceedings '93 IEEE Symposium on Research Frontiers in Virtual Reality, pp. 41-45.
[18] Irvin Rock, The Logic Of Perception. Cambridge, p.70. MIT Press, 1983.
[19] Emanuel Sachs, Andrew Robert, David Stoops, 3 Draw: A Tool for Designing 3D Shapes. '91 IEEE Computer Graphics and Applications, pp. 18-25.
[20] Chris Schmandt, Spatial Input/Display Correspondence in a Stereoscopic Computer Graphic Work Station. SIGGRAPH '83, pp. 253-262.
[21] Ivan Sutherland, A Head-Mounted Three Dimensional Display. Fall Joint Computer Conference, pp. 757-764, 1968.
[22] Colin Ware, Danny R. Jessome, Using the Bat: A Six-Dimensional Mouse for Object Placement. '88 IEEE Computer Graphics and Applications, pp. 65-70.
[23] Colin Ware, Steven Osborne, Exploration and Virtual Camera Control in Virtual Three Dimensional Environments. Computer Graphics 24 (2), pp. 175-183, 1990.
[24] Colin Ware, Using Hand Position for Virtual Object Placement. The Visual Computer 6 (5), pp. 245-253. Springer-Verlag 1990.
[25] David Weimer, S. Ganapathy, A synthetic visual environment with hand gesturing and voice input. CHI '89, pp. 235-240.
[26] Robert C. Zeleznik, Kenneth P. Herndon, Daniel C. Robbins, Nate Huang, Tom Meyer, Noah Parker, John F. Hughes, An Interactive 3D Toolkit for Constructing 3D Widgets. SIGGRAPH '93, pp. 81-84.
FIGURE 1
FIGURE 2
FIGURE 3