We built a virtual spray paint system that allows users to composite photos in an artistic manner.
In its simplest form, spraying paint essentially reveals portions of an image.
In the more advanced modes, paint can also come from multiple images in a slideshow or follow the user's movements.
This application can run with either an immersive or a desktop interface.
Immersive Interface
The immersive interface allows the user to stand and spray as if they were painting on a wall.
Rear Projection Screen
We built a rear projection screen out of a frosted shower curtain and PVC pipe.
The rear projection is necessary to allow the user to paint on a large, wall like surface with blocking the projected image.
The quality of the rear projection screen is surprisingly usable, though there are some bright spots.
It also requires turning off or dimming the lights in the room.
Wiimote Spray Can
We also built a spray paint can so the user can feel like they are painting with a can.
The can uses a Nintendo Wiimote to provide buttons and a wireless, Bluetooth interface to the computer.
We did not use the accelerometers in the Wiimote.
Tracking
We used Optitrak cameras to track the spray can and the user's head.
In theory, it would be possible to use the Wiimote's camera and some external IR lights for tracking the spray can.
Modes
Colored Paint
The user can paint on the screen using simple solid colors.
Since the interface is large and inprecise, we did not want to use a standard color wheel interface for selecting colors.
Instead, we created an interface that allows the user to add or subtract red, green, or blue from the current color by simply spraying at large colored targets.
Image Paints
Paint can also come from source images.
Spraying paint essentialy reveals the colors from the source image.
If paint hits the canvas at position 0, 0, then the canvas takes on the color from the source image at 0, 0.
Stencils
Instead of paint coming from exactly the same positions in the source image as the canvas position, we implemented a movable stencil.
The user can select an image, then draw a stencil on it.
The stencil can be repositioned anywhere on the canvas.
Painting in the stencil will paint the portion of the source image the user initially selected.
Slideshows
Instead of paint coming from only a single source image, it can come from a slideshow of source images.
This allows the user to create things like a background that changes from day to night.
Headtracked Images
In addition to tracking the can, we track the user's head.
This allows us to create source images that move around with the user.
Future Work
Given more time, we would like to have implemented the following:
Shaking the can to activate the menus
Ray casting for the spray can
Layered paint. This would allow the user to erase paint and see the paint under it.
Video paints
The eyes follow me! People often comment how the eyes appear to follow them in a regular painting.
We could implement it so the eyes literally follow the user.
We could also implement paranoid following - the eyes only follow the user when the user is not looking at them.
Sample Images
Source Code
Source code is available here as a mercurial repository. Login with the username public, password public.
