Multi-touch Flash Application using touchlib

The Natural History Science Museum of Los Angeles wanted to create a large kiosk that many guests could operate at the same time. This would make using a conventional mouse-based kiosk impossible because a mouse has a single point of control. Ideally, the designers at the museum wanted a large screen that can have objects manipulated by multiple control points (i.e. multiple “mouse” pointers) allowing for a shared experience. Cost was also a factor, so 8 foot touch screens were not an option.

An open source movement exists for software libraries designed to manage multiple touch events on an ordinary computer. It works by registering the x and y coordinates of blobs seen via a web cam on a translucent glass screen. These blobs are then sent to the event model as custom touch events and handled by the application. Touchlib is the software we used for our project and the experience was overwhelmingly positive.

The software is released under the Open Source New BSD license, which gives you the possibility to use it and modify it in every circumstance. It can be found here:
http://code.google.com/p/touchlib/

One of the main quality attributes of the application was that it had to be easily extensible. Flash Actionscipt 3.0 was the language of choice for the application since staff at the museum already knew the language, and it was the most platform agnostic. This became critical during deployment when the target machines moved from being PCs to Mac Mini computers.

Flash integration was easier than I had anticipated. Once the touchlib library captures the x and y coordinates from blobs registered from the web cam, it sends them via UDP to a port registered by a lightweight java server running in the background. Flash can listen to the port for UDP events and call them as needed. All this is done in about two lines of Flash code:

First, import the touchlib Flash library — import app.core.action.RotatableScalable;
Then, start listening to events over UDP — TUIO.init( this, ‘localhost’, 3000, ”, true );

There is a substantial number of Flash commands and events already written out and stashed in the utility library ready for use. We overloaded some of the behaviors to prevent runaway scaling of images and added some custom behavior to prevent text from being scaled too small. Most of the additonal functionality had to do with inteface animation and user controlled interface design. Since the Flash app is constantly listening for UDP events it is wise not to overload it with other events that are not crucial — be sure to clean up Timer events or EnterFrame events that are not needed.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s