The Physical Lab of the Interaction Design program owns four selfmade multitouch tables mainly for research and exhibition purposes.

Informations about the hardware setup of these tables can be found on the developer blog of Tim Roth.

I was able to use a commercial software package from Evoluce which includes a tracking software (Multitouch Input Manager) and a Flash Actionscript library developed for handling multitouch inputs (Multitouch Flash Framework). Once implemeted, it's really easy to handle every kind of touch events detected by the infra-red camera. Instead of normal actionscript objects like MovieClips this multitouch extension allows to initialize MultiTouchObjects which automatically are zoomable and rotatable with at least two fingers. Anyhow, most of the potential of this library I never used because the idea behind my project is not to focus on visual representation but on the recognition of different gestural inputs.

The algorithm represents the main part of the whole project. It is completely written in Flash Actionscript 3.0 and makes use of the FLOSC extension to send OSC messages out to the sound engine. Thus two tasks are processed by the Flashplayer: a.) the evaluation of touch events and the according generation of OSC messages and b.) the visual output of the application.

As soon as a touch contact has been detected, the initialization phase is coming into force which allows the registration of 'touch unities' by the support of a time buffer. Once this phase has expired, further touches can be received as long as a certain value for movement tolerance won't be exceeded by the other fingertips.
If they were moving or further touches could be registered, the initialization phase is completed and the detected gesture is ready to be executed. From that moment, all the changes of the gestures states and parameters are registered permanently while the algorithm is accordingly generating data outputs. The complete touch abortion of the surface cancels a gesture but in many cases the last data signal produced by a gesture is used to provide a closing 'touch up trigger'.

If a gestures duration has been short, the same gesture can be repeated. By doing so, an iterative gesture will be initialized but only if the time span between the previous touch abortion and the following touch contact has been short as well. As long as these conditions are complied, the initialization phase won't come into force again but an iteration counter will be incremented by every new execution of the gesture.

Because the sound engine I planed to use, Ableton Live, is not able to receive or send OSC messages, I was in need of a link between the Flash application that supports sending OSC and Ableton Live that only is able to receive MIDI messages. My first idea was to use the commercial software OSCulator which is an easy tool to translate OSC to MIDI but because of many reasons I decided to develop such a tool on my own (one reason was that I wanted the application to run on Windows and OSCulator is only compatible with Mac OS X). To program such an OSC-MID-bridge I chose Processing wherefore I imported the library extensions oscP5, rwmidi and proXML.

The first two libraries ensure the communication between the two applications, the last represents an interessting way to store the OSC/MIDI message relation. By writting all my OSC/MIDI mappings into an XML file it became a dictionary for the translation of OSC messages into MIDI messages which is easy to modify beause it is not part of the Processing application itself.

Here's a little code example that represents the node structure:

For generating sounds I'm using the sequencer software Ableton Live together with serveral software synthesizers of Native Instruments Komplete 5. Ableton Live comes with an easy to use mapping function that let users intuitively connect their hardware controllers with certain sound parameters of the software. 

The possibilty of using effect racks and its macro parameters was very helpful to configurate a more complex, software internal mapping setup. MIDI data that is sent from the OSC/MIDI bridge then controls many parameter values simultaneously. Also the concept of splitting the surface into two sides which allows the user to execute unimanual gestures with both hands matched well with Ableton Lives structure: For each side the same set of tracks is used whereby they're all routed through a subgroup (and its effects). According to this, MIDI messages generated by bimanual gestures control parameters of Ableton Live's master output.