Because the user is allowed to use both hands to carry out gestures, the graphical user interface is split vertically into right and left-hand areas to allow for parallel initialization of two gestures at the same time, one for each hand.
Furthermore, the graphical user interface provides a visual monitoring tool. A highly visible symbolic display is placed in the upper center and shows how many fingertips are detected on the right and left side.
Beside the possibility of executing two unimanual gestures at the same time, the split screen concept also allows for the detection of bimanual gestures as a collaboration of both hands.
From a programmers point of view, three types of gestures emerged:
- gestures defined by one touch unit
- gestures defined by further touch events
- and gestures defined by iterative patterns
Examples for these three gesture types are listed below (from left to right)
A. GESTURES DEFINED BY ONE TOUCH UNIT
Touch units are fingertips that are positioned on the same side of the touch sensitive surface within a very short time span. Once multiple fingertips are registered as a unit they can change their average position and their distance to each other.
The movement of the fingertips influences the pitch of the sound output: The nearer the user moves a touch unit towards himself, the more the pitch drops. The heighest pitch can be produced by moving into the upper left or upper right corner, respectively.
By spreading the fingers the touch units size increases which has a direct influence on the amount of voices used in generating the sound output for of the corresponding side.
The user is able to trigger additional control commands by applying certain motion patterns to touch units.
- For example, by an impulsive upward or downward movement the current octave can be increased or decreased.
- By permanently moving one finger on the horizontal axis the sound output gets rhythmized depending on the distance between the left and right turning points just previously detected. The smaller this distance becomes, the more aggressively the sound output will be sliced.
- By imitating a throw gesture, a reverb effect is triggered at the moment contact is lost with the touch surface. Additionally a panorama parameter is mapped to the touch units direction of movement.
B. GESTURES DEFINED BY FURTHER TOUCH EVENTS
When a short break is detected between a first touch unit and further fingertips, another gesture type is recognized.
- According to this two step principle this gesture triggers a grain delay effect with its duration and intensity correlated to the angle and execution velocity of the gesture.
C. GESTURES DEFINED BY ITERATIVE PATTERNS
When a gesture is applied for a very short time, the algorithm is ready to receive certain patterns again. If a user does this, an iterative gesture can be detected.
- Every time the user knocks on the surface, the sound output gets suppressed by feeding a side chain compressor with the audio signal of a kick drum. So instead of an anticipated knocking sound the output gets muted whenever the user applies this iterative pattern.
If the same gestural behaviour is recognized on both sides of the surface, the algorithm is also able to interpret certain gesture combinations as one bimanual gesture.
- A simple implementation is the crossing of the hands while both are handling touch units. Whenever the system detects that both touch units have left their initial sides and crossed over to the other, the distance in-between the two touch units determines the degree of a distortion effect.
