Recently, I was trying to find ways to use the Kinect as a sensor to trigger the interactive sounds in max. It would be interesting to let the audience trigger some sound when doing the action, and I heard from the DDM group-mates that they are also trying to use Kinect to trigger some changes in the visual, and it would be better if the changes in the visual are accompanied by the sound trigger. So I looked up a lot of information and videos, but I found that most of the cases of directly connecting Kinect data to Max are very old, and many of them use software called SYNAPSE for Kinect (link to SYNAPSE webpage below), but it doesn’t support Kinect for Windows. Some later cases also have different problems for me, they do not support Max8 or Mac, making it difficult for me to proceed.
At this point, I changed my way of thinking, because I heard that Kinect can access touch designer smoothly, so I can try to make touch designer as a relay station to transfer data from Kinect to Max to achieve interactive sound triggering, and in this way, the relationship between visual and sound is also closer. After some research, I found a related tutorial (link below) that makes transferring data from Touch Designer to Max very easy.
At the same time, I’m also curious about if I can change the direction of the data transport, which is from max to touch designer, since visual part also need the data from ultrasonic and we don’t have to use 2 sensors if we can send the data from max to touch designer. Meanwhile，I heard my teammates found a way to control the video speed by the amplitude of the audio wave, and my opinion was why not combine it with the interactive sound of going up and down stairs, using it as the file to control the visual, then it will be a great linkage of the sound and the visual.
In odder to achieve that, I learned the OSC in protocol and after some research and practice, it succeeded and worked very smooth.
So far, we can achieve at least two things:
Kinect -Touch Designer-Max: to achieve the sound triggered by the audience’s movements.But since our sound was already rich enough, we didn’t end up triggering the sound by this method.
Max -Touch Designer: We have used Max and ultrasonic sensors to precisely implement the interactive sound is triggered whenever the audience goes up and down the steps, so we can use these triggered sound effects(use a snapshot to get the level between ±1) to control the speed of the video in Touch Designer.
Chenyu Li, 3.29
Test records and logs：
3.28 find out the method and test on one computer(TD and Max in same IP address)
3.29 communicate with group mates on touch designer, confirmation of feasibility and preparation of experiments
3.31 Test out with touch designer group mates with different computer, after succeed, communicate with David and put it in to the whole max patch, and test out again
4.3 scale the number of output with touch designer group mates for a better visual effect
If you’re reading this in order, please proceed to the next post: ‘Sonification #1 – Data Collection, Treatment and Processing’.
