In our recent exploration of TouchDesigner, we identified Kinect as a promising tool to offer audiences increased freedom and opportunities for interaction with on-screen visual elements.
Kinect’s Versatility in Various Domains
As noted by Jamaluddin (2021), Kinect was initially designed for gaming purposes, but its application has expanded into other fields. The release of Microsoft’s SDK has enabled its use in medical, robotics, and other sectors through academic research projects, showcasing its adaptability and potential for innovation.
Integrating Kinect with TouchDesigner
By incorporating Kinect into our TouchDesigner workflow, our design team can utilize the motion data captured by Kinect to generate interactive and responsive visuals. Kinect’s real-time data can be seamlessly processed and manipulated in TouchDesigner, enabling the creation of more captivating point cloud effects based on audience movement.
Acquiring Kinect and Initial Testing
To our surprise, we were able to acquire a Kinect device from Ucreate in the library, prompting us to begin developing with this tool alongside Arduino. I followed an online tutorial by The Interactive & Immersive HQ (2020) to complete the initial Kinect connection test (see Figure 1). This straightforward tutorial provided essential information about both Kinect and TouchDesigner. Key takeaways included installing the Kinect SDK first, using CHOP objects to import data from Kinect into TouchDesigner, and understanding the various applications of this data, such as triggering color changes, deformation, and more.
Figure 1: Interacting with particles in TouchDesigner using Kinect, with the Kinect device placed on a ketchup bottle.
Next Steps and Collaboration
Moving forward, the Kinect device will be handed over to Yijun Zhou for further development of gesture-reactive effects. Please refer to the subsequent post for more information on our progress.
If you’re reading this in order, please proceed to the next post: ‘Kinect Phase 2’.
Allison Mu
Reference
Jamaluddin, A. (2021) 10 creative and Innovative Uses of Microsoft Kinect, Hongkiat. Available at: https://www.hongkiat.com/blog/innovative-uses-kinect/
The Interactive & Immersive HQ (2020). Generative visuals with particles & kinect in touchdesigner with Crystal Jow. YouTube. Available at: https://www.youtube.com/watch?v=mY7DavB0z2c