In recent years, adding interactivity to projected content has become increasingly popular in various fields and revolutionized the way audiences interact with the subject, opening new possibilities for creativity and innovation.
The primary goal is to create a more engaging and memorable experience for the audience by allowing them to actively participate in the content being presented. This can be achieved through a variety of methods, ranging from touch-sensitive projection surfaces and motion tracking to mobile device interaction and augmented reality.
General interactivity
Touch-sensitive projection surface – Use an interactive whiteboard or touch-sensitive screen overlay to turn projection surface into a giant touchscreen. This allows users to interact with the content directly, such as clicking buttons, manipulating objects, or drawing on the surface.
Motion tracking – Integrate motion tracking technology like the Microsoft Kinect or Leap Motion to track user movement and gestures. This allows users to interact with the content without touching the projection surface, for example, swiping through slides or controlling a character with body movements.
Mobile device interaction – Create a companion app or web-based interface that audience members can use on their smartphones or tablets to interact with the projected content. This can include voting, answering questions, or controlling aspects of the video.
Augmented reality – Use AR technology to overlay digital content onto the physical environment, enhancing the user experience. Users can interact with the content through their mobile devices, creating a more immersive experience.
Voice recognition: Incorporate voice recognition software to enable users to interact with the content using voice commands. This can be done using platforms like Amazon Alexa, Google Assistant, or Apple’s Siri.
Physical controllers – Provide users with physical controllers, such as a gamepad, joystick, or custom hardware, to interact with the content. This can add a tactile element to the experience and allow for more precise control.
Social media integration – Incorporate social media platforms like Twitter, Facebook, or Instagram to enable users to participate in real-time discussions or share their thoughts about the content.
Gamification – Introduce game elements like challenges, quizzes, or leaderboards to encourage user engagement and interaction.
Real-time data visualization – Display real-time data, such as audience opinions, poll results, or sensor data, to create a dynamic and engaging experience that changes based on audience input.
Collaborative tools: Provide tools that allow users to work together, such as shared drawing boards or document editors, to encourage collaboration and interaction between audience members.
Video Interactivity
Interactive video software – There are several interactive video platforms available, such as HapYak, Wirewax, or Kaltura, that allow for creation of clickable hotspots, branching narratives, or embed quizzes and polls within the video. Users can then interact with the video content itself by clicking or tapping on the screen.
Custom web-based video player – Develop a custom HTML5 video player that allows users to manipulate parameters of the video, such as speed, color, or filters. This can be done using JavaScript libraries like Video.js or Plyr, which provide APIs to control various aspects of the video playback.
Real-time video processing – Implement real-time video processing techniques using tools like MSP, OpenCV, WebGL, or Three.js to apply effects, filters, or transformations to the video based on user input. This can create a more dynamic and interactive experience, as users can see the changes they make to the video in real-time.
Video control interface – Develop a separate user interface (UI) that allows users to control different aspects of the video, such as volume, playback speed, or scene selection. This can be done through a web-based interface or a companion mobile app that communicates with the video player.
Interactive overlays – Add interactive overlays on top of the video, such as buttons, sliders, or dials, that allow users to control the video parameters. This can be achieved using web technologies like HTML, CSS, and JavaScript, or by using specialized tools like Adobe After Effects or Vuforia for augmented reality experiences.
Data-driven video content – Create a data-driven video experience where the video content changes based on user input or external data sources. This can be done using tools like D3.js for data visualization or custom software development to manipulate video content in real-time.
Tablet interactivity
Remote control interface – Develop an app or web-based interface for the tablet that functions as a remote control for the video. Users can control playback, volume, scene selection, or other parameters of the video directly from the tablet.
Data input and visualization – Allow users to input data or make selections on the tablet, which then update the video being projected in real-time. This can include adjusting parameters like color, filters, or playback speed.
Augmented reality (AR) interaction – Develop an AR app that uses the tablet’s camera to recognize the projected video and overlay digital content on the tablet screen. Users can interact with this digital content, which can then affect the projected video.
Multi-device synchronization – If multiple users have access to tablets, you can create a synchronized, interactive experience where users can collaborate, participate in quizzes, or control different aspects of the video simultaneously.
Motion or gesture control – Use the tablet’s built-in sensors, like the accelerometer and gyroscope, to enable users to control the video through motion or gestures. For example, tilting the tablet could change the playback speed, or shaking it could trigger a specific event in the video.
Social media integration – Incorporate social media platforms within the tablet app or interface, allowing users to participate in real-time discussions, share their thoughts, or submit feedback about the video.
Specific applications
TouchOSC – a modular OSC (Open Sound Control) and MIDI control surface app for iOS and Android devices. You can use it to send control messages to your video player or custom software, which then needs to be programmed to respond to those messages. This can help you control the video parameters or trigger specific actions using the tablet. https://hexler.net/products/touchosc
Vuforia – an augmented reality platform that allows you to create AR experiences for iOS, Android, and Unity. You can use it to develop an AR app that recognizes your projected video and overlays digital content or interactive elements. Users can then interact with these elements using the tablet. https://developer.vuforia.com/
Resolume Arena – a live video mixing software that supports OSC and MIDI control. You can use a tablet app like TouchOSC or Lemur to send control messages to Resolume Arena, allowing you to manipulate video parameters, effects, and layers in real-time. https://resolume.com/
Watchout – a multi-display production and presentation system. You can use it to control and manipulate video content on multiple screens or projectors. Watchout supports external control through protocols like TCP/IP, DMX, and MIDI, which means you can use a tablet to control the video parameters remotely. https://www.dataton.com/products/watchout
QLab – a live show control software for macOS that supports video, audio, and lighting control. You can use an iOS app like “QLab Remote” to control QLab from your iPad or iPhone. This allows you to trigger video cues and adjust parameters remotely using the tablet. https://figure53.com/qlab/
Isadora – a media server and visual programming environment for macOS and Windows. It supports OSC and MIDI control, allowing you to use a tablet app to send control messages and manipulate video parameters in real-time. https://troikatronix.com/
