Jenna Kyle's Tell Me

To promote artist Jenna Kyle’s new single “Tell Me”, Director Jay Harwood directed and edited a video meant to be interactive.

Created as a side project, the result was a video watching experience where viewers were able to interact with its original content by using their mouse or touch events, the webcam, and other methods of input. The viewer’s input would control and be mixed with the existing video content, allowing viewers to effectively be part of the music video experience.

On the technical side, this was a project with a lot of fun parts. It was fully made with PixiJS, which allowed building GPU-accelerated graphical user interfaces in a manner similar to Flash, but of course using JavaScript (or, well, TypeScript here specifically) and WebGL.

The result was a generic interactive video player system that allowed separate effects to be triggered during different playback times, taking arbitrary inputs into consideration and presenting the final result. These effects not only took traditional user inputs such as mouse and touch events into consideration, but also grabbed the viewer’s webcam stream. Live face detection was performed in real time so we could properly mix the viewer’s face with the music video in the correct position.

Overall, the idea was that viewers would be able to create customized viewing experiences - not only by using their webcams, but by manipulating parts of the video - and then share those with their friends, or share specific snapshots of the experience to social media.

In the end, this project was never published and thus remains unfinished. The sharing feature likewise remains disabled, although it contains code for capturing images (viewable in its debug version).

Regardless, it was a good learning experience in creating an interactive, GPU-accelerated experience using a Flash-like display list, and even a good experiment in publishing a website using Parcel rather than the traditional webpack.

Previews