Demo Video
Make sure you unmute the video if you want to hear what it sounds like!
The Vision
Ever since I learned about projection mapping, I have been wanting to try it for myself. I’m particularly inspired by this creator named Roelof Knol, who’s projections are always interactive. In this project, I wanted to project a guitar string on my wall and touch its image to play nice chords.
Current Status
It works nicely with a glove. I have to play around with thresholding and lighting a little more to get it to work with just my finger. Check out the video at the top of this document to see.
The String
I used Processing to create a virtual string. If a contour gets close to the string it plays a chord and vibrates. Cute little particles emanate from the string each time it’s plucked too.
Chords are randomly picked from a bank I created, however, subsequent chords are picked using strategies inspired by common practices in music theory. After the string is plucked, one of seven strategies is picked. The chord pool is then reanalyzed for chords that fit the criteria defined by the selected strategy. Occasionally when plucked, the string ignores the strategy and picks a random chord (so we don’t get stuck in loops).
See “pluck5.pde” from the GitHub repository.
Detecting When It’s Plucked
I convert my camera feed into binary (black and white) and detect the largest black spaces. (In reality I invert the image and detect the largest white contours.) When a thin shadow (your finger) passes over the string it plucks the string starting at the x-value of the contour.
Check out “center_contours.pde” from the GitHub repository.