Embodied Controls for Mixing and Composition allow you to play and record an instrument while simultaneously mixing it with pre-recorded tracks, using five special features:
Dynamic Volume, Dynamic Reverb, Musician Spatialization, Mix Control, Track Panning
You can now download the code (along with extensive installation instructions and a user manual) from my Github repository at:
http://github.com/delshimy/REMC
Disclaimer: While I have done my best to test the code extensively, I may have missed a thing or two. As a result, your feedback is invaluable! Feel free to contact me with any questions, comments or suggestions you may have
System Features
Imagine you are playing in a band with two other musicians, each of whom is assigned pre-recorded track. You will have access to a graphical user interface, seen in the figure below, that offers a top down view of all the band members, including yourself, as avatars. You are represented by the red avatar, while the other “virtual” musicians are represented by the that have been assigned the labels ‘1’ and ‘2’.
You can then play your instrument and interact with the other band members’ tracks according to the following system features:
Dynamic Volume
As you move towards one of the band member’s avatars, you can experience the pre-recorded track associated with that band member as gradually increasing in volume. The converse holds true as you move away from that band member’s avatar.
Dynamic Reverb
As you move away one of the band member’s avatars, you can experience the pre-recorded track associated with that band member as gradually increasing in reverb. The converse holds true as you move away from that band member’s avatar.
Musician Spatialization
This features allows you to experience the pre-recorded tracks as spatialized sound sources within your own space. In other words, the track of the band member whose avatar is to your left will appear to come through the left headphone, while the track of the band member whose avatar is to your right will appear to come through the right headphones. The spatialization effect is determined by your body orientation, and changes accordingly.
Mix Control
This feature allows you to change the mix of your instrument with the pre-recorded tracks by changing your body’s orientation. Turning your body to the left will move the sound of your instrument, along with that of the band member whose avatar is to your left, entirely to the left headphone. The track of the band member whose avatar is to your right will be heard unaccompanied through the right headphone. The converse holds true when you turn your body to the right.
Track Panning
You can isolate each of the pre- recorded tracks by changing your body’s orientation. Turning your body to the left will allow you to hear only the track of the band member whose avatar is to your left, entirely through the left headphone. The track of the band member whose avatar is to your right will become silent. Your own instrument will continue to sound the same, coming through both headphones. The converse holds true when you turn your body to the right.