UNIVERSITY H.C.I. PROJECT
Musical Trigger Gloves.
A Musical and Theoretical Interface.
Dave Corsie 1628876
MSc Audio Technology
7MU007 Musical Human-Computer Interaction 2016/17
The Initial Concept Idea
The original idea was to be able to play music without an instrument, using gloves with buttons/triggers/contacts on the ends of the fingers. These would be connected to a computer and pass signals to let the computer know what notes to play. The initial inspiration came from watching the progressive rock band Marillion in concert some years ago. The singer, Steve Hogarth, used a cricket bat to play keyboard sounds. This cricket bat had some triggers installed in it, and he had it slung around his neck, in the style of a guitar. The cricket bat was played with two hands, using one on the handle, and the other on the striking surface of the bat. Only a handful of triggers were used, and these set off pre-configured samples/arpeggios during the live concert. The use of a cricket bat on stage as an instrument caught my imagination, and shows that triggers can be installed in almost anything for a strong visual impact. Another inspiration came from watching Imogen Heap in concert at a WOMAD festival in 2010. Imogen uses a variety of high tech equipment in her music, which is mostly electronic vocal pop. Imogen has been in consultation with developers of the Mi.Mu gloves, and she was using a very early version in 2010. The gloves have been developed since then, and are now in production. They have been used by a variety of musical artists, including Ariana Grande on her 2015 world tour, as well as other performance artists and theatre practitioners. More information about the Mi.Mu gloves can be found here; http://mimugloves.com/
The Mi Mu gloves use sensors and gyroscopes to sense movement and direction, and this makes for a more visual experience, as the artist makes music using movement of the hands in air. Sounds and textures need to be pre-programmed, and then activated using the gloves. The concept for this project was entirely different.
The idea of the gloves was to be able to play in a more traditional sense, albeit without an instrument to hand. Triggers/sensors in the fingertips would tell the computer what notes to play, and a variety of ideas were tried in order to get the sensors to be pressure sensitive, enabling a more expressive performance. Instead of each fingertip corresponding to a set note, the idea was to have the left hand chose the key, or root of a chord, and the right hand play the degrees of the chord, ie root, third, fifth, seventh, etc. This would enable more choice of notes than the fingers could give using one note per finger. In order for there to be some more variation, sensors were also used on the palms, and when several sensors were used together, a full range of chromatic keys was made available on the left hand, while the degrees of the chord/key were triggered on the right hand.
Construction and Testing
In consultation with Mat Dalgleish, course leader, the original design for the sensors was to use pressure sensitive pads made from carbon impregnated foam. The idea was that as the foam was compressed, the voltage going through the pad would change, as more air is expelled, and the molecules get closer together, forming a stronger conductive medium. Initial tests showed that this concept could work, but the foam was not quite dense enough to give a wide signal variation, and therefore show much pressure variation. A variety of different thickness and density foams were tried, but there was not a strong enough or stable enough signal for this to be a practical proposition for this concept.
Fig 1. Testing of foam pressure pad on end of finger.
Unfortunately, the idea of pressure sensitive pads was abandoned at this stage, and conductive copper tape was introduced, which would give a strong electrical connection. The copper tape was tried in conjunction with the foam to see if a better connection could be established, but the results were still not stable enough or broad enough to give consistent or reliable results.
Figure 2. Testing of foam pressure pads with conductive copper tape.
Conductive thread was used to connect the pads, which was sewn into the gloves along the back of each finger, then terminated at the wrist. Wires were then attached to make a connection to the computer via an arduino circuit board.
Figure 3. Initial testing of copper tape.
Software/Coding
Along with the Arduino circuit boards, Pure Data software was to be used for converting the electrical signals into the correct notes to be produced when certain triggers were connected. As the two hands needed to be used together (one hand to set the ‘key’, while the other chose the notes to be played), there was a lot of calculations to be done, working out all the different permutations that could be used to get the full range of notes.
Video 1. Test of pressure pad.
Figure 4. Table of possible combination outcomes with unique numbers for each.
After some time spent figuring out the best way to get as many of the chromatic notes available within the number of finger combinations, a palm sensor was also added. This gave some more options, and allowed the use of all twelve chromatic keys to be used on the left hand. Another sensor on the thumb knuckle gave options to have different types of 7th chords. With another palm sensor on the right hand, further degrees of the key/chord could also be added
Figure 5. Finger combinations for left hand. Figure 6. Finger combinations for right hand notes.
Once the finger combinations had been sorted out, enabling the gloves to act in a musical fashion (could also be used to demonstrate musical theory as well as performance), the unique numbers could then be allocated to specific midi note information. Each midi note has a specific value, starting with A0 being 21, up to C8 being 108.
Figure 8. Midi note values in grid format.
Figure 7. Midi note numerical values.
While this concept works well in theory, at time of writing, it has not been fully tested and operational.
Practical Use and Further Development
The gloves were conceived with the idea of being able to play music without an instrument at hand, using a computer or other sound generator to hear the sound produced. While it can be used for performance, it would also suit well to help with understanding music theory.
The idea is still a relatively new concept, and new ways of conceiving it are being developed. The use of micro pressure switches could be used as a way to bring back some touch sensitivity. Further triggering options are also being explored.
Wireless is probably a better way to use the gloves, as they are a little cumbersome with several wires attached. Bluetooth is getting better, and is finding its way into many new ideas. The quality of Bluetooth audio has improved greatly, and would be a good addition to the gloves, if it could be made practical. Alongside a computer, having a dedicated app to run the gloves would make them more user friendly, as well as more portable. Tablets, iphones and ipads could be a good market for the gloves, and if an app could be developed that incorporates the facility to record, then this would really open up the market for these gloves. Most people use headphones with phones and tablets, so this could also be a way to play music anywhere, as the gloves themselves would become the instrument, being triggered by touching anything.
Conclusion
While there are some other musical gloves in production, the concept of these gloves is unique. The potential for them is at a very early stage, but they could be developed much further with time and investment. The prototypes would serve the purpose of showing what the concept is, but would need refinement and better quality materials to show their full potential. The programming and coding could also be refined, and a new way of figuring out possible finger combinations is already in progress.
The limits of these gloves are not just within the musical sphere. Essentially, if they are looked at as a pair of gloves with sensors, then the coding could be altered to make them do a variety of different tasks. In this project they have been put to use in a musical environment, but they could just as well be used for sensory exploration and development, or in a variety of educational areas, where touch plays a part of learning.