PRE2017 4 Groep2

Subject

A semi-professional guitar player, called Joop, has suffered from a stroke. This caused the right part of is body to be numb for the most part. He cannot play the guitar anymore. This project is all about him and starting a journey to get him back on the stage.

Information about Joop

Joop is a guitarist, who has been playing in three bands. He also performed. The kind of music used to play was music in the genre of like Pink Floyd, Led Zeppelin and Van Halen, including them. He played this using a plectrum and a lot of feeling coming from his left arm in bending the notes. This is the music we strive for to bring back to the guitar for Joop. In the summer of 2017 he suffered from a stroke. The stroke caused brain damage, which in turn led to the loss of functionality of the right side of his body. Fortunately, Joop does not experience any pain.

Parts of the right side of his body that lost functionality:

• He has some difficulty with talking.
• He cannot use his elbow or wrist or fingers.
• He cannot use his knee or ankle.

Parts of the right side of his body that are limited in movement:

• He can move his right arm just from his shoulder.
• He can still walk with a rather stiff leg.
• He can still pressure his index finger and thumb against one another.

In the six month following the incident, Joop went a lot forward with his health. After the six months the improvements stagnated.

Objective

The focus of this project is to make a start of making it possible for Joop to play the guitar again. This includes finding out what are the possible ways to create an interface for him, with which he could play again. These interfaces need to be natural for him and should fit within his style of playing.

The goal for this project is to make a clear overview of the movements Joop can make. Joop should be able to use these movements for a couple of hours at a time, comfortably. These movements should also be natural and contribute to bringing his previous style to life again or make him feel that he can use his emotions in his music. This means that Joop should be able to use these movements or combination of movements to play fast songs and solos from Van Halen but also produce controlled and meaningful sounds for music like Pink Floyd. Playing fast notes is not the only challenge. Playing chords should also be possible which presents different challenging parts than playing fast. Normally only a single string is played but with chords, several strings can be chosen at the same time. Of course, I will also come up with ideas for this but I will not experiment with this.

Approach

The process that is used for this project is mostly iterative. The first step is a literature review, which is done outside of the iterative loop.

The iterative step is the test phase. For all possible ways for Joop to play the guitar, there need to be little tests. These are shown under the section tests. They will be carried out one after the other. These tests will have a conclusion on the used movement in the test, whether they are useful or not.

Literature Review

In this project a foot pedal has been made and a mechanism which replaces the strumming hand. The system on the guitar strums each string of the guitar while the player frets each string[1]. The signals from the foot pedal are send wirelessly to the unit on the guitar, which actuates the strings based on the signal. The strings are played by knobs that go up or down over the strings controlled by actuators. These knobs are attached to the actuators by a spring. These knobs wear out in a couple of hours. Currently, the actuators can only play the strings at one volume.

This is a paper that shares a mechanism on the neck of the guitar which replaces the fretting hand. This done by a mechanism that pushes down the strings when commanded to using a foot pedal[2].

This is a company that makes custom instruments voor people with physical disabilities. They do this by, for example, using conductive paint. That makes for a kind of piano like playing experience[3].

This company has made a universal stand. This stand is also suitable for a guitar[4].

“Experiment with “open tunings” on the guitar – a guitar in open tuning requires less complicated fingering of chords and can be played using a slide to bar all strings on a fret to produce the notes desired. The guitar can be played lying face up on one’s lap, with the right hand fingers picking strings, and the left hand using the slide bar to the required fret. This may sound complicated to a non guitar player, but a guitar teacher who is familiar with open tunings and playing blues style guitar will understand this method. It’s important to keep an open mind and have an adaptive approach to the training. Usually where there’s a will, there’s a way. Playing a guitar with a slide is used in many styles of music and it’s kind of a shortcut to playing sooner. For one-armed guitarists – tune the guitar in open “g”, place face up on lap, place steel or glass slide on left pinky to bar frets. Attempt to strum with remaining fingers – or if the other arm has any gross movement, with that arm.”[5]

Experiments with Joop

While the right side of Joop's body cannot be used for playing guitar, the rest of his body is completely fine. Therefore, other parts of Joop's body need to take over the functionality of his right hand. Some experiments need to be carried out to see what other movements are useful and natural for Joop.

Foot piano

The name of the experiment says it all. A keyboard to be played with a foot is being used to see how easy it is to get used to this kind of movement. A tube around the largest toe is used to play the individual notes.

Joop played like this for two times 15 minutes a day for a week long.

Hypothesis

At the end of the week, the ability to play the piano with the foot is there but the speed is quite low.

Actual outcome

After the one week practice, Joop was able to some songs on the piano with his foot but he needed to go slow to be able to play the songs.

Conclusion

The side-wards movement in his foot is not fast enough. So, it will not be possible to have kind of keyboard, with a key for every string. Although the playing experience was very comfortable.

Foot pedal

In this experiment a MIDI foot pedal will be used to 'select a string'. This is done by having a different note played at different angles of the pedal. This is done by using the software Max[6] This is needed to see if the up and down wards motion of the foot has the comfort of the previous experiment and the speed that is needed to play fast notes. It is set-up so that when a certain angle is held on the pedal, one keystroke on the keyboard can generate a note. The first key on the keyboard will trigger the equivalent note of an open string. Every half step higher on the piano will add a half step to the selected string, just like a guitar. The goal is to see how reliably Joop can pick the angles of the pedal with his foot. For the pedal six ranges are used to imitate the six strings on the guitar. Every range has the same size.

Products used

• Expression pedal[7]
• MIDI keyboard [8]

Hypothesis

Joop is able to use this pedal for a couple of hours without any strain. Although it will be getting used to, a week is enough time to play Pink Floyd songs, with an accuracy of at least 80%.

Actual outcome

TBD

Conclusion

TBD

Future experiments

During the meetings with Joop, there were always thought of new ways to control the guitar. To try out all these ideas, an experiment should be conducted. In this section all these ideas will be explained and a possible method of testing will be presented.

Summary of the movements

All the movements that show great promise from the tests or have been thought of while conducting interviews, are in this section.

Bought gear and software

In this section will be a list of stuff that is bought for the tests with the links shops where they were bought. Not everything on this list used in the experiments but they are probably necessary for future experiments.

• The software tool Max MSP[9]
• Roland A49 MIDI keyboard[10]
• Roland EV-5 Expression pedal[11]
• Behringer XM8500 Microphone[12]
• XLR to 3.5mm jack[13](bought at keymusic but was not on their webshop)

Max MSP patch

In this section an in-depth explanation will be given for the program and the program made with it, that is used for the second experiment.

Max MSP is a program which lets you combine different objects to create programs that can be used for anything, from audio to video processing.
For experiment 2 it is used to process midi messages and output a new one. A picture of the program is shown below. The picture is divided into sections. Every section has a function. Notice that every text label in the pictures are in Dutch. Our user is dutch, therefore all the labels are also in Dutch.

The application behind the scenes

The first and second section are the places where the inputs and outputs are chosen. Nothing interesting happens there.

In the third section, the played note from the input is added to the base note from the guitar string and is outputted to the chosen device from section 2. The add object only outputs a new value if the first input of that object is triggered. Connecting the on-set to that input ensures that only on on-sets the note plays. The user can, i.e. a knob, with Control Change value of 74, to set the standard velocity of the played notes. This means that the notes cannot be played dynamically. Still, the velocity of the played note is routed to this section. This value is then set to either 0 or the value set by the knob.
At the bottom of the section, staff-lines are shown. On these staff lines, the played note is displayed.

The fourth section is where the real magic happens. At the top a Control Change signal comes in and routed to the scale object, which inverses the signal. The signal is then send to six split objects where the signal is only send through by one of the six objects based on one of the six ranges. The ranges are also displayed just above the split objects. The split object triggers a specific open string, seen at the bottom. This signal is then send to the add object in previous section.
Two buttons are also present, which put the open strings on the two most used tunings.
Notice how the first and the last ranges are not the same length as the others. This is due to the fact that the pedal is not perfect. During the first couple of degrees of travel there is no signal output. The same goes for the last part of travel when the pedal is pushed down. That is the reason why those two ranges seem very small, while in reality every range is basically the same now.

Lastly, in the fifth section a signal is send on closing the window. This triggers a message that stores all values present in all the number objects in this patch. Saving these settings can also be done by clicking the button in this section. Futhermore, on start up all these settings are reloaded using the loadmess object. This object also triggers presentation mode. This mode is explained in the next section. Lastly, here you can also see some text which is used to explain the buttons and numbers in the presentation mode.


Click to go to the full resolution

Presentation mode

In this mode, all the wires are removed. Only objects selected by the developer are visible to the user. The sections marked in the picture below correspond to the section in previous picture.

In section 1 and 2, the user can easily pick the in- and output devices for using the program. This is fairly straight forward.

In the next section, the note that is played will be displayed. The button above the staff-lines indicates an on-set.

The fourth section contains the tuning of the open strings. The tune of a string can be changed by clicking on the staff-lines or using the number box on the bottom. The two buttons at the top of the section will the tuning to the major or minor scale, respectively.

In the last section, the user can save the settings of the program. This mainly includes the tuning of the strings. The user can also view the volume of the played notes.