PRE2016 3 Groep2

This is group 2's page

Group Composition

Group 2 has the following members:

• Steven Ge
• Rolf Morel
• Rick Coonen
• Noud de Kroon
• Bas van Geffen
• Jaimini Boender
• Herman Galioulline

Project Description

In this project, we will explore music generation using machine learning. Specifically we will consider how taking user preferences into account in generating music via machine learning could create a unique and customized product for consumers of music. Applications for this include: generating similar music to famous dead artists (for fans of those dead artists), generating music scores for movies, and a new type of tool to help inspire artists with new musical ideas.

Our project will explore if there is indeed a large need/desire for this type of machine learning music generation. Furthermore, we will also see if such a machine learning algorithm is in fact feasible to implement and provide acceptable music for each of the applications.

Applications

One idea for application includes the concept that there are music consumers who have a particular taste for certain artists who are no longer alive (Bach for example). And these users are restricted to a fixed set of songs already composed by this artist with no variety - but desire new or more music of that same artist. A possible solution to this could be having a machine learning algorithm capable of generating songs containing the key characteristics that uniquely identify that artist. This would be done by training machine learning algorithm on that specific artist's songs. Assuming this is feasible, this could become an emerging market to help classical music lovers relive the past.

Another example of an application would the be movie industry. In the movie industry millions of dollars are spent on generating soundtracks for different scenes. However, overtime certain sounds have become key identifiers for evoking certain types of emotions in the audience (clashing cords to create distress and comfort for example). So it happens that when a new score is developed for a movie, the music is different, but several elements in the music must remain the same in order to create the desired effect. Thus, this could potentially be done much more efficiently and cheaply by training a machine learning algorithm for each of the desired effects. For example, having an algorithm to just generate "scary" music trained on the music played during scary scenes in movies.

The final example of an application would be some type of musical idea generator or inspiration for musical artists. Already musical artists (mainly techno, or house music) use everyday sounds as samples for their beats or tracks. And so a machine learning algorithm could provide a similar source of interesting ideas for those types of artists. This would be done by having the artist would feed the algorithm different kinds of music (either known songs, or some composed samples) and listen to the generated music for some type of inspiration or interesting idea. And then this could be repeated as many times as necessary until a desired idea or sample has been generated.

Machine Learning

To implement out Machine Learning idea to be able to mimic a certain artist we plan to use a Machine Learning algorithm called WaveNet. WaveNet is a deep neural network that takes raw audio as input. Already, it has been used successfully to train on certain dialects to create a state-of-the-art text-to-speech converter. Specifically, WaveNet has been shown to be capable of capturing a large variety of key speaker characteristics with equal fidelity. So instead of feeding it voice data, we intend to feed it musical input in order to train the network to generate our desired music.

WaveNet is an open source project written in python and is available on GitHub. This easy access will not only allow us to modify WaveNet if we find it necessary but also ensure that there is a community available to help if needed.

Training Data

For training data we plan to use 3 main artists: Mozart, Bach and Beethoven. The reason for these choices is that firstly there is no worry about copy right issues on any of these artists music. But also that they have written hundreds of songs all of which are fairly long (compared to today's music). Thus, the high number of hours of music available from these artists will ensure that the machine learning algorithm is trained on a sufficient amount of data. Depending on how the project proceeds this may be subject to change.

Interviews

We plan to conduct a survey of regular people to get a perspective on not only how useful and interesting they would find this idea of having a machine learning algorithm mimic a certain artist's playing style but also what else they would possible find useful (genre generation or mixture of artist styles).

Planning (subject to change)

GANT chart

The GANT chart of this project

This GANT chart is created to keep this project organized. It shows in which activities this project consist, when these activities take part and when an activity needs to be finished. It also contains a list of milestones. This list shows in which parts this project is divided. The major activities of the GANT chart (somewhat) correspond to these milestones. During the course, the percentage of completeness in this GANT chart will be updated.

week 1

Deciding on the subject of the project Working on first presentation

week 2

First presentation done Working on second presentation Create a project plan (the GANT chart) Create a task division Start researching Data Engineering Start researching Tensor flow Start working on wiki

week 3

Second presentation done Start interviews

week 6

Interviews done

week 7

Start creating prototype

week 8

Researching done

week 10

I think the GANT chart is incorrect (Written here so I wont forget)

Division of Tasks

For the USE part of this project, we have given each member of the group a task. Each task involves interviewing people related to the creation of art/music with artificial intelligence and/or people related with the music industry.

• Steven Ge: Make use part go smoothly (and conduct extra interviews)
• Rolf Morel: Interview a deep learning professor
• Rick Coonen: Interview a musician (or any other person that creates music)
• Noud de Kroon: Survey the populus
• Bas van Geffen: Interview a professor at a conservatorium
• Jaimini Boender: Interview Tijn Borghuis
• Herman Galioulline: Interview a producer (A person that makes money from the music industry, but does not create music him-/herself)

For the technical part of this project, we have split the group into two teams. The Tensor flow team will work on the Neural network (which for which we use Tensor flow) and the Data Engineering team will gather test data and find a suitable encoding.

Team Tensor flow:

• Noud de Kroon
• Bas van Geffen
• Herman Galioulline

Team Data Engineering:

• Rick Coonen
• Jaimini Boender
• Rolf Morel
• Steven Ge