PRE2015 3 Groep4

This is the wiki page of Group 4 of USE Project: Robots Everywhere.

For this course, we've made a prototype of a "Smart Bedroom" using an Arduino unit connected to a laptop. Our idea of a Smart Bedroom is a bedroom filled with electrical modules that are controlled by a central computer, which automatically commands them to simulate the sunrise and morning sounds in order to wake up the user in the most natural way possible. So the Smart Bedroom basically replaces your alarm clock and makes sure you're well rested and cheerful in the morning. The sunrise is simulated by a combination of a light sensor mounted on the outside wall of the bedroom and automatically controlled blinds and lights. Another module of the Smart Bedroom is a thermostat that gradually heats up the room when it's time to wake up the user. Morning sounds are provided by your smartphone, which also acts as a user interface for the entire system. Furthermore, the smartphone records the sounds the user makes when sleeping, most importantly his/her breathing, in order to determine how deep he/she sleeps, which is necessary to find out when it's a good time to wake him/her up. Through research, we found that waking up during light sleep is much more comfortable than during deep or REM sleep. Setting the time period during which you want to wake up is also done via the smartphone.

For our prototype, a few concepts had to be simplified or even scrapped entirely. Naturally, the thermostat and blinds were off the table, which also eliminates the need for a light sensor. None of us have experience in making apps, so we decided not to use a smartphone as well. Instead, the laptop acts as a kind of user interface. The prototype consists of three scripts written in MatLab which is aided by an Arduino unit. The Arduino contains a microphone that records the user's breathing sounds and an LED that produces light. The laptop translates the sound measured by the Arduino into a graph showing the sleep deepness and uses that to decide when to start the wake-up procedure, which consists of gradually turning on the LED and producing morning sounds with the laptops speaker. When the user has woken up, which he/she indicates by pressing any button on the keyboard, the opportunity is given to provide feedback about the sound volume and light brightness, which is taken into account the following nights.

Group Members

• 0888465 - Thomas Bardoel
• 0942166 - Wesley van den Broek
• 0887435 - Jeroen Ermers
• 0888864 - Luuk Ladegaard
• 0887879 - Roel Montree
• 0835870 - Jeroen Verbakel

The Smart Bedroom

A visualization of our initial idea of a Smart Bedroom.

Inspiration

We started off this project with a dream: a high-tech automated bedroom that wakes up the user in the most natural way possible, but still in time for him/her to go to school for example. We were heavily inspired by the smartphone app called Sleep Cycle. This app uses your smartphone, which would be put beside your bed, to record the sounds you make while sleeping. This involves breathing as well as movement. The data is then used to calculate how deep you sleep. The app wakes you up within a chosen time period during light sleep, which is more comfortable than being woken up during deep or REM sleep. To wake you up, the smartphone plays a selected tune which sounds very relaxed and natural, like birds singing in the morning for example. When you've woken up, the phone provides information about how you've slept, most notably a graph showing the deepness of your sleep through time. This information can be compared with that of previous nights and of other people thanks to the internet.

Walkthrough

The picture to the right visualizes the Smart Bedroom as we imagined it. The bedroom is controlled by a central computer which can be wirelessly communicated with using your smartphone. When going to bed, you use your phone to set the time period in which you desire to wake up. It will then start to record the sounds you make, just like with the Sleep Cycle app. The data is sent to the central computer, which translates it into a sleep graph, which is used to determine if it's the right time to start the wake up procedure. This procedure, which we call the 'smart alarm', involves a number of things. First of all, the smartphone starts to play a tune similar to the ones of the Sleep Cycle app, which gradually increases in volume so as not to startle the user. Secondly, the central computer commands the thermostat to raise the temperature of the room, as it's more pleasant to wake up to a higher temperature. This could also begin earlier, at the beginning of the wake up time period for example, depending on how long it takes to heat up the room and how much influence it has on the sleeping user. Finally, the Smart Bedroom controls the brightness of the room in order to simulate a sunrise. If it's possible, this is achieved by slowly opening the blinds, as sunlight is a more natural and free source of light. But in case it's too dark outside, which is determined by a light sensor mounted on the outside wall of the bedroom, a lamp is used instead. The blinds and the lamp could also work in conjunction, with both modules contributing to the brightness of the room. The factors that contribute to waking up the user are indicated by red symbols in the picture to the right. If the system couldn't find a suitable moment to wake up the user or the user hasn't woken up yet by the end of the wake up time interval, the lights and blinds open up completely and the smartphone starts to produce an alarm sound similar to that of a generic digital alarm clock. So basically, the smart alarm is replaced with what we call the 'standard alarm' in order to wake up the user at all costs. After the user has woken up and stopped the alarm by pressing a button on his/her phone, he/she can provide feedback about the sound and light level. The feedback could for example be given as a selection between "The sound was too soft", "The sound had the right volume" and "The sound was too loud". This information is then taken into account during the following nights to ensure that the Smart Bedroom fits the needs of its user.

Speculation

These were the most important points of our Smart Bedroom, but it has the potential to perform a wider variety of tasks. Things that have come to mind were: the ability to display information about the user's sleep and share this over the internet, just like the Sleep Cycle app does; not only helping the user with waking up, but also with falling and staying asleep using the various output modules; the ability to give advice, like telling the user it's a good time to go to bed based on gathered data and so on. Furthermore, we thought about for who this system would be most beneficial. The most obvious benefiters are the people who need to go to work or students who need to go to school early in the morning and suffer from sleepiness. Another group of benefiters could be people with insomnia, as the Smart Bedroom might have the possibility to help them fall and stay asleep. We also contemplated whether the system would still work when there are more than one person in the bedroom. According to the Sleep Cycle app, it would actually work as long as the smartphone is placed significantly closer to its user than anyone else, so the Smart Bedroom should still be able to function properly. However, it would only be able to monitor one user at a time, and it's unavoidable that all people in the room will wake up when the smart alarm jumps into action, so using the system on multiple people is not recommended. Hence, for the remainder of this project, we will only focus on the situation in which there is only one person present in the Smart Bedroom.

Scenario

Here's a short story to illustrate our idea:

John is a student at the University of Technology Eindhoven. He has trouble waking up in the morning, so he decided to invest in a Smart Bedroom. When John goes to sleep, he sets the wake-up time to anywhere between 7am and 7:30am on his phone. Now it is time to sleep. His phone checks his breath and movements to see in what phase of the sleep cycle he is. Around 7am, the Smart Bedroom notices that John almost completed a sleep cycle. Since John will have completed this sleep cycle during the wake-up interval, the Smart Bedroom lets him sleep a little longer. When John is close to finishing his sleep cycle, the Smart Bedroom instructs the windows to let more sunlight pass through and also raises the temperature. When getting closer to finishing the sleep cycle, more and more sunlight will pass through the window and the temperature will keep on raising. This allows a much more natural wake up than usual. The sleep cycle has ended but the smart bedroom notices that John is still asleep even though all sunlight is going through the window and the temperature is raised to the maximum level allowed, so it decides to play a relaxing sound which builds up in volume to force a wake-up. Finally, John wakes up at 7:20 am and he is not as sleepy as usual. He is able to get out of bed without any problems and is ready for his day at the university. John also rates his sleeping experience on his phone and the alarm will use this information for the next night.

Use aspects

The Smart Bedroom is a dream for many people. Here, the USE-aspects of the Smart Bedroom will be discussed.

• The user will benefit from the smart bedroom because this integrates his blinds and lighting into his smartphone. This is the first step towards home automation. By crossing the barrier of combining your living space with your mobile devices, it becomes very easy to expand on this.
• The Smart Bedroom helps to battle tiredness, and therefore also stress, which can also be considered a benefit for the society. Besides tackling tiredness in general, the Smart Bedroom is great for people with problems getting asleep.
• Due to the integrated lighting and blinds in the bedroom, it becomes possible to sleep during the day. This possibly opens up all kinds of new lifestyles, such as working in multiple shifts during 24 hours and decreasing the morning and evening commute by spreading the starting hours throughout the day.
• A good night's rest greatly improves productivity, which is good for both society and enterprise. Through the Smart Bedroom, we can open a whole new market in home automation, enabling many new jobs, startup companies and other benefits to the enterprise. If you once again consider working night shifts, you'll see this is beneficial to enterprise as well, in decreasing the size of offices, as not everyone has to work there at once.

Users

Primary users:

• The person who sleeps in the smart bedroom

Secondary users:

• Other users sleeping in the smart bedroom
• People living with the primary user

Tertiary users:

• Scientists (data collection on sleep)
• Employers (more productive personnel)
• Engineers (that install and service the smart bedroom)
• Medical personnel (for people that need monitoring when sleeping)

User needs

Listed below are the needs of the primary, secondary and tertiary users. The needs colored in blue are the ones that have been fulfilled with the prototype, although only partially in some cases.

Primary users

The person who sleeps in the smart bedroom

• The user should be able to set the time. The option to synchronize the time with the time according to your smartphone would be very helpful.
• The user should be able to see what time it is in the dark. The user could simply open his/her phone, or there could be an option to continuously display the time. This would, however, drain its power fairly quickly, so it's not recommended.
• The alarm clock should not have a snooze option. Being able to go back to sleep after the alarm has woken you up is proven to be bad for your health.
• The alarm should have a volume control. This could simply be the phone’s volume control.
• The user should have the possibility to choose between three options: no alarm, a standard alarm which wakes you up at a specific time and the smart alarm.
• In the case of the standard alarm, the user should be able to choose when he/she wants to be woken up and what sound the alarm should make: a repeating pervasive tone, the radio or perhaps a song which is stored on the phone.
• In the case of the smart alarm, the user should be able to indicate in which time frame he/she wants to wake up. If the system can’t find a suitable moment to wake up the user, the standard alarm should go off at the end of this time frame.
• Perhaps the user should be able to indicate when it’s a suitable moment to be woken up. However the user probably doesn’t know a lot about sleep patterns and such, so it’s not a necessary option.
• The user should be able to choose what the standard and smart alarm control, like the lights, the blinds and the temperature of the room. These could also have different settings depending on the user’s preferences.
• The system should be able to deliver understandable information to the user about how he/she slept that night, the nights before and in average. The user should be able to decide whether or not he/she wants the information to be (anonymously) shared on the internet for statistical purposes and/or be available for his/her doctor.
• The system shouldn’t badly influence the user’s sleep in any way.
• The system should be adaptable to a changing bedroom layout, which it probably is because most is done by the user’s smartphone, which is of course portable.

Secondary users

Other users sleeping in the smart bedroom

• Because the smart alarm wakes its user up based on the sounds the primary user makes during sleeping, it perhaps will not work when there are more people sleeping in the same room. In addition to that, in most cases there is no moment where it’s for everyone ideal to wake up, so using the smart alarm would be useless. The users could still use the standard alarm, of course.

People living with the primary user

• These people do not want to be involuntarily woken up by the alarm. For this reason, the sound the alarm makes should not be too loud to wake up the people living with the primary user. It should be noted here, that the alarm should still have the capacity to go very loud, as some people might require a louder alarm to wake up.

Tertiary users

Scientists who collect the data on how people sleep

• The scientists should have a direct link to the database of shared information about people’s sleep.
• The information needs to be complete (so as much information as possible), structured and easily exportable to other statistical programs.

Employers of the users of the smart bedroom

• The employers benefit more from active and well-rested employees.

Engineers that install and service the smart bedroom

• The system should use mainstream components like normal screws, standard European cable inlets and outlets etc.
• When installing the system, the components should arrive in an orderly fashion, so similar parts are grouped together in a plastic bag for example, and a detailed instruction manual should be included. If necessary, the system should be delivered with a few tools which are specifically designed for installing it.
• The system should come with a detailed troubleshooting manual which suggests what could be wrong and how it could be fixed if there’s a problem with it.
• If necessary, the company behind the production of these smart bedrooms should hire and train personnel who are specifically employed to deal with this system.

Medical personnel for people who need monitoring when sleeping

• The medical personnel should have a direct link to the information of their patients.
• The information needs to be complete, structured and easily exportable to other statistical programs.
• In case the medical personnel has no experience with handling data, the information needs to be presented in a way which is easy to understand, with graphs for example. The presentation should still be able to show important aspects which are vital for the personnel’s diagnose of the patient’s sleep patterns.

Final product

A visualization of the prototype for the Smart Bedroom.

Components

Our final product for this course is more of a prototype of the Smart Bedroom. The various components of the prototype are visualized in the picture to the right. As one can see, the system consists of an Arduino unit connected to a laptop. The laptop fulfills the tasks of central computer and user interface, as well as sound output through its speaker, with the help of a number of scripts written in Matlab. The Arduino contains a microphone for recording the user's breath and a LED for the light output. The blinds and the thermostat as described with the dream are off the table, as we didn't have the time and resources to implement them properly. Furthermore, we had quickly decided not to use a smartphone, because none of us have enough experience with creating apps for the difficult task at hand. Nevertheless, this prototype is a great representation of what the Smart Bedroom could potentially be.

Walkthrough

The prototype works very similar to the Smart Bedroom as we imagined it. When going to bed, you specify in what time interval you want to wake up in one of the scripts and click the "run" button. The Arduino records your breath and sends the data to the laptop, which uses Matlab to calculate how deep you sleep and when to wake you up. When it's time to wake the user up, the laptop starts the smart alarm, which consists of playing a recording of birds singing in the morning and turning on the LED. Both the sound volume and light level will linearly increase from 0 to a certain maximum during the first 5 minutes. The smart alarm is also initiated about 4 minutes before the end of the 'wake up time interval' if the laptop hasn't found a suitable moment to wake up the user yet, which is a bit more comfortable than being woken up directly by the standard alarm. The standard alarm replaces the smart alarm if the user hasn't woken up yet by the end of the time interval and consists of playing an alarm sound similar to that of a digital alarm clock and setting the LED to maximum brightness if it hasn't reached that already. This will definitely wake up the user, who has to press any button on the laptop's keyboard in order to stop the smart or standard alarm.

Our final product exists out of several components, which have each his own function. The data during the night will be gathered with an Arduino and sound sensor. Gobetwino logged the real-time data to a .txt file on the computer. Matlab, a calculation program on the computer, will process this logged data real time. Finally, Matlab will determine the best wake-up moment. At that moment, Matlab will start the alarm which will build up in volume. The computer will be used to play the sound. Matlab will also turn on an LED, which will also build up in brightness. When the user is awake, the Smart bedroom asked the users experience, and will use this feedback the next nights.

The sound will be like our ideal situation. Before the outer-and-outer wake-up time, there will be played a relaxing sound of singing birds. And at the outer-and-outer wake-up time, there will be a more irritating standard alarm to be sure the user will wake up. However, the lights aren't like in our dream scenario. In our dream scenario, the smart bedroom will control the total room. Then, the Smart alarm can open the blinds more and more if it's light enough outside. Otherwise, the Smart bedroom can put on the lights of the room more and more. But our final product can't control the blinds or lights in a room. Our final product mimic this actions by using an LED. This LED will be glow more and more, like the brightness in the room will do in the dream scenario.

Userfriendlyness

Our final product isn't a completely finished product which is ready for consumer use. It's a prototype which isn't really userfriendly. Now, the user has to do many actions before the Smart Bedroom is ready. The user has to do the following steps now:

• Make a .txt file.
• Make (or change) a command in Gobetwino to log the gathered data on the made .txt file.
• Fill in the right path and filename to the .txt file in Matlab.
• Set the alarm and wake-up interval at the right time by using the notation in Matlab.
• Connect the equipment in the right manner. (same port as defined in the Matlab script)
• Upload and start the Arduino-code. (while Gobetwino is closed)
• Open Gobetwino. (the logging of data will start now)
• Run in Matlab the script 'runscript.m'

So, this is much to many actions for a consumer. But a number of actions isn't the only thing why our product isn't ready for consumer use. The interface is also very primitive. For example, now you've to fill in the wake-up time and interval in the script.

Planning

Links

Below are several links which have each have another function. In the first link 'System input and output' a overview is shown of our dream and the product we realy gona made. In the link 'Measurement plan and experiments' is the measurements located. The results of the experiments we did, the made trade-offs and justification of the method of measuring can also be found at this link. In the next three links, 'input', 'algorithm' and 'output', the whole code is described. In the link 'input' is the arduino code described for sampeling the sound. In the link 'algorithm' are the matlab scripts explained which finaly determined the ideal moment of waking up. And the link 'output' described the working of the output script and the made trade-offs. Finally, in the link 'conclusion', not only the conclusion will be given, but also the futher improvements and the reflection.

In the link 'literature' can be found some background information about some specific topics. For example about the sleepcycle in genaral, the ideal moment of waking up and the influence of light, temperature and sound during your sleep. The information about this topics is used by making decisions during the construction of the algorithm.

• System input and output
• Measurement plan and experiments
• Input
• Algorithm
• Output
• Conclusion
• Literature
• Logbook

Appendix

In this appendix can be found a zip-file of all needed scripts to use the Smart Bedroom by yourself. (if you have the right Arduino, sound sensor and logging program) The zip-file contains all scripts, including the feedback-mechanism. Only thing which is missing is the audio file of singing birds which will wake up the user in the morning. This file was to big to upload on the wiki-page. So if you want to run the complete script including wake-up sound, you've to put a file of singing birds in the same map as the other scripts. Give that file the following name: Birds-singing-in-the-morning.mp3

File:Code Definitive V4.zip