PRE2018 3 Group7

Group members

Name	Student ID	Major
Jules Vliem	1256122	Electrical Engineering
Stijn Verhoeven	1238464	Mechanical Engineering
Tim Jonkman	0963139	Mechanical Engineering
Ruben Schmeitz	1233052	Mechanical Engineering
Jessie Maassen	1266500	Computer Science

Introduction

As the amount of elderly increases, the demand for caregivers is much higher and therefore a shortage of caregivers must be compensated. One of the anticipated needs of these elderly are physical exercises. In order to motivate elderly to do exercises, someone else needs to tell them to do any exercises at all and encourage them to get movement. Besides, there has to be someone that tells the elderly that gives tips about how the exercises can be improved. That is where the research and robot of this project come into play. This project is about the research, design and possible solutions of a robot that helps motivating elderly to stay in a good shape and to help them to do the exercises correctly. Not just physically, but also mentally. During the process, we will take a closer look at the user, society and enterprise perspectives of this research and how they are related.

Problem Statement

Elderly, especially the lonely elderly, often sit whole days long without any movement. This impacts not only the physical condition of a person, but it also has its effects on the mental condition of these people. To solve this problem, caregivers can help elderly by talking directly to them and encouraging them to do exercises and to make sure they execute it, they can help the elderly. However, since the amount of elderly increased with high speed over the years, there is a shortage of caregivers and therefore the anticipated needs of these elderly cannot be provided by the current amount of caregivers. Already thinking about possible solutions, robots can be used to solve this problem. But who decides if this works? And why should elderly accept a "device" to help them? These are questions that we will address in this project as well. Nowadays, this is a problem that is discussed a lot. It is not only important to find a solution just for now, but also the future is really important, because when the amount of elderly increases even more, the relative shortage of caregivers is even higher and therefore there is desperately more need for extra assistance.

Week 1

Objectives

In order to apply and work with a robot that helps motivating elderly to do physical exercises, a set of objectives must be made and a main goal must be established. The main objective of this project is to find a way to increase the motivation among elderly to do physical exercises. With this main objective, a subset of objectives can be formulated that directly relates to the robot but also to the project itself.

- There is communication between the robot and the user;

- The interface is as simple as possible so that elderly can easily use the robot;

- The project is focused on the user, society and enterprise perspectives;

- The user must be convinced to make use of the robot;

- The robot must be applicable at larger scale.

Users

Primary users

The primary users of our technology are elderly people who need to perform certain exercises which remediates impairments and promotes mobility and function. Our technology is aimed at improving the physical health of elderly people, while providing them with the comfort of their own homes. Elderly people however are often characterised as being skeptical about technology. That's why it's important to focus on how this feedback technology can be integrated in the households of the elderly people effectively. As little aversion as possible towards the technology should be provoked. Firstly it has been shown that people tend to react better to commands and feedback given by real robots than virtual agents. That's why it is important that we don't just make a virtual platform that can check the physical movements of the elderly people and give feedback to it, but that its functionality is combined with a physical robot. Another thing the elderly people require in order to improve their physical activity by given feedback is the way the technology communicates with the person. The right form of persuasion has to be determined so that the elderly person don't feel a form of coercion, but merely a form of good motivation.

Secondary users

The secondary users are the care givers and the family and friends of the old people. The family and friends might feel less anxious about the physical health of the elderly person they care for, when they know that they are performing regular exercises in a appropriate way. One functionality that can be added for example is the option to check on the robot, whether the elderly has actually performed the exercises and how well they did. This way they will be able to keep track of their activity and see if progress has been made. The functionality of tracking progress can also be useful for the caregivers hence they can see how certain physical exercises are affecting their patient and if maybe some exercises have to be changed or added. A problem that comes with this added functionality, that should be considered, is the invasion of privacy.

State-of-the-art

But for how to check if the elders do their tasks, not a lot of studies have been done.

A couple of studies have been performed on Robot-Human relationships. Overall, results show that socially assistive robots positively affect user experience and motivation compared to standard smart environment interfaces such astouch screens In their KSERA research, Torta et al. have used Nao as an interactive humanoid robot.

Another example of an experimental humanoid care robot is the robot TAIZO which has been used by Matsusaka et al.. Matsusaka et al. have performed a research on the effects of voice command on health exercise demonstration in a robot-human collaborative demonstration.

The Taizo exercise robot

Leire Lopez-Samaniego and Begonya Garcia-Zapirain have even gone further. Lopez-Samaniego and Garcia-Zapirain have written an article about a robot based tool for physical and cognitive rehabilitation of elderly people using biofeedback. Their approach didn't use a humanoid robot, but they've used a Lego based EV3 brick robot carrying an iPad instead. According to Lopez-Samaniego and Garcia-Zapirain "the main strength of this project is the combination of biofeedback techniqueswith Lego robots and serious gaming with the aim of providing both cognitive and motor rehabilitationfor elderly people."

Not only researches for physical motivation have been performed. Broadbent also describes a couple of healthcare robots. A specific one to mention is the robot seal "Paro". "The seal is intended to provide sim-ilar positive effects to a real pet, including psychologicalcomfort, physiological arousal, and social communication." Kazuyoshi Wada et al. also have performed research on the seal robot Paro. A part of their conclusion states "interaction with Paro improved the mood state of the subjects, and its effect was unchanged throughout during the five weeks of interaction." Subject being here the elderly users which have interacted with Paro for a period of five weeks.

Most of the research in the care robot domain has been done on the (ethical) consequences that it brings with it

But research on the best way to control and boost performance of the elders by a exercise robot, not a much has been done.

Approach

In order to meet the criteria that was set in the objectives,an approach had to be made. The approach includes a rough summary of the planning (which can be seen below) and an explanation on how we want to achieve the milestones. So we decided to divide the project in three parts:

1. Research:

In this part research has to be done with use of literature, survey(s) and personal interview(s). The results of the research have to give answers about the following questions:

• How do elderly react to care robots which are helping with their health?

• What are the best methods to motivate someone who is doing health exercises?

• Which exercises are the most effective ones?

• Is scanning with use of a camera the best way to analyze the movements?

• Is it profitable to implant such a system in a robot?

2. Product shaping:

The product shaping is the part in which the product will be conceptualized and programmed. In this process the RPC's have to be decided first, in order to chose the program, hardware and the approach of the programming language itself. When the product is finished an user manual will be written to make it easier for the users to understand how it works and what the functionalities are. Besides an user manual a logo and brand name will be assigned to the product itself.

3. Documentation

The wiki have to be updated and look like a report at the end of the project. To accomplish this someone will be assigned to check and edit the page every week. Besides the wiki, a presentation have to be made in anticipation of the last few weeks.

Every week the members of this project will be divided in three groups and work on the three part described above. 2 Persons will work on the research part, 2 on the product shaping and 1 will do the documentation (if needed). If we decide to cancel the product shaping part due to the complexity, the project will become a research project in which we will research the pro's and con's of a system which helps elderly with their health exercises. When the complexity of programming such a system can be handled the research part will support the product part, because the results can be used in the actual product. For instance the answer to the question: 'What are the best methods to motivate someone who is doing health exercises?' influences the output of the program.

Planning

Week	What have to be done	Responsible member(s)
1	1. Problem statement+ objectives 2. State of the art 3. Users+ what do they require 4. Approach+ Planning milestones+ deliverables+ Who does what	1. Tim 2. Jules & Stijn 3. Jessie 4. Ruben
2	1. Research question 1+ making an enquete/ interviews 2. Product shaping: RPC's + pick software	1. 2.
3	1. Research question 2 2. Product shaping: Start programming + pick hardware	1. 2.
4	1. Research question 3 2. Product shaping: Logo+ brand name+ programming 3. Check/organize the wiki page	1. 2. 3.
5	1. Research question 4 2. Product shaping: Programming+ implant the use aspects found during the research	1. 2.
6	1. Research question 5 2. Product shaping: Finish programming 3. Check/organize the wiki page	1. 2. 3.
7	1. Research question 2. Product shaping: Write manual 3. Check/organize the wiki page	1. 2. 3.
8	1. Research question 2. Make a presentation 3. Finish the wiki page	1. 2. 3.

Week 2

Milestones

To make it easier for the group to get to the deliverables (which are stated below) some milestones have to be set. The milestones are:

1.Research:

• Doing research on every question
• Take a survey
• Interview some potential users

2. Product shaping

• Define the RPC's
• Pick the software and hardware
• Program the product
• Implant the results of the research into the product
• Brand name, logo and user manual

3. Documentation

• Organize the wiki page
• Grammar and spelling check
• Write presentation text
• Make a powerpoint

Deliverables

At the end of the project the following things have to be deliverd:

• Report/finished wiki page
• Results of the research
• Product
• Presentation

Who will do what

In the planning a column is made for assigning every task to certain people. Only week one is filled in, because we have to determine who is going to do what in what particular week after the first week. On the other hand we already assigned the members to be responsible for some deliverables:

Deliverable	Responsible member(s)
Report/finished wiki page	Ruben & Tim
Results of the research	Jules
Product	Jessie
Presentation	....

Survey

In order to get a clear picture of our users, a small survey was set up. The survey consisted of three parts. Part 1 is about the respondent him/herself. Part 2 is about the physical therapy the respondent might have and the exercises they get to perform at home. The last part, part 3, is about what the respondent thinks of the technical solution we have in mind. The main focus of the survey is to get a global insight of the usergroup and their global thoughts about robots in this area.

The survey has been spread out in MTC Bernheze. MTC Bernheze is a Medical Training Centre (MTC) located in Heeswijk-Dinther, The Netherlands. Locals can attend here a physical therapist. MTC Bernheze also has a separate room in which fitness exercises can be held, mostly for the patients of one of the four therapists, but also for people who want to do some form of exercising in the fitness room.

Since one of the members of our group has close (business related) relations with Ziekenhuis Bernhoven (hospital) located in Uden, The Netherlands an attempt has been made to also spread out the survey at their department of physical therapy. They responded that they'll examine the survey whether or not it is suited for their patients. Till date no further response of filled in surveys have reached us.

The survey has been held in Dutch, as our research group are native speakers of the Dutch language.

Link to the survey: https://bit.ly/2SPZB6Z.

Literature

In order to create a proper survey, first some literature study has been performed. With the KSERA project , Torta et al. have performed some research on some findings of the same usergroup in combination with a Nao robot.

This bar chart indicates what rating the usergroup gave to four different ways Nao tried to get the users attention. It seems clear from this figure that waving is in most cases the 'best' of the four. Whether or not this is truly the case can't really be investigated through a survey, therefore this part hasn't been added to our survey. When testing our Prototype robot we can ask participants to fill in another survey in which we for example can ask if they indeed thought that waving was clear and friendly.

Results

As mentioned previously, the surveys main goal was to get a clear picture of the intended usergroup and their main thoughts about the technology we have in mind. In total we had 24 (N = 24) respondents on the survey, but in some results not all the submissions were used (e.g. if a person doesn't have (had) gotten treatment from a physical therapist, it is impossible that they didn't have gotten exercises to be performed. All these filters are explained at their individual subheaders.

Section 1

The first section is about the respondents in general. We can use this data to compare our sample group with the 'real' user group.

Q1.1 Gender

In question 1.1, no filter is applied ([math]\displaystyle{ N=N_{tot}=24 }[/math]).

Q1.2 Age

In question 1.2, no filter is applied ([math]\displaystyle{ N=N_{tot}=24 }[/math]).

Section 2

Q2.1 Treatment

In question 2.1, no filter is applied ([math]\displaystyle{ N = N_{tot} = 24 }[/math]).

Q2.2 Exercises

In question 2.2 a filter has been applied. People who don't receive treatment from a physical therapist, won't be able to get exercises from a physical therapist. Therefor, these submissions are filtered out. Which results in [math]\displaystyle{ N = 17 }[/math].

Q2.3 Frequency

In question 2.3 the same filter as in the previous question has been applied. Yet, the value for [math]\displaystyle{ N }[/math] is different, as not all people answered all questions (meaning they left some blank).

Q2.4 Execution

In question 2.4 a filter has been applied. People who didn't got exercises to do from their physical therapist, cannot perform these exercises.

Section 3

Q3.1 Motivation

In question 3.1 the same filter as in the previous question has been applied.

Q3.2 Feedback

In question 3.2 a filter has been applied. In this question we only looked at the people who don't always perform their exercises as prescribed by their physical therapist.

Motivating people to perform excercises

To design a robotic system that helps motivating people to do excercises, we need to understand the theory behind motivating people. Research suggests that 50% of the people that actually starts with excercises will dropout within the first six months[]. The research for motivation can be divided into a few main categories:

- Demographic and biological factors;

- Psychological, cognitive and emotional factors;

- Behavioral attributes and skills;

- Social and cultural;

- Physical environment or physical activity characteristics;

The demographic and biological factors state that men are more involved in physical activities than women and that the presence of obese has a negative impact on the motivation to do excercises.

For the psychological aspect the most important factor is the confidence of a person. When a person is aware that he/she can successfully execute certain excercises, the motivation to do this will increase as well. Therefore it is very important that someone (in this case a robot) tells a person to do excercises and to empower them that they will succeed with excercises.

Behavioral attributes and skills are important as well. How does the daily life impact the motivation to do excercises? It seems that certain behavioral attributes might impact the motivation (sleeping, drugs usage, alcohol usage, etc.) and diet is a very specific one. A person with a healthy diet is more motivated to do excercises than a person that is living unhealthy. Behavioral skills are more related to knowledge. When people know what they are doing (know what gains are made when performing a specific excercise) it helps to motivate them. In this project it could be useful to apply this to a robot so that a robot is able to tell the user what he or she is actually doing in order to motivate the user to stay in shape.

Certain social and cultural aspects increase the motivation to perform excercises, but when these excercises are performed in groups, it will be eassier for people to do these excercises together. Therefore, a friend is of course important to support the user to do it, but this can also be achieved in fitness groups, bootcamps or other forms of group training.

Last but not least, you have to deal with a physical environment. It is eassier to do some physical training when you live close to a gym or fitness centre. Besides, satisfaction is the most important factor of all. Without satisfaction, the user will lose motivation and therefore stop performing excercises. Being satisfied with the work-out equipment, the place and the person supporting the user will increase the motivation as well. Watching others (personal trainers for example) to be an example of how an excercise should be performed, will help the user as well.

After all, all factors are connected to satisfaction and self-confidence. Without them, it is hard to motivate a person to attend some physical training.

Feedback for elderly

A literature study should provide a (hopefully) optimal way to provide elderly feedback. When we're going to demonstrate the actual technology to some elderly, we're planning on comparing the data from our survey and the literature to the real test users to whom the technology has been demonstrated to.

Jacko et al. have performed a research on multimodel feedback. They've concluded the three most effective ways of feedback are haptic feedback, visual feedback and a combination of haptic and visual feedback. Less effective feedback seemed to be auditory feedback or a combination with auditory feedback. Apparently, users waited for the "participants waited for the auditory feedback to conclude before continuing to complete the task." Here we meet one of the limitations of Nao in combination with haptic feedback. The time needed for Nao to successfully provide useful feedback is way bigger than the time needed for Nao to successfully provide useful auditory feedback. Thus our hypothesis is based on the experience of Jacko et al., yet contradictory with their conclusion.

In addition to the research of Jacko et al., a student at Eindhoven University of Technology wrote a thesis as partial fulfilment of the requirements for the degree of Masters of Science in Human-Technology Interaction. Meijer wrote his thesis about appropriate feedback mechanisms in games for improving motor skills and cognitive abilities of the elderly. In his section about the influence of feedback (6.2) he stated that the reaction time of the elderly performing in his study increases when using solely haptic feedback compared to auditory and visual feedback solely. The same holds for using the trimodel feedback (haptic, auditory and visual feedback together). Meijer concluded " to some extent that auditory feedback was to be preferred over visual feedback when an elderly audience is the target, though the combination of auditory and visual feedback should be able to give the player the sense they are performing well, motivating and stimulating them to play more." Which of course meets our one of our RPCs, namely the constraint of being a user friendly interface. When users (elderly) are motivated and stimulated to play more with the interface (Nao), the type of feedback is in line with the attributes of the real-end-users.

One of the disadvantages of haptic feedback is that there is not a "real connection" with the user. This might lead to less motivation, because one of the main aspects of motivation is that someone close (family, friends and other close people) encourages you to do excercises or to tell you that you are performing the excercises well enough. This means that there has to be a connection between the user and the support assistant (could be anyone). This connection won't be established if a robot is simply calculating the position of someone's limbs. This disadvantage rests on the use of haptic feedback only. To suppress this disadvantage, the combination of haptic feedback with visual feedback will be applied. In this case the visual feedback gives a certain expression and for users of this robot, this is a human-like form of behaviour. This gives the user the idea that there is a connection between the user and the robot.

A big advantage of haptic feedback is that it is much eassier to check whether a person is doing the excercises correctly with high accuracy. This makes it very attractive to use this form of feedback.

Besides, there is the technical side of implementing feedback systems to virtual agents. In order to know how to apply feedback, you must know how human-like a robot should be and to what extend this technology should be applied. This aspect directly relates to the section about motivating people, where realism of a virtual agent is very important for humans in order to accept feedback at all. And should the technological part be available as open source so that other robots can make use of this technology as well?

RPC's

In order to determine what we have to do to decide when our product is finished, it is best to make a list in which we state the requirements, preferences and constrains (RPC's). Those RPC's are solely for the product we will deliver at the end of the project. The list is as followed:

• Requirements

1. The product should be motivating; it should make use of motivating feedback. The product shouldn't simply say "this is wrong" and "this is right" but aim to improve the satisfaction and self-confidence of the user during the exercise(s).

2. The product should provide feedback in the way that our research has determined as most efficient. This means both vocal feedback given by a physical robot (instead of a virtual platform) and visual feedback.

3. The product should be used as extension for existing care robots.

4. The instructor who gives the patient the exercises has to easily implement the new exercises into the program(should take not more than half an hour).

• Preferences

1. The product should be provided together with a manual. It is important that the elderly know how to use the product independently.

2. It has to be tested by a real user. The real user in this case should be an elderly person.

3. The product should be a motivating as possible. It should increase the satisfaction and self-confidence of the user as much as possible.

4. The product should provide accurate feedback in as much cases as possible.

5. The product should crash in as few cases as possible.

• Constrains

1. The product has to provide feedback on at least 1 exercise aimed at health recovery.

2. The product has to be easy to use. Elderly should be able to start up the program without the help of someone else. They should not depended on other people to start up the program since they should be able to practice their exercises independently whenever they want.

3. The movement scanning should be able to track the movements of the person at a accuracy of 85%. (In our product we make use of a kinect V.2. to achieve this. The kinect, namely, has an average accuracy of 88% regarding simple movements such as sitting up and standing down)

Week 3

The programming environment (Choregraphe) of the Nao robot

Before testing with the robot the code for the robot can already be simulated in Choregraphe (the coding environment of Nao). The ways the robot will interact with the people can already be programmed as well as the movements of the Nao. So a few exercises need to be made for the Nao to perform.

Hardware options for scanning movement

Detecting of the movements of the person in question is an important part of our product. In order to do this we need hardware which could make a skeleton of the person which is being detected. We searched for some hardware which could do this and listed the best options:

Hardware	Easy to program	Price	Easy to use (for the user)	Example	Other pro's/con's
Kinect Xbox 360	+/-	++ (€40 with adapter)	++	https://www.youtube.com/watch?time_continue=27&v=_jrrmf9fj7E	Need windows 7 in order to program.
Kinect Xbox One	++	+/- (€201 with adapter)	++	https://www.youtube.com/watch?v=bydLSVVuaRM	Very accurate
Xsens MVN Awinda	+/-	-- (€600)	--	https://www.youtube.com/watch?v=ggLge1Rw2z4	It is very accurate
Orbbec Astra	++	+/- (€140 with adapter)	++	https://www.youtube.com/watch?v=RWA4cHWxTN0	Very easy to program with the free software
TVico	+/-	+/- €211	++	https://www.youtube.com/watch?v=lKutlO8Ibmc
OpenPose (software which needs a normal camera)	+/-	++ (free for personal use + normal camera costs)	++	https://www.youtube.com/watch?v=C1Sxk6zxWLM	Not sure if personal version is good enough

So if we are not able to use a body tracking system of the TU/e we are destined to choose one of the option above. In my opinion the Orbbec Astra and Kinect 360 are the best options which use a depth sensor. The disadvantage of those two are the prices (respectively 40 and 140 euros). If we don't get a budget and do not agree about the fact to pay ourselfs, OpenPose is the best option.

Week 5

We were able to hire a xbox kinect (model: 1414).

Example exercises

In the last meeting with the teachers, we got as feedback to perform a literature analysis or ask a professional about exercises we could use in a real life demonstration.

During the holiday break, we went back to MTC Bernheze (where the survey also has been held) and contacted Mike Bosch, lead therapist of the practice, who is specialized in sports physiotherapy, manual therapy, ultrasound and Dry Needling. He's been an active physical therapist since 1992 and has gained experience in hospitals, but also in 'regular' practices.

We asked Mike to bring up a selection of five different exercises which are most common around our user group. Eventually we would scale down that selection to three exercises we wanted Nao to perform at least. If the possibility might exist (meaning we have time left etc...) we can also choose to implement the other two exercises as well.

Squat

Narrow squat

The first exercise is just a regular squatting exercise. For elderly balance might be tricky, thus performing this exercise with a chair isn't prohibited. The chair can be used in two different ways: (1) as a grip support (2) as an object to fall on. The first way (as a support) is clear enough by itself, yet the second might need some explanation. In (2) the squat will be performed above a chair. In this case, when the legs fail to hold the patients weight or the patient experiences imbalance, the patient can drop him/herself onto the chair to prevent a complete fall to the ground.

In the animation a squat has been performed with a narrow stand, but a more wider stand is also applicable. The most important parts to validate are the upperleg movements.

The patient might be encouraged to speed up the tempo of their squats a little bit (keep in mind that balance is the most important part of the exercise).

Knee lift

Knee lift

A second exercise is the knee lift. At first glance, this exercise seems quite simple, but after a while elderly might get exhausted. Additionally, for extra balance, a chair might be used in the same way as (1) at the squat exercise.

The exercise doesn't need to be performed as quick as in the example image. At a slower rate the exercise is fine as well. The most important parts to validate are the height of the knee compared to it's original position and the general balance of the patient.

The patient might be encouraged trying to lift the knee even higher or faster.

Lunges

Forward lunges

A third exercise are the lunges, forward lunges in particular. Since for elderly this might be difficult, the exercise can be simplified with a smaller forward step. The bigger the step taken, the more difficult the exercise gets. If somehow this exercise isn't though enough, the difficulty can be increased furthermore with the use of additional weights (e.g. dumbbells).

The most important parts to validate are the overall balance. If the patient doesn't perform the lunges in a correct way their balance might be dropped.

The patient might be encouraged to take a little bit bigger steps (max knee bend is [math]\displaystyle{ 90^{o} }[/math] between upper and lower leg), or to try additional weights, or to perform the exercise quicker.

Arm raise

Arm raise with small weights

A fourth exercise is a simple arm raise. The arms start vertically aligned next to the body and are being raise up sideways until they reach a [math]\displaystyle{ 90^{o} }[/math] bend between the body and the arms. Additionally the exercise can be performed with small weights (e.g. for elderly 1 or 2 kg) to increase the difficulty.

The most important part to validate is that the arms are kept fully stretched (absolute distance between the hand and the shoulder should stay the same), both arms should perform the exercise simultaneously and both arms should reach the [math]\displaystyle{ 90^{o} }[/math]-position at the same time.

Squat and lift combo

Squat and lift combo