PRE2016 1 Groep1

Group members

Marissa Damink 0858700 (M)

Jochem Meijer 0899769 (JO)

Nicky Alexander 0886116 (N)

Jesse van Kempen 0907453 (JE)

Jasmijn Kleij 0906848 (JA)

Introduction

Our current society is facing the societal dilemma of ageing, given that the ratio of care takers and caregivers is out of balance and will be expanded even more in the future.

Considering the fact that technological advancements in health care are taking place, such as better treatment for diseases, also contribute to the increase of elderly people, because their lifespan is extended.

Due to this, there will be lack of caregivers in the future and therefore it will almost be inevitable that robots are partially taking over tasks. These tasks can not be of great complexity of course, as robots are unable to think autonomous like humans. From this point of view, we have come up with a good use for robotics in which the relatively few human caregivers left in the future can focus on important and risky care tasks which should not be given in the hands of robots, like for instance the use of robots in operating theatres.

Our idea will be realized by improving the current patient hoist in hospitals and nursing homes, in which patients who are not able to relocate themselves anymore, are transferred by lifting them. The patients can be transferred between for instance a bed and a chair or to places like restrooms and sitting rooms etc.

However, unlike already existing patient lifts, our improved patient hoist (I.P.H) can work partially autonomous and will be equipped with an interaction device. Due to this, the new patient hoist is able to detect the emotions of patients. The human-robot interaction will play a significant role in our design cycle.

This improved hoist can be beneficial for all sorts of groups. The human caregivers for example can benefit from it, because the patient hoist is more manageable. They do not have to lift patients themselves anymore and they can exert less power to move the hoist, which is now partially autonomous. This is better for their own health by for example preventing back pain from heavy lifting and pushing. Due to the interaction device, the hoist can detect how a patient is feeling during the lifting process and is able to alert the caregiver whenever there is something wrong with the patient. Constant eye contact between patient and caregiver and better awareness of the environment of the hoist for the caregivers can also be obtained by the interaction device during the transportation process.

Hypothesis

To guide our research we have set up several hypotheses. These are divided in main hypotheses with several sub-hypotheses. Some of these will be investigated through surveys and others will become clear from interviews.

Caregivers need to be supported by the patient hoist

- Caregivers need to be supported by cameras for navigating the patient hoist.

- Caregivers need a patient hoist that can navigate itself and avoid obstacles.

- The caregiver would like to be warned by the patient hoist whenever the patient is anxious.

- Patients appreciate it when they are reassured by the patient hoist whenever they are anxious.

- Caregivers need a patient hoist that can provide information about the weight or the 'BMI' of the patient.

Patients and caregivers attach value to interacion during the process of lifting.

- Patients need a certain level of interaction with the caregiver during the process of lifting. (So more need for touch/eye contact/speach, or no need?) (Survey)

- During the interaction between patient and caregiver eye contact is more important than speach for a pleasant interaction. (Survey)

- During the interaction between patient and caregiver eye contact is more important than physical contact for a pleasant interaction. (Survey)

- Patients do not mind if the caregiver is absent for a certain part of the lifting process.

Objectives

The main objective of the project is to improve the existing patient hoist. To be able to establish the shortcomings and disadvantages of the current patient hoist it is of great importance to get enough inside information using personal interviews and surveys. After analysis the hypothesis stated above can either be confirmed or disconfirmed and are, next to the now known shortcomings and problems, used to draft requirements for the new design of the patient hoist. The new design will be sketched and if possible a prototype will be made, otherwise a 3D-model will be made of the new design. Either way feedback will be gathered to verify whether the adjustments are successful.

Scenarios

To give you all a proper view about the current use of the passive patient hoist, the current working method will be described in the next two scenarios:

Scenario 1: - Inside 1 room -

“Transporting from bed to chair inside a room with the passive patient hoist”

Mr. A is a 75 year old inhabitant of Parc Imstenrade, a home for the elderly. Mr. A, who can not stand up by himself and therefore not able to relocate himself anymore, lies in his bed in room 23. Caregiver Lilly is going to put Mr. A in his chair near the window so he can see the nearby public garden. She will use the passive patient hoist to do so.

Lilly takes a lifting mat of the proper size for Mr. A and puts 2 whalebones in the backside of the lifting mat for better support for Mr. A’s back during the lifting process. She brings the bed to the right working height so she can work more easily without risking back problems. Lilly explains to Mr. A. what she is doing and is going to do at every step of the lifting process so he will not be surprised by any actions. First she turns Mr. A to his left side and pushes the lifting mat underneath him. Then she turns him to his right side so the mat can be equally divided underneath him. As soon as Mr. A lies back on his back again, Lilly lowers the bed, because the working height is too high for the patient hoist to lift Mr. A from his bed. Now the bed is at proper height and Mr. A lies correctly in the lifting mat, the patient hoist is called over. Lilly pushes the legs of the hoist beneath Mr. A.’s bed and guides the yoke of the hoist to the right position above Mr. A. By using the remote control of the hoist Lilly lowers the yoke so it can be attached to the lifting mat. The yoke is attached between Mr. A.’s legs and along both sides at shoulder height to the mat. The arms of Mr. A will stay inside the yoke and the mat during the lifting process. Once the mat is well attached to the yoke, Lilly lifts the yoke by using the remote control again. During the lifting Mr. A. is brought from a lying position to a sitting position in the mat. This is not very comfortable for Mr. A. because the lifting mat cuts a little bit between his legs. Once free from the bed, Lilly can move the patient hoist with Mr. A. towards his chair near the window. Note that this is not an easy task for her, because she has to push hard enough to get the hoist into motion, mind the environment of the hoist to prevent bumping up against things and hold one hand at the yoke so it will not shake too much, so Mr. A. will not get motion sick or scared due to the trembling. As they arrive at the chair, Lilly brings the hoist to the right position in front of the chair and moves the legs of the hoist from each other with the remote control so the hoist can be placed around the chair. She places the yoke with Mr. A .inside exactly above his chair so his back and the back of the chair are exactly in the same position. When chair and yoke are in the right position Lilly presses the remote control again to lower the yoke. Mr. A. is then gently lowered into his chair with the lifting mat underneath him. Once Mr. A. sits in his chair the yoke is released from the mat and the hoist is slowly driven away from the chair by Lilly while preventing a collision between the yoke and Mr. A. The whalebones in the backside of the lifting mat are removed, because they are not comfortable for Mr. A.’s back while sitting. Lilly asks Mr. A. whether he wants the lifting mat to stay underneath him or not. He does not want it underneath him, therefore Lilly pulls the mat from between his legs to the outside of his hips towards his back to remove it underneath him. Now Mr. A. can enjoy his sightseeing.

 

Scenario 2: - Transportation -

“Transporting from bed to general living room with the passive patient hoist”

It is morning in ‘De Tulp’, a home for the elderly and the 70 year old Mrs. B. who suffers from paraplegia lies in bed. She is ready for nurse Jane to transport her to the general living room where she can sit next to her close friend Mrs. C. to gossip about the cute Mr. D of next door. Due to her illness Mrs. B. is not able to replace herself anymore so she will be transferred from her bed to the chair in de living room next to Mrs. C. by the passive patient hoist. Nurse Jane will use the same method as caregiver Lilly to bring Mrs. B. from her bed in the yoke as was done with Mr. A. in scenario 1. When this is done and Mrs. B. hangs above her bed in the yoke, Jane will drive the hoist out of Mrs. B.’s room over the corridor to the living room. Just like Lilly, Jane will explain what she is doing to Mrs. B. during the lifting and transfer process so Mrs. B. will be at ease and not surprised by any actions during her transfer. Jane will need to coordinate the whole process very carefully, because she needs to exert enough force to push the patient hoist. She has to take into account the surroundings of the hoist to prevent accidents during the transfer. She also frequently has to keep the yoke steady with one hand during the process so Mrs. A will not get motion sick or scared due to the trembling of the yoke. Before the transportation process of Mrs. B. with the hoist really can start Jane has to turn the yoke with Mrs. B. inside manually into the right direction. Due to this Mrs. B.’s face points forward, while the hoist and Jane are behind her. Mrs. B. is now able to see in what direction she is pushed by Jane. Jane can now start the transportation by pushing the hoist towards the room door. At the door Jane needs to be extra careful and make sure that she does not bump into something. This is necessary, because her sight about what is happening directly outside the door is not sufficient enough. She also needs to push the hoist with more power to drive over the little threshold. Once outside Mrs. B.’s room, Jane needs to focus again extra carefully to provide Mrs. B. a safe ride and to prevent any collisions with other humans or items in the hallway. Keep in mind that Jane and Mrs. B. can not make any eye contact during the transportation process so Jane has to observe very attentively (as far as possible) whether something is going wrong with Mrs. B. The use of speaking is now the only solution for any interaction between them. Once arrived at the door of the living room, the threshold situation repeats itself and precise coordination of Jane is required. Jane puts Mrs. B. in the chair next to Mrs. C. by using the remote control to open the legs of the hoist and to lower the yoke just like was done to put Mr. A. in his chair in scenario 1. Mrs. B. en Mrs. C. can now finally continue their discussion about Mr. D.

Literature study

State-of-the-art

A patient hoist is a mechanical device for lifting a patient out of bed into a (wheel)chair. There are two kinds of hoists, the active and the passive one. The active hoist is meant for patients who still can stand up, but cannot move very well. It lifts the patient from a bed to a standing-up position on the hoist, after which the patient can be moved to a chair. The passive hoist is used for patients who are too weak to stay standing up. It moves the patient in a sitting position. (1)

Most passive hoists consist of a frame with a sling attached to it. The patient gets secured in the sling by the nurse and the hoist lifts the patient upwards. After that the hoist can be moved by the nurse towards the bed or chair and afterwards the hoist lowers the patient onto the bed or chair. Finally, the nurse releases the patient from the sling. There are lots of different models for passive hoists. Most hoists move the patient along a vertical axis, after which the whole device can be moved by the nurse. Some work with a rail which can transport the patient over a horizontal axis, after the patient has been lifted up. These rails can be integrated into the room, or can be moved from bed to bed. An active hoist does not move the patient from a lying position to a sitting position like the passive lift, but moves the patient from a sitting to a standing position. It can be used to transport the patient, but sometimes the active hoist is only used to help the patient to stand after which he/she can walk him/herself, possibly with the help of a walker. This helps patients to maintain their mobility (2).

A lot of research has been done about the use of patient hoists in hospitals and nursing homes. Johnsson et al. (2004) (5) proposed a model that simulated the balance between the nurse and the patient during a transfer task. The movement between the patient and the nurse should be harmonious, otherwise it may cause injuries. Research has proven that a lot of difficulties exist with the current patient hoists. Even when a hoist is available nurses do not always use it. Several reasons are: “(..) lack of time and availability, difficulty of use, space constraints, and patient preferences.” (6). By including the nurses in the buying process and buying a powered lift instead of a mechanical one, a nursing home can improve the chance of the hoist actually being used. Furthermore a hoist cannot decrease the risk coming with lifting, but it can still reduce it significantly. (6) Despite several researches it is not clear what patients prefer. Some studies show that patients prefer a mechanical hoist, while other studies concluded that patients were more comfortable with a ceiling lift (5).

There have been several attempts to improve the passive hoists with help of robotics. One of the earliest attempts was in the period 1990-1992 in a study of Patrick A. Finlay (3). In this study it was recognized that several specification were needed. First the robot should be able to move the patient without causing injuries. Of course, every patient is different and especially patients that have to be moved with the hoist can have numerous of physical problems the robot has to take into account. The robot should also be able to collect the patient from a lot of different positions and move them to lots of other possible positions. Third, the robot should be able to move through a hospital without accidents. Finally, the robot should not work too slow, because nurses and patients otherwise find the waiting time not worth to use the robot. The decision was made to make it a nurse controlled device, to get the patients and nurses to slowly get used to it. A initial design for the robot, which was called the Patient Assistant for Mobility (PAM) was already made up: “The patient surface of PAM is made up of an array of slats or tines which, using a patented method of deployment, are able to be insinuated gently under the patient to support his/her weight and draw him/her onto the trolley surface. The patient surface has a Z-axis to adjust its height, is additionally articulated at the hip and knee joints, and can thus move to set the patient into a seated or other intermediate pose. (…) Sensors are used to monitor the stability of the platform, and as a useful by-product these are processed to provide a readout of patient weight. For patients with special nursing needs, selected tines can be disabled so that no contact is made with the corresponding part of the body. The patient surface has an autonomous acquisition capability, so that a single command enables a patient to be collected from a bed once the PAM is parked in approximately the correct orientation at the bedside. Articulation of the patient surface is normally in telemanipulator mode, but the PAM also contains a memory enabling details of patients and furniture to be stored, so that a collection and placement sequence can be played back whenever required.” (3)

Although this project started the demonstrator phase in 1992, it is unclear what happened with PAM after that.

Another attempt to improve the patient hoist was in 2007, in a project led by Lakshitha Dantanarayana (4). In this project the author developed a smart hoist in collaboration with the resident and carers of the residential care facility. Some adaptions they found to be useful were weight measurement, rear view mirrors, ability to monitor the environment and assisted manoeuvering. The researchers equipped a standard patient hoist with cameras in the front, just above the floor to provide information about the part of the environment that is blocked from view by the hanging patient and implemented strain gauges into the design to measure the weight of patients. Other improvements were cameras to monitor the environment behind the carers and robotic wheels. The patient hoist also had a navigation assistance algorithm implemented. The main change however was that the patient hoist was motor driven, but still moved by applying forces to the handles. The force was measured by the hoist and a similar movement was applied by the motors. This ensured the hoist could be used intuitively, and the learning curve was short. Several users trials were executed, and the researchers implemented the feedback in the final design. There was no clear conclusion whether the smart hoist was better than the traditional hoist, but several points for improvement were found.

Besides the hoist other solutions for lifting patients are researched, like the HAL exoskeleton and the Japanese lifting robot RIBA, but those projects are beyond the scope of our research.

(1) http://www.domicare.nl/tilliften-badliften-verrijdbaar/

(2) http://www.arjohuntleigh.nl/producten/transfer-oplossingen/actieve-tilliften/

(3) http://search.proquest.com.dianus.libr.tue.nl/docview/217008747/fulltext/AE31C6BAA874EE9PQ/1?accountid=27128

(4) http://onlinelibrary.wiley.com.dianus.libr.tue.nl/doi/10.1111/j.1365-2834.2011.01276.x/full

(5) Johnsson C., Kjellberg K. & Lagerstrom M. (2004) A direct observation instrument for assessment of nurses’ patient transfer technique (DINO). Applied Ergonomics35 (6), 591–601.

(6) http://ovidsp.tx.ovid.com.dianus.libr.tue.nl/sp-3.22.0a/ovidweb.cgi?QS2=434f4e1a73d37e8ce55b227ffaae5fa148541856cb5686bfad9f6431240b98c07524237e1720f34b02eb8fa2b88b7b97cae33df7331783914f69c8dde4ca12d06569431bde95318885c0300c9e4cb854a6adb57292dc4b9d5d515362ffaf06f20e36f3c9460a88b948e9e5dcf4c00bab436508a09afdce5366e7c719263f67d40a206676fd0c0a5686e4e1da71239e75c2ce0637418cb6107561917b1c775b62b2deac9baf23bbafabe516e9e595ddb3711a24619f5e9fff0b6524d61d8e55df726a6944468052b4402c3d9ae720be38f03cb8a56190a0535e3fa712b6c3423a74b470672b139464e79db6d185645a45cb60d8e15bd92bcee10aa989a2dfbadeef09927167e35fcee3135359c01e825083f314bc5221e7c8644d3bbcbe868deccbffbe8e7109265789c479c547d2e3fdb26a430178a76903ff214576e1d05cdf304d7473261f3c184d70eea1d98c1e23e9483a61fdd973d2af33538480ec1aee29fd33ec82224c9b14f035663bed6db11254245dfcda166d7bd10d7e687e549e4d066e920b3493fb

Human-Robot Interaction

Natural human-robot interaction plays an important role in effective nursing services system provided by service robots for the elderly and disabled people. This paper proposed a multimodal “human-robot integration” collaboration system, and set up a shared collaboration interface between human and service robot. Consequently, Users and service robots can naturally communicate and retrieve information from the collaborative interface with multimodality(e.g. head gesture, eye gaze) in an interactive dialogue approach. By this way, making the service robots fully understand human's intention, so they can collaborate and complete tasks well. Furthermore, some experiments were conducted, and the results suggest that it is effective to identify user's intention in light of the advantage of different modalities, the shared collaboration interface can provide more information both from human and robots to improve the naturalness of human-machine collaboration. The proposed methods can provide a new way for exploiting human-service robots cooperation.

http://ieeexplore.ieee.org/document/6999239/

USE aspects

This subject is in many ways related to USE-aspects. First of all there are many different stakeholders and users. The primary users are off course the patients and the nurses. The secondary users are the people who pay the hoist, in this case the hospital management and the government. The tertiary users are the maintenance people. The preferences of all these users have to be taken into account. For instance, the patients want the hoist to be comfortable, while the nurses want it to be easy to use. The hospital management wants the hoist made as cheap as possible and the maintenance people prefer the important components easy to repair or replace. This can lead to conflicts since it can be difficult to take all these preferences into account at the same time. We mainly focus on the preferences of the primary users. The main question is how the hoist can be improved so that it is less uncomfortable for both patient and nurse. An important aspect that will be taken into account in this research is the interaction between the patient, the hoist and the nurse. The patients are a vulnerable group and being transferred with a hoist can be terrifying. So important questions are: 'How does a patient want to be comforted?', 'Does this differ between different groups of patients?' and 'What does the nurse want?'. It can also be a bothersome process for the caregiver. These are the kinds of questions we want to answer in our research. As you can see the hoist is not only a technical problem. A lot of these are actually more related to social or use aspects.

Research

Interviews

Interview questions

Before we took the interviews we made a list with questions about we wanted to know from the nurses and patients. After that several interviews were taken and the list was adapted a little to answer new questions that had arisen during the research.

Vragen voor verzorger versie 1:

1. Hoe lang werkt u al in de zorg?

2. Hoeveel ervaring heeft u met het verplaatsen van cliënten uit bed?

3. Ervaart u het tillen als een lastig en zwaar onderdeel van uw baan?

4. Wat komt er, buiten het fysieke gedeelte, nog meer bij het optillen van een cliënt kijken?

5. Maakt u wel eens gebruik van een tillift, zo ja, welk type?

6. Hoe ervaart u het gebruik van de tillift?

7. Welke onderdelen van het proces maken tillen lastig voor u?

8. Hoe is de interactie tussen u en de cliënt?

9. Hoe gaat u ermee om als de cliënt angstig wordt?

10. Waaraan herkent u dat een cliënt angst ervaart?

11. In hoeverre is sociale interactie van belang?

12. Is er behoefte vanuit uw kant voor een tillift die de cliënt meer kan geruststellen?

13. En als deze weet wanneer de cliënt pijn ervaart zich hierbij kan corrigeren?

14. Wat zijn voor u de voordelen van de huidige tillift?

15. Wat kan er volgens u verbeterd worden aan de huidige tillift?

16. Heeft u moeite met het rijden met de tillift? Gebrekkig zicht?

→ Wat zou u ervan vinden als de tillift ondersteunt wordt met camera’s voor het navigeren?

→ Wat zou u ervan vinden als de tillift zelf kan navigeren en obstakels kan ontwijken?

17. Zou u het handig vinden als de tillift u bepaalde informatie kan geven? Als ja, wat voor informatie? Bijv. gewicht kunnen wegen….

Vragen voor verzorger versie 2:

1. Hoe lang werkt u al in de zorg?

2. Hoeveel ervaring heeft u met het verplaatsen van cliënten uit bed?

3. Maakt u gebruik van zowel een actieve als een passieve tillift? Zo ja, welke heeft op het huidige moment de meeste verbeterpunten/bij welke treden de meeste problemen op?

4. Hoe ervaart u het gebruik van de tillift?

5. Welke onderdelen van het proces maken tillen en verplaatsen lastig voor u?

6. In hoeverre is sociale interactie van belang?

7. Op welke momenten tijdens het hele proces (tillen en verplaatsen) is er weinig tot geen contact tussen u en de patiënt? (Is er verschil tussen til- en verplaatsproces?)

8. Is er behoefte vanuit uw kant voor een tillift die de cliënt meer kan geruststellen?

→ bijv. mimiek aflezen en seintje geven als er iets is (facial recognition)

→ Oogcontact met verzorger via beeldscherm (360 graden camera)

→ Beeldscherm die handelingen beschrijft, vertelt, animatie etc..

9. Hoe verloopt het transport proces? (houding patiënt ten opzichte van verzorger)

10. Heeft u moeite met het rijden met de tillift? Gebrekkig zicht?

→ Wat zou u ervan vinden als de tillift ondersteunt wordt met camera’s voor het navigeren? (bij transport)

→ Wat zou u ervan vinden als de tillift zelf kan navigeren en obstakels kan ontwijken?

→ Wat zou u ervan vinden als de tillift een motor heeft/beter te besturen/duwen valt?

11. Zou u het handig vinden als de tillift u bepaalde informatie kan geven? Als ja, wat voor informatie? Bijv. gewicht kunnen wegen…

12. Wat zijn voor u de voordelen van de huidige tillift?

13. Wat kan er volgens u nog meer verbeterd worden aan de huidige tillift?

Vragen voor patiënt:

1) Hoeveel ervaring heeft u met de tillift? (van en naar welke locatie?)

2) Hoe ervaart u het tillen door de tillift?

3) Bent u wel eens bang/angstig tijdens het tilproces?

4) Bent u tevreden over de tillift of niet? Wat is er bijvoorbeeld fijn of juist vervelend etc?

5) / Wat zijn voor u de voor- en nadelen van de tillift?

6) (Maakt u liever gebruik van de tillift om verplaatst te worden of vindt u het fijner als verzorgers het tilwerk verrichten?)

7) Hoe is de interactie tussen u en de verzorger bij het tilproces?

8) Vindt u de sociale interactie met de verzorger van belang/belangrijk tijdens het tilproces?

9) Wat zou u ervan vinden als de tillift (meer) autonoom zou werken? (dus verzorger niet bij hele tilproces nodig)

10) Wat zou u ervan vinden als de tillift zelf voor de sociale interactie tijdens het tilproces kan zorgen (u op uw gemak stellen, uitleg geeft over wat er gaat gebeuren etc.)

11) -->En op uw signalen kan inspelen als iets bijvoorbeeld niet prettig voor u verloopt? (verkeerde houding, drukt op zere plek etc.)

12) Wat kan er volgens u verbeterd worden aan de huidige tillift?

Collected interviews

In total five interviews were taken. All of them were with nurses who had experience with the passive patient hoist. The first two interviews were taken with Petra and Lianne, both nurses from Buurtzorg, which is a private care institution. Another two interviews were taken with Linda and Rachelle, both interns in their last year of their nurse education. The last one was taken with Lisan who works in a nursing home with patient who suffer from dementia.

Collected interviews

Results of interviews

Several problems emerged from the interviews. Here the most important points that were made clear in the interviews are summarized.

Interview 1:

This interview states that interaction between patient and caregiver is very important to establish good collaboration and trust between them. Everybody should feel safe and comfortable while using the passive patient hoist. Whenever the patient feels scared or not secure a good explanation about the taken actions should be provided, so trust and feeling safe can be created within the patient. Therefore it is a good idea to implement a device that can recognize emotions of the patient to assist the caregiver at his work. The hoist is not always experienced suitable for both patient and caregiver. The patient often dislikes for example the lifting mat, because it can cut in the patient’s legs. The caregiver on the other hand finds it difficult to move the hoist over a unsmooth floor or in small rooms. Further, the caregiver does not stand very close to the patient while operating the hoist.

Interview 2:

From this interview can be concluded that the use of the passive patient hoist is an improvement in comparison with lifting of patients manually by the caregiver in which for example back pain can be reduced or prevented. However the hoist is not perfect and consists flaws. Maneuverability of the hoist has proven to be a main issue. When working with demented patients interaction between them and caregiver is a bit different than with not demented patients. Direct commands and explanation about the taken actions are given to the demented patients to reassure them, but sometimes an explanation has no use for them, because they do not understand it. Therefore they are sometimes distracted with a stuffed animal to smoothen the lifting process. At the same time these patients do not always express whether they are feeling uncomfortable or scared. Caregivers try to make something out/read the facial gesticulation of the patients to estimate how they are feeling. A device that could determine their emotions and one that could distract or reassure them during the process would be of good use.

Interview 3:

This interview shows that using the passive patient hoist is not an easy task in nursing homes. In these homes are often small doorsteps present which makes it difficult for the caregiver to drive over. During the transport process the visibility for the caregiver is deficient due to the post of the hoist. To prevent this limited view the caregiver walks alongside the hoist. However, this is also not very convenient, because it makes it more difficult to push the hoist with sufficient power. A device that improves this navigation process is reflected as a good idea. Another practical aspect is the lifting mat. This mat is not very pleasant for patients, because it sometimes cuts in the legs of the patient during the lifting process. Further, during the process these mats are provided with whalebones at the backside for extra support for the patient. This whalebones need to be inserted before the process starts and removed by the caregiver whenever the patient sits, because it is not comfortable for the backbone while sitting. A big issue with these whalebones is that they get lost very easily, because they do not have a fixed place at the hoist. During the lifting process interaction is again important. Whenever a patient feels uncomfortable he is asked what is wrong and all actions are explained extensively. It depends on the patient, whether they are comfortable with more interaction or less.

Interview 4:

Social interaction between patient and caregiver remains a crucial element during the lifting process. A lot of eye contact is made between them and they converse during the process. Whenever a patient is not acquainted with the hoist or if the patient is scared the caregiver will communicate more and try to make more eye contact as well to reassure the patient. When the patient is familiar with the hoist less interaction between him and the caregiver is needed to put the patient at ease. A technical aspect of the hoist is that the battery is heavy and it’s lifespan is very short. Another hoist like the ceiling hoist would be more suitable in smaller rooms and steering is much easier. However it is not a practical solution for existing care homes, because rails need to be constructed all over the nursing home.

Interview 5:

The main pont that can be concluded from this interview is that interaction is an important, but also a difficult point. Especially since the patients are suffering from dementia they do not really understand what you are saying. Still it is important to keep communicating about what you are doing and to comfort the patient. This is also possible by touching the patient, by eye contact or by distracting them with a stuffed animal. Over time you learn to recognize whether the patient is uncomfortable, because you get to know them. It is very important for nurses to get to know their patients. The main problems with the hoist are that you have to do several things at the same time and it is heavy to turn and move the hoist when it is loaded with a patient. It is especially difficult to navigate the hoist safely through small spaces while watching the patient.

Survey

Design enquete

Verbanden/ hypotheses:

- Both patients and caregivers prefer eye contact most during the process of caregiving during transportation in the “til lift”.

- Patients like to be comforted by voice interaction between patient and caregiver of between patient and someone else during the process of caregiving during transportation in the “til lift”.

- Patients like to be comforted by bodily contact during the process of caregiving during transportation in the “til lift” and also caregivers prefer to have some extent of body contact during care giving.

Opzet voor de enquête

Link to survey: https://docs.google.com/forms/d/e/1FAIpQLScinjbrL8rMoxs1CQOP0JCgIO3u_tavH2K1LGRcKIa9q_bFVA/viewform

Method

In the previous week some data for the enquête was already gathered, a powere analysis was conducted. The standard deviation at that moment was around 0.7, so this is the value that we used for the power analysis, together with a power of 0.8 and an alpha level of 0.05.

Code: tab rol //see sample size tabstat oog voice lichamelijkcontact, by(rol) stat(mean sd min max) total col(stat) long format // see standard deviation power twomeans 2 2.5, sd(0.7) power(0.8) alpha(0.05)

This showed that we need 24 to 32 respondents to be able to see a significant result in our data. We used a mean difference of 0.5 points difference on the Likert-scale. The Likert-scale is the scale of 0 to 5 that we used in the enquête.

Then we started collecting data again, especially of patiënts, because there was not enough data of this group. A sample size of 29 participants for each group was gathered.

The data was prepared for analysis, for example by giving proper names to the variables i.e. After that, the new standard deviation is determined.

The follow hypothesis were analysed.

- What way of interaction receives most preference?

- H0: there is no difference in the need for interaction between patients and caregivers

- H0: there is no difference in preference for eye contact between patients and caregivers

- H0: there is no difference in preference for voice interaction between patients and caregivers

- H0: there is no difference in preference for body contact between patients and caregivers

Commands used in Stata:

//welke manier van interactie krijgt de meeste voorkeur?

tabstat oog voice lichamelijkcontact, by(rol)

//H0: er zit geen verschil in de vraag naar interactie tussen patienten en verzorgers

ttest interactie, by(rol)

//H0: er zit geen verschil in voorkeur voor oogcontact tussen patienten en verzorgers

ttest oog, by(rol)

//H0: er zit geen verschil in voorkeur voor stem tussen patienten en verzorgers

ttest voice, by(rol)

//H0: er zit geen verschil in voorkeur voor lichaamscontact tussen patienten en verzorgers

ttest lichamelijkcontact, by(rol)

Results

Using the new standard deviation (0.7 again), the sample size (29 participants per group) and again an alpha level of 0.05, the power is calculated. The value of the power is 0.76, this value is oké. In an ideal case we collect 5 to 10 more patient participants, but we continue with the analysis anyway because the power is good enough.

Code: power twomeans 2 2.5, sd(0.7) n(58) alpha(0.05) //to compute power with given sample size -> 29 participants per group

The results are:

Total mean for eyecontact is 2.22, for voice-interaction is 2.21 and for bodycontact/interaction is 2.96. The difference between patient and caregiver in eyecontact is 0.20, for voice interaction it is 0.31 and for body contact it is 0.41.

For the ttest of interaction by role of patient of caregiver, the t value is 2.55. There are 61 degrees of freedom. The p-value is 0.006.

For the ttest of eye contact by role of patient of caregiver, the t value is 1.06. There are 61 degrees of freedom. The p-value is 0.146.

For the ttest of voice interaction by role of patient of caregiver, the t value is 1.762. There are 61 degrees of freedom. The p-value is 0.042.

For the ttest of body contact by role of patient of caregiver, the t value is 2.46. There are 61 degrees of freedom. The p-value is 0.008.

Discussion

The results mean:

What way of interaction receives most preference?

Command: tabstat oog voice lichamelijkcontact, by(rol)

Eyecontact receives most preference. With bodycontact you see that patients do not really like it. The value of preference is higher, which means their preference is lower. Also, the difference with preference of body contact between patients and caregivers is quite high.

What also stands out is that the preference for voice interaction is a little less with patients that with caregivers. This means that caregivers like talking to their patients more than that patients prefer to talk to their caregivers. This is not what we hypothesized so we will look into this, using interviews and a ttest on voice interaction.

H0: there is no difference in the need for interaction between patients and caregivers

command: ttest interactie, by(rol)

There is a significant result, which means the preference for interaction between patients and caregivers is not the same. The p value is 0.006, so H0 is rejected.

H0: there is no difference in preference for eye contact between patients and caregivers.

Command: ttest oog, by(rol)

A ttest is conducted where the preference in eyecontact is compared between patients and caregivers. There is no significant result because the p value is 0.146. This means H0 cannot be rejected, so there is no difference in preference for eye contact between patients and caregivers.

H0: there is no difference in preference for voice interaction between patients and caregivers

Command: ttest voice, by(rol)

A ttest is conducted where the resulting p value is 0.042. This value is below 0.05 which means that H0 can be rejected, so there actually is a significant difference in preference for voice interaction between patients and caregivers. However, this resulting p value of 0.042 is not very far below 0.05, so we cannot be totally sure that this difference is significant.

H0: there is no difference in preference for body contact between patients and caregivers

Command: ttest lichamelijkcontact, by(rol)

This final ttest shows a large significant result again. We got a p value of 0.008. This means H0 can be rejected and there is a difference preference for body contact between patients and caregivers. Patients prefer this way less than caregivers.

Noldus company for face detection software is contacted for a free trial software. Via telephonic contact it is promised that we will receive this before the end of the week.

Analyse

From both the survey and the interviews we can conclude several things. First of all we will try to answer the question whether our hypotheses were true or not.

Caregivers need to be supported by the patient hoist

From the interviews we can conclude that they don’t really need to be supported by the patient hoist, they can manage without, but it would help much. Nurses have indicated that especially a hoist that helps with navigating through small spaces would be of great help. Thus we can confirm the hypothesis that nurses would need to be supported by camera’s for navigating. It is less clear whether the second hypothesis, “caregivers need a patient hoist that can navigate itself and avoid obstacles” can be confirmed. Nurses have mentioned they would like to have help with navigating but none them have mentioned they would like the hoist to move on itself. At least one of them, Lisan, has mentioned an aversion for a self-moving hoist. She was convinced a nurse should always be present. So we conclude that this hypothesis is untrue. Not much consensus was found on the third hypothesis: The caregiver would like to be warned by the patient hoist whenever the patient is anxious. Some nurses mentioned that it could be handy, but other nurses mentioned that this is the task of the caregiver and by getting to know the patient a caregiver could identify whether the patient was anxious perfectly well themselves. So we will neither confirm nor reject this hypothesis. We can also neither confirm or reject the hypothesis “patients appreciate it when they are reassured by the patient hoist whenever they are anxious”, because it was not possible to interview a patient. It is a very vulnerable group, so nursing had rather not us interviewing them. The last hypothesis was: ”Caregivers need a patient hoist that can provide information about the weight or the 'BMI' of the patient.” One nurse mentioned it would not be necessary, because they weigh the patient already once a month. But since the other nurses never gave a clear answer to this question we neither confirm nor reject it.

Patients and caregivers attach value to interaction during the process of lifting.

We have researched this hypothesis with a survey. From this and the interviews it can be concluded that most patient and care givers do indeed attach much value to interaction. In the interviews it was mentioned as one the most important aspects of the tilling process. From the survey can be concluded that both patients and caregivers prefer eye contact most during the process of caregiving. This is also what we hypothesized. We also see that both caregivers and patients like to have voice interaction to some extent. However, the preference for voice interaction is significantly larger for caregivers than for patients. Patients do prefer voice interaction, but to a much lesser extent than caregivers do, and also to a much lesser extent than we hypothesized. For body contact there is a large significant difference in preference for caregivers and patients. Patients actually dislike body contact during care giving. This is not what we hypothesized. From these results we conclude that eye contact is the most important factor of interaction during caregiving. We will use this to optimize interaction during the process of caregiving during transportation in the patient hoist. Voice contact is less important to patients, but we could also take this into consideration with our design for optimizing the patient hoist.

Problems

With these hypotheses answered we can now identify the main problems of the current design and propose requirements which the improved design must meet.

A lot of nurses have mentioned trouble with moving the hoist. A lot of these problems were caused by wheels that got stuck. The swivel caster wheels, which are currently used, do not roll as smoothly as hoped. This is caused by a rough surface or the orientation of the wheels. Therefore the first requirement for the new design is that wheels are used that do not get stuck.

Another problem that many nurses encountered was that the hoist was heavy to push when loaded with a patient. It was also very difficult to turn the patient or the arm of the hoist when also focusing on the wellbeing of the patient. That is why we set up the following two requirements: Less force should be needed to push the hoist and less force should be needed to move or turn the patient when loaded in the hoist.

From our hypotheses we also concluded that nurses do have trouble with moving the hoist around in small spaces and would like some assistance with that. It is difficult for the nurse to keep an eye on everything, mainly because the patient is blocking a significant part of the field of sight of the nurse. This brings us to our fourth requirement: The hoist should give a better overview to the nurse during the transfer.

A conclusion we drew from the survey was that patient and nurses value eye contact during the transfer for a great deal. However, on longer distances where the patient is facing towards the moving direction there is no eye contact at all. This can lead to a less comfortable experience for the patient, that’s why another requirement holds that during the whole process eye contact between the patient and the nurse is guaranteed.

Another problem we researched was whether the nurses would like assistance with recognizing whether the patient is uncomfortable. Although some mentioned that nurse would learn this in time, for inexperienced nurses recognizing anxiety can be a challenge, because the patients are not always able to express their feelings well. That’s why we think the following requirement can still be an improvement to the hoist: The hoist should be able to recognize anxiety in the patients.

Our last requirement is: The hoist should be able to comfort the patient. We were not able to research whether the patients would like this, but especially since our design could be a bridge between the current lift and full automation, we still think it can add something important to the hoist.

Requirements

In this section the best solutions for the requirements stated earlier will be discussed.

Requirement 1: Wheels that do not get stuck are used.

A solution for this is to use solid spherical wheels, which are part of a ball bearing. Since these wheels can move freely in every direction they will not get stuck due to orientation and can as well move over surfaces which are rough. Since these wheels do not have an axis around which they turn it is difficult to support them with an electromotor. Therefore these wheels are mainly useful as replacement for the front wheels of the patient hoist.

Requirement 2: Less force should be needed to push the hoist.

The nurses mentioned in the interviews we held that they had trouble moving the hoist around in the rooms, because it was so heavy, especially when the patient was lifted. Sometimes the floors made it even more difficult to push the hoist forward. So we concluded that the improved hoist should have the ability to move without the nurse applying al the force. This especially important when the hoist is used for larger distances, like moving the patient to the toilet or to another room. Our solution is to build in an electromotor to drive the wheels. This means the hoist requires a stronger battery. (calculation of the kind of battery). We decided to not use a remote to steer the improved hoist, because that would add extra complexity to the hoist. The nurses would have to learn yet another thing. We want to keep the learning curve short, so we would user intention recognition to steer the hoist. This means the hoist anticipates on the intentions of the nurse by measuring how force is applied. For the nurse it means she can use the hoist the same way she was used to. More explanation of this can be found at requirement 3.

Requirement 3: Less force should be needed to move or turn the patient when loaded in the hoist.

The patient loaded in the hoist should be easy to turn. From the interviews we know some nurses had trouble with turning the patient in the right direction, while also making sure the hoist does not bump into something. The patient is turned by hand, which means all the force applied comes from the nurse. We want to make that easier by also using an electromotor to turn the harness to which the sling is attached. For this we also want to use user intention recognition, for the same reasons as mentioned above. To recognize the intention of the nurse the hoist needs force control system. Usually force control is used in robot that have to pick up objects. It makes sure the robot doesn’t squeeze anything. It does this by measuring the opposing force the object applies to the robot when the robot picks up the object. A feedback loop is used to ensure the robot responds in the correct way to the opposing force. This force control can be implemented in the hoist, so that the bigger force the nurse applies, the longer the robot continues this movement. To measure this force, force or strain sensors have to be implemented on every joint of the hoist. A computer measure this force and with the implemented logic determines the motion and velocities. Major motion patterns have to be identified and implemented in this computer, to ensure the hoist continues its movement in the same direction as the nurse applied the force in. The final block diagram will look something like this:

Requirement 4: The hoist should give a better overview to the nurse during the transfer.

This problem can be solved by the use of a camera, which is placed at the front of the lift and films in the moving direction of the lift. Transferring these images to the display of the nurse, he/she can easily see what is in front of the lift and if there are any obstacles that could be possibly dangerous. Another addition would be sensors that indicate how close the lift is to a certain object, for instance a wall, so the nurse has an even more complete overview. These sensors, often used in cars for parking assistance, use super sound to determine the distance between the sensor and an object. If a minimum distance is exceeded the nurse will get a signal on het screen and the hoist will break automatically to prevent collision. But since this partly takes away the control the nurse has, we decided to go with the camera’s. Maybe in the future when nurses and patients are more familiar with automation and robot technology, parking sensors could still be implemented.

Requirement 5: During the whole process eye contact between the patient and the nurse is guaranteed.

One way to realise this is by the use of cameras and displays. A patient, who is sitting in the lift, can be filmed by a camera. This images are transferred to a display that is attached to the back of the lift where the nurse is standing, who is also filmed. The patient also should have a display in sight, so eye contact can be established at all time. However, when you look at a screen, you do not look in the camera. This is because they are at two different places. For this reason, eye contact via videos is difficult. However, there has been a lot of research on improving video-mediated communication. As stated in “eye contact and video-mediated communication: a review”[1] video-conferencing has become a popular video technology because it improves real-time communication. Video-conferencing is a richer form of communication than email or telephone, however, the authors says that video-conferencing is not quite as informative as face-to-face communication. We did some research about ways to improve video-conferencing and eye contact during use of webcams. Franc Solina and Robert Ravnik proposed a method related to the mona lisa effect [2]. This method establishes eye contact during video calling and it used big screens and cameras mounted above the screen [3]. However, a still better method must be possible. Stereo matching method is a method that could solve our problem. With stereo matching a color image is used in a stereo camera as an input. This is configured by two cameras, they create a reference point [4]. However there is a problem with stereo matching, since most of the stereo matching methods only consider paired images in specific orders. To treat video images of stereo maching as a static image causes flickerings [5].

Recently, a patent for “method and apparatus for providing eyecontact function to multiple points of attendance using stereo image in video conference system” has been proposed [4]. The patent presents an invention to improve eye contact. It improves eye contact images between remote attendances during video calling. Since this invention is not on the market yet, we will not include this in our demo, but for future use this aspect is very important. This method uses the same ideas as stereo matching, two cameras are used. But in advance to that, a depth image camera is also used, this is to calculate the occlusion region. With this information, a object mask for an image is created. The completed object mask is the way in which eye contact can be improved. Figure 2 shows an illustration of this.

Requirement 6: The hoist should be able to recognize anxiety in the patients.

There are several ways a machine can recognize emotion in humans. One of the most obvious ways is off course the same as most humans do it, by reading a person face. Software that is able do this already exists and is called facial expression recognition software. Reading the face by looking at the position of, for instance, the mouth and the eyes has proven to not work well enough, because every person is different. That is why these software analyse the relationship between points on the face. When somebody curls the corners of his/her mouth, the software register that the point of the corner of the mounts moves. With a complex algorithm it concludes that the person expresses, to a certain amount, happiness. It is even possible for these software to recognise micro-expressions. Most of these software show the extent to which a person shows one of the seven basic emotions in percentages. The seven basic emotions these software uses are: joy, sadness, anger, fear, surprise, contempt, and disgust. Often a Neutral emotion is also added to the possible emotions. More complex emotions are always regarded as a combination of these seven emotions. It is therefore more difficult to recognize these emotions in the numbers the software generates. But since the software only has to recognize fear, it could be accurate enough for our goal. Off course there are also other ways to recognize someone emotions. Another aspects researches often take into account are respiration and heart beating. MIT has written a paper about a new technology that can detect the emotion of a person by analysing reflecting radio frequency signals off a body, which means that no sensors need to be applied. The EQ-radio sends a RF signal through the room, recognizes the person from the other objects, analyses its reflections and recognizes his emotional state. This all is done by a new underlying algorithm that extracts individual heartbeats from the wireless signal. The accuracy of the emotion recognition is comparable with state-of-the-art emotion recognition systems that needs a person to be hooked to an ECG monitor. For our design we will use the facial expression recognition software. The main reason for this is that this technology is already much developed and many different software are available. This makes it easy to implement in the design. The only things needed for this are software, camera’s and a computer. Since the computer is also needed for other aspects of our design a computer will be implemented anyway. The camera is also already implemented for making eye-contact with the nurse. We decided to go with the facial expression recognition software of Noldus. After contacting them we obtained a trial version of their software.

http://eqradio.csail.mit.edu/files/eqradio-paper.pdf

Requirement 7: The hoist should be able to comfort the patient.

We already stated the hoist should be able to recognize fear in the patient, but we also want the hoist to react on it. We already mentioned a screen attached to the hoist on which the patient can keep contact with the nurse, when the patient is moved. This screen can also be used to comfort the patient. One thing we concluded from the interview that it is important to tell the patient what is happening to them. So a feature is implemented in the hoist that it can tell, either with spoken words or written on the screen, what is happening to the patient at the moment he/she is scared. The nurse should be able to disable this feature at any moment. Another thing we learned from the interviews is that a lot of patient who have to transported by a hoist have dementia. They don’t understand what the nurse is saying to them. Therefore they won’t understand the hoist either. An option for them would be, that the screen on the hoist would show an animation which can distract the patient. This should also be a feature that the nurses should be able to disable at any moment.

Design 1

A first design in Photoshop CS6 has been drawn after the requirements were taken into account. The numbers are attached to the changes that are made to the passive hoist, type 'Wendy'.

1. Screen display with built in camera, which is able to live record the care giver. The software on the display is able to show information about the patient in the hoist.

2. Force sensors. To measure how much power the care giver is currently applying.

3. Emergency stop button for the care giver.

4. Electronic powered motor.

5. Gyroscope to prevent unneccesary oscillations for the patient.

6. Camera live recording the patient.

7. Screen to make eye contact with the care giver.

8. Holder for the stiffeners.

9. Camera to create more overview for the care giver.

10. Wheels for better manoeuvrability

The improved patient hoist (I.P.H.) in our design 1 is equipped with two interfaces, that will now be evaluated further. There is one interface for the patient (nr. 7 in Design 1) and one for the caregiver (nr. 1 in Design 1).

Interface patient:

The interface for the patient is sketched in figure 2(?). This will be a display, like for example a tablet. To fill up this display three options can be chosen by the patient and caregiver. In this case they have both control over how they are supported by the added technology.

The first option is that the caregiver is visible for the patient on the display so eye contact between them can be obtained. This is an improvement in comparison with the current patient hoist in which, like in scenario 2, there is no direct eye contact possible between patient and caregiver during the transportation.

The second option consist of the first option with an extra window on the display. This window shows a short animated film clip with a speech function in which the actions of the lifting process are explained to the patient. This option is implemented, because results of the survey and interviews showed that patients like to be informed about what is happening and going to happen during the process. As the patient wishes this option can be switched on or off by the caregiver.

A third option can be implemented to the first option in case the process is executed with for example a demented patient. In this case the display contains an extra window in which an animation is showed to distract the patient during the process. Consequently the process will go more smoothly and pleasant for both patient and caregiver.

All options can in fact be put on or off by the caregiver in compliance with the patient leading to actually four options for the interface of the patient.

Interface caregiver:

The interface for the caregiver is shown in figure 3(?). This will also be a display, like for example a tablet, which will be attached at the backside of the patient hoist at face/shoulder height of the caregiver (nr. 1 in Design 1). This display can be changed of position (up and down) so even a tall or a small caregiver can use it properly. On this display the patient lying in the yoke is visible for the caregiver, so eye contact between them can be accomplished. In another window an overview of the environment just in front of the hoist will also be visible on the interface. Due to this the caregiver can easily see what is in front of the hoist and if there are any obstacles that he cannot see for himself from behind the hoist. Further, on the right side of the interface a kind of traffic light system will be visible. This traffic light displays the well-being of the patient during the lifting and transport process. The traffic light is connected to the camera of the patient (nr. 6 in Design 1) which can due to the implemented software of Noldus recognize facial emotions of the patient.

Green light shines when the patient is feeling good and comfortable. In this case the caregiver does not need to change his procedure, the patient is doing just fine in the yoke. Orange light lights up whenever the patient is a little uncomfortable, but the caregiver does not necessarily need to take any actions or alter the transport process for the patient. The caregiver just needs to be more alert, because the chance of changing from orange to red light is higher. The red light shines when the patient is not feeling good, he is scared or uncomfortable. In this situation the caregiver should give attention to the patient and figure out why he is feeling uncomfortable and take action to change this feeling.

The distinction between which color light should shine when will be determined by the results of the test plan for the used facial recognition software of Noldus.

Noldus

In this section the facial expression regocnition software 'FaceREader 7.0' from Noldus will be looked into. First a measurement plan is discussed, after which the results are presented after analysis. Lastly conclusions regarding this software will be drawn.

Measruement plan

The goal of this measurement plan is to create a guideline for the two experiments that will be carried out. This will result in a smoother transition between acquiring the materials necessary for the set-ups, performing the experiments and exporting the data.

With the first experiment we will be verifying if the impact of different circumstances results in a deviant output. What will a difference in distance, angle or light/dark ratio do to the results?

With the second experiment we will be verifying the performance of the Noldus software by testing a range of basic emotions, by using certain movie segments. Experimental subjects (varying in age) are positioned in front of a camera, while being exposed to fragments of themed video material. Key is to execute the experiments with comparable circumstances.

Experiment 1

List of equipment:

1. Laptop with Noldus software ‘FaceReader 7.0’

2. Webcam

3. Lightsource

4. Chair

5. Table

Experiment 1.1: varying distance, constant angle, constant light/dark ratio

Steps to take:

1. Turn on laptop and start ‘FaceReader 7.0’

2. Start ‘new project’

3. Select ‘new participant’

4. Select the connected camera

5. Take place in front of the laptop, with the face pointing directly towards the webcam

6. Start an analysis

7. If the software recognizes the face, increase the distance between participant and webcam

8. Repeat till the maximum allowable distance is reached

9. Write down this distance

Experiment 1.2: constant distance, varying angle, constant light/dark ratio

Steps to take:

1. Turn on laptop and start ‘FaceReader 7.0’

2. Start ‘new project’

3. Select ‘new participant’

4. Select the connected camera

5. Take place in front of the laptop, with the face pointing directly towards the webcam

6. Start an analysis

7. Let the participant turn her/his head slowly to the right, until the face cannot be detected anymore

8. Write down this angle

9. Repeat this also for the directions, left, up and down

10. Increase the distance between participant and repeat until the face cannot be detected anymore

Experiment 1.3: constant distance, constant angle, varying light/dark ratio

Steps to take:

1. Turn on laptop and start ‘FaceReader 7.0’

2. Start ‘new project’

3. Select ‘new participant’

4. Select the connected camera

5. Take place in front of the laptop, with the face pointing directly towards the webcam

6. Start an analysis

7. Make the room as dark as possible and slowly turn on the lights

8. Write down when the face is first detected and at which point (if necessary) it cannot be detected anymore

9. Repeat for turning the lights of

Experiment 2

List of equipment:

1. Laptop with recoding software

2. Webcam

3. Lightsource

4. Chair

5. Table

Experiment 2.1: Varying experimental subject with constant circumstances

Steps to take:

1. Turn on laptop and start ‘FaceReader 7.0’

2. Start recording software

3. Place the experimental subject in front of the laptop, with the face pointing directly towards the webcam

4. Start the first fragment with emotion ‘Happy’

5. After two minutes, stop the fragment

6. Start an analysis

7. Ask the participant how he felt during the experiment and write it down

8. Repeat this for the fragments with emotions ‘Angry’, ‘Sad’, ‘Fear’, ‘Disgusted’ and ‘Surprised’.

Selected fragments for experiment 2

Literature of movies

Researchers interested in emotion have long struggled with the problem of how to elicit emotional responses in the laboratory. In the article of James J. Gross and Robert W. Levenson, five years of work to develop a set of films that reliably elicit each of eight emotional states is summarized. These eight emotional states are amusement, anger, contentment, disgust, fear, neutral, surprise and sadness

We will only use the movies for the 6 basic emotions, this is based on the possibilities of the Noldus software, and on Eckman’s faces (bron Eckman). These emotions are happiness, anger, disgust, fear, sadness and surprise. We also take into consideration the movies of neutral, because the Noldus software can also measure this and we want to test it.

Movies

We selected some movies that are approximately around 2 minutes of duration, because this is the maximum duration that our software can analyse.

Neutral:

Abstract shapes: https://www.youtube.com/watch?v=MHNlkCWjznM

Amusement/ happiness:

Ronald Goedemondt: https://www.youtube.com/watch?v=TSyofAODLxQ

Anger:

Protest against police: https://www.youtube.com/watch?v=JwiFgzcz0GE

Bullying: https://www.youtube.com/watch?v=8A93UJdENbY

Fear:

The Shining: hallway scene https://www.youtube.com/watch?v=79p57JJrMwo

Sudden shock: https://www.youtube.com/watch?v=zwdpTdmGy_M

Disgust:

Arm amputation: https://www.youtube.com/watch?v=U1VaYbrgW8s

Sadness:

Father dies: https://www.youtube.com/watch?v=FAhrqKqK_cA

Surprise:

Mind reading: https://www.youtube.com/watch?v=YzTdJkQRqG8

Discussion of movies:

What is important to take into consideration is that the tests can be shocking, so participants have to be informed very well on forehand, and they have to agree to the research after being well informed. We selected some movies that are approximately around 2 minutes of duration, because this is the maximum duration that our software can analyse.

For amusement we didn’t think the movies were very up to date anymore. So we watched them and decided to pick a movie that is quite similar to us. We picked another comedian: Ronald Goedemondt, instead of Robin Williams.

For anger, the movies were not findable anymore so we took a movie of protests against police armies, just as the title: “Cry Freedom (CRY)- police abuse protesters” suggests, and a movie about high school bullying. These movies can be a bit shocking.

For fear, we added another movie ourselves that is not in the research. However, we think that it will elicit very clear facial expressions of fear, so it is useful in our tests. So there are two movies for fear that we will show. For surprise it was very hard to find an appropriate movie. The movies from the article were not findable anymore and we were not sure about any significantly surprising other videos. So the results of the Noldus software of surprise might not be that good, it might be mixed up with happiness. Luckily, surprise is not a very important emotion in our research for optimizing the hoist. So we do not expect large problems because we can conclude that the Noldus FaceReader software is appropriate if the other emotions do show significant facial expressions matching the expected predicted facial expressions.

Tests

We selected some movies that are approximately around 2 minutes of duration, because this is the maximum duration that our software can analyse. Then test 2 of our test design plan was conducted using these movies. Participants were recorded when watching the movies. Also, after watching each movie their feelings were asked and written down. Using the Noldus software these faces can be analysed.

These tests were conducted for two reasons, the first reason is to validate the Noldus software and the second reason is to establish appropriate boundaries for the traffic light principle that we want to use to optimize interaction for our hoist.

Validation of Noldus

Method:

The recordings of the participants are analysed using Noldus software and their feelings were asked and written down. By combining these two factors and the literature about the movies, we will determine whether the Noldus software gives valid data about facial expressions.

We can conclude that the Noldus FaceReader software is valid if the highest mean emotion is the emotion that is expected from reporting of participants and from the literature.

The recorded participant’s faces during watching the movies are analysed by the software. Then the project data is exported via a .txt file to an excel file so the data can be used by other programs.

Figure (1) shows what the output of the Noldus software looks like. For each time frame and for each emotion the presence is given and a number is given to show the height of the emotion. A MatLab script is written that analyses all the data of the participants at the same time and determines the mean emotions and the maximums of the emotions that were found during analysis.

Then these outcomes are shown graphically. The graphs are shown with and without taking the neutral emotion into consideration. Because only happy, sad and scared emotions are important for our hoist, these are the emotions we analyse, because only these emotions are important to be valid in the Noldus software for our application of it.

Results

Experiment 1

By daglight the software was able to detect the face of the participant untill a distance of 2 meters between the webcam and the participants face. By daylight and 0.5 meter distance between the participant and the webcam the software was able to detect the face that was turned right untill an angle of 45 degrees was reached. The same holds for turning the face to the left. Turning the face down the software was able to detect the face untill an angle of 20 degrees, whereas the software kept on detecting the face when lokking up untill the neck of the participant was fully streched. Increasing the distance led to an increase of allowable error in every direction. By changing the dark/light ratio is was observed that the software is very sensitive to light. At daylight there are no problems, however in a slightly darker room, where the curtains are closed, the maximum distance between the face and the webcam decreaeses whether fast, as well as the allowable angle. Turning light right into the camera has the same effect.

Conclusion

By daglight and an angle between the face that should be detected (in the allowable range of left/right: 45 degrees, down:20 degrees and up: 60 degrees) and the webcame the software is able to detect the face without any problems. As soon as more or less light is available the softwares performance is decreasing. A solution for this could be to attach a light near the camera that shines on the face of the patient in the patient hoist, so the software is able to detect the face, even when the room is dark.

From the second experiment it can be concluded, that detecting specific emotions like fear and anger are difficult, since they are hard to distinguish. Positive and negative emotions however are detected rather well by the software, which is why valence will be used to determine a patients state of emotion, as will be further explained in the following section.

Emotion detection

As explained above, distinguishing different specific emotions lead to complications, which are undesirable. So, instead of focussing on these specific emotions and whether this one emotions exceeds a cetrain internsity, valence is used to dertemine the state of a patients emotion. (Arousal is not taken into account, since the software recieved from Noldes is not able to output the activity of the Action Units, which are needed to be able to calcultate arousal). To calculate valence the intensity of positive emotions, like happy, as well as negetive emotions, like anger, fear, sad and disgust, are needed for every time sample (surprise is not taken into account since it can be positive and negetive) [1]. Consequently, valence is the substraction of the intensity of happy and the highest negative intensity, leading to a range for valence from -1 to 1, where -1 is purely negative and 1 purely positive. Using this, the idea of a traffic light system on the nurses display can now be realised, as can be seen in the figure below. The matlab code can be found the the appendix.

Where it is green when the valence is higher than 0.3, red if it is less than -0.3 and yellow when the valence is in between. This treshold can be varied for every patient personally.

[1] http://www.noldus.com/files/file_manager/downloads/whitepaper/FaceReader_Methodology.pdf

Discussion

We have obtained a design for an improved hoist. Although this design well-founded, there are also several limitations to our research. Some of them can be solved in further research.

First of all, we were not able to interview patients and we also have a limited amount of surveys filled in by patients. This is due to the fact that patients who have experience with the hoist, are a very fragile group. Often they are old and have dementia. Most nursing homes would not let us ask questions to this patients, because that could upset them or they would not understand us anyway. Nurses could tell us a lot of what they thought to be the desires of the patients, but it would be better if we could have asked the patient ourselves. Some of our solution might have been rejected or patient could have had very specific problems with hoist the nurses did not know about. The improvements in our design are therefore mainly based on the view of nurses on the problems of patients. This can be different from the problems the patients experience themselves.

Secondly, there were a couple of problems with the Noldus Facial expression recognition we used. For instance, the version we obtained did not have the possibility to export the data real-time. This meant we could only export and analyze the data at the end of recording. It was possible to see the data in the program itself real-time, but for using it for the interface of the nurse, we needed to export it to our script, which was not possible. The traffic-light system we designed could only work afterwards. This problem, however can be easily solved for anyone who would like to implement our design by obtaining the right software. Another problem we encountered was that the software was not able to export the data on arousal, only on valence. Our script for the lights of the traffic light could be more accurate with this data. At the moment it only uses valence, which means that every negative emotion can change the traffic light. Ideally the traffic light would only respond to fear, but for that data on arousal is needed. This problem could also be solved by obtaining the right software.

Furthermore, an important limitation of our research is that we did not test the software on people who are partly paralyzed or have a disfiguration in the face. It is possible that the software does not work well on these kind patients, which can lead to false signals towards the nurse. This problem can solved however by measuring heart-beat or respiration as mentioned in the requirements. The technology using radio frequency signals (see requirements) the MIT has invented can also work for this, but this technology rather new and needs to be developed further, before it is possible to be implement it in the hoist.

Appendix

Matlab code

This file shows the used Matlab code to analyse the output of the Noldus software. File:Analysis.pdf

This file shows the used Matlab code to generate the traffic light system. File:Trafficlight.pdf

Collaboration

Planning

Short version planning

• Gathering information (week 1-3)

1) Interviews (week 2-4)

Making interview (week 2) (1h)

Conducting interviews (week 3-4) (10h)

Processing results interviews and conclusion (week 4) (4h)

2) Surveys (week 2-4)

Making survey (week2-3) ( https://docs.google.com/forms/d/e/1FAIpQLScinjbrL8rMoxs1CQOP0JCgIO3u_tavH2K1LGRcKIa9q_bFVA/viewform )

Conducting survey (week 3) (8h)

Processing results survey and conclusion (week 4) (4h)

3) Literature research (week 2-4)

State-of the-art (9h)

Human-Robot interaction (9h)

Specific (...h)

4) 1st design (week 4-5)

Design/sketch (perhaps 3D-model) (week 5-6) (4h)

Description (week 5) (10h)

5) Potentially prototype (week 5)

Reading in(4h)

Programming (7h)

6) Feedback (autorative) (week 4-6) (6h)

7) Potentially testing prototype (week 6) (10h)

8) Final design (week 7) (5h)

9) Wiki

Week 2-6 (Every member 1 hour per week for adding and restoring information etc)

Week 7 (each member 2 hours)

Work done so far

Week 2 presentatie 1 groep 1

Problem and subject/solution:

Our current society is facing the societal dilemma of ageing, given that the ratio of care takers and caregivers is out of balance and will be expanded even more in the future.

Considering the fact that technological advancements in health care are taking place, such as better treatment for diseases, also contributes to the increase of elderly people, because their lifespan is extended.

Due to this, there will be lack of care givers in the future and therefore it will almost be inevitable that robots are partially taking over tasks. These tasks can not be of great complexity of course, as robots are unable to think autonomous like humans. From this point of view, we have come up with a good use for robotics in which the relatively few human caregivers left in the future can focus on important and risky care tasks which should not be given in the hands of robots, like for instance the use of robots in operating theatres.

Our idea will be realized by improving the current patient hoist, which will transfer patients, who are not able to relocate themselves anymore, in hospitals and nursing homes by lifting them. The patients can be transferred between for instance a bed and a chair or to places like restrooms etc.

However, unlike already existing patient lifts, our improved patient hoist (I.P.H) can work partially autonomous and will be equipped with some kind of interaction device. Due to this, the new patient hoist is able to react to the patient and can adapt or stop it’s action whenever the patient expresses he is uncomfortable or whatever. The human-robot interaction will play an significant role in our design cycle.

This improved hoist is beneficial for all sorts of groups. The human care givers for example can benefit from it, because they do not have to lift patients themselves anymore which is better for their own health by preventing back pain from heavy lifting and they do not need to assist the transportation all the way anymore which is necessary with the current patient lifts so they can spare precious time and focus on their other care tasks. Another advantage is that more patients can be helped by human care givers when some parts of their tasks are taken over by robotics.

Objective

The main objective of the project is to improve the interaction between a patient and a care robot. To be more specific, it can be divided into three sub-objectives.

The first one is regarding facial expression recognition, which is closely connected to the recognition of certain emotions. We will try to find out what techniques are already used. Also we will go further into the principle of Eckman’s faces, where there are a few basic emotions. If there is not an already existing technology, a new one has to be given thought to. This is important in order to make the robot able to detect facial expressions. Once these expressions or emotions are detected by the care robot, it can react properly to let the patient feel more comfortable. This reaction can vary from a verbal response to a physical action, where the care robot adjusts its lifting mechanism accordingly.

Secondly, it is of great importance to know what the general needs of patients are to feel comfortable as well as the needs of the nurses, so they are able to deliver good and comfortable care. By interviewing these groups of people certain knowledge will be gained, which should also be taken into account by designing the robotic system.

Which leads to the third and final sub-objective, where the optimal design specifications are discussed. On the one hand this includes the mechanical aspect of a comfortable and safe lifting movement and on the other hand the specific look of the care robot; whether it should be more human like and risking that the ‘uncanny valley’ becomes an issue or designing it in a more abstract way.

Use cases

This subject is in many ways related to USE-aspects. First of all there are many different stakeholders and users. The primary users are ofcourse the patients and the nurses. The secondary users are the people who pay for the robot, in this case the hospital management and the government. Also the family members of the patients are secondary users The tertiary users are the maintenance people and cleaners of the hospital. Because they have to take care of the robot, or work around it. The preferences of all these users have to be taken into account. For instance, the patients want the robot to be comforting, while the nurses want it easy to be used. The hospital management wants the robot to be made as cheap as possible and the maintenance people prefer the important components easy to repair or replace. This can lead to conflicts since it can be difficult to take all these preferences into account at the same time. We mainly focus on the preferences of the primary users. The main question is how you let a robot assist them without affecting their dignity and making them uncomfortable. An important aspect is the interaction between the robot and the patient. Do the patients want to be approached by the robot first or do they want to ask the robot for help themselves? Another aspect is how the robot will understand if he’s hurting the patient or not. Maybe he can ask the patient or scan their faces to see if they are in pain. But in this case another question arises: What if the patient is not able to communicate in that way anymore. Maybe the patient had a stroke, which paralyzed a part of his face, and thus the robot cannot read the face anymore. The patient could also has lost the ability to speak clearly to communicate with the robot. These problems should all be taken into account.

Approach

By creating a clear approach, the list of objectives can be answered with back up of different sources. Each method contains a brief description of the task that will be performed.

Interviews One of the objectives is verifiying what kind of needs both patients and care takers have. This will be tested by interviewing a number of nursary homes and private individuals, thus assesing a varierty of problems that can occur, as different institutions can have different sets of problems.The results of these interviews will be taken into account when the design of our ‘’Lift Robot’’ is made.

List of nursary homes around Eindhoven ( https://www.zorgkaartnederland.nl/zorginstelling )

Experiments Furthermore, to test the recognition of emotions, as this is one of the key concepts that will make our ‘’Lift Robot’’ unique, we will conduct an experiment. This experiment makes use of a webcam and software, which is able to spot the micro-expressions of the face of an user, and validate in what state the user is currently. If the outcome of this experiment is postive, thus being a good method to recognize the emotions of the patient, we will use it in our ‘’Lift Robot’’. This makes it possible to tell if the patient is at ease or experiencing a form of discomfort, where the ‘’Lift Robot’’ will react upon.

Companies with software: -> http://emovu.com/e/ -> Software Marissa

Literature study As some of the information of our subject may already been researched, we will conduct a study in literature, by assesing databases on scientific research like the library of the TU/E and Focus. This will give us a better idea about current robots that can perform a similair task and get an idea of their imperfections.

Milestones In the upcoming week this approach will be extended with milestones, setting dates for each of our methods and dividing the tasks so that we will make optimal use of our time.

Conclusion: We think the I.P.H will be an advantage for all kind of groups. Although users can have diverse opinions about the use of robotics during the interaction with humans. Some fear there will be less human personal contact, others are more comfortable with the human-robot interaction, because it increases their independence on other humans and streamlines the care process.

Our believe is that using this system will make human caregivers be able to work more efficiently.

So our goal will be to satisfy as many stakeholders as possible, but the main focus will lie on the preferences of the primary users. Of course it should be comfortable for the users by improving the interaction between patient and ‘care robot’.

To do so we will look at current state of the art by a literature study to complete our design. We will also be taking into account the opinion of the users to make it as comfortable as possible by taking interviews and conducting a survey by the patients and care takers. We will also try to test the interaction equipment by experiments which will be chosen out of the results of the interviews and surveys.

Further, in the upcoming week this approach will be extended with milestones, setting dates for each of our methods and dividing the tasks so that we will make optimal use of our time. We strongly believe that our improved patient lift can be useful for human beings.