PRE2016 3 Groep10

Advanced Elderly Emergency System (A.E.E.S.)

This is the wiki page of group 10 of the USE course: Project Robots Everywhere at the Eindhoven University of Technology. Here all aspects of this project are thoroughly described, in which the AI part of a wearable fall-detecting device for elderly people is designed.

Design team

The design team consists of a diverse team of 6 people from different majors of the TU/e:

Name	Student ID	Department
Lennard Buijs	0959903	Mechanical Engineering
Bram Grooten	0885158	Applied Mathematics
Ken Hommen	0911594	Industrial Engineering
Pieter van Loon	0861532	Software Science
Steef Reijntjes	0944701	Electrical Engineering
Man-Hing Wong	0944285	Electrical Engineering

Problem statement

Problem

In the Netherlands there are increasingly more elderly people. For them it can be more difficult to balance themselves when standing or walking, so it regularly happens that they fall. When they do, they often have trouble standing up. Also, they have a higher chance of injuries, with hip fractures being the most common. The problem especially arises if these elderly are severely injured by a fall, and therefore can't get up to reach for help.

Solution

A device that should be worn by, mostly, elderly that detects when one has fallen. It can automatically send a warning to ‘ICE’-persons or even call 911 (or 112 in Europe). Automatically sending its location along with it. A microphone and camera can in this case be used to observe the situation even faster. By connecting the device to the internet, this all can be made possible even faster. Also the device asks questions to the owner to determine the severity of the situation once a fall has been detected, which can be answered by simply talking back.

Objectives

To effectively and successfully end this project, a list of objectives is created to ensure weekly improvements, which will help in obtaining a final product. List of objectives:

• Research state-of-the-art technology
• Establish list of requirements
• Design the system that follows every requirement, this would be the ideal deliverable
• Create a final presentation, explaining the final design

Goal

The goal of this project is to help as many elderly as fast as possible when they have fallen, while minimizing the amount of false positive calls to the emergency services.

USE aspects

In the section below, all needs and demands of the users, society and enterprise (USE) are listed and discussed. The needs and demands that play a role in the final design are specifically indicated.

User

In particular, the users of the Advanced Elderly Emergency System are the elderly. Since the number of elderly is increasing, more elderly people will need a way of calling for help when an emergency occurs. Nowadays, an elderly person may have an arm wrist or necklace which has a button to call for help. However, in the future, there will be too many elderly for the number of nurses or doctors. By asking questions autonomously, the doctor's job will be easier, which means it will also correspondingly help the enterprise. The chance of getting helped after getting injured from falling will be significantly greater.

Primary users

Elderly that regularly experience falling.

• The system should not negatively influence physical actions performed by the elderly in any way.
• The user should be able to manually press the button for a certain amount of time when help is needed.
• The user should be able to set a time, before a medical instance will be warned, in case when no response can be sensed by the system.
• Data information that has been collected and transmitted should be secure and private.
• The system should give relevant, understandable questions to the elderly and respond morally correct in a humane way.
• The system's volume can be changed in a preferable way.
• The system should notify the elder in time when the system is not working properly i.e. due to low battery.
• To avoid disaffection by the owner, the system will once a year speak to the elder, so the A.I. voice will not be forgotten.
• The system should notify the elder in time when there is need to update the system software.
• Falling movements and its impact should be accurately measured, in such a way that a fall will be detected with high accuracy and as few false responses as possible will be given by the system.
• The elderly person can manually choose whether there is the possibility to contact a medical emergency service (i.e. 911 or 112), or that only contact persons should be contacted (emergency service will never be contacted in all situations then).

Secondary users

The secondary users are the next of kin of the elderly (i.e. the people who are living together with the elderly, or are responsible for their care).

• Calling for family or others who are responsible for their care by accidentally pressing on the button, or by pressing it without any purpose, should be prevented and not occur.
• Geolocation data and other information what can be obtained by the system should be held private by next of kin.
• Information that has been sent to family or next of kin should be accurate.
• The next of kin are not obliged to become integrated in the emergency system service by playing the role of the contact person.

Tertiary users

Medical health instances that should stay alert all the time and can provide help in emergency cases.

• Geolocation data and other information what can be obtained by the system should be held private. Only these trained medical health instances and the relevant person himself may have a look insight.
• Information what has been sent to the local emergency instances should be accurate.
• Accidentally calling for health instances by pressing on the button should be prevented and not occur.

Society

Due to time managing reasons, the society group of USE has not been discussed and analyzed. It has not been involved during the entire period of time of the project.

Enterprise

The government acts as the enterprise, since the government has to take care of the health insurances of society.

Responsibility

Who should be responsible when the wearable system does not operate properly in critical situations?

Interview with elderly

On Friday, March 10th we went to Vitalis Berckelhof, an elderly home in Eindhoven, to interview the elderly about their thoughts on our idea. First, questions has been proposed on the system they were currently using. Then we explained our design and asked their opinion on multiple parts of it.

Part 1: Current model

The first question we asked was if they actually used some sort of emergency button or system on a regular basis. We discovered that all people in this retirement home received an emergency button when they move in. For the use of this system they pay around 10 euros on top of their rent. The button is attached to a necklace, and should be worn at all times. Some people concede that rule and wear it day and night. The necklace was designed in such a smart way so that it would break open if too much pressure is applied to it, in order to make sure the person doesn’t suffocate. Other people would hang the necklace around their walking frame next to their bed during the night. Or only wear it when they are in their room, since that’s the only place where the system really works. This is because next to the button, the people in the elderly home also have a little box attached to the wall in their living room. This is where the responding sound from a nurse would come from, and where the microphone is to respond.

Thus we already noticed two main parts where this system can really be improved. One, there is no implemented form of fall detection. It already exists, as we explain in our research section, but it is not yet implemented into society. The button is of course of great use, but if someone is directly unconscious after a fall, he or she won’t be able to press the button. A man we interview actually told us this had happened to him about two weeks ago. He fell in his apartment and apparently was laying there for almost 2 hours when the nurses came by. He was one of the people that didn’t use the button at all, but he told us that he would be willing to wear a fall detection device attached to the belt.

And two, the system only works when people are in their own apartment! This is because the communication after the button is pressed, goes through the box on the wall. This could be really improved by moving the functionality of the box to a smartwatch or even right into the little button on the necklace. A location sensor or GPS should then be implemented as well, since the location of the ‘box’ isn’t constant anymore.

Next, we asked questions like: "How do you like the button and the emergency system?", "Have you ever used it?", "And if you did, why did you press the button and what happened after you did?" Many people liked the system, but weren't really enthusiastic about it. It seemed like the elderly got used to the fact that they have this system available. Almost everyone that we interview who wore their button on a daily basis, also pressed the button at least once already. Some used it for when they fell in their apartment, for example in the shower. The shower is too far away from the box in the living room, so the woman couldn't talk with a nurse. But they came pretty quickly once they didn't get any response. Others used it a bit more regularly, when they needed attention, it seemed. For instance, when an elder is scheduled to be visited and it takes an hour or two longer, some men and women press the button. Obviously, they are able to talk to the nurses through the box in this situation and the nurses can assess the low severity of it.

We also wanted to know if people had ever pressed the button accidentally. This barely happened, because of the smart design of the button. It was already dented in a little bit, such that even when elders would wear it during their sleep and then roll over the button, it wouldn't be activated. When asked if the elderly would like to be able to contact more or other people with the button and the box, they mostly said no. If they want to contact their family, they will use their phone, most of them told us.

Finally, we were interested in the question of responsibility when the system fails. We received varying answers to the question: "Imagine the system fails when you press the button, who would then be responsible?" Some said the elderly home (Vitalis), others thought the manufacturer of the system, and some really didn't know and told us that the system has always worked until now. This question might be so hard because the responsibility is in fact spread out over many people and organizations.

Part 2: A.E.E.S.

After the proposed questions on the current devices that the elderly wear, we started explaining how our intelligent design looks like and what its functionalities are. Briefly, the fall detection component and the fact that it consists a small, wearable communication system has been explained to the elderly, whereupon questions has been asked to determine what the elderly think of our design in contrast with their current device.

The interview started with questions about what the elderly think of the A.E.E.S. in general. Later on, its appearance and functionalities were questioned. Relevant questions as ‘Are you willing to wear a small wearable device on your hips with a related wristband or a smartwatch?’ and ‘How do you think our intelligent design could be used to improve the the functionalities of the current system?’ has been asked. The interview ended with questions to acquire some input from the elderly what our team could use to improve our design. What questions could best be proposed by the A.I. in case of emergency or help?

From the interview, it could be stated that the elderly have different opinions about our design. Most elderly were enthusiastic about our design, thinking that it could be a great improvement, while others are confident with the current design and think there is no need to use another device. They are familiar with its simple operation. Also, it could be concluded that most women are complaining about the spot where the fall detection component has been put, while men does not see this as a problem. The fall detection component is going to be placed on the hips, attached on a belt. Women do not agree with this, since women do not wear a belt that often. They prefer a system what is fully integrated into one design, instead of seperate components. This makes it correspondingly more comfortable and less time consuming for the elderly to put their emergency system on.

Considering the functionalities of A.E.E.S., all elderly what has been interviewed told that our new design could be very practical. It gives the opportunity to call for emergency or help at any location, whether it is in your appartment, or outside. Nowadays, elderly can only make use of verbal communication through the system what is attached on the wall of the living room, so when an elder got injured outside, there is a great chance that they cannot call for help in any way. A couple of interviewees were telling that they experienced several times that they were just waiting till an employee by coincidence walks by and can provide help, after they just fell on the ground. The elderly noticed and complained that it takes a long period of time to reach the injured ones by the employees. It can even take 1 to 2 hours!

Research

Medical Alert Systems

Currently there a few different wearable emergency devices for elderly. All of them have slightly different functions. Firstly, there is the Medical Guardian. The Medical Guardian is considered the best medical alert system available. The Medical Guardian (Premium) is a medical alert system that can be worn as an arm wrist, belt or necklace and has the following features:

• Fall detection that will call for help when the patient has fallen
• A button that will call for help and contacts you with an employee of Medical Guardian
• GPS-tracking
• Heat-sensor in case of fire

Other devices and further explanation on the Medical Guardian can be found further in this chapter.

Fall detection

The fall detection is most important to this research, since it is one of the main features of the Advanced Elderly Emergency System. Currently, most fall detection systems utilise accelerometers and gyroscopes. With these sensors the movement of the patient can be measured. A sudden change of direction can then be also be measured and calculated.

Most sensors have a certain waiting time before calling for help, because it checks for movement after a possible fall. If the patient moves, the system will not call for help. When an AI is added to the system, this waiting time can be reduced a lot, because the AI can ask the patient whether he or she has fallen.

In the A.E.E.S. the fall control system from 'FallWatch' will be used. A great part about this design is that it can be attached to the skin and is waterproof, so it can even be worn while taking a shower for example. This European project is further explained in the section about Fall Watch.

Speech recognition

Speech recognition is another important feature of the Advanced Elderly Emergency System. There are many applications for speech recognition. A very wellknown software is Siri. Siri is able to recognise one's voice and hear what the person is asking.

Current devices and designs

ZEMBRO

“Zembro bracelet is the first intelligent bracelet specially designed for the elderly. It looks good. And better yet: it’s good company. It offers you the freedom, independence and peace of mind that is so crucially important.” An elegant bracelet, it features smart functions and is connected to an app. Zembro personal alarm keeps you in touch with each other. At any time. Including those moments when it is so crucially important. And that gives everyone peace of mind. Because you’re always nearby. Even when you’re not in the neighbourhood. The Zembro bracelet makes the connection with your family or friends even stronger. And isn’t that the most beautiful gift that you could possibly give someone?

LIMMEX

Limmex in general:

The watch is always within reach. It works outdoors just as reliably as it does at home. Simply press one button and the Limmex Emergency Watch will set up a telephone call for you. Thanks to the built-in loudspeaker and microphone you can directly speak to your person(s) selected. You define which telephone numbers should be called. In an emergency situation every minute counts. Communicating with people in an emergency can be fraught with problems and cost valuable time and delay from the rescue services. All models come with an integrated high performance GPS-module (A-GPS). Thanks to the GPS-function the Limmex wearers can quickly pinpoint their location via satellite when they are outside of a building.

The Limmex Watch also sends an SMS to the person called after the alarm has been raised. The SMS contains a link to a map indicating the location of the watch at the time of the emergency call. You can also email the map link to anybody you wish. Your watch will only be located after an emergency call has been made. This means that the Limmex Watch battery lasts longer and that you decide when you want it to be located. If there is no GPS signal reception (e.g. inside buildings), GSM tracking is used over the mobile phone network.

Limmex for elderly:

Most elderly persons would like to live for as long as possible in their own home. A great number of them still have good health but nevertheless realize that the risk of falling, feeling unwell, dizziness or suffering from other health problems increases substantially with age. Many of them would like to own a device which offers them security without limiting their individual lifestyles. Traditional domestic emergency call systems only partially fulfill the requirements of this target group. Monitoring is only guaranteed inside the home – this is in spite of the fact that many older people remain mobile well into old age. In addition, traditional emergency call systems are complicated to operate because of their costly installation and their great number of technical alarms – and their design is unsatisfactory. With Limmex it is possible to solve most of these problems: the product offers security also outside the home, it is technically considerably easier than current solutions and due to the convincing watch collection it meets even high demands on design and aesthetics.

MEDICAL GUARDIAN

Get Help in 3 Simple Steps

Step 1

Press the button on your Medical Guardian Pendant. The small pendant can be attached to a belt, worn around the neck, or around the wrist. While you are home, the device should be worn at all times. Because it is 100% waterproof, the pendant can be worn in the shower or bath. When pressed, the device sends a wireless signal to the device's base station, in your home. This base station alerts the monitoring centre of an emergency.

Step 2

Communicate with a certified medical operator. Within moments of pressing the button on your Medical Guardian you will be contacted, via a two-way radio in the device, to a certified and trained emergency responder. The operator will assess your situation and help you determine what type of help you need. Able to access your file, the responder can view your medical profile, medications, preferred doctors, local hospital, and contact information for friends and family members. This means you don’t have to worry about an ambulance being sent to your home anytime you use the Medical Guardian system. You can arrange to have a neighbour come over, have a family member called, or anyone else you designate as a contact. If you are unable to respond, operators will send local emergency personnel to your home.

Step 3

Help is on the way! Completely portable, the Medical Guardian medical alert system can go anywhere you do. You can bring the security and peace-of-mind of Medical Guardian with you when you are on vacation, visiting family, or even permanently moving within the United States. Just plug the base station into the wall, notify the monitoring centre of your new location, and you are done. Medical Guardian provides service in all 50 states, allowing you to travel with confidence.

Design by Falin Wu et al.

Table XXX: Test results of the design with different motion types

A possible implementation of fall detection sensor is proposed in "Development of a Wearable-Sensor-Based Fall Detection System" by F. Wu et al. The advantages of using this sensor are:

• Can be worn on the waist, but does not have to be fixed in position
• Only uses a single triaxial accelerometer for fall detection
• Is power efficient, making the elderly able to travel without constantly having to recharge it.

This design is an acceleration threshold based design, which will be further explained below in the section Fall Detection Algorithm. The reliability of this sensor is fairly high. In Table XXX the test results of the system test are shown. As you can see, the sensor detects a fall 220 out of 240 times, resulting in an average sensitivity of 91,7%. However, also notable is the number of false alarms when a person is jumping, walking or resting. Most elderly people that need a medical alert system will not regularly be jumping, but to limit the number of false alarms it is necessary to experiment with the number of a_threshold.

Fall Detection Algorithm

Figure XXX: State diagram of fall detection algorithm

The fall detection algorithm consists of five states. The state diagram is the same as the original design by F. Wu et al., with the exception of state 5. The state diagram can be found in figure XXX.

State 1: Initial state

The acceleration a of the elder will be measured by the triaxial accelerometer. This acceleration will consist of a_x, a_y and a_z. The total acceleration can be calculated with the formula:

|a| = sqrt(a_x² + a_y² + a_z²)

g_before will be set to the current acceleration.

State 2: Acceleration Acquisition state

In this state the program will check if the current acceleration is higher than the set number a_threshold. It is necessary to do some experiments to find the right number for this a_threshold. When this acceleration is reached, the program will progress to state 3.

State 3: Impact state

When a person falls, it will reach a state of impact, after which he or she will be laying still. At the point where the person is laying still, the acceleration picked up by the accelerometer will equal g. Since it takes a while for a person to finish their falling, a t_threshold is necessary to minimize false alarms. This time will be set to at least two seconds. When the person has come to a stop and two seconds have passed, the program will progress to state 4.

State 4: Lying state

In the lying state, the system checks whether the person that has fallen has changed its angle relative to the ground. When the person starts by standing, he or she will have a rotation angle of 90 degrees after the fall. The rotation angle can be calculated by knowing the direction of the gravity g_before and the direction after g_after. g_after will be set to the acceleration after the fall, and the rotation angle can then be calculated using the formula:

θ = 2 arctan() (insert picture for formulas)

When the rotation angle is approximately 90 degrees, the program will progress to state 5.

State 5: AI state

In this state, the AI with speech recognition takes over. When the AI has done its job, the program will go back to state 1.

Hardware Analysis

Measurement hardware used in this system include a motion sensor and a GPS and GSM service unit. Together with a microprocessor for processing data and a speaker/microphone set for human interaction, these units comprise the entire system which will be powered by a low-voltage battery (1200 mAh with 3.7 V is enough to last a couple of days).

• An ADXL345 can be used as motion sensor and will typically cost €3.50.
• A SIM808 GPS GSM GPRS which is used for GPS and GSM service which costs around €40.00.
• A TI MCU MSP430F1611 microprocessor will control the whole system and imply the detection algorithm. This costs approximately €17.00.

When looking at ways to implement a smartphone/smartwatch, an extra bluetooth module has to be added to establish a bluetooth connection between said device and the microprocessor. This means that the GPS and GSM service is no longer necessary and allow for a cheaper system.

This requires a different microprocessor and perhaps a bigger battery to support full functionality. Other possibilities include making an internet connection between the smart device and the fall detection system. This also requires a module to establish a Wi-Fi connection or an entirely different processor which has integrated Wi-Fi functionalities.

European projects

FallWatch

FallWatch

Objective

Falls among the elderly occur frequently. Approximately 30% of people over 65 fall accidentally each year, and for those over 80 this rate raises up to 50%. Falls are an important factor in the health limitations of the elderly due to a high morbidity caused by physical and psychological consequences, loss of independence and even death. Late or non-existing responses cause 2 million persons having to be hospitalized and approximately 85000 deaths can be attributed each year to this fact. There are a number of alarm devices trying to satisfy the need of an early intervention in case of fall, but existing fall detectors do not have a remarkable social impact neither a significant market penetration: just 4.5% of Europe's potential end users have any of the existing social alarms. Two facts hinder their effectiveness of the existing fall detectors, capping therefore the market demand: bad ergonomics and lack of reliability. FallWatch project proposes a new generation fall detector device included in a whole fall detection system and is thought as an effective way of minimizing the consequences of falls in the elderly. FallWatch will automatically detect a fall and immediately contact the health services to allow an early intervention. FallWatch will produce, on the one hand, a miniaturized fall detector device of a scale reduction by 50% of existing devices, and on the other hand, biocompatible materials for the device’s package and for an adhesive patch to comfortably stick the device to the user’s skin. FallWatch system will solve the reliability problem by using both wearable device and environmental infrared sensors at the user’s home and developing a specific data fusion algorithm that will cross check the variables to avoid false alarms. In order to achieve the ergonomics and reliability goals, FallWatch project will work in different technological areas which are widely explained along this proposal.

Novel device enhancing care for older generation

Fall detectors, used mostly by the elderly, are limited by bad ergonomics and unreliability. A group of researchers proposed a new-generation device encompassing a whole fall detection system that enables early intervention and minimises the consequences of falls for this vulnerable population.

Funded by the EU, the objective of the project 'A wearable miniaturised fall-detection system for the elderly' (FALLWATCH) was to develop a wearable and radio-communicating fall-detection device. The first of its kind, the Vigi'Fall solution comprises a miniaturised embedded electronic device patched to the skin.

Seeking to overcome deficiencies of existing fall-detection products, FALLWATCH project partners advanced the concept of a comprehensive system able to manage a fall event from the moment it happens. From detecting the event to assessing its cause to initiating medical intervention, Vigi'Fall is a 'context-aware' rather than a standalone system.

Embedded on the holder, the device continuously measures kinematics and classifies the situation according to low, medium or high activity. Another system component, an in-home control box, monitors ambient activity from movement detectors, classifying the situation on its own three-degree scale (inactivity, average or exceptional).

Miniaturisation of the device involved complex technological routes for electronics (miniaturised multi-chip module (MCM)) and battery miniaturisation. Other technical work included development of a fall-detection algorithm, the construction and testing of an electrochromic display, the development and manufacture of a functional mechanical biocompatible package, and the selection of silicon material for the housing.

The prototype has been delivered and integrates the whole system, covering MCM, battery, electrochromic device, package and patch. Vigi'Fall has been validated on a first version of the prototype; following the industrialisation phase, it will be refined to produce a commercial first version of the device.

FATE - FAll DeTector for the Elderly

FATE

The ultimate goal of the FATE project is to widely validate an innovative and efficient ICT-based solution focused on improving the elder's quality of life by an accurate detection of falls in ageing people, both at home and outdoors. This has been done by implementing an accurate, portable and usable fall detector that runs a complex and specific algorithm to accurately detect falls, and a robust and reliable telecommunications layer based in ZigBee and Bluetooth technologies, capable of sending alarms when the user is both inside and outside the home.

The system is complemented by secondary elements such as a bed presence sensor or the i-Walker, an intelligent robotic walker, with the entire system ensuring successful prevention and detection of falls in all circumstances.

The system has been tested and validated in 3 pilot studies involving real living scenarios, one in each of 3 different EU countries (Spain, Italy and Ireland), in close collaboration with the relevant public authorities (regional authorities in Spain, municipalities in Italy and National authorities in Ireland). For the sake of an efficient and significant validation, the pilot test selected more than 200 individuals with high risk of falling.

I-DON'T-FALL

The main goal of I-DON'T-FALL is to deploy, pilot and evaluate a range of innovative technnological ICT solutions for fall detection and prevention management.

The I-DON'T-FALL platform will provide specificcaly tailored fall technological solutions while medical experts and professors will be given a wide range of tools which will enable them to customize fall solutions specific for the end-user's needs. I-DON'T-FALL accomplishes this by testing integrated and fall management solutions while altering certain techincal solutions targeting specific needs while finding root causes, risk factors and cultural factors associated with fall incidents.

The I-DON'T-FAll project contributes to prolonging the time that the elderly can live independently at home by integrating ICT based safety and FALL-prevention/detection services.

iStoppFalls

The aim of iStoppFalls is to develop and implement ICT-based technologies which can be easily integrated in daily life practices of older people living at home, and which allow for continuous exercise training, reliable fall risk assessment, and appropriate feed-back mechanisms, based on discreet measuring technologies and adaptive assistance functions.

iStoppFalls will integrate an AAL system that predicts and prevent falls by monitoring mobility-related activities and other risk factors of falling in real life. Offering this enables tailoring individualized exercise programs coached by iStoppFalls.

One component of iStoppFalls is the 'Senior Mobility Sensor' which will evaluate quantative information on frequency, duration and type of mobility activities and qualitative information on balance function and muscle power. This inertial sensor system is integrated in the form of a necklace which can be worn without restrictions. It also provides information on the effect of the training exercises for daily life of the user and in turn gives feedback needed to adjust the exercises in a most beneficial way.

A 'Fall preventive exercise game' will provide real preventive exercise training where data is gathered by a Kinect sensor together with biomechanical modeling and optinoal heart rate data.

The 'eHealth platform' correlates these two types of information and provides sufficient data to perform a trend analusis of these entities which in turn provides evidence for valid fall prediction and prevention.

Finally, the 'iTV application' presents advanced reasoning based on all data to the users. This includes the users further in the technology.

Advantages of the A.E.E.S.

Here will be explained why the A.E.E.S. is better than just using an emergency button and a call center, and here will be shown what the A.E.E.S really adds in new functionality.

The shortage of people working in health care will be supported.

The Dutch population is aging. In a decade the ratio: 'people above 65 years per people under 65 years' will increase a lot. Therefore, a lot more jobs will be created in the healthcare sector. However, less and less students are deciding to choose a study in the healthcare sector. This means that there will be a shortage of people working in the healthcare sector. The A.E.E.S. is a solution for this problem, as it is a possible replacement for people working in call centers.

The call center is not needed anymore, making it cheaper.

When the AI is strong enough to fully take over all the functions from the call center, the call center will not be needed anymore. Since it will be unnecessary to pay for a subscription to a certain call center, it will end up to be cheaper for the user.

'False Alarms' will be less problematic for emergency staff.

In the normal medical alert system, the emergency call center will be called when a fall is detected. This makes the call center take unnecessary calls every time a there is a false alarm. With the A.E.E.S. only the AI will be triggered, meaning that no person will have to react to the false alarms yet. Only when our system really doesn't understand the answers that the person gives and calls an emergency service, the staff is unnecessarily bothered.

It has improved efficiency

It takes no time for the AI to be triggered, whereas it may take a few seconds to get into contact with the call center.

It can work around false positive sensor data much better

It is quite challenging to make a fall detection system that perfectly detects when someone falls, and when someone does not (for example just laying down quickly). Because it is so hard, the chances are that existing fall detection systems will have some false positives. The design by F. Wu et al. given in section Design by Falin Wu has sensitivity of 97.1% and specificity of 98.3% in their test results. This means only 1.7% of non-falls have falsely been identified as a fall. To make sure the emergency services aren't called right away even in these cases, our system comes to use by asking the questions.

According to the European Project "FallWatch" around 30% of people in Europe above the age of 65 fall each year. Assuming this percentage holds approximately the same in the Netherlands, we can estimate the amount of falls per year. There are 3.1 million people over 65 in the Netherlands which would mean about 930,000 people fall each year. About 97,400 people over 65 visited first aid in a hospital after falling, in the Netherlands in 2015, thus we have a rough estimate that about 10% of the people who fall, also visit the first aid. The other 90% apparently was not severe enough for a trip to the hospital. This is a much bigger number than we expected. Thus, when we detect those 90% of non-severe falls, we need to figure out the situation isn't serious enough to call 911. When this successfully works, the emergency services won't be overloaded with calls from people with an A.E.E.S.

We attack this problem by using symptom analysis. After we have confirmed with the elder person that he or she has fallen and needs help, we ask what is wrong. We determine if the situation is severe enough for 911 by the answers to this question. If someone for example answers: "I have pain in my chest," he or she could have fallen because of a heart attack. When the situation is not that bad, but the elder still wants help, we call a provided contact person like a neighbour or child of the person. More about this method is explained in sections Calling the appropriate contact and Symptom analysis.

Approach

Requirements

These are components that each part of our design will most likely need.

The fall-detection-device

• Wearable, for example one of the following:
  • Belt clip
  • Wrist band
  • Necklace
• Sensors to detect falling:
  • accelerometer
  • barometer
  • gyroscope
• Air quality measuring sensor

The AI (smartwatch app)

• Microphone, for speech recognition
• Speakers, so it can speak to the user
• GPS to track location
• Internet and phone connections
  • Call emergency services
  • Contact next of kin
  • Share location with next of kin
• Button to manually trigger emergency state

Design options

Figure XXX: The four possible design options.

These are the possible options of designs that would could produce. The fall-detecting-device attached to one's belt will detect the fall. This belt clip could:

• have its own microphone and speaker to communicate with the user.

• or the belt clip could be connected to a smartwatch app. This smartwatch will then need to have a microphone and a speaker. Many smartwatches exist which don't have a speaker, thus we would need to look for a specific watch that does have one.

• or it could connect with a smartphone app. This is simpler in design for us, and most smartphones both have a microphone and speakers. However, many elderly won't have their smartphone with them at all times, if they even have a smartphone.

• or we could make our own wrist band with a built-in microphone and speaker to communicate with the user.

It seems like we should definitely make the belt clip, and see what we can add because of our limited time. But, the functionality of this belt device alone already exist, as we have seen in our research. To really add something to the state-of-the-art, we need to combine it with some form of AI. Thus another option is starting with the AI functionality.

We have decided to go for the third option, thus we will make a smartphone app. We actually think the design with the smartwatch is the best idea, because of the wearable functionality. Elderly don't always have a smartphone with them. But since none of us have a smartwatch, we will design a smartphone app with the same behaviour that the smartwatch app would have. We won't be making the actual fall-detecting-device, because as we said earlier, this already exists. It is not yet implemented into society, though.

AI functionality

The AI in the device is activated when the device registers a falling motion, or when the user presses the action button. This AI will then immediately contact emergency services to inform them that an elderly person fell. The emergency services then know the place/situation of the elderly person, and they will act accordingly by for example sending out an ambulance. After this, the AI will ask the person if they are okay, and eventually if the AI should contact family/friends. If the AI does not receive feedback from the user, the AI will send a message to family or friends that something is wrong with the user.

Specific questions from the AI

Suppose a fall has been detected (or the user has manually pressed the button). Then our device immediately triggers the AI and it will ask:

"Are you okay?" (or something in this trend)

• If the person answers "yes", we ask one question again, just to make sure. Like: "Do I need to call emercency services?" The answer will then decide if we take all the next steps of calling emercency services and the next of kin, or just returning to normal state again. In a machine learning environment, even when the answer to the second question is no, it still is valuable feedback to our system. Our AI could learn in what ways it detects a fall, and which are most of the time not severe.

• If the person answers "no", we notify emergency services immediately, while going on with the conversation with the elder. We will ask questions like: "Do I need to call emergency services?" and if the answer is "no", another "Are you sure?" will follow. If the person is sure, we will cancel the emergency call.

• If the person doesn't answer within 10 seconds, we will notify emercency services. After 3 seconds, for example, we will repeat the first question a bit louder. The elderly might not have heard it. When there is still no response, we could ask a different question after 6 seconds, like: "Do I need to call emergency services?"

When we have made sure that the elder needs help, we thus contact emergency services, like an ambulance. Then we send our location details

Thoroughly asking questions

The answers that an elderly person gives might not be very clear. Therefore, the AI should be able to figure out which questions it needs to ask to get the best information. For example, in a case from the company CSIservice, they once received an alarm from a woman saying she had a stomach ache. At first, people might underestimate the situation when hearing "stomach ache." But because the employers of the alarm centre are trained to keep asking questions, they discovered how serious the situation was. An ambulance was called, and it turned out that the woman had intestinal aneurysm.

Final design

Design description

The home screen of the App

A.E.E.S. is a wearable emergency system, which can mostly be worn by elderly. This system consists of two parts. Those parts are each individually responsible for different functionalities of the whole system. The system consists of an fall detection component and a speech recognition component, all integrated into one product. Artificial intelligence is used when designing the speech recognition component. The system should ask questions to the relevant person that just fell. AI can be used to have a conversation with the person based on the situation, and further on determine the state and severity of the current situation. Based on the state of this situation, it can perform actions fully autonomous by notifying health instances or other personal contacts in cases when needed. This can speed up the process to help the wounded elderly, or even save their lives.

During the project the design team is mainly going to focus on the prototype software development of the speech recognition component. This will be done on Apple's iOS by means of an application. The prototype of the software of A.E.E.S. will be used to propose questions to the elderly that just accidentally fall on the ground to determine the state and severity of the situation. This application will consequently make use of the responses of the elderly. The application makes use of a self-made database, where a limited amount of diseases with their corresponding symptoms are registered. On this way, artificial intelligence makes use of natural language processing to recognize and 'understand' the responses of the elderly. The main purpose of the application is to find whether the situation is severe enough the call the emergency services, or that just calling a contact person should be sufficient to resolve the problem. Therefore, the symptoms (with each its own severity) has to be analyzed. This will give the persons who are responsible for the help and/or care of the elderly more information beforehand, and exclude the use of emergency services when not recommended. The A.I. is responsible for this decision-making. Finally, the design is going to be used to reduce the number of false positives.

Basis of the AI

Decision tree as a basis of A.E.E.S

First of all, AI will be needed to recognise what the user says. But the next step is finding out what to do with the acquired information. As a start, we have developed a decision tree for this 'background AI'. As you can see in the figure below, it is not done yet (and it never will be) because at the end we want the AI to ask open questions, and we can get all sorts of answers out of this. We could divide the answers into categories like: fall related, other health issues, or unrelated. Also, the background AI, very much like someone currently from a callcenter, needs to rigorously keep asking questions in order to make sure nothing's wrong.

Calling the appropriate contact or service by estimating the seriousness

With the detection, and using other methods like thoroughly investigating the situation with many questions, the AI could make a better estimate of the seriousness of the situation. Depending on the outcome (the severity of the situation) the A.E.E.S. will decide who to call. If for example the system thinks that the person is having a hart attack, it will call 911. But if the person only has difficulty standing up again, it is better to call a contact person like a son, daughter, friend or neighbor of the elder. This contact information needs to be provided by the user when purchasing the system.

The following table shows examples of situations in which the system would call a provided contact person or directly call 911 (or 112 in Europe). When this product is really brought onto the market, these situations will need to be discussed with the authorities of course.

Personal Contact	Emergency Service (911/112)
difficulty standing up	heart attack
possible bone fracture, or none-severe like finger or arm	severe bone fracture, like open wounds or legs
trouble breathing, oppression	unconsiousness
panic attack	large open wound
concussion
person telling us not to call 911
large probability of a false positive

Person not responding

If there is no response to the questions at all, it can be quite difficult to assess the situation. The person might be unconscious, but could also be unwilling to answer. This is a situation in which extra data from the fall detection device can help. If the device still notices movement after the fall, the situation might not be that bad. But for safety we think its a good idea to inform 1 or 2 provided contacts with information via a text message. The message would contain information like: "The A.E.E.S. detected a fall of name of the person at (link to) location. The system has detected movement after the fall, but name doesn't respond to questions."

If the fall detection device actually doesn't notice any movement after a detected fall, then there is a good chance that the person is unconscious. The A.E.E.S. then calls 911 and speaks with an officer itself, after explaining that it is an AI helping the elderly with fall detection. It can't easily send the location of course, but the GPS functionality within the device knows the coordinates, which can be given to the officer.

The first question that is asked when you call 112 in the Netherlands is: "Do you want to speak with the police, fire department, or an ambulance?" After an answer is given you are directly connected with the right authority. The AI will most of the times just need to answer "ambulance." Then the questions like who, what, and where will come. This is the point where the AI will explain itself shortly and say: "I have detected that name has fallen, and he/she doesn't move or respond anymore." Then it would give the coordinates of the location.

Robustness

Of course we need some robustness of the hardware, to make sure the person didn't just throw his or her detecting device and the smartwatch away, and then walked away and thus didn't hear the questions. For this the smartwatch will have some kind of mechanism to detect if it is still being worn, like a body heat sensor. Another situation that could come up is when the microphone or speaker malfunctions. To make sure this isn't discovered when they are needed, the smartwatch will make a short test sound with a high frequency so that the owner does not hear it. If the device detects it's own sound, it knows there's nothing wrong.

Another kind of robustness in the software in necessary. We want the elderly to be comfortable talking to the AI in the smartwatch. This can be achieved by regularly talking to the elderly. Unfortunately, this doesn't happen if they don't fall often. Thus, it might be a good idea to have a little small talk with the owner about once per month, for example. The frequency of these talks could of course be adjusted if the elder is a bit lonely and wants to have more conversations, or if he or she doesn't want to talk.

Natural Language Processing

One way in which we could detect the different situations given in the table above is by natural language processing. The AI should then closely inspect the answer that is given to determine in what category the situation should be sorted, or what other follow-up questions need to be asked to make sure we make the right choice. We can implement this in our app in a simple manner by looking what words the answer sentence(s) contain. If it has "heart attack" in it (and no form of negation) then we could conclude to call 911.

Symptom analysis

In the symptom analysis we will use the natural language processing abilities, but combine them with a database of diseases and their symptoms. The system can greatly be used for injuries caused by falling. The additional data package with all information and symptoms of diseases can be added to the artificial intelligent machine. This way, the intelligent system can also be used for injuries caused by health problems.

The system asks the person "What's wrong?" or "Where does it hurt?" to hopefully get symptoms as an input. It then interprets the symptoms and checks whether a great majority of the given symptoms correspond with the information of a certain disease. The AI can perform a diagnosis and give an idea of what might be wrong. It then decides whether to call the emergency services (911) directly, or call a provided contact person first. Of course, when this contact speaks to the patient, the contact person can still decide to call 911.

Further on, the A.E.E.S. can even give a recommendation about what the person can do best. For example, it could happen that an elder makes use of the system because of stomach ache. Usually, the system asks questions to give an expression of the seriousness of the situation, but with the introduction of a data package with all disease symptoms, it can ask for symptoms explicitly to determine the situation and health even better and more accurately.

An small example of how such a data set would look like is given in the table below.

DISEASE	Heart Attack	Severe Bone Fracture	Bone Fracture	Severe bleeding	Unconciousness	Concussion
SYMPTOMS	Pain in chest	Hurting bodypart = leg, back, hip, neck	hurting other bodyparts	Lots of blood	Lowered heart rate (FallWatch)	Headache
	Pain in neck	Can't move bodypart	Can't move bodypart	A cut	No Answers	Confused behaviour
	Pain in upper arm	Pain in body part	Pain in body part	blood still flowing		Nausea
	Pressure in upper back	Displacement of body part	Displacement of body part			Dizziness
	Dizziness	swelling	swelling			Hard to speak
	Lightheadedness	bodypart turns blue	bodypart turns blue			Ringing in the ears
	Fainting
	Nausea
	Shortness of breath

This table has been implemented into the simulation app of A.E.E.S. Later on, it should of course be extended to a much larger database of diseases and their symptoms.

After the call / when no call is needed

According to NIH SeniorHealth two stages exist when someone has fallen. In the first stage, right after the a fall, it is important to reassure the specific person. In the second stage the person should, when possible, get up from the fall. These two stages will be elaborated in the following passages.

Right after a fall

It is important for the person that has fallen to get help as fast as possible. Our device can help by asking specific questions and giving advice. The first three steps that should be taken by the person that has fallen will be described now.

• Take several deep breaths and try to relax. When our device detects a fall, it will ask emergency questions and will be able to say to the fallen person to relax, as is described before.
• The fallen person should remain still on the ground for a few moments, this will help to get over the shock of falling. Our device will help with this, by telling the person to do so.
• The third step for the person that has fallen is to decide if he/she is hurt. Our device will ask injury-related questions to the person. Now can be decided if the person should get up from the fall or lay still until help arrives. This is because getting up too quickly or in the wrong way can make the injury worse.

Getting up from a fall

When our device, based on the conversation with the fallen person, decides that no help needs to be called, the patient can try to get up from the fall. Our device will help with process by following this template.

• If the fallen person and our device decide the person can get up safely without help, the person needs to roll over onto his/her side.
• Now, the person can rest again while his/her body and blood pressure adjust. Our device will tell to slowly get up on hands and knees, and crawl to a sturdy chair.
• Put your hands on the chair seat and slide one foot forward so that it is flat on the floor. Keep the other leg bent so the knee is on the floor.
• From this kneeling position, the person can slowly rise and turn their body to sit in the chair.

Less false positives

Because of the question-answer system built into the device, resulting scenarios will be subjected to investegation by the device. This investagation ensures a higher accuracy when claiming a fall has been detected.

This question set, which includes very straight to the point questions like 'Did you fall?' and other less directly fall-related questions like 'Are you okay?', provides insight into the current situation of the person.

Even when a fall is detected, a question regarding whether or not the user has actually fallen will be asked. This way, when the person did indeed fall, absolute certainty can be established.

When a fall was detected but the user tells the AI that he or she did not fall, further actions need not necessarily be taken. By asking the questions, determining an actual fall is much more accurate as opposed to only relying on the sensor since these false positives in the sensor readings are filtered out by also factoring in given answers.

Possible expansions

These are all expansions to the project, which would greatly improve the A.E.E.S. Unfortunately, we didn't get to these topics, or decided not to because of their complexity.

Detecting tone of voice

By analysing the tone of the users voice it should be possible to make some assumptions about the severity of the situation.

Learning users patterns

The system could learn from when an user often triggers the system without there being an emergency. It can for instance update its thresholds or also take time of day and or the users patterns into consideration.

Possible other Implementations

While this device can be very useful when implemented in the care of the elderly regarding falling, it is also possible to slight alter this device to easily implement it in other areas which will benefit more people.

Literature

Other pages with important information on this subject:

URLs

Medical guardian emergency system

https://www.medicalguardian.com/product/premium-guardian

The Best Medical Alert Systems of 2017

http://www.toptenreviews.com/health/senior-care/best-medical-alert-systems/

Speech recognition library

http://arjo129.github.io/uSpeech/

Appendices

This section gives an overview of the progression and planning of our design project. This mainly concerns organizational as well as technical tasks, decisions and ideas that have been performed within our project environment to maintain a successfull and appropriate end result. Explanations on how certain decisions has been made by our design team can be found in the following, relevant sections:

Appendix A

Project progress (log): Gives an overview of the general and specific tasks for each member every week.

Appendix B

Planning: A schedule of the general tasks.

Appendix C

Interview: Nursing home: Gives an overview of the proposed questions and answers during the interview.

Appendix D

Application code: The full Swift-code of the iOS-application of our emergency system is given in this section.