PRE2018 4 Group5

<link rel=http://cstwiki.wtb.tue.nl/index.php?title=PRE2018_4_Group5&action=edit"stylesheet" type="text/css" href="theme.css"> <link href='https://fonts.googleapis.com/css?family=Roboto' rel='stylesheet'>

Group members

Group Members	Student nr.
Erik Wubbels	0917805
Bas van Kampen	1236216

Planning

Abstract

This wiki contains the research phase and explanation of a device that detects falling and communicates with an online platform that allows the sensor data to be accessed. The device is also able to communicate with the user via voice control, in order to verify fall detections.
A two-part prototype was made. One part is the measuring device, the other part is the platform.
The measuring device monitors accelerometer data until it passes a threshold. The accelerometer data is then sent to the platform, and voice control communication starts.
The platform receives the accelerometer data and the voice control answers, and stores them in a database. A web client is then able to access this data and view it in a graph.
Possibilities to expand the platform, such as sensor addition, are described.

Brainstorm Ideas

When starting this course, the first thing we did was brainstorming about multiple possible projects. The results of this brainstorm are shown below:

- (Using datasets from Eindhoven for) Internet of Things (IoT) actuation throughout the city
- Mobile Cloud Robotics as a service with OCCIware
- Robots to help elderly and physically impaired people with daily tasks
- Learning with augmented reality glasses / AI mentor
- Consumer dental x-ray scanner device (+service), from home
- Wearable for elderly people to quickly arrange quick responders in case of falling

After doing some minor research on each topic, we decided to combine designing a wearable for elderly people for fall detection with connecting it to the Internet of Things. This way, a standalone device can be coupled to the internet to allow sharing data with secondary users, such as caretakers, doctors, etc. This also allows more complicated (but more accurate) data analysis to ensure no false positives or false negatives occur. An additional profit of connecting the device to the internet, is that is becomes "infinitely" expandable. When more devices are created for the same purpose, they can be connected to the same platform, giving users the choice of which devices they want to use.

Initial problem description

This subject of this project is supporting elderly people living their live independently by assisting them with a wearable AI device. The great strides in the rise in longevity of people, have resulted in an ageing populations in most developed economies. This developed creates new challenges, for the individual as well as society. Many individuals want to live as independently as possible, but their physical and/or mental impairments make that more difficult. It is also difficult for their loved ones, since dangerous situations arise. For both individuals, their families as society as a whole, the associated increase in healthcare costs is a challenge. One of the big problems for all involved is falling. Elderly people are more likely to fall for biological reasons, and their falls can cause bigger damage as well. Additionally, for elderly people it is more risky to undergo surgery, and recovery (such as learning to walk again) is more difficult - for more feeble individuals surgery is not even possible, resulting in a big permanent disability and pain. With more elderly people living on their own, instead of in retirement homes, there is a bigger risk of people lying helplessly on the floor for hours or longer. This is a horrible experience, and it can increase injuries, and even result in death in extreme cases.

This project started with the objective to support these elderly individuals, as well as their families and hence society as a whole, by developing a wearable device to detect falling, connect it to the IoT and notify the people responsible for taking care of the falling person. This is a solution in the core of robotics: it combines a device with various mechanical aspects, interaction with the environment, and autonomous behavior. We want to minimize negative health consequences of getting older through measuring activities and presenting feedback.

A wider range of problems our elderly users face has been identified. These will not be the main focus of the project. However, since a part of our aim is to develop an online platform, other devices can be easily coupled. These problems include:

- Measuring health-parameters and (earlier) diagnosing of diseases and injuries:
    - automatic fall detection
    - PPG: measuring of blood flow and oxygen-levels, to detect and monitor cardiovascular, respiratory diseases
    - heart-attack detection
    - sleep (problems)
    - breath analyses
- GPS-tracking / Bluetooth to prevent wandering
- Context-aware medicine planning/reminders
- Exercise training with feedback

Goal of the project

The main goal of this project is to develop a platform to which an accelerometer can be coupled. This is mainly a proof of concept. The accelerometer is chosen because it is a simple sensor to work with, but in reality, any type of sensor (and even multiple types of sensors) can be coupled with the platform.

The platform should be able to receive data from the sensor, store it, and process it.
For this project, it will show whether the user has fallen, is in pain and/or is in need of help.
Clients will also be able to see the accelerometer data of a fall.

The device with the accelerometer should be able to measure data, process it offline, and send the results to the platform.
This offline processing can be useful for fast responses.
At later stages, it may be preferable to process data online, so that more complicated processing methods become available, increasing the reliability of the result.

USE aspects

In this section, USE stakeholders will be described, and the link to the project will be explained.

Users

Primary users The primary users of the device will be elderly people. More specifically, elderly people who live independently. According to the CBS, the amount of elderly people living independently is increasing. [1] It is important to encourage this trend, since care homes are expensive to maintain. [2] Reducing the amount of elderly people that move to care homes will keep these costs down, especially due to the growing population, which may cause a shortage of care homes.
The statistics provided by Valsymposium show that the amount of serious injuries caused by falling is quite high, and deserves attention. [3] The device will be beneficial for the elderly people that use it. It will encourage a more active lifestyle, it may prevent fear of falling to a certain extent, and it will reduce the amount of serious injuries caused by falling by alerting other people when a fall is detected.

Secondary users The secondary users will be the users of the platform, and the people that will be alerted when a user of the device needs help.
The platform may be used by family, care givers, doctors, etc. It may be used to visualize measurements to find trends or patterns in falling of the user, and care given to the user can be changed based on these measurements.
The extent to which secondary users can use the platform depends on the extent to which the user of the device is willing to share data. The user may decide to share no data and only let the device alert secondary users when they fall. The user may, on the other hand, also decide to share all data to a range of secondary users.

Tertiary users The tertiary users may be defined as healthcare institutions and repairmen. They will not have much demands. The data that is stored in the cloud must be easy to access for the users that are authorized. This will allow healthcare institutions to monitor the user if a lot of falls occur, and can help deciding on the future of the user.

Society

The population is growing. This means that there are more elderly people in our society. It is important for our society to encourage independent living for elderly people, otherwise nursery homes will not be able to take in all these elders. As mentioned before, maintenance of nursery homes is expensive, so if we do not limit the amount of people living in such homes, the costs will go up, and will probably increase more, since new nursery homes will have to be built.

Enterprise

Living safely independent reduces costs for hospitals and healthcare institutions. Accidents will happen less and when an accident occurs it can be addressed quicker resulting in faster care for the user and reduced costs, since injuries can be treated faster and in an earlier stage, decreasing the risk of permanent damage.

RPCs

Requirements

Specificity of 100%
The device should never fail to detect a fall. It should preferably be as accurate as possible, but falsely detecting a fall is not a very big problem, since the user can deny the detection. False negatives, on the other hand, are a big problem. If the device doesn't detect a fall, it doesn't do its job properly and is not a useful device.

Battery life of at least 24 hours
A battery life of at least 24 hours will allow the user to use the device throughout the day, without having to recharge until they go to bed.

Easy to wear
The device should be comfortable to wear, and may not interfere with daily tasks and activities.

User interface that suits elderly users
The device is targeted towards elderly users. They are often not familiar with complicated user interfaces, and may not be able to use the device properly if the interface is not simple. Preferably, no user interface is needed, other than voice control.

Voice-control
The user should be able to control the device with their voice. If the user falls, they may not be in a state where they can press buttons. Voice control is suited for these situations.

Usable when offline
The user may not always be at a location where internet is available. If they fall in such locations, the device should still be able to alert others.

Preferences

No calibration needed
Preferably, the device should work at any time without the user having to worry about the calibration of the device.

Only one device needed
Preferably, the user only needs one device, the fall detector itself, and doesn't need other devices to perform any of the tasks, such as alerting a designated person when a fall occurs, or sharing data to the internet.

Constraints

Designed for elderly people
The device should be suited for use by elderly people. This means that the user interface needs to be very clear and the device should be comfortable to wear. <br?

Limited budget
The device should be relatively inexpensive. There is no real constraint for the price of the device, but during the project, there is no budget (so a constraint of 0$).

Limited time (Only a constraint for the project)
The research and prototype must finished within 7 weeks, with an average work load of 20 hours per person per week.
For us, this is about 400 hours of work (150 hours for two group members, and some hours made by two group members that didn't finish the project).

Deliverables

Prototype
A physical prototype that measures falling, sends the data to the cloud, and delivers feedback to the user or caregivers.

Wiki page
A website that contains all information about the project.

Presentation
A presentation in which the research done is explained and the prototype is demonstrated.

State of the art

AI supported living for elderly

The Elderly’s Independent Living in Smart Homes: A Characterization of Activities and Sensing Infrastructure Survey to Facilitate Services Development
Source: https://www.mdpi.com/1424-8220/15/5/11312
Summary: an systematic review on ai-support for elderly people, it is about smarthomes, but we can adapt this to our wearable device. The analysis-unit / building block is ‘activity’, to which particular measurements and assistance-forms can be applied. The right kind of sensors, preprocessing and evaluation need to be chosen. This article has a classification of main activities of elderly people living independently, and suitable sensors and data processing for these.

Related: ai-system that uses constraint‐based scheduling technology to actively monitor a pattern of activities executed by the person. Detects temporal constraint violations which are used to trigger meaningful and contextualized proactive interactions:
https://onlinelibrary-wiley-com.dianus.libr.tue.nl/doi/full/10.1111/j.1467-8640.2010.00372.x?sid=worldcat.org

Integrated e-Healthcare System for Elderly Support
Source: https://link-springer-com.dianus.libr.tue.nl/article/10.1007%2Fs12559-015-9367-3
Summary: An unobtrusive integrated e-healthcare system for elderly support (gerontech) for monitoring biomedical parameters of a person in real time, anywhere and in any situation. The data is send to a smartphone or tablet, and can be shared with care-takers. Continunous monitoring gives a wealth of health-data-history for better diagnosing and preventive care, as well as quicker response in emergencies. It also discusses the easy use of contacting medical assistance / consultation from home, which is useful for elderly people with walking disabilities.

Current progress of photoplethysmography and SPO2 for health monitoring
Source: https://link-springer-com.dianus.libr.tue.nl/article/10.1007%2Fs13534-019-00097-w
Summary: A photoplethysmograph (PPG) is a simple medical device for monitoring blood flow and transportation of substances in the blood. It consists of a light source and a photodetector for measuring transmitted and reflected light signals. Clinically, PPGs are used to monitor the pulse rate, oxygen saturation, blood pressure, and blood vessel stiffness. Wearable unobtrusive PPG monitors are commercially available.

This article reviews the issues and applications for monitoring oxygen saturation, such as detection and monitoring of cardiovascular disease(s), sleeping disorders, respiratory diseases. Some can be avoided by these (daily) measurements. The measurements are not very robust, so patients need to be explained very well how to do them correctly.

An autonomous robotic exercise tutor for elderly people
Source: https://link-springer-com.dianus.libr.tue.nl/article/10.1007%2Fs10514-016-9598-5
Summary: Interesting application of using ai – here of a physical robot, not just a wearable device – to help elderly people learn new physical exercises, and to help them train more. ‘ambient assisted living’ is the notion to sustain the mental and physical health of elderly people in the comfort of their own homes. Perhaps we could replace the robot with an instructional videos on tv or laptop or tablet, and the placement of a camera to observe the motions of the elderly trainee, while using our device to measure the activities and use the ai-aspect to interact by providing feedback.

Related: review article about the benefits of training for elderly people, and specifically what kind of exercises are useful:
https://link-springer-com.dianus.libr.tue.nl/article/10.1007%2Fs40520-017-0863-z

Medicine Reminder and Monitoring System for Secure Health Using IoT
Source: https://www-sciencedirect-com.dianus.libr.tue.nl/science/article/pii/S1877050916000922
Summary: This article shows how medicine reminders and monitoring systems that are hooked up to the Internet of Things (IoT) can be beneficial for prescribing the correct medication to patients. It also provides possible ways that these systems can work. Monitoring medicine intake of the users allows doctors to have insight in the users commitment to the medication, and to decide whether or not the medicine is working properly. The reminders also prevent accidental skipping of medicine intake that may be caused by dementia or similar reasons, and the financial factor that this skipping may induce (for example, changing to more expensive medicine while this is not actually needed). The data of the medicine intake patterns of the users can be stored on the cloud (provided that the connection made is secure, as to not corrupt the data). This data can later be accessed by the user and by doctors to review past activity�

AI empowered context-aware smart system for medication adherence
Source: https://www.emeraldinsight.com/doi/pdfplus/10.1108/IJCS-07-2017-0006
Summary: This article discussing the downside of many (proposed) electronic pillboxes, which use often use time-based reminders. However, these reminders can come at inopportune moments for the pill taker, resulting in suboptimal medication adherence. This article proposes a method of AI-empowered, context-aware reminders. From measurements of the user and the/his environment. This can integrated into our device very well, it seems, only the/a electronic pillbox is optional, although most elderly people have prescription medication, so it is not that strange of an option to pay attention to in our development.

Recognition of Activities of Daily Living with Egocentric Vision: A Review
source: https://www.mdpi.com/1424-8220/16/1/72/html
Relevance: Sensor using camera. Helping to support the egocentric view. It uses a model based on different stages.

Can be intrusive due to camera's monitoring users behavior. So think about possible privacy complaints. Not the best option'

Wearable and Portable eHealth Systems
Source: https://ieeexplore-ieee-org.dianus.libr.tue.nl/document/4312668/references#references

Relevance: Different studies on personal health measuring systems(PHMS)

The validity and reliability of consumer-grade activity trackers in older, community-dwelling adults: A systematic review

Source: https://www-sciencedirect-com.dianus.libr.tue.nl/science/article/pii/S0378512218301828

Short summary & relevance: How good do these trackers work. Errors are measured when elderly people walk slow, however overall the results where highly accurate. Food to think about regards which tracker system is best for different kind of users. Users with certain chronic disorders have multiple dips in their activity, so not always true. However using these kind of step-wise trackers to self monitor the physical activity of the user is a good idea. It is important to keep these users active, especially elderly, because it is one of the biggest strategies to reduce age-related morbidity.

Multimodal Wearable Sensing for Fine-Grained Activity Recognition in Healthcare

Source: https://ieeexplore-ieee-org.dianus.libr.tue.nl/document/7155432

Relevance:

A multimodal system. Using different sensors, think about accelerator and gyroscope, to sense the user its position. GPS and Bluetooth beacons to detect someones position. Important to distinguish between different scenarios. Although the sensors can detect if a person is sitting or standing it has to distinguish the difference between for example sitting on the ground, which could be a sign of a bad situation(falling), or sitting on a sofa. So at this moment only basic ADLs can be distinguished. A way to get rid of this problem is to use a camera. However, a lot of users do not prefer this solution due to the privacy that is been violated, constant recording of what you are doing.

In the table beneath are some other studies which uses different sensors to sense the different attributes of the users.

Fall detection

Emergency Fall Incidents Detection in Assisted Living Environments Utilizing Motion, Sound, and Visual Perceptual Components
Source: https://ieeexplore-ieee-org.dianus.libr.tue.nl/document/5623343
Summary: This article discusses an advanced system with various sensors and processing to measure and verify falling, accessing the severity of the situation and arranging suitable help. The method uses semantic representation of the patient's status, context and rules-based evaluation, and advanced classification. The article also discusses various advanced classification techniques that have been and their accuracy and efficiency in detecting an emergency situation.

Inertial Sensing-Based Pre-Impact Detection of Falls Involving Near-Fall Scenarios
Source: https://ieeexplore-ieee-org.dianus.libr.tue.nl/document/6905812
Summary: This article discussed two improvements of ‘general’ accelerometer fall-detection. First, it includes ‘near-fall’ scenarios in its analysis. Second, it uses a different way to measure near-fall and fall scenarios, a vertical velocity-based pre-impact fall detection method using a wearable inertial sensor. The conclusion is that this detection method was more accurate in their own experiment, compared to an accelerometer, in detecting fall scenarios from near-fall scenarios. In other words, this method is claimed to solve the issue of ‘false positives’ that ‘mere’ accelerometers have.

Related: This article has a good introduction about fall detection, with many useful references: https://link-springer-com.dianus.libr.tue.nl/article/10.1007%2Fs10015-017-0409-7

Related: increase accuracy with barometric measurements: https://ieeexplore-ieee-org.dianus.libr.tue.nl/document/5559476

Related: increase accuracy with surface electromyography: https://ieeexplore-ieee-org.dianus.libr.tue.nl/document/6399498

Analysis of Public Datasets for Wearable Fall Detection Systems
Source : https://www.mdpi.com/1424-8220/17/7/1513/html
Relevance: In the study they looked at different datasets and compared them regarding the fall data. At the end the conclusion was that it is very difficult to determine an abstract and invariant threshold to detect falling. It is important to establish the different Activity of daily lives (ADLs) in the evaluation of the fall detecting systems FDS. So to sum up, movements should be put into groups of the same mobility to properly detect it with a sensor like an accelarotor in combination with a gyroscoop.

Fall Detection Using Smartphone Audio Features
Source: https://ieeexplore-ieee-org.dianus.libr.tue.nl/document/7093113
Summary: This study explores the best ways of using audio features of a smartphone to detect a fall. A database of different smartphone recordings (some recordings of falling sounds, some recordings of none-falling sounds) was processed in MATLAB to obtain the spectrogram of the recordings. Then four ways of recognizing falling sounds were explored: K-Nearest Neighbor Classifier, Support Vector Machine, Least Square Method, and Artificial Neural Network. Results of the study show that Artificial Neural Networks were the most successful in recognizing falling sounds, with an accuracy of above 98%.

Fall Detection Monitoring Systems
Source: https://link-springer-com.dianus.libr.tue.nl/article/10.1007%2Fs12652-017-0592-3#Sec2
Summary: This article covers multiple types of fall detection monitoring systems. It divides the system into three categories: wearable systems, ambient systems and camera systems. Some properties of the system are summarized in the table below.

Wearable systems are our main focus, so only wearable systems will be discussed further. Wearable systems generally use embedded sensors to monitor movement of a person. These sensors are usually accelerometers. The article also mentions that smartphones can be used as sensors to detect motion. Advantages of using a smartphone include the fact that most smartphones have a lot of sensors built in, including accelerometers, gyroscopes, proximity sensors, etc. Problems of using a smartphone include poor battery life and real-time processing speed (compared to committed systems). Current disadvantages of wearable systems include:
- Wearable systems can be intrusive.
- Wearable systems can run out of power (since they are all battery powered).
- If microcontrollers are used, the software cannot be updated. (Smartphones don’t have this problem.)
- Most microcontrollers implement only threshold classification, which means they don’t learn from their mistakes. (Machine learning algorithms solve this.)
- Wearable systems lack context, which allows more false positives to be generated compared to other types of systems.

Continuous Heart Rate Monitoring using Smartphone
Source: https://ieeexplore-ieee-org.dianus.libr.tue.nl/document/8053379
Summary: In this article, a method to continuously monitor heart rate is explored. The article mentions an existing method to measure the heart rate using the light and camera of a smartphone. This method records the blood flow in the finger of the user and detects the heart rate based on the blood flow. For continuous monitoring, this is not practical. In this article, an embedded system is developed to allow continuous monitoring. The system uses a sensor that converts biopotential analog measurements, made by electrode pads, to digital output signals that can be processed more easily. The signal passes through a microcontroller, which detects when heartbeats occur. During a 10 second time period, the amount of beats are tracked. After this period, the data is extrapolated to calculate the heart rate of the user. This data can be communicated to a smartphone via bluetooth, wifi, etc. A systematic overview of the program is shown below. While this method of monitoring heart rate is very simple and effective, the design that the authors of this article came up with is not fit for continuous use. An improvement in design is necessary to make the system usable.

Wearable Heart Monitor Catches Undiagnosed Atrial Fibrillation
Source: https://search-proquest-com.dianus.libr.tue.nl/docview/2074920876
Summary: The results of this study show that wearable heart monitors are capable of catching undiagnosed heart disease in patients. The information was collected in real-world settings and the device does not interfere with day-to-day routines of patients and doctors. The used device is an FDA-approved electrocardiogram (ECG) monitor, the wireless iRhythm ZioXT patch. The patches are able to detect the disease before symptoms show.

Portable breath monitoring: A new frontier in personalized health care
Source: http://interface.ecsdl.org.dianus.libr.tue.nl/content/25/4/63.full.pdf+html?sid=df2bbd55-8488-451b-9960-ea27c47f12ee
Summary: Portable breath monitoring is a way to analyze the user’s health by inspecting the composition of the user’s breath. Breath consists of many different molecules. A deviation of the average breath composition may indicate a problem in the user’s health. Often, knowing what molecule is excessively present in a person's breath allow doctors to make an immediate link to a certain physiological problem or a disease. (A table with examples of this is given below.)

Sensors that can measure the amount of a certain molecule have been miniaturized to the point where they fit onto a 3 by 4 millimeter board. This means that breath analyzers can now be made portable very easily. As soon as enough breath profiles are collected for certain diseases, breath monitoring can be standardized as a method to identify diseases. In addition, breath analysis could be critical in emergency situations where the patients are unable to report their condition, or in early stages of lung injury, disease or toxic exposure.

Non-academic but useful sources about the problem of elderly people falling

Elderly people fall quicker (biology) https://www.healthdirect.gov.au/what-causes-falls

Falling is more dangerous for an elderly person, also because they often cannot undergo surgery. (medicine) https://www.msdmanuals.com/home/older-people%E2%80%99s-health-issues/falls/falls-in-older-people

Falling elderly people is a real problem, in the US 1 in 4 falls every year! More facts about the size of the problem: (healthcare and social costs) https://www.cdc.gov/homeandrecreationalsafety/falls/adultfalls.html

Cloud Computing

Mobile Cloud Robotics as a Service with OCCIware

Philippe Merle-Christophe Gourdin-Nathalie Mitton - 2017 IEEE International Congress on Internet of Things (ICIOT) - 2017. https://ieeexplore.ieee.org/document/8039054

Summary: This study proposes combining of cloud computing and robotics into a system called Open Mobile Cloud Robotics Interface (OMCRI). Using an extension of Open Cloud Computing Interface (OCCI), a standard and gateway for hosting mobile robot resources. Then it illustrates the use of these technologies in three off-the-shelf robots: Lego Mindstorm NXT, Turtlebot, and Parrot AR. Drone.

Cloud robotics: Current status and open issues

Jiafu Wan-Shenglong Tang-Hehua Yan-Di Li-Shiyong Wang-Athanasios Vasilakos - IEEE Access - 2016. https://ieeexplore.ieee.org/abstract/document/7482658

Summary: Taking a deeper look into the combining of cloud computing and robotics this study analyzes the subject while looking at combining multi-robot systems with improved energy efficiency, high real-time performance and low cost. And finally showing potential value of these systems through different practical applications. Big data, cloud computing, open source resources, cooperative robot learning, and network connectivity are the major technologies being analyzed.

A High Reliability Wearable Device for Elderly Fall Detection

Paola Pierleoni-Alberto Belli-Lorenzo Palma-Marco Pellegrini-Luca Pernini-Simone Valenti - IEEE Sensors Journal - 2015. https://ieeexplore.ieee.org/abstract/document/7087338

Summary: The study proposes a fall detection system consisting of an inertial unit that includes triaxial accelerometer, gyroscope, and magnetometer with efficient data fusion and fall detection algorithms.They discuss different solutions and their pros and cons. Then they explain the measuring solutions they used, for detecting a fall through the measuring of the human body as well as the measuring the acceleration and orientation of the person using it. The device they describe is worn at the belt to provide undisturbed movement to the person using it.

It then goes on to discuss the different algorithms they tested and compare their reliability and performance to each other.

An overview of wearable applications for healthcare: requirements and challenges
Vivian Motti-Kelly Caine - Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers - UbiComp '15 - 2015. https://dl.acm.org/citation.cfm?id=2809436

Summary: This article goes into the combination of mobile applications within wearable devices. Discussing seven examples of the applications that have been put to practice as well as the challenges they still bring within healthcare. Particularly focused on promoting healthcare and behavior change. Going into the intuitiveness of interfaces, privacy control, customizability, data collection, data analysis and encouragement.

A Study On Cloud Robotics Architecture, Challenges and Applications

G. Arunajyothi- - International Journal Of Engineering And Computer Science - 2016. https://ieeexplore.ieee.org/abstract/document/6201212

Summary: The study looks at the capabilities of shared information and computation through a cloud robotic architecture. Proposing machine-to-machine (M2M) communication combined with a machine-to-cloud (M2C) communication, making it possible to increase efficiency among the machines as well as sharing different appliances through the cloud.They propose the protocols for these technologies and something they call Elastic Cloud Computing Architecture with three different models: Peer-Based, Proxy-Based and Clone-Based. They go on to discuss the communications challenges and minimizing loss of a message through delay as well as potential security risks due to it being in the cloud, proposing different protocols.

Reliable MAC design for ambient assisted living: moving the coordination to the cloud

Source: https://ieeexplore-ieee-org.dianus.libr.tue.nl/document/7010519

Relevance: Talks about ways to use cloud computing to enhance ambient assisted living(AAL). Ways to transform information from wearable(s).

Analysis

User Analysis

The following questionnaire will be distributed in care homes in Eindhoven in order to determine what features the device needs.

File:Questionnaire care homes.pdf

Note: The person responsible for the questionnaire dropped out of the course before analysing the results. The filled out forms were never shared with the group, so this data is missing.

Audio Analysis

Recordings of audio can be processed to obtain the spectrogram of the recording. Frequency analysis can determine whether or not a recording is of a falling sound. There are multiple ways to do this. Study has shown that creating an Artificial Neural Network (ANN) to analyse the spectrograms is the most reliable method. [4] The ANN consists of a set of nodes that give weighted output, and finally arrive at two final nodes. One of these final nodes gives the correlation of the audio to a falling event, the other node gives the correlation to a non-falling event. The study shows that the sensitivity, specificity and accuracy of this method are all over 98%.

There are limitations to audio analysis. First of all, the recordings always have to be compared to example recordings of falling sounds, so if a person falls in a different way than any of the provided reference recordings, the system will not recognize the fall. Secondly, the system may recognize the sound of falling objects as that of a falling person. Lastly, the environment has a big influence on the recordings. If the environment is noisy, a method like blind signal separation may be used to isolate the sounds of events and remove background noise, but there is a limit to how noisy the environment can be before the systems stops working properly.

Implementation
To implement this method, the system has to be trained with training data to learn what recordings of falling people sound like. Codes for ANNs already exist, so these do not have to be programmed. (Note that if these systems are intended to be commercially used, it may be beneficial to program these anyway in order to adapt and optimize the code for this specific situation. For a prototype, codes for Arduino based ANNs are available. [5] ) All we need to do is to find a database with falling sounds and sounds that have a similar spectrogram but are not falling sounds, and label them with the correct answer (falling or non-falling). Then, after feeding them into the ANN, the program should be able to recognize falling sounds.

Note: This is a concept that may be explored later, if the prototype works before the end of the course. This method demonstrates how the platform can be used for analysis too, rather than only receiving and storing data.

Accelerometer

Triaxial accelerometers measure the acceleration in the x, y, and z direction. With these measurements, the angle of the device can also be determined. There are several options for fall detection algorithms that use the parameters.

The simplest option is to check whether a parameter (or a combination of parameters) exceeds a predetermined threshold. This threshold will of course be chosen so that it is likely that the measurement indicates a ‘falling event’. This method is very easy to implement, but will not likely yield the most reliable results.

A more specific method would be to compare the parameters with known values of these parameters during a fall. Acceleration during a fall has certain characteristics. [6] First, the acceleration will suddenly drop when the person is falling (due to weightlessness). Then within a short time period, on impact, a large spike will show in the acceleration graph. When measuring the magnitude of acceleration, these events can be found, regardless of the orientation of the accelerometer. This method requires two subsequent measurements to pass a certain threshold before a fall is detected, which makes it more reliable while still being a relatively simple method.

A different method is to make use of machine learning, which is not a threshold based method, unlike the two methods described previously. For this method, a database with categorized measurements of falling events and non-falling events that have similar characteristics as falling events is provided. An artificial neural network (ANN) uses this data to teach itself how to recognize a falling event. If a new measurement is then fed into the ANN, the probability of the event belonging to a category is calculated for each category. If the category with the highest probability is a category of a falling event, a fall is detected.

For maximum accuracy, these methods can be combined. [7] One way to do this is to only toggle on the ANN after a threshold of a certain parameter is passed. The method where fall characteristics are measured could be used as an activation for the ANN. However, this may mean that falls that for some reason do not follow these general assumptions are not detected. For this reason, it may be safer to use the simple threshold detection method described first. To make a reliable decision, a collection of fall events may be analyzed in order to determine what the optimal method is. Alternatively, both methods could be tested so that the results can be compared, and a decision can be made based on these results.

A flow chart of this method is shown below. In this chart, SVM denotes the gravity weighted magnitude of the vector and angle denotes the angle between the y axis of the accelerometer and the vertical direction. The calculation of these parameters and thresholds is explained in the section 'Final Concept'.

Cloud Platform

Once the wearable registers a fall, it has to notify the right contacts, either relatives if the fall is minor, or professional care, if the fall is more severe or when there is no help needed it will have to register this as well. The device sends messages to a client device that can then serve the responses visually to the receiving party.

Once the device registers a fall, it then saves a data object that it has to send to the client on the other side. To do this fast and reliably, a Message Queueing Telemetry Transport (MQTT) broker is used. The MQTT protocol is a way of sending data over the internet, it is beneficial compared to regular HTTP protocols, in not having the extra overhead data. The protocol also ensures that every message gets delivered, by sending a confirmation ping back to the sender once a message has arrived.

The protocol uses a subscribe & publish system. You can subscribe to different topics that you assign by publishing to a new one. For instance, publishing ‘no pain’ to the falling/city_name/user1 topic could be subscribed to by subscribing to the ‘falling/#’ topic, with # denoting every folder beneath it, ‘falling/city_name/#’ could be used in a similar way, excluding all the other users from the subscription.

As demonstrated, these topics are used to identify different users of the system and it allows for multiple users to make use of the system. Making this even more accessible is the fact that all the messages get sent through a single broker that has no other load or programs to run except for sending the messages, you can use multiple clients to make distribution of data more efficient.

When the message arrives at the broker, it gets sent through to the client, which then displays it into the web client with the accompanying graphs and fall description.

User Interaction

Once a fall is detected, the device will start interacting with the user. It will ask questions and act based on the answers it receives.
The decision tree used for this project is shown below. The squares indicate the output of the device.

Final Product

Measuring Device

Once development started, we decided to use a smartphone as the measuring device. Smartphones contain all parts that are needed for measuring and sending data. This does not mean that the device cannot be made without using a smartphone. On the contrary, a device that consists only of the necessary parts will be more efficient regarding power consumption and will be smaller, thus more comfortable to wear. These aspects do not matter for demonstration purposes.

The smartphone app is able to connect to the server via a web-socket. This allows live data transferring. The app also measures the data from the smartphones accelerometer. This data is processed to obtain the angle of the device, and the (gravity weighted) magnitude of the acceleration.
The angle (in degrees) can be calculated with some simple trigonometry*:
angle = arctan( sqrt(Ax^2 + Az^2) / Ay ) * 180/pi

The gravity weighted acceleration magnitude (GWAM) gives the magnitude of acceleration times a weight that is proportional to the angle. This means that when the phone is upright (angle 0), the GWAM is zero. If the phone is sideways, the weight should be maximum, because when a person falls, they usually land flat on the floor.
The GWAM is calculated using the formula*:
GWAM = sqrt(Ax^2 + Ay^2 + Az^2) * abs(angle/90)

The threshold for these values can be determined using basic physics. The impact force of a human after free fall is equal to the potential energy divided by the distance over which the velocity decreases to zero. (F = mgh / d). Assuming the center of gravity of a human being is about 1 meter high, and the human tries to catch him- or herself, so that the distance of which the impact takes place is 30cm, the impact force is equal to F = 3.33 m g. The accelerometer measures acceleration, not force, so we divide both sides by m (keeping in mind that F = ma, so dividing by m yields a). This yields: a = 3.33 g. This means that the acceleration on impact will be about 30 m/s^2.
In reality, the threshold will need to be lower, because this value of 30 only occurs at the exact moment of impact, while the accelerometer measures in intervals of approximately 50ms. Tests shows that a threshold of 20 m/s^2 is low enough to detect falls, but high enough to prevent unnecessary fall detections.

* Latex parsing doesn't work, so the equations are given in text format.

Server

The server receiving the messages from the device through the broker, save the incoming messages through a python script, subscribing to the relevant topics. This could be a hospital, care home or an individual running a server. It then pulls the data from the database and translates the json objects that were sent into readable arrays rendering graphs to be inspected and analysed based on medical research.

There is the possibility of having the server (and the broker) being a physical machine that you could set up in a local network within a professional environment and the option to have it be a web server that is accessible for everyone you want to know about what’s happened during a fall.

At the moment there is no authentication on the server, but it could be easily implemented through different frameworks or adapted to existing systems in place. To have it running in a care home, you would need to implement the application into the inhabitants smartphones or distribute a smart wallet to run the application. Then they would need to set up a broker and have the different devices publish to their own topics, with a client machine in the current system add a database table, subscribe to the top level topic and run the script while rendering the data locally.

Results and analysis

The smartphone transmits the last few seconds before a fall. This data is processed by the online platform and 4 graphs are made.
The top left graph contains the separate x, y and z components of the acceleration. This is what the accelerometer reads.
The top right graph contains the magnitude of the acceleration. This is calculated by the app.
The bottom right graph shows the angle of the phone with respect to the vertical direction. This angle is also calculated by the app.
The bottom left graph shows the gravity weighted acceleration magnitude. The phone uses the calculated angle to determine the weight that the acceleration magnitude will be multiplied with.
The graphs of a fall look like this:

When a fall is detected, the smartphone will ask questions to the user. These results are also sent to the platform. The responses of the user are shown above the graphs:

The threshold used for the gravity weighted acceleration magnitude is 20 m/s^2. This value seems to be adequate for detecting falls while not being triggered by normal daily events.

Conclusion

The prototype shows that this device can be easily made and used. Even with the simple threshold method used for the prototype, falls are being detected relatively accurately. The main goal of this project was not to make a high quality device, but to show that using a combination of offline and online data processing can be achieved fairly easily and is beneficial compared to offline devices. The fact that we managed to produce a working prototype as a group of two people in less than 3 weeks proves this assumption. The platform can easily be expanded. To add a new device to the platform, only the topic on which the data is sent needs to be changed. This can be hard coded into the software, or this can be determined via user input. Secondary users can then select which topics they want to receive data from. All of this can be made even simpler by adding a graphical user interface to the server, with which these requests can be handled.

Discussion

An important part of the research is missing. The demands of potential users were going to be determined by distributing a questionnaire. We did this at a care home in Eindhoven, ZuidZorg. The questionnaire was answered by about 100 participants, but the group members responsible for analyzing these decided to quit. We never received the filled out questionnaires from them afterwards.