PRE2020 1 Group1

Group members

Name	Student ID	Department
Davide	1255401	Electrical Engineering
Lieke Nijhuis	0943276	Innovation Sciences
Ikira Wortel	1334336	Electrical Engineering
Wout Opdam	1241084	Mechanical Engineering

Problem statement and objectives

COVID-19 is a disease that as of 5 september 2020 has 26.171.112 confirmed cases and 865.154 confirmed deaths worldwide. This pandemic has caused many countries to go into lockdown, and strict measurements are being taken to lessen the spread of the disease. The virus easily spreads itself in crowds, and this is where the idea of a center for a COVID-19 detection system in choke points and crowded places was born.

The objectives of this center are:

- Detection of COVID-19 symptoms in a crowd

- Testing people that might be infected

- Applying a form of social control in order to make people follow the rules better

- Creating a general feeling of awareness and safety

The project

This project is part of the course 0LAUK0 Robots Everywhere. The project concerns a COVID-19 detection system for choke points and crowded places.

Approach

A literature study will be performed on the feasability of a COVID-19 detection system for choke points and crowded places. This literature study will include an investigation of possible privacy issues that can arise when such a system is deployed in public places.

A design for the hardware of the system will be made and a pseudocode will be written for the software behind the system.

Also, a cost-benefit analysis will be performed to get a view on the financial feasibility of the system.

Deliverables

The deliverables for this project include:

• Literature/feasability study of a COVID-19 detection system for choke points and crowded places, including a privacy-issue study.
• Design of the hardware of the system.
• Pseudocode of the software of the system.
• Cost-benefit analysis.
• A wiki page.
• A presentation video.

Planning and milestones

In the figure below, the project planning is given. The left most column contains the milestones for the project. The column to the right of the first column contains all tasks that need to be completed in order to achieve that milestone.

The planning for the project.

Here, an overview will be given of the tasks that need to be completed per week (corresponding to the figure above), and who is responsible for them:

Week 1

Task	Person	Completed
Brainstorm about subject	Everyone	Yes
Create layout for wiki page	Wout	Yes
Make planning for project	Wout	Yes
Write problem statement and objectives	Lieke	Yes
Describe users	Ikira	Yes
Describe user needs	Wout	Yes
Read and summarize articles for State of the Art	Everyone	Yes
Write State of the Art	Davide

Week 2

Task	Person	Completed
Read/investigate similar systems and describe on wiki
Describe psychological effect of visible detection system	Lieke
Formulate RPC's	Everyone
Make sketches of camera with microphone on pole	Wout
Make sketches of IR measurement station	Wout
Make sketches of notification screen for 1.5m violation	Wout
Discuss required software functionalities	Everyone
Write pseudocode for cough detection via camera and microphone	Ikira

Week 3

Task	Person	Completed
Read/investigate similar systems and describe on wiki
Describe psychological effect of visible detection system	Lieke
Describe possible use cases on wiki (questionaire?)
Perform cost-benefit analysis and describe on wiki	Lieke
Research possible privacy issues and describe on wiki	Ikira
Discuss and finalize sketches	Everyone, Wout
Write pseudocode for 1.5m violation detection via camera	Ikira

Week 4

Task	Person	Completed
Describe possible use cases on wiki (questionaire?)
Perform cost-benefit analysis and describe on wiki	Lieke
Research possible privacy issues and describe on wiki	Lieke
Make CAD design of camera with microphone pole	Wout
Make CAD design of 1.5m notification screen	Wout
Make CAD design of IR measurement station	Wout
Write pseudocode for notifying medical staff of coughing person	Ikira

Week 5

Task	Person	Completed
Describe possible use cases on wiki (questionaire?)
Perform cost-benefit analysis and describe on wiki	Lieke
Research possible privacy issues and describe on wiki	Lieke
Conclude on feasability and describe on wiki	Ikira
Make CAD design of camera with microphone pole	Wout
Make CAD design of 1.5m notification screen	Wout
Make CAD design of IR measurement station	Wout
Write pseudocode for IR temperature measurement station	Ikira

Week 6

Task	Person	Completed
Conclude on feasability and describe on wiki	Ikira
Describe CAD designs on wiki	Wout
Put pseudocodes on wiki with description and user interface screenshot examples	Ikira
Discuss what should be in video	Everyone
Make a script for video	Lieke
Make slides and visuals for video	Lieke
Write conclusion
Write discussion
Write recommendation
Check wiki completeness (titel, introduction etc.)	Everyone

Week 7

Task	Person	Completed
Record video	Lieke
Describe peer review	Ikira
Finalize layout of wiki	Everyone
Final check: spelling, citation style etc.	Everyone

Week 8

Task	Person	Completed
Hand everything in	Everyone

Users

In this section the users and their needs are identified.

Who are the users

User

The users of the robot are the people walking through choke points and in crowded places and the medical staff that is informed of the people that are suspected to have COVID-19.

The people in crowded places will be checked to see if they are coughing, if not then there is no problem, however if they are coughing they could then be picked out by the medical staff and get checked. This will prevent people who are unaware of that they are infected from spreading it further. Especially since they are in a place where it is easy to infect other people.

Furthermore the robot can also check if a distance of 1.5 meters is kept. Since sometimes people do not notice how close they are actually standing to other people they could keep a bigger distance if they were told and so the spread can also be slowed down.

Society

The main stakeholders for the society would be the government, the uninfected and infected people.

The government is a stakeholder since they would need to spend money to place the device and are responsible for the regulations around COVID-19.

The infected people are a stakeholder since they can be informed about having COVID-19. Which could change their behavior and limit the spread of COVID-19

The uninfected people are a stakeholder since when there is a detection device for COVID-19 less people will be walking around unaware of being infected which results in infected people taking more precaution not to spread it further. Since it also detects the distance of 1.5 meter people will better keep their distance and slow the spread.

Enterprise

Since the robot should be developed the stakeholder would be the companies developing the product.

What are the users needs

Here the needs of the above identified users regarding the COVID-19 detection system will be given.

User

People in crowded places

• Privacy: The system must comply with all privacy regulations.

• Proper detection: If people are monitored, essentially diminishing their privacy, then they should also benefit from the system. Therefore, it is important that the system properly detects people with COVID-19 symptoms.

• Hindrance: The system should not cause an excessive amount of hindrance for the public on locations where the system is installed.

Medical staff

• Ease of use: The system must be easy to use for the medical personnel. No special extra training should be required, a user manual should provide enough information to operate the system.

• Safe to use: While operating the system and working with people that possible carry the virus, the medical personnel should not be exposed to too much risk of catching the virus themselves.

Society

Government

• Comply with regulations: Like any product sold, the systems must comply with all regulations. This includes regulations that protect the citizens like health and privacy regulations.

• Ease of installation: The installation should be easy and should not require any drastic changes to the instalment location.

• No hindrance: The system should not hinder visitors and personnel from conducting their activities.

• Increasing attractiveness: The system should make it more attractive and safe to visit the location compared to the situation in which the system was not in place.

Infected people

• Proper detection: The system should minimize the number of false positives and false negatives. People with good intentions that are unaware they carry the virus should be notified.

• Privacy: The system must be discrete since it can be embarrassing for people to be picked out for a COVID-19 test and it can cause unnecessary panic at the location. The privacy of the people being tested should be guaranteed such that for example bystanders cannot obtain the identity of the person being tested.

Uninfected people

The requirements for uninfected people are the same as for people in crowded places.

Enterprise

Company developing the product

• Attractive product: The product has to be attractively priced in order to sell more units and make a profit. Therefore, it should not be too expensive to manufacture. It also needs to have decent looks to make it marketable.

• Comply with regulations: The manufacturer does not want to get into trouble for selling this product. It must comply with all regulations.

As can be seen, for multiple stakeholders it is important that the product complies with regulations (health, privacy etc.). This is one of the most important requirements. The above-mentioned user requirements will be converted to more concrete requirements for the design of the system in the RPC’s.

State of the art

Screening System for detecting COVID-19 symptoms and safety rules compliance

To study the feasibility of an automatic system which screens subjects for COVID-19 virus using cough sound some research needed to be done. Several research topics have been investigated such as: COVID-19 detection trough cough sound, conventional techniques to diagnose COVID-19, thermal imagery for mass fever screening and infrared technology for temperature sensing in humans.

An alternative method to screen subjects for COVID-19 would be through cough samples. Currently, research is being done by a couple of groups in the UK and USA. Resulting high accuracy of the algorithm has encouraged a mass collection of cough samples, even through the use of mobile APPs.

General guidelines and techniques to diagnose the COVID-19 virus. Some of the discussed techniques for diagnosis are molecular tests, CT scans and emerging protein tests.

Several articles discussing the efficacy of infrared skin thermometers and infrared imaging in COVID-19 screening are presented. The conclusion is that handheld infrared thermometers have a low sensitivity and should not be used as a screening system. While discussions arise when assessing the usefulness of mass fever screening using thermal imaging. Many factors need to be taken into account such as: camera resolution, ambient temperature, distance from the camera and even marketing.

Articles

AI4COVID-19: AI enabled preliminary diagnosis for COVID-19 from cough samples via an app. (A. Imran, et al. 2020). Summarized by Ikira.

This paper discusses an alternate way of testing for covid-19, since the current testing:

- Requires people to move through public places which could spread the virus

- Puts medical staff at risk for the personal

- Costs a lot

- Not everyone goes to a medical facility to get tested

The solution they came up with is to make an smartphone app that does a preliminary test based on your cough to see if you have covid-19 or not.

The reason for choosing this method is because:

- Prior studies have shown different respiratory syndromes have distinct features

- Coughing is the way the virus is mostly spread.

- There are less medical conditions that cause cough than fever

The steps taken for the making of the smartphone app are to first test if covid-19 has specific coughing characteristics (Results show there are). Second to train an engine that can distinguish a cough from background noise. third to train an engine that can distinguish between the different types of coughs and fourth to check the feasibility of the app. The accuracy of the proposed app is encouraging enough to support large scale testing.

Semantics and types of cough. (K. F. Chung, et al., 2009)

Semantics of cough was very confusing and unorganized since also classification of cough type was dependent from methods used and form clinic-to-clinic.

Definition of cough

• First definition: Three-phase expulsive motor act:

1. Inspiratory effort (inspiratory phase)
2. Forced expiratory effort (compressive phase)
3. Rapid airflow (expulsive phase)

• Forced expiratory manoeuvre against a closed glottis producing a sound

Types of cough

Coughs identified by their cause (e.g. asthma, reflux). Coughs are defined by their characteristics (e.g. wet, moist, chesty). Coughs can be acute (i.e. <3 weeks), subacute (i.e. 3-8 weeks) or chronic (i.e. >8 weeks). Coughs can occur in isolated episodes or in epochs. See Table 1.

Clinical analysis of coughs

Only quantitative data available is cough count. Coughs can be counted using automated audiovisual recordings. Automated cough counting does not distinguish between isolated and epoch coughs. More sophisticated analysis which measure pressure, airflows and abdominal muscle electromyograms can give a quantitative intensity of the cough. But this analysis is better suited for a research clinic instead of an investigative clinic.

Estimated effectiveness of symptom and risk screening to prevent the spread of COVID-19 By (K. M. Gostic, et al. 2020). Summarized by Lieke. This article uses a mathematical model about the spread of emerging pathogens from 2015 combined with the current knowledge about the epidemiological parameters of the COVID-19 spread to estimate the extent in which traveler screening will be useful. Because COVID-19 has a long incubation period many cases are fundamentally undetectable, which will lead to screening missing more than half of the infected people. This number might increase a bit when the epidemic is stable instead of growing, because the times since exposure will be more evenly distributed.

Real-time tracking of self-reported symptoms to predict potential COVID-19 (C. Menni, et al. 2020). Summarized by Lieke.

Symptoms and indicators of COVID-19 include loss of taste and smell, anosmia, fatigue, persistent cough and loss of appetite.

Asymptomatic Transmission, the Achilles’ Heel of Current Strategies to Control Covid-19 (M. Gandhi, et al. 2020). Summarized by Lieke. Infection control relies heavily on early detection, but because COVID-19 has a long incubation time this does not work. This is why it is important to continuously test especially nurses on COVID-19, because it can be detected before the symptoms start to manifest.

Crowds by Example (A. Lerner, Y. Chrysanthou, D. Lischinski, 2007). Summarized by Lieke.

Crowds can be modeled in several different ways, and in this study the people are tracked using video input, and modeled from that. Afterwards, the pedestrians were asked what influenced their path and to what extent. Using this information a model can be made. The influence factors are, to name a few, the topography of the terrain, the presence of obstacles, proximity to other people, the pedestrian’s personality and state of mind. Setting up some equations based on the experiment a model can be made that is able to simulate a crowd.

Diagnosing COVID-19: The disease and tools for detection (B. Udugama, et al. 2020). Summarized by Lieke. To detect COVID-19 in the hospital there are several tests that can be used. Molecular tests are useful because they can target and identify specific pathogens, although nucleic acid testing is the primary way of testing for COVID-19. CT scans can be used but have a lower specificity because the imaging features overlap with other viral pneumonia. Emerging diagnostic tests for COVID-19 are protein tests, more developed nucleic acid tests and detection at the point-of-care, such as lateral flow antigen detection and microfluidic devices.

Detecting the Coronavirus (COVID-19) (P. Pokhrel, et al. 2020). Summarized by Ikira. In December 2019 a new virus was discovered, the SARS-CoV-2. This virus causes the disease COVID-19 which has a medium reproduction rate(2.25%) and a medium mortality rate (5.7%). However contrary to the looks of only being a mild virus, when it remained untreated the sick people began to overwhelm the healthcare system which spread panic. SARS has a lot of similarities with COVID-19 and was eradicated in 2003 by syndromic surveillance, isolation of patients and not physical contact. If this will also work for COVID-19 will remain to be seen.

The characteristics of COVID-19 appear 3-7 days after infection and include fever, coughing and body weakness. People of old age or with underlaying illnesses have a higher mortality rate. People of ages are susceptible to SARS-CoV-2 which transmits via contact or bodily fluids.

One of the most important ways to limit the spread is by early detection so precautions can be taken. The current ways of detection COVID-19 are via a CT-scan, Nucleic Acid based methods and Immunoassays.

CT-scans contain specific characteristics of COVID-19 which helps to identify if the person has COVID-19, however these characteristics may overlap with other infections and give false positives. The test is more accurate if the test is executed during specific stages.

Nucleic Acid based method is a way of detection of RNA viruses in a clinical setting. Based on this method the CDC developed a real time RT-PCR diagnostic unit. Although this method is less sensitive than the CT imaging at certain stages, it specificity is much better making it a better detection method.

Immunoassay detects viral protein antigens or serum antibodies of people exposed to SARS-CoV-2, this method can detect the antibodies 3 days after infection it however also ha a lot of drawbacks.

The now emerging detection techniques are isothermal amplification for nucleic acid targets and lateral flow based detection of nucleic acids an proteins.

Comparison of Non-Contact Infrared Skin Thermometers (T. Fletcher, et al. 2018). Summarized by Ikira.

Non-contact infrared skin thermometers are being used more as a means to asses body core temperature and signs of diabetic foot ulcerations In the paper 3 different thermometers were tested. All the manufacturers claimed less than 0.3 °C in the 22-40 °C range.

For infrared measuring devices there are changes in the measured temperature depending on the size of the target. This is also called the size-of-the source effect. Something else that influences the measurements is the difference in distance or the target from the measuring device. This is important since for these measuring devices there is no way to have the exact same distance each time and is thus included in testing these devices.

The results of this paper show that the tested measurement equipment did not come close to the before specified 0.3 °C and could reach up to 6 °C. The paper furthermore showed that the size-of-source effect and the distance matter a lot in the accuracy of the measurement device.

Comparison of Infrared Thermal Detection Systems for mass fever screening in a tropical healthcare setting (M. R. Tay, et al. 2015). Summarized by Ikira.

Fever screening detection systems such as infrared thermal detection systems (ITDS) have been used before for rapid identification of potential cases of infected people. ITDS use the difference in the ambient temperature and the temperature of the subject.

Three different ITDS were used in Singapore to check if people had a fever. For these people an oral check was taken as a reference. In the paper sensitivity is described as people that have a fever are also detected as having a fever and specificity is described as people without a fever are also not seen as having a fever. Of the three tested ITDS they all had high specificity , 2 showed low to medium sensitivity while one showed high sensitivity. The ones that showed the lower sensitivity were older models while the newest model showed the higher sensitivity.

Crowd temp screening is a joke, says one expert (K. Field, 2020)

Accuracy of the thermal camera depends on several factors (including human variability) and in ideal conditions the maximum accuracy is of ±2 deg C. (which to detect feverish patients is not enough).

The human body is a living and closed loop system it will fluctuate its temperature on various conditions (i.e. ambient temperature, energy expended, and alcohol consumption).

According to FDA guidelines the ideal part of the face to detect accurate temperature would be the inner canthus (tear duct). To measure the temperature of such a small area needs at least 3 x 3 pixels (5 x 5 for better results). Therefore to film such a small detail in a crowd environment a thermal camera should have a resolution of at least a million of pixels (around 12 mega pixels) which do not yet exist.

"Are you just deploying this gear for ‘feel good’ needs, or are you truly looking to measure and eliminate feverish people?"

"I could deploy a doctor that says he can tell your temperature is not normal by looking at your nose--and that might be just as accurate"

The expert's opinion: If you can’t do it right--don’t bother.

Remote monitoring of breathing dynamics using infrared thermography (C. B. Pereira, et al., 2015)

Technique using thermal imaging in combination with a robust algorithm to detect breathing and to measure breathing rate (BR). The algorithm is a able to compensate for the movement of the subject.

The system shows accurate estimations of the breathing rate even under non ideal conditions (i.e. movement of the subject). For accurate measurements, the ambient temperature must be controlled. For further improvement of breath detection movement of the shoulders should be included in the algorithm.

Privacy Issues with Smart Technologies

Some research on privacy issues was done as a possible alternative idea. Nevertheless, privacy issues do tie quite strongly to the current subject. Since people are being recorded both on video and audio the question arises, what is done with the data? A more researche ethical could be conducted. Some of the article discuss ethical use of surveillance technology, cyber security smart camera networks and in smart homes.

A Systematic Approach Towards User-Centric Privacy and Security for Smart Camera Networks (T. Winkler, et al. 2010). Summarized by Ikira.

Fully digitized systems analyze video footage and sets of an alarm if there is suspicious activity. There a however still issues of privacy for this. Since the footage is stored, people can retrieve it and study the behavior of people. The current solutions for this are to blur out faces or to extract encrypt sensitive image regions.

Desirable properties of a secure an privacy-preserving camera system would be:

- Detection of sensitive regions

- Blanking

- Obfuscation and scrambling

- Abstraction

- Encryption

- Multiple privacy levels

For user acceptance it is important that the user gives consent to being filmed and can control it when they are being filmed by for instance placing signboard that the area is being filmed. Furthermore it is also important that the users can give feedback.

The paper sketches concept of a level approach for trustworthy smart camera’s. As a basis for the camera there will be hardware security around a microchip called the trusted platform module. To support the different privacy levels, motion detection is used. The lowest level of security is where only a background is visible while the person and their immediate surroundings are cut out by a black box. The next level would is done with edge detection, here only the person himself is visible and nothing of the surroundings. This is followed by the last level where the person and their immediate surroundings is visible. In other words only the black box that was cut out for the lowest level is visible.

A Smart Home is No Castle: Privacy Vulnerabilities of Encrypted IoT Traffic (N. Apthorpe, et al. 2017). Summarized by Ikira.

Smart Home devices collect data not only when you are on the internet, but also about your daily life. There is a privacy concern if a passive network observer deduce behavior in the change in internet traffic. Some people say that due to encryption no sensitive information can be seen, however it can be argued that meta data and traffic patterns can reveal sensitive information. In this paper a strategy to recognize user behavior with traffic load of internet of things devices is given. Four Smart home devices are used for this. From all of these devices a network observer could infer behavioral patterns from observing the traffic load. Thus it would be necessary for devices to mask the true rate of devices. For the research in this paper a few assumptions were made. First of it is assumed there is only a passive network threat. Second packet contents are not used and finally it is assumed that the observer can obtain and study those devices. The way to analyze these traffic rates is to first separate traffic into sperate packet streams, then to find which stream belongs to which device. With this the traffic rates can be examined and the behavior of the consumer can be deduced.

All Eyes on You! Impact of Location, Camera Type, and Privacy-Security-Trade-off on the Acceptance of Surveillance Technologies. Summarize by Ikira.

Crime surveillance technologies(CST) have increased in the past years. Critics fear that the recording and storing if the data will invade the privacy of the people. The acceptance of the people for these CST are important. This paper discusses the acceptance of the people towards different types and locations of the CST, at which turning points acceptation becomes rejection, the trade of between safety and privacy. To research this questionnaires were taken.

The results show that people find the locations of the CST the most important followed by the crime reduction, handling of the recorded data and last the camera types. The paper further shows based on the before named subjects the how much people accept it the CST. The most rejected was surveillance in your own home. To research the trade of between security and privacy 3 scenarios were made, 1 with high security low privacy, 1 with average security and average privacy and 1 with low security and high privacy. The high security came out the best of these scenarios.

Autonomous robots against COVID-19

Autonomous Robots Are Helping Kill Coronavirus in Hospitals (E. Ackerman, 2020). Summarized by Lieke.

UVD Robots is making robots that are mobile arrays of powerful UVC lights that kill microorganisms. UV lights on carts are used to disinfect areas for some time, but they are dangerous for human skin and when they are operated by humans it is prone to human error and it might skip a few places. This newly developed product is able to autonomously disinfect a room between 10 and 15 minutes, with the robot spending 1 or 2 minutes in five or six different positions to maximize the surface it disinfects.

General robotic technologies in the medical field

Can robots handle your healthcare? (N. Davies, 2016). Summarized by Wout.

The demand for healthcare robots will rise due to an ageing population and a shortage of carers. Japan is a leading country in the field of healthcare robots. They develop their robots in close coordination with elderly and patients.

Safety is an important aspect for healthcare robots. They must be absolutely fool proof. In order to safely operate around humans, robots need to feature things like quick reflexes, communication capabilities and interaction capabilities via video and audio. This translates into a wide range of sensors, cameras and actuators the robot needs to have. Next to a robot’s features, their looks are also important. Robots that try to impersonate actual humans, but still show imperfections, can make users uneasy.

An area in which robots can do a better job than humans is being consistent in the quality of the care that is provided. After a long difficult shift, humans can be prone to errors while robots are not affected by exhaustion or other emotions.

For simple routine healthcare tasks, robots can be a good solution.

How medical robots will change healthcare (P. B. Nichol, 2016). Summarized by Wout.

A possible major field in future healthcare is medical nanotechnology, and more specific nanorobots. These robots will work inside the patient at cellular level. The most important stakeholders in robotic healthcare are not the people that develop the technology, but the people that will use it.

There are three main ways in which robotics can be deployed in healthcare: Direct patient care (e.g. robots used in surgeries), Indirect patient care (e.g. robots delivering medicine), Home healthcare (e.g. robots that keep the elderly company).

Acceptance of Healthcare Robots for the Older Population: Review and Future Directions (E. Broadbent, et al. 2009). Summarized by Wout.

It is projected that the proportion of people older than the age of 60 will double between 2000 and 2050. This increases the need for healthcare. Alongside this there is also a shortage of people working in the healthcare industry. In western countries, most of the older people want to remain living independently in their own homes for as long as possible. To facilitate this, smart solutions need to be found, for example healthcare robots.

Of great importance is the acceptance of healthcare robots by the people in need of care. There are three requirements for acceptance of a robot to occur: motivation for using the robot, sufficient user-friendliness and a feeling of comfort with the robot.

Healthcare robots need to be able to adapt to individual differences since not all patients are equal.

Basic healthcare robots have been developed which can assist with simple tasks like: assist in walking, reminding of doctors appointments, carry objects, lift patients, providing companionship. Mitsubishi sold the first commercially available mobile robot in 2005. This robot is designed to help in the home.

The older the age, the less willing people are to use robots. Older people are more fearful of new technology. However, they are more willing to accept assistive devices if this helps them maintain their independence.

Studies have shown that people prefer robots that do not have a human-like appearance. For companionship robots an animal-like appearance is preferred. Other examples of desired characteristics of a robot for the elderly are slow moving, safe, reliable, small and easy to use. Older people also appreciate having some form of control over the robot because this reinforces their sense of independence.

Medical Robots: Current Systems and Research Directions (R. A. Beasley, 2012). Summarized by Wout.

An example of a great success in medical robotics is the surgical robot da Vinci. The biggest impact of medical robots has been improving surgeries that require great precision.

Medical robots can be employed during brain surgeries. By using medical images (e.g. CT-scan) a robotic arm can orient the tools in the correct direction, or the surgeon can specify a target which the robot guides the instrument towards with submillimetre accuracy. Medical robots are also used to assist in spine surgery.

Another application field of medical robots is orthopaedics (e.g. hip or knee replacement). An example of such a robot is Robodoc, which is used for milling automatically according to a surgical plan. Other devices, like the RIO, are designed such that the robot and the surgeon hold the tool simultaneously. The RIO ensures that the surgeon makes the correct movements.

There also exist devices which are controlled by the surgeon (technically not robots). The surgeon uses joysticks to manipulate the tools with great precision. An example of such machines are the Zeus and the da Vinci. These systems filter out hand tremors and enable the surgeon and patient to not be at the same location.

InnoMotion is a robot arm which can accurately guide a needle using CT or MRI imaging.

Robots exist that can help a surgeon guide a tube through a patient’s blood vessel. The surgeon can steer the tube and force feedback sensors tell the surgeon how much pressure is being applied.

Robotic arms are also used in radiosurgery, in which a specific area inside the patient is targeted with beams of ionizing radiation from multiple angles.

Robots are also used to perform CPR. A band is placed around the patient’s chest which is pulled tight by an actuator to compress the chest. The tightness of the band is adjusted based on the patient’s chest size.

Microprocessor-controlled prosthetics are prosthetics in which the patient is assisted in the movement or are controlled by the patient to perform movements which have been lost (e.g. exoskeletons).

Another area of medical robots consists of assistive and rehabilitation systems. These include things like a controllable arm holding a spoon or a grasper which can be attached to a wheelchair.

In the future, medical robots will become smaller and cheaper. More functionality will be added. Also, robots will be used more for medical training. Medical robots need to provide solutions to real problems, otherwise there is a change that they are displaced by other medical advancements.

Medical robots go soft (E. Strickland, 2017). Summarized by Wout.

New kinds of medical robots are being developed of which the parts that touch the patient are soft. This is to prevent damage to organs and blood vessels and makes sure the human body does not surround the object with scar tissue which can diminish its effectiveness.

An example of a soft medical robot is a sleeve that surrounds the hart. This sleeve contains rings which can be pneumatically inflated to detect and force a heartbeat in a certain tempo.

Another example is a device which can be implemented in the body to give of small dosages of a drug. This device is made of a soft gel with a mechanism inside which can be activated from outside the body.

A final example is a soft gripper-hand made of hydrogel. The hand can be actuated by forcing water into the hand. This hand can be placed on a robotic arm and used for grabbing organs inside the human body without damaging them.

Robots in Health and Social Care: A Complementary Technology to Home Care and Telehealthcare? (T. Dahl, et al. 2013). Summarized by Lieke.

This article discusses several specific robotics applications.The robots that were considered were classified in 9 categories: Surgery robots, Robots supporting Human-Robot interaction tasks, Robots providing logistics in care home environments, Telepresence and companion robots, Humanoid robots for entertaining and educating children with special needs, Robots as motivational coaches, Home assistance robots for an ageing society, Human-Robot relationships in medical care and society, and Human- Robot relationships in medical and care situations.

Between these categories, there are a few ingredients for success that can be found. The general ingredients for success include: Adequate level of personalization, Appropriate safety levels, Proper object manipulation and navigation in unstructured environments, Patient/user safety, Reliability and robustness and Sustainability. In addition, robots that serve motivational or social purposes also need to be personable, intelligent and highly interactive.

However, there are also some concerns with using robots extensively. For example, humanoid robots need to have a set-up etiquette, like following the eyes of a talking person. What also needs to be kept in mind, especially using social robots with elderly people, is that the extensive use of robots could lead to a potential reduction of human contact, social isolation, an increasing feeling of objectification and a loss of privacy. Furthermore, cost effectiveness plays a great deal in the practical use of robots.

Multimedia Companion for Patients in Minimally Conscious State (MCS)

Novel imaging techniques have enabled communication with minimally conscious patients. The idea is to create a multimedia system for patients in MCS. This system would project films, music and videos of family members. This system can communicate with the patient only with binary (yes or no) questions. Communication is used to let the patient navigate the multimedia system (i.e. choosing what to watch or do). The yes or no answers are given by the patient's ability to modulate brain activity. The patient's response is decoded by sensing which areas of the brain intensify in activity.

System:"Do you want to watch the news? If YES imagine playing tennis. If NO imagine walking in the rooms of your house."

Patient: The patient imagines to play tennis.

The motor area of the patient's brain activates.

Activity is recorded by the system.

The system plays the latest bulletin.

Articles

- First technique using functional Magnetic Resonance Imaging (fMRI) and PET.

The Aware Mind in the Motionless Body (M. M. Monti, A. M. Owen, 2010) & Detecting Awareness in the Vegetative State (A. Owen et al., 2006)

Case Study: Martha

Martha, a young woman of not even 30 years old is victim of a car accident. She survives the accident and slips into coma; her eyes are closed, her body is motionless and unable to breath autonomously. After a week her eyes open, her body is able to breathe again autonomously and even makes small movements. For example, she yawns, she stretches her arms and slightly moves her body in her bed. It seems she is even able to sleep and wake up.

Vegetative State, Comatose state, and sleep

When a patient in a comatose state is able to sleep and wake up again, she/he is said to be in the Vegetative State (VS). A patient in a vegetative state might have his/her eyes open, can breathe autonomously, can move his/her body, can produce urine, and can vocalize sounds. The big question, are patients in vegetative state conscious? Consciousness is made of wakefulness and awareness. A healthy person is both awake and aware, when not asleep. A comatose patient is neither awake nor aware. While a patient in vegetative state is awake, but is not aware. Now around 40% of patients diagnosed in the vegetative state seem to be aware. Therefore, these misdiagnosed patients are said to be in a Minimally Conscious State (MCS). Often while diagnosing patients the “Lack of evidence of consciousness is, in this situation, equated to evidence of lack of consciousness” (Monti, Owen, 2008)(i.e. False Negative). To distinguish patients in MCS from VS one of three factors must be met:

• Awareness of the environment

• Sustained, reproducible, purposeful or voluntary response to visual or tactile stimuli

• Language comprehension and/or expression

Novel neuroimaging techniques are used to detect markers of consciousness, such as Positron Emission Tomography (PET) and functional Magnetic Resonance Imaging (fMRI).

Experiment 1: brain response to speech of patients in VS

Two patients in VS showed brain activities similar to healthy volunteers in response to speech and picture of faces. The two patients in VS listened to simple and complex (semantically ambiguous) sentences. The complex sentences activated, in the VS patients, the same brain area as in healthy volunteers. Furthermore, similar experiments were conducted with healthy volunteers heavily sedated. In the heavily sedated volunteers, no activation of the same area was recorded. This shows a higher level of awareness (some language comprehension) in the VS patients than the sedated volunteers. Nevertheless, simple brain responses cannot be taken as a factors of awareness nor consciousness. Much of sensory processing is automatic. Such as lines depicting a face or speech of a familiar language. Therefore, the type of stimuli and the type of brain activity recorded must be attentively chosen.

Experiment 2: voluntary modulation of brain activity in VS patient

A young woman victim of a car accident slips into coma, then wakes up again, but is diagnosed in the Vegetative State. The patient was given spoken instruction. The first instruction was to imagine playing tennis. The second, to imagine walking in her own apartment. When the patient was instructed to imagine playing tennis intense activity in the motor area of the brain was recorded. While when she was asked to imagine walking in her apartment activity in the parahippocampal gyrus (memory retrieval area), posterior parietal cortex (spatial reasoning area) and some motor areas. Healthy control volunteers, given the same instructions, reproduce the same brain activity of the VS patient. Unless the volunteers did not understand the instructions or decided to not comply with them. Therefore, in this experiment the idea of an automatic brain response is discredited.

- Second technique using Transcranial Magnetic Stimulation in combination with high density Electroencephalography (TMS/hd-EEG). This technique is much more practical and portable than fMRI or PET.

A perturbational approach for evaluating the brain’s capacity for consciousness (M. Massimini et al. 2009)

It does not confirm if a person is conscious or not. The technique weighs the capacity of the brain for consciousness. The technique does not require any input from the user. In this article the technique has only been used to differentiate brain response, to TMS excitation, in three conscious states: wakefulness, long-wave sleep, and REM sleep. More research must be done with sedated patients , and patients with brain injuries (i.e. Minimally Conscious State or Vegetative State).

1. Start from a theory describing how consciousness is created. Integrated Information Theory of Consciousness (IITC) suggests that consciousness is created by a physical system.

2. Measure the brain’s capacity to generate consciousness. Measurement is done by poking (exciting) the concerned physical systems, with TMS, and record the response, with hd-EEG.

Results: Brain responses to TMS during wakefulness are similar to those during REM sleep, showing effective connectivity between brain areas (i.e. the stimuli travel between various regions). While long wave sleep shows loss of cortical integration (i.e. the stimuli remains local).

Workload

An overview of who has done what per week of the project.

Week 1

Name	Student ID	Hours this week	Tasks
Davide Fabbro	1255401	9	Intro videos + first brainstorm (1.5 hours) + Studied and wrote summaries for papers: , , , , , (6 hours) with small research on , (0.2 hours) + second meeting (0.5) + Reorganizing state of the art section (1 hour).
Lieke Nijhuis	0943276	9	Intro videos + first brainstorm (1.5 hours) + Studied and wrote summaries for papers: , , , , , , (6.5 hours) second meeting+ sending e-mail (45 minutes) + Describing problem statement and objectives (15 minutes)
Ikira Wortel	1334336	10	Intro videos + first brainstorm (1.5 hours) + Studied and wrote summaries for papers , , , , , and (7 hours) + extra meeting discussing chosen subject (0.5 hour) + Who are the users (1 hour)
Wout Opdam	1241084	12.5	Intro videos + first brainstorm (1.5 hours) + Start with wiki layout (1 hour) + Studied and wrote summaries for papers: , , , and (5 hours) + wrote about approach and deliverables (0.5 hour) + Made planning for project (2.5 hours) + extra meeting discussing chosen subject (0.5 hour) + Describing user requirements (1.5 hours)

References