PRE2018 4 Group1

Problem Statement

In modern society independency and mobility are of crucial value for ones everyday life. Everybody has the requirement to do whatever one wants and is able to do so. Independence and being mobile is important in everything you can depict, like having a good job, going to that job, walking to your next destination and choosing by yourself where to go next. To the majority of society mobility isn't a big issue. However, for a small part of the population this mobility and thereby independency is.

Visually impaired people have a huge disadvantage over people with visual capability. Almost all the concerns one can think of is not or slightly possible to people who can't see. But perhaps the most unpleasant concerns are the once just described; they need more resources to achieve a similar level of independence and mobility as people with regular vision. Through the ages blind people has been assisted with the design of a walking stick, assisting lines provided on public space and through Braille. One of the most user-friendly assisting aids for blind people is the training of guide dogs. Guide dogs are specifically trained to aid in a better mobility for the blind user and is much often seen as a companion as well.

Nowadays users ask for new input for improving the independence of people through a technological perspective. Our society is forever improving and becoming more and more reliable on technology. This project aims to design a robotic service dog, which will exist of a combination of already existing technologies like infrared sensors to avoid collisions or speech recognition . These technologies will be evaluated on their costs, efficiency and other aspects which influence the way in which the user will experience and interact with the robotic service dog. A regular service dog will act, like any other dog, as a companion for the user. The robotic dog which will serve as a substitute for the regular service dog will therefore also need to have the ability to be there as a companion for the user, especcialy if the user is alone.

This project will propose a design for the robotic service dog, which will use existing technologies to be able to serve as both a service dog and as a companion robot for visually impaired people.

Group 1

Group members	Student number	Study	Email
Lotte van Gessel	1237708	Applied physics	l.s.v.gessel@student.tue.nl
Piers da Camino Ancona Lopez Soligo	1015467	Biomedical Engineering	p.h.d.camino.ancona.lopez.soligo@student.tue.nl
Sander Poot	1017804	Biomedical Engineering	s.a.poot@student.tue.nl
Timon Heuwekemeijer	1003212	Software Science	t.m.heukemeijer@student.tue.nl
Jan van Leeuwen	1261401	Applied physics	j.a.v.leeuwen@student.tue.nl

Subject

Researching and redesigning robotic substitutes for service dogs that help blind people navigate

State of the Art research

Robots that can adapt like animals

Relevance

When designing a robotic service dog, a problem that can occur will be that the dog can become damaged while the user is out with the service dog. The following article is about how a robot can cope with problems in a similar way animals do.

Summary

In this article, a trial and error algorithm is proposed so that the robots can adapt to damage is under two minutes in a similar manner as animals. Current recovery typically involves two phases. The robot first needs to diagnose itself, which is followed by selecting the best plan to fix the problem. Problem is that it could be that not every situation is foreseen by the designer, and the robot does not have the right diagnosis or contingency plan to cope with the inflicted damage. Animals deal with injuries in a more trail and error based way. A similar algorithm could be implemented in robots to learn the robot different behaviors to injuries without the limitations to the engineers possible damage scenario’s. The current state-of-the-art algorithms for this are not suitable since they can’t cope with the curse of dimensionality. Other algorithms take up about 15 minutes and need some human demonstrations of some kind. Animals can do it in 2 minutes, so for robots it would be more practical to do so in a similar time. The main difference between animals and robots is that animals know the search space of behaviors and can therefore adapt intelligently. Robots would need to do the same in order to achieve a similar behavior. Robots used in the article store knowledge in a behavior-performance space. This helps them to cope with injuries by quickly discovering the behavior that would help in the injury at hand.

Ethorobotics: A New Approach to Human-Robot Relationship.

Relevance

The robotic service dog should, besides it’s service duty, also serve as a kind of companion robot in order to build a good relationship with the user.

Summary

This article proposes a new approach to the relationship between the human and the robot. It focusses on the uncanny valley hypothesis. The uncanny valley states that humans will be more likely to interact between things that are more human-like, but when they become too similar they tend to avoid them.   It is stated that biological agents need to be able to make a difference in one of the following categories: same vs different species, familiar vs unfamiliar conspecifics and familiar conspecifics vs individuals. The division in these categories determine the way in which both agents will interact with one another. The way in which humans can divide in these categories is learned by some specific pattern of cues. This learning can take place in a certain period in the development of the person. The uncanny valley hypothesis offers two options for social robots. They should either achieve perfect similarity to humans, or humans need to be exposed to social robots from their first year of life on, so they can become more used to the robot. Both options have some problems. It is then argued that the hypothesis can be extended to a symmetric landscape, in which there is also a part after perfect similarity.

The ethological approach proposed is focused on the function of behavior, related to the environment in which the species is evolved. It states that instead of aiming for human similarity, we should aim that the robot is most suited to the function and environment that it will need to operate in. This way, the uncanny valley is avoided on both sides of the landscape. A robot designed in this way, would have its own niche in the environment. This approach has the following benefits:

• Robots have their own evolution, without interfering with that of the human they interact with
• There is no competition between humans and robots
• If the robots function is no longer needed, or it does not live up to the humans expectations, it can simply be seen as irrelevant and go extinct.

The interaction between robots and humans can be linked to that of humans and dogs. In this relationship, the human interact with a morphologically very different species, which also behaves differently, in a sophisticated way. Dogs have a certain social competence, something that a social robot will also need to get so they can be integrated in society. This approach of ethorobotics suggests that social robots should be seen as a new species, and have their own niche in the environment at hand. In this way, the similarity to humans is irrelevant. Only resemblances to humans will be those needed to do its function.

Legged robots

Relevance

The robotic replacement dog will likely look like a dog and thus it will move with legs. This is more difficult to do than for example wheels.

Summary

There are several advantages to legged robots. The first, is that they are more adaptive on rough terrains. s. The travel trace of a legged robot consists of a series of discrete footprints, not a long continuous rut like that for a wheeled or tracked robot. Secondly, a legged vehicle possesses greater mobility and flexibility in that all its legs have many degrees of freedom (dofs).Third, a legged vehicle isolates its body from the underlying terrain and smoothly. propels its body independent of terrain details and foot placement. Fourth, a legged robot is more familiar to humans than a tracked or wheeled robot since legs are the common way to get around in nature. This breeds a familiarity and allows for Zoomorphism which can lead to a stronger bond between human and robot.

Sensing Surrounding 3-D Space for Navigation of the Blind

Relevance

To make the dog usefull, it has to have state of the art 3D space navigation since it doesn't only have to move itself through but also help another person navigate

Summary

A range of adaptive technologies and devices has evolved since the 1960s to assist people who are blind in dealing with a variety of situations. The primary drawbacks included inconsistencies in feedback depending on various conditions (such as weather), possible disorientation caused by overuse of the sound space, and the fact that the information such devices provided was redundant to what the individuals could discern on their own in a more efficient manner using a cane or guide dog. The main drawbacks of existing assistive devices are the cumbersome hardware, the level of technical expertise required to operate the devices, and the lack of portability. These technological advances do not facilitate unobtrusive indoor navigation and learning from the environment. This limits employment and social opportunities for blind and visually impaired individuals. In summary, these technological advances target specific functional deficits but largely neglect social aspects and do not provide an integrated, multifunctional, transparent, and extensible solution that addresses the variety of challenges (such as independence) encountered in lives of blind people everyday.

Existing Technology Aiding Visually Impaired

Relevance

In order to develop a product that aids the visually impaired, we need to know what already exists on the market. This enables us to improve upon this.

Summary

There exists a "co-robot" cane that can guide blind people through an indoor environment, using algorithms that allow it to navigate and estimate the pose of the user. There is also a pair of glasses for the blind that helps them navigate, and this too has proven to be effective in indoors situations.. The sensors used here are of low cost and have easy integration, so this device can be affordable for most users. There are more electronic devices that aim to aid blind people in their mobility, giving feedback by using audio feedback and/or vibration.

A smart Walker for the Frail Visually Impaired

Relevance

Our dog has to help the person to be able to walk on it's own. Therefore this dog has to have some sort of ability to track a path

Summary

In this article a device is created to help Visually impaired and people who have some physically impairment. The device is made in such a way that is as comfortable for the user as possible, therefore it chooses the most smooth path. The device is a walker with build in path finding software. This walker can also be of aid for the visually impaired as I suppose they would feel more at ease. The device has sensors all around it to scan the environment. It does this by sonar reading. For the navigation an variation VFH+ algorithm. This algorithm seems to work fine in local environment. A very smooth path was obtained when using this device. Further development on this device is to have this aid be able to work inside buildings and have it communicate with other devices to accomplish more advanced tasks.

Applying Mobile Robot Technologies to Assist the Visually Impaired

Relevance

On the market there exist already technologies to help the visually impaired, we can learn from the problems from these technologies

Summary

Guide dogs are very capable to help the visually impaired, however they require an extensive training. Furthermore fully trained guiding dogs are very expensive. Also the people have to take care of a living being, feeding it etc.

Therefore technologies have been developed to help the visually impaired. However up until this point in time the following problems have occured:

1) The user has to scan his/her environment actively to detect obstacles 2) The user has to perform additional measurements when an obstacle is detected, which is very time-consuming 3) The feedback the robots give is acoustic feedback which is very deceptive to interference from the environment. A lot of noice.

Another way to give feedback to the user, instead of using acoustic, is using physical feedback. It has been proven that users react better on this kind of feedback.

Guide Dog Robot (MELDOG)

Relevance

This article provide a study from 1977 stating that a more suitable (robotic) system is required to guide blind people in their mobility and thereby increasing their independence. For this, the Guide Dog Robot MELDOG is designed with the implementation on how to navigate, how to avoid obstacles and to warn the specific blind person. This article is very mathematical and came up with a complete plan on how to design such a system.

Summary

For blind people, independence is one of the most strongest desires to have. Blind people are always seen as helpless, it is seen as an handicap and there mobility is for course less strong than non-blind people, which will always put them in a situation as were there will be seen different or even be discriminated. A guide dog aids to increasing this mobility by supporting in three different ways: 1. the blind person's next step, 2. his/her directional orientation, 3. his/her navigation along reasonably long travel path on both familiar and unfamiliar terrain

In this paper, the design concept of the Guide Dog Robot MELDOG is described, as well as the navigation method using an organized map and landmarks. Next up an obstacle detection/avoidance system based on the ultrasonic environment measurement and man-machine communication is described. For this to be achieved, the robot needs to require: (a) A guiding system for itself by using an organized map of the environment and registered landmarks in the environment. (b) How the robot finds obstacles which are not registered on the map and avoids them, (c) How the robot informs its blind master about the route and the obstacles detected.

Animal-Assisted Therapy and Loneliness in Nursing Homes: Use of Robotic versus Living Dogs

Relevance

In order to be able to rely on a robotic guide dog in the first place, a comparison has to be made between a living and a robotic dog. In this study the robotic dog AIBO has been used to treat loneliness in elderly patients in long-term care facilities (LTCF). Does a robotic dog decrease the level of attachment to people of the elderly people and thereby reduce the level of loneliness as well?

Summary

The method for this study was facilitated using the animal-assisted therapy (AAT). This was done by the use of Sony’s AIBO, the robotic pet and a living pet. They used 12 participants for AIBO, 13 for the living dog and 13 for the control group. The control group did not receive AAT, so the expectation was that in this group the level of loneliness would be the highest. The experiment lasted for 8 weeks and the participants where asked about it during a questionnaire.

The results show that elderly living in long-term care facilities who received scheduled AAT with either a living or robotic dog were significantly less lonely than those who did not receive AAT. Unfortunately, no difference was found in effectiveness of a robotic dog in comparison to a living dog and that the attachment to either the living dog or AIBO did not reduce the level of loneliness for participants. However, participants who received the living dog had an higher level of attachment than those who received AIBO, but the difference was not statistically significant. Next to that, AIBO wasn’t used in its full potential. It was for instance not allowed to its voice or walk around. Therefore one cannot conclude that the robotic dog is better or worse in treating loneliness than a living dog.

Service Dogs and People with Physical Disabilities Partnerships: A Systematic Review

Relevance

The new robotic service dog won't only replace a means to see but also a companion. It is important to know the impact and relation between the blind person and their service dogs.

Summary

Occupational therapists have recognized the benefits that service dogs can provide people with disabilities. There are many anecdotal publications extolling the benefits of working with service dogs, but few rigorous studies exist to provide the evidence of the usefulness of this type of assistive technology option. This systematic review evaluates the published research that supports the use of service dogs for people with mobility‐related physical disabilities.

Articles were identified by computerized search of PubMed, CINAHL, PsycINFO, OT Seeker, the Cochrane Database of Systematic Reviews, SportDiscus, Education Research Complete, Public Administration Abstracts, Web of Knowledge and Academic Search Premier databases with no date range specified. The keywords used in the search included disabled persons, assistance dogs or service dogs and mobility impairments. The reference lists of the research papers were checked as was the personal citation database of the lead author. Twelve studies met the inclusion criteria and whereas the findings are promising, they are inconclusive and limited because of the level of evidence, which included one Level I, six Level III, four Level IV and one Level V. All of the studies reviewed had research design quality concerns including small participant sizes, poor descriptions of the interventions, outcome measures with minimal psychometrics and lack of power calculations. Findings indicated three major themes including social/participation, functional and psychological outcomes; all of which are areas in the occupational therapy scope of practice. Occupational therapists may play a critical role in referral, assessment, assisting clients and consulting with training organizations before, during and after the service dog placement process. In order for health care professionals to have confidence in recommending this type of assistive technology, the evidence to support such decisions must be strengthened.

The Tactile Ethics of Soft Robotics: Designing Wisely for Human–Robot Interaction

Relevance

The robotic service dog will have to be able to have a certain amount of interaction with the user, in order to serve a companion like duty.

Summary

Soft robots are more adaptive, have a more flexible way of moving and are equipped with a more sensitive interface for the things it has to interact with. Soft robots are less hazardous in social interactions. Because of the increasing amount/demand for assistive and companion robots in our society, tactile HRI is good to consider in a design. This suggest a balance between tactile engagement and emotional manipulation by the robot. Soft robots can offer a new insight to achieve a better sensory intake by robots. To determine the way in which the robot presents its interaction with other agents, it is necessary to take into account the following three layers; outward appearance, behavior and the experienced behavior by the user. How the robot functions and presents itself is key to understand the role it employs within the relationship between the user and the robot. A framework is presented based on the following categories; bonding, specifying function and modeling in relation to social norms It is possible for humans to form a unidirectional bond with a robot that does not look or act like a human. Bonds to be formed could be strengthened by certain voice abilities or certain shapes the robot has. The robot will not have a certain humanoid form or appearance, but moving in a human or animal like way could contribute to the affection of a human to the robot. The function a robot will have in society should be a key consideration in the way it is touched by the human. Softness of robots could be a way to implicate vulnerability of the robot for the user. A soft outside appearance could be a way to invite more interaction with the robot.

LEGO Mindstorms NXT for elderly and visually impaired people in need: A platform

Relevance

This is a way in which (simple) robots have been used to help visually impaired people in need. LEGO Mindstorms can be a good platform to use, since it is relatively cheap.

Summary

The problem of the ageing society is ever increasing. Most of the robotic aids that have been developed already are of a mechanical nature. There is already a lot of research done to develop robots to help elderly and visually impaired. However, common problems these run into are that the users mostly do not have the technological know how to easily understand how to use the devices. This article aims to use LEGO Mindstorms NXT to be implemented to substitute a service dog, to facilitate health monitoring, to be a assistive robot for the user and to provide a user-friendly interface. In the NXT, several features are added in order to make it able to function as a service robot. For instance, a touch sensor was put on the robot to make sure it notices when it hits something. Also, a reminder GUI was implemented in which the user could enter several things it needed to be reminded of. The robot uses rechargeable batteries in order to cope with the issue of power consumption, and has implemented WiFi instead of Bluetooth to enable faster data transmission. The research states that the NXT could be a good low-cost user-friendly instrument to be used for these kind of purposes.

Automatic Generation of Indoor Navigation Instructions for Blind Users using a User-centric Graph

Relevance

The robotic service dog will also need to be able to help the user in their home environment. This research proposes a technique in order for the robot to be able to help the user navigate through indoor environments

Summary

Visually impaired people have a big challenge when they need to navigate through unknown, indoor environments. It limits the independence of the individual since they will always have to have someone with them to help with the navigation through the building. Indoor navigation consist of both localization and path planning. GPS is not accurate indoors. This paper proposes an algorithm that generates a map of the indoor environment, and accounts for the users wayfinding ability. It assumes that users do have the well known white walking cane at their disposal. In the research there are in total 31 actions proposed which the user can perform in order to get to its next position. These actions are then put into a graph, which guides a user through every possible action. Every action has a certain weight, which is used to give the user a lower cognitive load. These actions are then eventually translated into verbal instructions for the user so it knowns which action to perform. The system is tested on blindfolded people who do have visual capabilities. The next step will be to use to system on blind/visually impaired people.

Robotic Assistance in Indoor Navigation for People Who are Blind

Relevance

In this study a small mobile robot, namely Pioneer 3DX, has been used for robotic assistance in indoor navigation for blind people. With the robotic device the user could hold onto the robot and was thereby designed in a way that it would harness a guide dog. They tried to make the interaction with the robot as comfortable as possible as well.

Summary

The user could hold onto the robot by the implementation of a D-handle which as placed perpendicular to the robot’s travelling direction. During the experiment, the users requested interactive features for the robot themselves. A few features where used to optimize the interaction relative to the users:

• Sound generation for identifying the robots location. This was utilized so that the user would know where in the environment the robot would be.
• Speech for user-robot interaction. Text-to-speech (TTS) software was used and users got the ability to choose their preferred voice.
• Retaining user information for evolving HRI. The robot was equipped with a database for user information. In this way the robot could remember the specific people they helped.
• Periodic user checking. With this implemented the user could easily stop a performance with the robot and perform another task or take a break while on a journey with the robot. Included was a software that the robot would ask the user every five minutes if they need anything.
• Charge awareness. The robot has the ability to check the battery level before undertaking a task.

These are just a few examples on how to enhance the user’s experience while interacting with the robot. There are however more features that need to be implemented in order for the robot to really take over a living guide dog. Moreover, it is important to consider the acceptance and the use of the robot more. The device has to be user-friendly, but this paper describes only the initial steps to that.

Robotic travel aid for the blind: HARUNOBU-6

Relevance

This research has developed a robotic travel aid RoTA to guide the visual impaired and compared it with a living guide dog.

Introduction

This RoTA has been a project since 1990 to guide the visually impaired in Japan and they wanted to design a system to help these people other than the long cane or the guide dog because they are not rich in auditory, sensing and memory of the map. RoTA is equipped with vision system, sonar, differential GPS system, dead reckoning system and a portable GIS(Geographic Information System). RoTA is superior to the guide dog in the navigation function, and is inferior to the guide dog in the mobility. It can show the route from the current location to the destination but cannot walk up and down stairs.

Guidance

The robot uses the GIS for the guide information. This guide information should give the system the information about the environment. The robot gets two kinds of signals to guide the user through the environment; a sign pattern and a landmark. The sign pattern is used to correct the location and heading errors. It has to activate and guide the fixed actions of the robot. The landmark is used to verify the current location. Next to this it uses map learning to make a digital map. This digital map is represented by a list of sections.

Conclusion

Described in this article as well:

• Pedestrian detection by rhythm (where walking people were detected
• Danger estimation at an intersection (where upcoming danger could be detected)
• Experimental results (where the robot was tested alongside a user)

The most important aspect of this article is however that the HARUNOBU robot has the potential to replace the guide dog by using all the technical aspects HARUNOBU has. Nonetheless the system will require more field testing.

Evaluation of the robot mobility aid for the elderly blind

Relevance

This article focusses on people how are both frail and visually impaired, and the particular problems they have. They normally suffer in a greater extent than the people how are only visually impaired and have great difficulty using mobility aids like a guide dog. Thereby this robotic research aids a physical support as well. In Europe over 65% of all the blind people are older than 70 years old. The focus thereby lies mainly at supporting the elderly.

Summary

Mobility is the ability to avoid obstacles and move through a known space safely. Mobility for elderly visually impaired is other than for the visually impaired only. Elderly normally experience difficulty walking, tiredness and start walking slower. They normally need a walking frame, rollator and a reciprocal frame and this will become a problem when they need a guide dog for their visual impairment as well. The design was a prototype whereas the user got personal adaptive mobility aid (PAM-AID). It was not seen as a guide dog, but more as a support for the user to walk around, but still be able to independently commit exercises. With this, they tried to remove the necessary human contact and increase the independence of the user without being fully dependent on the robot as well. The prototype was supported by a user interface containing a command confirmation and with a warning device for an approaching object. The detection of these obstacles was done using sonar sensors. The user could hold onto the prototype using a handle.

The needs of the infirm blind and visually impaired are different from those of the able-bodied blind, and this is why the study on both must be seen differently from each other. This manifests itself in the need to combine both a walking support and a mobility device. The concentration must lay with developing the user interface and control systems required to provide a reliable mobility aid in a dynamic environment.

The users and their Requirements

Users are very important part of the design process. They are the ones who will use the product in the real world. They will find faults with the product. They will hopefully be helped by the product. To allow for a good design, it is very important to know the users.

Primary users and their needs

The primary users are (severely) visually impaired people. There are an estimated 302.000 people who are visually impaired in the Netherlands . This might be only 1.8 % of the total population but these people experience great obstacles in their daily life that can be partially solved by better technology. What many see as easy tasks can be very hard for visually impaired people. Navigation, going from one place to another, can be impossible without proper planning. Even something as simple as a walk for fresh air can become impossible. To do this, they will require an aid which can be for example a white cane or a guide dog. They both have their advantages and disadvantages. The white cane has a limited range and cannot prevent the user from walking into traffic or anything else dynamic. The dog is a living being which can get sick and die. But normal service dogs are not only a tool to visually impaired people to navigate without sight, they are also companions.This is why we have chosen to make a robotic service dog instead of a 'normal' enhanced aid. We want to do this by making a robotic service dogs which will not only make navigation easier for visually impaired people but also will serve as a companion. Thus, while they are great helps to visually impaired people, they can be improved upon. The needs the visually impaired people have for their aids can be broken down in several aspects:

• Navigation: It has to help the visually impaired people move from one place to another.
• Safety: It has to help the visually impaired people be safe around dangerous situation for example train tracks
• Reliability: The robotic service dog will have to be able to work for a longer time before the battery dying or the robotic service dog would break down. Otherwise it wouldn't be much help for its owner. Also, when the robotic service dog does break down, the maintenance should be easy.
• Clear communication: The robotic service dog has to communicate anything it sees to its owner. This has to be done without ambiguous signals. Normal service dogs and their owners do have a system of signals which is adequately conveys information about the system but it can be improved upon by the robotic service dog.
• Companionship: Everyone gets lonely once in awhile and people with a handicap more than others. So, visually impaired people often craft strong bonds with their service dogs. To prevent them from experiencing more loneliness, this relationship is of key importance while designing the new robotic service dog.
• Adaptability: The home environment is always changing. The robotic service dog will have to be able to adapt to these changes. Also, it would be great if the robotic service dog would be able to navigate through unknown territory through GPS.

Secondary users and their needs

The secondary users are family, caretakers and friends. Their needs are mostly related to the safety of the visually impaired people.

• Safety Their concern for the visually impaired people matches the safety concern the visually impaired people have for themselves. What might be different is that if something does happen to the owner of the robotic service dog, it might call the owner's loved ones for help or the emergency services if a more severe accident has happened.

Approach

We want to achieve a list of available technology which could be combined into a prototype of the robotic service dog and to make a design of it. We want to keep the users in the loop. So, we will have to find first hand accounts of visually impaired people about their bond with service dogs and which obstacles they face being visually impaired. It would be better if we would make our own questionnaire where we could ask question that are important for us but since we have limited time and distribution opportunities. We thought that we could gain more from using already existing data. Only after having gathered sufficient data to understand the user and the relationship they would have with the design, we will continue with making concrete requirements for the robotic service dogs. Then, we will find available technology that matches the requirements. From this, the preliminary design will be made. This will be refined by looking at it with the user needs in our mind. If fault is found, we return to the technologies and go through the design process again until it is the best we can do.

Deliverables

As deliverables we will provide this wiki which documents our project. This will contain the design for our product concept, as well as the literature study that backs it up.

Milestones

As our milestones we have the following:

• Must have features are researched, including ethical research where appropriate
• Should have features are researched, including ethical research where appropriate
• Can have features are researched, including ethical research where appropriate
• USE analysis completed
• Cost analysis completed
• All features combined in one product
• Product has a final (visual) design
• Discussion about this product completed

Following these milestones, we believe we have made an effective planning that will allow us to finish this project in a timely fashion.