PRE2018 3 Group15

Group members

Name	Student ID
Sanne van den Aker	1258788
Leon Cavé	1240614
Robin van Tol	1246240
Heleen Fischer	1223688
Laura Barendsz	1245706

Introduction

In 2017 2.5% of Dutch parents thought their child between 6 and 12 has an autism spectrum disorder (ASD). This means around 40000 children in the Netherlands have difficulty interacting on a social level and maintaining structure. Since we know there is no conventional 'cure' for ASD, a lot of research has been done on finding the right therapy to help these children through life. As we now live in a technology based era, therapy has also shifted in this direction. In the last ten years, a lot of research has been done on the help of social robots during therapy. Especially NAO is a popular robot on this front. Even though there are promising results that show social robots can be very effective, there is still a long way to go before they can be fully implemented in therapy and maybe even daily life.

Problem Statement

Children with ASD often struggle with social interactions. It has been shown however that this becomes easier when guided or mediated by a social robot. A possible explanation for this is that they are less intense and have a more procedural way of interacting. Because of this, interactions are always very similar and sameness is appreciated by these individuals. However, there are still a lot of problems to be dealt with in this field, such as voice recognition and also gaps that have to be filled in, caused by lacking knowledge. Thus we want to see whether it is possible to help these children in interactions with the people around them by the help of a social robot.

Objectives

For this project a test group is needed, in our case, we will need children with autism in the age group from 6 to 12. To reach this group we will go to a middle school for children with autism. For our project we will use a Nao robot which will be lent to us. The robot’s task is to comfort the children by helping them create a routine. The robot will do so by reminding them of certain tasks.

To properly test the robot a rubric will be made which will be discussed with the test group. This will give an insight on the actual practicality of the robot and will help answer the question if it is a helpful aid for children with autism. Different tasks will be measured by grading the specific task.

To achieve this goal, we will use a Nao robot which is provided for us, the robot needs to be programmed to perform the tasks that are set out and the program needs to be altered according to the test results.

There are a few solutions already available for children with autism, however we would like to achieve something new in that it is multi purpose. The robot should help a child on multiple levels while remaining approachable by its appearance.

The project must be finished in 8 weeks, in order to help us meet this deadline we made a planning.

Users

For our main users we confined to children between aged 6 and 12 (this might be changed during the project) which is the age for primary school in the Netherlands who have autism spectrum disorder (ASD). Their needs will be defined as the needs they have of improving their social interactivity skills by means of education to help cope with their neurological disorder. As stated below children with ASD are more vulnerable to emotional and behavioural difficulties;

“Children with developmental disorders such as specific language impairment (SLI) and autism spectrum disorders (ASD) appear to be more vulnerable to emotional and behavioural difficulties than typically developing children.” Wenche Andersen Hellanda, Turid Helland (2017)

This is something to be considered in the education of children with ASD, however, we will (for now) confine ourselves to the educational needs of children with ASD to help them increase their social skills by means of a robot. Many studies have already been done on the use of robots for educational purposes and many proposed methods proved an effective tool to improve the social skills of these children (see literature study). Some of these studies even stated the robots to be more effective at attracting attention while teaching. Thus these robots seem to be in line with the needs of the main users when implemented correctly.

Approach

The goal of our project is to help young autistic children with their social behavior. We want to do this with the help of a robot Nao.

To determine the needs of the children a literature study will be performed. This study also includes research about what is already done, so the state-of-the-art is explored.

With the information obtained the code for the robot will be written. This will be done in either in TiViPe or Choregraphe depending on the complexity of our project.

When the code of the robot is finished we will test it with autistic children and teachers of an elementary school. The results and feedback will be implemented after this. At the end of the project, the robot will be demonstrated with all the adjustments mentioned by the children and teachers.

Milestones

• Stage 1 planning completed.
• State of the art research completed.
• Scenarios and test plan completed.
• Nao obtained.
• Code completed and tested.
• Interview completed.
• Wiki page completed.
• Final demonstration completed.

Planning

• Week 2:

• Introduction; Sanne
• Problem statement; Sanne & Laura
• User’s and RPC's; Leon
• Milestones; Robin
• Add planning and headings to the wiki; Robin
• State-of-the-art literature study; Everyone
• Write smart objectives; Heleen
• Approach; Laura

• Week 3:

• Literature study; Heleen & Laura
• Write scenarios; Robin & Leon
• Interview Emilia; Everyone
• Find people for the interview; Sanne

• Week 4:

• Literature study; Heleen & Laura
• Start programming; Robin & Leon
• Test plan; Sanne

• Week 5:

• Make a working version; Robin & Leon
• Test plan; Sanne, Heleen & Laura
• Update wiki; Heleen & Laura

• Week 6:

• User test + interview; Sanne, Leon & Robin
• Adjust code; Leon & Robin
• Process results; Heleen, Laura & Sanne

• Week 7:

• Process code feedback; Leon & Robin
• Write a conclusion and update wiki; Heleen & Laura
• Make presentation; Sanne

• Week 8:

• Provide presentation; Leon, Sanne & Heleen

Literature review

Currently, these are the summaries of the state of the art articles

1-5. Autistic children have problems with several things like self-initiated interactions, turn-taking activities, imitation, emotion recognition, joint attention and triadic interactions. There are already existing solutions or ideas to help those children. An example is an app which gives them the structure they need. But there are also experiments done with the Nao robot we would also like to use. Human Robot Interaction (HRI) helps with suppressing the autistic behavior. Especially the children with moderately impaired intelligence.

6. Parents and therapists are enthusiastic to work with robots such as NAO as it encourages children to participate on a social level. Especially for children with ASD as NAO is not impulsive and gives a certain structure. The main problem at this time is related to voice recognition.

7. Game-based therapy have positive effects on training social skills, especially with lego. Children in the experiment did not like a slow speaking robot. The repetition of the rules also irritated them. Therapists prefer one on one therapy with a robot and a child, instead of a pair of children as a robot cannot adapt well to situations happening between the two children. One clear outcome is that children (with ASD?) react more strongly to the verbal than non-verbal communication, thus the speech should be improved most. Major challenge is keeping children with ASD focused, especially when the task at hand takes long. However, robots are considered exciting by the children, which makes it easier to keep their focus.

8. For this study children with ASD were paired with their siblings. The idea was that a robot mediated lego therapy, which is supposed to improve collaborative behaviors. However no significant changes in behavior was found (as opposed to lego therapy without a robot mediator). Even though robot therapy is advised for children with ASD, it comes with limitations. One of the main limitations is that the robot has a limited behavioral repertoire, so no differentiation in prompt levels could be made. The article thus recommends the increasing of the variability in prompt levels to respond to children’s individual needs.

9. The goal of this study was to see whether children with ASD performed better when NAO is more humanlike and when his intonation differs. It looked also for a difference in likeability. There was no evidence that intonation has significant effect on performance. However, as the children in the study where already familiar with NAO and it’s voice Jasmijn, the children may not have experienced possible positive influences of a different intonation as individuals with ASD like things to be the same (thus preferring Jasmijn). Bodily appearance does also not influence the performance. It does however has a large effect on the affective state of an individual. It also helps when the bodily appearance matches the intonation. So mechanical + monotonous or human-like + normal. For these congruent combinations, the happiness also increases.

10. The article mentions that there are cases where individuals with ASD react better to robots and prefer interaction with robots over humans. However it also says that studies that come to these findings have very inconsistent results. A lot of studies are too exploratory or theoretical. It concludes that the robots have great potential in a clinical setting, with advantages like the appeal of technology to children with ASD, the ability of a robot to have a simple social behavior (that is easier to understand and focus on) and a robot can be easily adapted for each individual. However further research should be done to determine the validity. Work should be evaluated by experts with clinical expertise as most studies are now focused on the technology development and not necessarily clinical application.

11. Bibi Huskens, Rianne Verschuur, Jan Gillesen, Robert Didden & Emilia Barakova (2013) Promoting question-asking in school-aged children with autism spectrum disorders: Effectiveness of a robot intervention compared to a human-trainer intervention, Developmental Neurorehabilitation, 16:5, 345-356, DOI: 10.3109/17518423.2012.739212

This research aimed at investigating applied behavior analysis (ABA)-based interventions either done by a human trainer or by a robot to promote self-initiated question asking in children with autism spectrum disorder (ASD). In an ABA analysis it is studied how behaviour works in real life situations, with the goal to increase behaviour considered “good” and decrease behaviour that are harmful or affect learning. This research focused mainly on two research questions: (a) are an intervention conducted by a robot and an intervention conducted by a human trainer effective in promoting question-asking in children with ASD? and (b) which of these two interventions is more effective in promoting question-asking in children with ASD? The research in this article was done with the help of six children participants which were between 8-14 years old, had a ASD diagnosis, an IQ of above 80 on a standardized test and where not able to initiate a question after a statement of a trainer. A NAO robot was used and each child had two interventions by both a robot and a human trainer while being under video surveillance. The results of this article state that both robot and human-trainer ABA-based intervention is an effective tool that increased self-initiated question asking in children with ASD between baseline and the first intervention and was maintained during the follow up. The article states that no conclusions could be drawn with regard to the differential effectiveness of the robot or human-trainer interventions.

12. Felippe Sartoratoa, Leon Przybylowskia, Diana K. Sarko (2016) Improving therapeutic outcomes in autism spectrum disorders: Enhancing social communication and sensory processing through the use of interactive robots.

This research focuses at examining a range of socially interactive robots which are currently the most used for therapeutic purposes and their therapeutic effects. The researchers discussed how enhanced sensory processing and integration of social cues into these robots may underlie the benefits that these robots bring about. They state these robots and their interactions might provide therapeutic benefits by allowing the audiovisual cues in social interactions to be experienced in a simplified version of human interaction. The research focusses mainly on two parts: The deficits in autism spectrum disorder (ASD), consisting of a sensory perception and neurobiology part, and the spectrum of socially-assistive robots, where multiple types of robots are examined. They conclude that ASD comes with perception impairments related to multisensory integration of more complex stimuli. Furthermore they state that ASD comes with several neurobiological overarching impairments such as one involving the mirror neuron system which facilitates imitation and social communication, or deficits in the amygdala which might underlie social cognition deficits. After examining the impairments that come with ASD the research focusses on the spectrum of available socially-assistive robots, where they investigate 4 types of robots: humanoid robots, cartoonish robots, animal robots and robots in robotic form. They conclude that humanoid robots were found to elicit enhanced generalization of social skills taught during therapy, however non humanoid robots attract the most attentional engagement and robots with a simple appearance increase levels of interaction and are more easily accepted by children with ASD. Next the research examined the application of neuroscience tools to improve the therapeutic value of the robots and concluded that the underlying mechanism behind the benefits of these robot interactions remains largely unknown and state that recent studies in their lab have begun to address this question. The research does not end with a general conclusion but with a written part about the access to social robot therapies and state that there is a lack of affordable, commercially available robots for in home use and that there is also a lack of longitudinal studies addressing following children’s process as they go through childhood.

13. Sang-Seok Yun, Hyuksoo Kim, JongSuk Choi, Sung-Kee Park (2015) A robot-assisted behavioral intervention system for children with autism spectrum disorders.

This research aims at examining the feasibility of a robot-assisted intervention system for the training of children with autism spectrum disorder (ASD) via human robot interaction (HRI) based on the discrete trial teaching (DTT) protocol. Their proposed robot architecture configures 4 modules: human perception, user input, the interaction manager and the robot effect, with which the robot can generate different training stimuli and can automatically cope with the child’s responses by using human recognition and interaction techniques. The article consists of 3 main parts: Robot system for autism treatment, experimental setup and results and discussion. In the robot system for autism treatment part they discussed their interaction architecture and its treatment protocols which focusses mainly on the social interaction skill of eye contact and reading emotion. Furthermore they discussed their robot automatic interaction system which involved sound, physical movement made by the robot, displayable content and even material reward. The experimental setup part goes into great detail about the setup used in the research, stating they used two different robots for their tests and a monitoring setup. Their results verify the effectiveness of behavioral intervention in autism treatment using a robot and they claim to be confident that the proposed system they used can be attributed to the positive effect of social skill training in children with ASD.

14. Jaishankar Bharatharaj, Loulin Huang, Christian Krägeloh, Mohan Rajesh Elara, Ahmed Al-Jumaily (2018) Social engagement of children with autism spectrum disorder in interaction with a parrot-inspired therapeutic robot.

This paper discusses a series of pilot studies in which ten participants (children with ASD) were to engage with a parrot inspired therapeutic robot. Hereby it was evaluated if these children with autism spectrum disorder (ASD) exhibited more social interaction when engaging with the robot compared to another human. The article states that parrots already have a widespread use in therapeutic sessions for example patients with post-traumatic stress disorder, the researches give this as a reason to design a parrot inspired therapeutic robot. The participants were gathered in a study room and three sets of outcomes were collected; one at baseline, one without human and robot interaction and one with robot or human interaction. Data was collected on 12 defined behaviours such as: looking at the person, going close to the person or touching the person. The study’s results indicate a significant improvement in children’s interaction abilities as opposed to sessions with human involvement and the 12 types of social interaction abilities monitored reported that the robot has the potential to act as a social robot to improve social interactions in children with ASD. The article ends with the notion that the sample group was small and the study was not conducted long-term and there were no follow ups, hereby stating they aim to address these limitations by increasing the scale and duration of their research.

15. Wing-Chee So⁎, Miranda Kit-Yi Wong, Wan-Yi Lam, Chun-Ho Cheng, Sin-Ying Ku,Ka-Yee Lam, Ying Huang, Wai-Leung Wong (2019) Who is a better teacher for children with autism? Comparison of learning outcomes between robot-based and human-based interventions in gestural production and recognition.

This study aims to compare the learning outcomes in children with autism spectrum disorder (ASD) and intellectual disabilities from robot based interventions on gestural use versus the human based interventions. A intervention protocol was designed, implemented and tested on a test group of 23 children aged six to 12 divided randomly in two groups; one for human-based intervention and one for robot-based interaction. Two NAO (Aldebaran Robotics Company) robots were used and programmed to speak and/or produce 14 gestures. The treatments consisted of pretests, four training sessions (twice per week), an immediate posttest and a follow up posttests after two weeks. The learning outcomes in gestural production and recognition did not differ between the robot- and human-based intervention groups. The research however suggested that the children in the robot-based intervention group were more likely to establish eye contact with the teachers than those in the human-based intervention groups. To conclude the researchers state that there does not seem to be a difference between the likeliness of children with ASD who received human-based gestural training to recognize gestures and children who received robot-based gestural training.

State of the Art sources

• [1] J Ricks, Daniel & Colton, Mark. (2010). Trends and Considerations in Robot-Assisted Autism Therapy. Proceedings - IEEE International Conference on Robotics and Automation. 4354 - 4359.

• [2] S. Shamsuddin et al., "Initial response of autistic children in human-robot interaction therapy with humanoid robot NAO," 2012 IEEE 8th International Colloquium on Signal Processing and its Applications, Melaka, 2012, pp. 188-193.

• [3] S. Shamsuddin, H. Yussof, L.I. Ismail, S.Mohamed, F.A. Hanapiah, N.I. ZahariInitial (2012) response in HRI-a case study on evaluation of child with autism spectrum disorder interacting with a humanoid robot Nao, Procedia Engineering, 41 (2012), pp. 1448-1455

• [4] S. Shamsuddin, H. Yussof, L. I. Ismail, S. Mohamed, F. A. Hanapiah, and N. I. Zahari, “Humanoid robot NAO interacting with autistic children of moderately impaired intelligence to augment communication skills,” inProcedia Engineering, 2012

• [5] Gijzen, T. (2016, February 9). Gratis app helpt mensen met autisme bij dagindeling. Retrieved from https://www.klik.org/Nieuws/Gratis_app_helpt_mensen_met_autisme_bij_dagindeling-160209130000

• [6] A. Amanatiadis, V. G. Kaburlasos, C. Dardani, S. Chatzichristofis (2017), Interactive social robots in special education, Proc. IEEE 7th Intl. Conf. on Consumer Electronics (ICCE), 210-213.

• [7] Barakova, E. I., Bajracharya, P., Willemsen, M. H. J., Lourens, T., & Huskens, B. (2015). Long-term LEGO therapy with humanoid robot for children with ASD. Expert Systems, 32(6), 698-709

• [8] Huskens, B., Palmen, A., Van der Werff, M., Lourens, T., & Barakova, E. (2015). Improving collaborative play between children with autism spectrum disorders and their siblings : the effectiveness of a robot-mediated intervention based on lego (R) therapy. Journal of Autism and Developmental Disorders, 45(11), 3746-3755.

• [9] van Straten, C. L. C., Smeekens, I., Barakova, E. I., Glennon, J., Buitelaar, J. K., & Chen, A. (2018). Effects of robots’ intonation and bodily appearance on robot-mediated communicative treatment outcomes for children with autism spectrum disorder. Personal and Ubiquitous Computing, 22(2), 379-390.

• [10] Diehl, J. J., Schmitt, L. M., Villano, M., & Crowell, C. R. (2012). The Clinical Use of Robots for Individuals with Autism Spectrum Disorders: A Critical Review. Research in autism spectrum disorders, 6(1), 249-262.

• [11] Bibi Huskens, Rianne Verschuur, Jan Gillesen, Robert Didden & Emilia Barakova (2013) Promoting question-asking in school-aged children with autism spectrum disorders: Effectiveness of a robot intervention compared to a human-trainer intervention, Developmental Neurorehabilitation, 16:5, 345-356

• [12] F. Sartorato, L. Przybylowski, D. K. Sarko (2017) Improving therapeutic outcomes in autism spectrum disorders: enhancing social communication and sensory processing through the use of interactive robots, Journal of Psychiatric Research,90, 1-11

• [13] Yun SS, Kim H, Choi J, Park SK (2016) A robot-assisted behavioral intervention system for children with autism spectrum disorders, Robotics and Autonomous Systems 76:58-67.

• [14] Jaishankar Bharatharaj, Loulin Huang, Christian Krägeloh, Mohan Rajesh Elara, Ahmed Al-Jumaily (2018) Social engagement of children with autism spectrum disorder in interaction with a parrot-inspired therapeutic robot, Procedia Computer Science, 133, 368-376.

• [15] Wing-Chee So, Miranda Kit-Yi Wong, Wan-Yi Lam, Chun-Ho Cheng, Sin-Ying Ku,Ka-Yee Lam, Ying Huang, Wai-Leung Wong (2019) Who is a better teacher for children with autism? Comparison of learning outcomes between robot-based and human-based interventions in gestural production and recognition, Research in Developmental Disabilities, 86, 62-75.