PRE2016 3 Groep6

Using Robots to Help Autistic Children
Project Robots Everywhere

For the final course of the use learning trajectory of Robots Everywhere students work in groups to create a robot design.
Combining scientific knowledge with social knowledge and taking into accound the USE aspects, it is a perfect application to finish this USE learning trajectory.

List of Participants

Roxanne Boehlé
Bo Drummen
Jolijn Hesen
Rachel Roumans
Sietske Wijffels
Fenna Wit

Technique on Heels

Problem Description

Our group acknowledged that therapy for autism based on robot is developing rapedly. For these therapies different kinds of robots are used and each robot has another approach.
Autistic people take a look at the world differently. They have a mysterious complex impairment in social interaction and communication.
Children with autism struggle with social skills. A robot would be a good solution, because they have infinite patience and do not judge.

Project

In this project we want to face the state of art of this sort of autism therapy specialised for children.
We want to study possibilities in helping these autistic children. We want to get a clear view of what a robot can do for autistic children by analysing existing robots and other autism research, keeping user and society in mind. And finally, we want to design a game that is suitable to help autistic children. This game should make use of a robot and help the child to develop their collaboration skills.

Objectives

Our main objective is to make sure children with autism can smoothly integrate in society, by improving their social skills with the help of robots.

• objective 1; Recognize facial expressions
  
• objective 2; Achieve skills in teamwork
  
• objective 3; Responding appropriately to different kinds of behaviors
  
  • Turn taking
  • Gaze behavior
  • Show interest
• objective 4; Adapt to changes in the environment
  

We want to focus on objective 2, and create a game that encourages teamwork

Approach

Our approach containes doing literature research to see what is out there and what not, mostly focusing on what is not.
Then, we plan to talk to specialists and/or parents with children with autism, to see what is needed or what they would look for in a robot. If possible, we would like to integrate these needs in a game which could be programmed in a robot.

Research question

In what kind of game with Nao can collaboration skills in autistic children be improved?

Subquestions

• What is ASD?
  
• What are the key factors in collaboration?
  
• What is the state of art in robottherapy for autistic children?
  
• What are Nao's functions and limitations?
  
• How can Nao be used in the game by simulating and stimulating?
  
• Which possible game concepts are relevant for our research?
  
• What are the requirements, preferences and constraints of such a game?
  
• In what way can we properly test our game?
  
• Why is our game useful for society?
  
• Future research?
  

USE aspects

This project in which we look for a robot that can be implemented in the of improving social interactions by autistic children is related to USE-aspects.

The primary users are autistic children. They can improve their social interactions by doing a game with the robot. In this game they get acquainted with teamwork through starting with working with the robot first.
The secondary users are the family or other people who are involved in the lives of autistic children. For example the classmates and teacher of the child, they will definitely notice the change in teamwork when these skills are improved. Or maybe they are also excited about the game and even implement these kind of games in their own lives. Also the therapists are stakeholders in this case.

All the preferences of all the users has to be thought of. For example, the best for the autistic child is to improve these skills on an easy and comfortable way. On the other hand, it should be easy to use for therapists.
The social environment of the autistic child benefit the most of the social skills improve fast, to make life easier with the child.
For society, improving social skills of every person is beneficial. This is because everybody is able to work and to add something to society and integrate. Different kind of companies will be involved in this; the company that provides de Nao (the robot), and companies can produce different kind of games for variety.

Theory

Autism

Autism spectrum disorders (ASDs) are a class of neurodevelopmental disorders defined by impairments in social functioning and communication, often accompanied by repetitive and stereotyped patterns of behaviour and interests (Landa, 2008). The impairments typically occur before 3 years of age. ASD include the diagnostic categories of autism, Asperger’s syndrome and PDD-NOS. ASD is pretty common: around 1 in 150 children (Landa, 2008) and even more than 1% of the population of The Netherlands (approximately 190.000 people) are diagnosed with an ASD (source: NVA, see references).

Signs and symptoms
ASD has a wide range of signs and symptoms. There a two main types of behaviours according to the National Institute of Mental Health (NIMH) (source: NIMH, see references): restricted/repetitive behaviours and social communication/interaction behaviours. Examples of the first type of behaviours are: repeating certain behaviours or having unusual behaviours, having overly focused interest, such as with moving objects and having a lasting and intense interest in certain topics, such as numbers or details. Examples of the second type of behaviours are: getting upset by a slight change in routine, atypical language development, poor nonverbal communication skills and having facial expressions that do not match with what is being said.

Therapies
To assist people with ASD, therapies are often already started at a young age. These therapies necessarily involve a team of people. At the Brigham Young University Comprehensive Clinic, this team includes a primary and a secondary therapist, a therapy supervisor, and the child’s caregivers (Goodrich, M.A. et al.,2012). As stated by Marwecki et al., ASD are not curable. The main goals of therapy are to decrease the symptoms, help affected persons to accept their situation, and provide support for their families. Through behavior therapy, desired behaviors can be conditioned and develop strategies for overcoming his or her deficits (Marwecki, Radle, & Reiterer, 2013).

Therapies with robots
Nowadays, more and more research is done in therapies with robots to help children with ASD. Mostly, if not all of them, show positive results. Below you can read about several robots that are already tested with autistic children. Important to note is that social engagement with a robot is not a goal but rather a means for helping autistic children interact socially with other humans (Goodrich, M.A. et al.,2012). Although we know that involving robots in such therapies is beneficial, there are no clear conclusions yet on why these robots succeed in establishing social engagement. However, Scassellati et al. stated that there are a couple of hypothesis why robots generate prosocial behaviour in many children (Scassellati et al., 2012): Perhaps the simplified social cues that robots present result in less overstimulation of the children; perhaps robots offer more predictable and reliable responses than those from a human partner with ever-changing social needs; perhaps robots trigger social responses without the learned negative associations that some children have with human-human interactions; perhaps the exaggerated prompts at robots provide are better triggers for social behaviour than the nuanced and subtle social prompts from a human partner (Scassellati et al., 2012).

Requirements robots
Research that was based on the three-way interaction of the child, clinician and robot found that robots that autistic children favour robots that do not look too much like real humans (Giullian, N. et al., 2010). However, it is suggested that the children can generalize what is learnt better if the robots are more humanoid. This leads to the suggestion that it would be beneficial if robots could change appearance throughout the therapy. In a conference paper of October 2010 (Giullian, N. et al., 2010) were suggested a couple of robot specifications: the robot must be robust, easy programmable, affordable, and appealing to children with autism in order to be useful in therapy.

Collaboration

To improve the collaboration skills of autistic children with a game a number of objectives are stated by Giusti, Zancanaro, Gal and Weiss (2011). One of them is:

• Embed specific interaction mechanisms to foster and promote collaboration between children.
  

They defined three key factors in collaboration:

1. Joint Performance: Collaboration is the performance of an action together (e.g., moving heavy objects together).
   
2. Sharing: Collaboration is the sharing of personal resources to achieve a common objective.
   
3. Mutual Planning: Collaboration is the elaboration and performance of a joint plan; if we want to solve the problem we have to coordinate our actions and our resources.
   

In pursuing this Design Objective, four different collaboration patterns to implement the three dimensions of collaboration described above were defined

1. Choosing together: in order to select an object, touch by some or all of the users interacting with the system is required (e.g., pressing a button together). This realizes the Joint-Performance dimension.
   
2. Constraints on objects pattern: collaboration that is required because of explicit constraints on an object; for example, an object is too heavy and the children cannot drag it on their own. This realizes the Joint-Performance dimension.
   
3. Different role pattern: collaboration that is required because the individuals have been assigned to play different roles; for example, a driver who needs to focus on driving while a navigator checks a map. This realizes the Sharing dimension.
   
4. Ownership pattern: collaboration that is required because the participants have ownership of different objects that need to be negotiated. This realizes the Mutual Planning dimension

State of the Art

ZECA:

ZECA robot ("Author Archives", 2017)

ZECA stands for Zeno Engaging Children with Autism. Zeno is a small humanoid looking robot that can display facial expressions. ZECA can either be sad, happy, scared, surprised, angry or neutral. In a study (Costa et all, 2013) it was found that adults and typically developing children could identify Zeno’s facial expressions and gestures which intent to convey emotion. So, Zeno can interact with children through nonverbal communication such as body movements and facial expressions. This speeds up diagnosis and perhaps it can even enable it to be carried out before a child can talk(Tucker, 2015).

Dr Dan Popa at the University of Texas at Arlington (Tucker, 2015):
"The idea is for the robot to instruct kids, give them some useful social skills and at the same time observe their reactions and calculate their reaction times. That calculation could form some kind of an autism scale."

Zeno has three modes in which it can be used (Tucker, 2015)

1. Scripted mode of interaction. Here pre-programmed sequences of motions are performed by Zeno.
   
2. Wizard of Oz mode. In this mode an operator or therapist can control the robot by tele-operations. In this mode, Zeno mirrors the motions of the instructor.
   
3. In the third mode, the child can take control of the robot. However, this could be unsafe, because for both the child and the robot. So this mode is intended for entertainment only.

In another study (Torres et all, 2012), researchers were able to achieve human-robot interaction through mimicking behaviors between Zeno and humans. Zeno was given smooth motion that resembles human-like movement, taking into consideration that Zeno has a lower number of degrees of freedom compared to humans. The ability of having smooth motion contributes to the human-likeness of a robot. With the algorithms for smooth motion, Zeno was introduced to one child with ASD and his reaction was positive. The child was immediately attracted to Zeno and was tentative and responded to the instructions given through him.

Salvador and his researches compared the ability to recognize emotion of children diagnosed with autism with those of typically developing children. (Salvador et all, 2015), They looked at the effect of incorporating gestures to the emotion recognition accuracy in both child groups. No indication of a significant impairment in the general emotion recognition of the autism group was found. However, a specific shortfall in correctly identifying fear was found for the autism group when compared to the TD children. Furthermore, it was found that gestures can significantly impact the recognition accuracy for both children with autism and typically developing children. This is either in a negative or positive way depending on the specific expression. This study concluded that the use of gestures for conveying emotional expressions through a humanoid robot is relevant in a social skill therapy setting.

NAO ("Wiki Therapist", 2017)

WikiTherapist

The WikiTherapist project aims to promote embodied interaction through interfaces that are able to develop an understanding of the intent of the natural movement of the human body. The movements are then simulated on robots as part of games. These games aim to improve the social interaction skills of children with ASD. The WikiTherapist project strives for a framework which enables the expression and interpretation of patterns of emotional movement (Costa et all, 2013).

The WikiTherapist project uses the humanoid robot NAO as an assistant in their behavioral therapies for children with autism. The NAO robot can engage with people in interactive behaviour through movement, speech, basic facial expressions and touch (Gillesen et all, 2010). So far, the NAO robot is able to recognize simple instrumental and emotional movements based on emotional primitives (Barakova et all, 2009). Furthermore, NAO is able to perform some complex movements. However, interaction scenarios with autistic patients is still in its developing stages.

The goal of the WikiTherapist project is to create a web based community of therapists and robot practitioners who co-create robot behaviors and scenarios of all types of complexity ("Wiki Therapist", 2017).

Furthermore, the WikiTherapist project strives to simplify the programming environment for therapists who will use WikiTherapist as a platform. This is realized through developing techniques to program robotic systems by demonstration of the desired behavior. This is called imitation learning. Imitation learning reduces the complexity of robot programming for therapists. Moreover, the developed imitation learning techniques can also be used as a training tool in the therapy sessions for children with ASD (Gillesen et all, 2010).

The current FACE robot, developed by Hanson Robotics (Mazzei et all,2010)

FACE

FACE stands for Facial Automation for Conveying Emotions. FACE is a humanoid robot which is able to express and convey emotions and empathy in order to help autistic children to better deal with emotional and expressive information. FACE is used in combination with a therapeutic setup comprising a sensorised shirt, video cameras, and an eye tracking hat (Mazzei et all,2010).

Subsequently, FACE integrates the therapeutic adaptive information derived from sensors used on children and in the surrounding environment. This information allows FACE to harmonize its own expressions and movements with the feelings of the user, through control and data processing algorithms (Costa et all, 2013).

robot bandit (Scassellati et all, 2012)

Robot Bandit Project

The robot Bandit project aims at facilitating human robot interaction in a natural way, subsequently increasing interactions of children with autism.
A sound speaker is mounted on the robot which allows the robot to play pre-recorded vocalizations. These communicate the emotional state of the robot (e.g., happy or confused).
It was found that more verbal children showed more interest, but also had more expectations for the social capabilities of the humanoid robot(Costa et all, 2013).

Bandit has two degrees of freedom. One in the mouth and one in each to the eyebrows. This allows the robot to have limited facial emotion expression without being biologically faithful.
Bandit can also turn its head from side to side and has six degrees of freedom in each arm. This allows him to indicate “no” and do pointing and make gestures (Scassellati et all, 2012).

KASPAR

KASPAR is designed in the UK by the University of Hertfordshire’s Adaptive Systems Research Group. He has the size of a small child. Unlike robots like Zeno and Milo, Kaspar has a neutral expression so that children can interpret him how they wish. (Tucker, 2015). KASPAR is used in tactile interaction scenarios with children with autism. KASPAR responds differently(e.g. happy or sad), depending on the way he is touched. KASPAR has a realistic face with significantly little actuation: It has two degrees of freedom in the mouth (open/close and smile/frown) and three degrees of freedom in the eyes (up/down, left/right, and open/close the eyelids). According to its designers, KASPAR’s minimally expressive face reduces its complexity as a social stimulus (Scassellati et all, 2012).

With KASPAR’s ability to provide immediate feedback to a child’s touch, it provides appropriate teaching considering physical social engagement. It reinforces suitable behaviors for tactile interaction with another agent. Despite seeing no significant differences when evaluating the tactile interaction in a study, more than 90% of the times the children touched the robot gently (Costa et all, 2013)

In an exploratory study (Costa, Lehman et all, 2013) children with autism interacted with KASPAR. The study investigated robot-assisted play, here KASPAR acted as a social mediator. The study aimed to increase body awareness with tasks that taught children about the identification of human body parts. It was found that children were to have a lot of interest in touching the robot, that the children started looking to the experimenter for a longer period of time and that KASPAR could prolong the attention span of the children.

An evaluation by teachers of the children participating in the experiment showed (Costa, Lehman et all, 2013), that they improved their ability to identify parts of the their body with their own hands. Additionally, some of the children that initially were not able to identify any of the body parts on themselves, showed an improvement of their knowledge.

In another recent study (Huijnen et all, 2016) researchers were looking at the potential contribution of KASPAR to the therapy and/or educational goals for children with autism. For this the researchers asked professionals and practitioners in the field to fill in online questionnaires and participate in focus groups. It was found that professionals expected KASPAR to be of added value to autism objectives like ‘communication’, ‘social/interpersonal interaction and relations’, and ‘play’, but also in objectives related to ‘emotional wellbeing’ and ‘preschool skills’. A top 10 (Table 1) was created of professionals’ expectations of potential added value for robot KASPAR for working on therapy and educational goals for children with autism.

Kaspar

Kismet ("Can Robots Feel Your Pain?", 2017)

Kismet

Kismet engages people in expressive social interaction. Kismet perceives a variety of natural social cues from visual and auditory channels, and delivers social signals to people through gaze direction, facial expression, body posture, and vocalizations (Breazeal, 2000).

In a study (Costa et all, 2013), the creators studied how children reacted to the robot when it generated a variety of social gestures and facial expressions in response to stimuli from the child. The result consisted of an ongoing turn-taking between robot and human.

MILO

Milo is a humanoid robot that engages with children with autism through research-based lessons that teach social behaviors. Milo is meant as a social mediator for educators, therapists and parents. Milo can teach children to understand the meaning of emotions and expressions, and demonstrates appropriate social behavior and responses ("Robots4Autism", 2017).

Recent research ("Robots4Autism", 2017) has shown that children working with a therapist and Milo are engaged 70-80% of the time compared to just 3-10% of the time with traditional approaches. It was found that children want to work with Milo again and again which increases their opportunities to learn and their developmental success.

Milo looks very similar to Zeno from ZECA. Milo’s expressive face is one of his most important features. Children with autism are asked to identify the emotion shown by Milo from multiple choices on an iPad. Milo’s eyes are cameras, recording feedback. The child wears a chest monitor that records changes in heart rate. This allows Milo to identify the child’s changes in emotion (Tucker, 2015).

What seems to be unique about Zeno and Milo is the way that their expressiveness defies long-held robotic conventions. They could eventually have an impact far beyond the diagnosis and treatment of autism. The inventors, RoboKind, envision a broader role for their robots in educating young children.(Tucker, 2015)

Leka
Leka is a robot that has fun and educational games to motivate social interactions by autistic children. It uses colors, movements and gestures, sounds and music, and apps and games.The robot focusses on social interactions, improve motor, cognitive and emotional skills.The robot simulates a friend for the child.
Because Leka is pre-programmed, the interaction with the robot is predictable. This adds a safe factor to be with the robot. Leka detects interaction through autonomous behaviors with sensors. This robot is customizable so every stage of every child can be adapted to.

Nao

Nao is a personalizable and interactive humanoid robot that has been evolving since 2006. A French company named Aldebaran Robotics are the makers of the Nao Robot. Currently Nao is in its 5th version. It is 58 centimeter in height and it weights 4.3 kilograms. Nao is controlled by NAOqi, a Linux-based operating system. With the robot comes a software suite which includes a graphical programming tool Choregraphe.

Functions

Nao can speak up to nine different languages. He is able to detect and recognize faces. He is able to recognize speech and understand what people say and his sensors and camera are albe to sense the environment very well. Nao is mainly used for entertainment purposes. He can play songs, talk with people or show them how to do their exercises. Therefore Nao is used as a help for physiotherapists for example.

Limitations

Nao is not able to walk very fast, because he will fall down if he does this. Beside that it is quite hard for Nao to pick up objects because his hands are not too powerful and he is not able to move his fingers seperately. Since we are not experienced with programming the Nao robot, this is also a limitation for us. We will have to stick to using the Choregraphe tool and use the movements and actions from that tool. These include standing up, sitting down, walking, talking, waving and some dances and other basic movements. And it is able to read data from the sensors.

Game design

In the literature can be seen that there exist a lot of robots that could potentially help in ASD therapy, but the implementation and acceptation is still missing. In this project, we tried to tackle this problem. To achieve this, we focussed on how to use Nao as optimal as possible during our game.

Nao is already seen as useful in therapy because it could be seen as a buddy. Nao is easy in social interaction, is always straight forward and says and acts in exactly the same way in repetitive situations. An example that desribes this is when a person is happy: everytime he/she is happy, this will be expressed in facial expressions and body language. But this will never be exactly the same. But when Nao does this, he will act exactly the same every time. This makes Nao more comprehensive and predictable for an (autistic) child.

First simulate, then stimulate

We want to improve the socials skills of autistic children by first simulate and then stimulate collaboration. In this way, an 'intermediate level' with Nao is created, before the skill has to be applied in human interaction.

The idea is that the game is built up into two successive phases: a simulating and a stimulating phase. In the first phase, Nao simulates a teammate, with whom the autistic child has to collaborate during the game. Together with Nao the child has to reach the end goal. In this way the child can learn the basics of collaborating. When the child is comfortable with Nao and can play the game well enough, it is time to take a next step. In the second phase of the game, the autistic child plays together with another child. Nao will also be present: not as a player but as a supervisor/mediator. Nao will stimulate the children by giving suggestions for questions, making sure they will not forget annything and encourage the collaboration. So the two children have to reach the end goal together, but Nao still plays an active role.

Game Concepts

City game

Arrangement of game:
The game includes 2 tables with a small border in between. In other words, each player can only see what is on his/her table and not what is on the other player’s table. However, the players can see each other, stimulating communication.

The idea for this game was based on a game that already exists that stimulates teamwork. The game is called Team Building (http://wij-spelen.nl/team-building.php) and it can be played in several ways depending on the number of players. For 2 players the goal is to build a city from a map together. However, each player has their own colored blocks and can only place blocks of this color. Only by working together they can build the city correctly.

Gameplay:
At one table there are pieces to build a city, on the other table there is a picture of what the city should look like. There can only be 1 player at each table and they are not allowed to switch. The players have to take turns asking questions to each other in order to achieve their shared goal: reconstructing the city. The idea is that the both players have to ask questions and give instructions in order to build the city correctly. The players can ask a supervisor (therapist) whether their city is correct. If not, they can continue to improve by communicating and working together.
The difficulty of this game can easily be adjusted through changing a few variables. For example, increasing the number of pieces and increasing the number of maps (with only 1 correct one). This will depend on the child and his/her needed therapy.

Robot:
This game can be played with a robot and an autistic child. However, the robot can only take on the role of the player with the map of the city, because it is too hard for the robot to grab objects and put them in the right place. After playing this game with a robot, the autistic child can play the game with another (autistic) child.

Logical puzzle - city

Arrangement of game:
The game consists of several pieces which represent colored houses, tags for buildings (baker, hair stylist, school etc.) and people. These pieces can be placed on a game board, which consists of a basic arrangement.

Gameplay:
2 players play together to build the city. Each player has half of the information about the puzzle. However, the information is divided in a way that the players need each other to achieve their common goal: building the city correctly. Examples of the information cards are:

• Billy lives in a red house
  
• The blue house stands next to the red house
  
• The baker lives in the blue house
  

This game can be made harder by changing the availability of information. For example, decreasing the number of clues. This will depend on the child and his/her needed therapy.

Robot:
This game can be played with a robot and an autistic child. However, it is difficult for the robot to put the pieces in place. But because the robot and child form a team together, the robot could ask the child to put the pieces in place. After playing this game with a robot, the autistic child can play the game with another (autistic) child.

Virtual Game 1

Arrangement of game:
The game can be played on one computer/tablet screen with two controllers (e.g. two buttons on a keyboard), or on two screens with separate controllers. The players can see and talk to each other, so they are still able to communicate.

Gameplay:
In this virtual game, two players have to navigate through an online 2D environment from start to finish. Each player controls one pion. The idea is that they cannot both reach a finish (there will be one finish spot per player) without helping each other. The only way to win the game is when both players reach a finish. This can be implemented in different ways. Collaboration is stimulated by integrating tasks that can only be done when the players work together. An example of this is for example when a finish is on a platform that is too high to jump on. Then player 1 can ‘use’ player 2 as a kind of stair to reach the finish. Other examples are that both players have to stand on a certain place in the field to open a door or that they have to work together to fight against an enemy. The ‘push buttons’ and ‘colors’ game below are variants of this game.

Robot:
In theory, this game can be played with a robot and an autistic child. Nao is not able to use the controllers used in this game so a real person should do this (probably the best way to do this is that the person who controls nao also controls the game). This person should best be in another room, where the child can’t see him. I.e., Nao acts like he touches the controllers but actually the game is played by someone else. The child and Nao should be seated on opposite sides of a table. In this way, chances are biggest that the child doesn’t suspect he is not really playing against nao. This approach makes this game hard to carry out and is therefore less suitable. After playing this game with a robot, the autistic child can play the game with another (autistic) child.

Virtual game 2: 2 players 1 pion

Arrangement of game:
Same as ‘Virtual Game 1’ above.

Gameplay:
In this virtual game, two players have to navigate through an online 2D environment from start to finish. The main idea is the same as the game above, but in this game the two players have to control the pion together. The pion has to be navigated in a 2D field, which means in our case that it can move in 4 directions (x, -x, y, -y). To achieve this you need 4 buttons. The idea is that each player can only control 2 buttons. For example: player 1 can only move the pion vertically and player 2 can only move the pion horizontally. In this way the players really need to work together to reach the finish.

Robot:
Same as ‘Virtual Game 1’ above.

Virtual game 3: Colors

Arrangement of game:
The game has to be played on one computer/tablet screen. Controllers (one button per player) should be attached to make sure both players can adjust ‘their’ color. The players are seated next to each other.

Gameplay:
In this virtual game, two players have to work together to mix two colors into another one. The idea is that there are two bars on the screen. The color of one bar is the color to be mixed and the other bar's color is the color created by the participants. An example of this is the following. At the start of the game, the first bar is a hue of orange and the second one is white. Participant 1 ‘owns’ the color red and participant 2 ‘owns’ the color yellow. By pushing on their buttons, each player can add a bit of his color to the white bar. The goal is to mix a color which is the same as the orange color in the other bar. In this way, the goal can only be reached when both players work together.

Robot:
This game can be played with a robot and an autistic child, assuming that Nao is able to click a bar. Important is that Nao is not delayed too much when having to click the bar. After playing this game with a robot, the autistic child can play the game with another (autistic) child

Lights Game

Arrangement of game
In this game two players are sitting at a different side of a table with a bar between them. This bar is low enough that they can look each other in the eyes, but not at each other's side of the table. On one side of the table, specific items should be placed on certain sensors. On the other side, a map of how the items should be placed is visible for the the other player.

Gameplay:
The player who is placing the items asks the other person where these items should be placed. If the right items are placed on the right sensors, the surface of the table should be the same as the other side of the table. When all the right items are placed, the players can check whether the map is the same by pushing a button. All lights will appear and form a smiley (or something else) when all the items are placed on the right sensors. The goal of this game is that they need to work together to achieve that the lights will be on.

Robot:
The robot in this game will be sitting at the side where the map is shown. This is because the robot cannot pick up things, and so he will only talk and push the button.

Board game

Arrangement of game:
In this game the robot and the child have a goal to play a board game. The robot and the child have the same goal, and they need each other to reach the goal. They are sitting around the board, and they can throw dices to move on to the next place on the board.
Gameplay
The goal of the game is to reach the end of the playboard. This can only be reached by cooperation between the players of this game. every turn you can move closer on to the end of the playboard. The robot can help the child by solving difficult math questions for the child, when the child can do the physical actions of the game that the robot is not able to do. Together they will arrive individually to the end of the game.

Robot:
The robot will have its own goal, and is in the same position as the child. Because it is not able to do physical actions well, it will compromise with mathematical skills.

Activity Scheme

Arrangement of game:
In this game two children are in a room and have to work together to reach a goal, which could be to build a city, a zoo or a theme park for example. The robot is present to control and help the collaboration.

Gameplay:
This is a game in which children try to reach a overarching big goal, by fulfilling sub tasks that together contribute to the bigger goal. These tasks are all stated in an activity schedule. The purpose of the game is to teach children with autism to complete the tasks cooperatively by following this schedule.

Since the game is played by young children, the activity scheme is photographic and consist of a map which the children should build up together. For each building on the map, one exemplar should be build. To reach this, the children have to discuss who will build which building of the map.

Robot:
First the robot will have a steering and controlling role in the game. It helps the children to decide who will do what task and it keeps in mind which tasks are already done and so should not be given to one of the children again. Later on in the game, the robot will interfere less and less with the game and the children have to use their new learned skills.

Cooperative Building

Arrangement of game:
Two children try to build something together with blocks, Lego or Knexx for example. They are together in a room, seated on a playmat for example. The robot is near them.

Gameplay:
The children build alternately on their shared construction. In this way, the children have to accept the things the other built. During the process, the children have to decide several important decisions of their construction. They discuss this together and the robot will lead this discussion. In this game the children learn to accept opinions of others and see that in teamwork also the other opinions are very important to make something everyone is satisfied with. Beside that, the children learn to discuss and find a compromise for the things they determine together.

Robot:
The robot gives turns to the children alternately and leads the discussions of the children.

Creative building

Arrangement of game:
The game consists of different cardboard figures in different colors. The cardboard figures are pieces of a house, like a window, a door, a roof etc. Multiple different houses can be created with the different cards, so there are multiple possibilities to create one house. The game needs two game players, which in the first stage are one autistic child and one NAO robot and in the second stage NAO is replaced by another autistic child. In the last face NAO only fulfills a coaching role.

Gameplay:
The Goal of this game is to let the child and NAO build a house together, where in the end, both the child and NAO are happy with. This will be achieved by giving the couple different assignments.

Examples of the assignments can for instance be:

• Discuss which color the floor is going to be.
• Who is responsible for making the first wall?
• Place the green triangle on top of the building. Etc

Of course the hidden goal of this game is letting the autistic child and NAO communicate about what their reaction on the different assignments will be. They have to discuss and to get to an agreement before they can really perform the assignment.

Robot:
Simulating:
NAO is not able to build something himself, so the therapist or the child has to move the cardboard pieces to the correct positions. NAO however has to discuss the different assignments with the child and has to make the game challenging enough for the child. It depends on NAO’s output, how challenging the game is for the child. (If NAO always agree on the child, it is not an discussion, but if NAO never admits, the discussion will never end, which is not the purpose of the goal)

Stimulating:
NAO will in this face fulfil the role of a coach. He will ask the questions and help the two autistic children to get to an agreement in the different faces of the game. The role of NAO also will be to have a reflection at the end of the game.

Requirements, Preferences and Constraints

Requirements
In the game, the players have to perform an action together. (1.1)
In the game, the players have to share personal resources to achieve the common objective (1.2)
In the game, the players have to elaborate on and perform a joint plan. (1.3)
The game has to be age appropriate for autistic children of age 7 till 9. (1.4)
The game has a clear goal, which is the same for all players. (1.5)

Preferences
The game has several levels and can hence be adjusted to different intelligence levels of the children. (2.1)
The game is played in real life and not online. (2.2)
Nao has an active role during the game, but not necessarily as a player (2.3)
Equal role of participants i.e. social interaction (2.4)

Constraints
Nao can only speak, walk, point or make other simple movements.
Nao cannot work autonomously.

Requirements per game concept

After formulating the requirements and reading research, we came up with some possible games to develop. These games can be find in the game concepts above. Per game we looked at the formulated requirements and discussed if the game could satisfy these. When not, we thought of the possible solutions for these. Some of these are explained below to get a view into our decision proces.

For the City Game we can meet most of the requirements. Unfortunately Nao can only play the one of the roles, which will reduce his active role.

In the Logical city puzzle we faced the problem of reading skills. Our target group has low reading skills, so the assignment and hints will be to hard for them.

For the virtual game 1,2 and 3 the requirements check looks a bit the same. The most positive about this game is that the only way to reach the goal is working completely together. Also in all the games Nao is not able to push the buttons, so it could not take an active role.

The light and the board game have only two requirements not fulfilled, so seemed that they can be succesfull in our research. But in both cases the social interaction is low, which in our eyes is an important requirement.

For the activity scheme we have a strong believe, based on the literature, that the autistic children will learn from it and will improve in their social teamwork. But in this game it is hard to experience the communication during the teamwork, because the children do their task independently and will not have independent resources and share them.

The cooperative building and the creative building are both games that have a high communication factor, but there is no clear end goal. Also the role of Nao in the cooperative building game is only as a leader and not active enough.

Our choice

We eventually chose for the City Game as our final concept because the most important requirements will be fulfilled in this game. We think this game suits our research the best, which the requirements check suggest. Due to the constraints of the robot, in this game concept Nao can only play the role of the participant with the complete map. The role of Nao is not necessarily active and the players do not perform actions together. however we think that the game concept can satisfy these two requirements by adding different rules and making different levels.

Our game

In the game both players get a map of a city. In this city there are several buildings and together they have to recreate the city.

Initially the game is played with an autistic child together with Nao. There is a therapist that supervises the game. First the therapist will explain the game to the child and Nao will introduce himself. This allows the child to get familiar with Nao. If all goes well, the autistic child could play the game with another (autistic) child to practice his/her collaboration skills.

City game

Arrangement of game:
The game includes 2 tables with a small border in between. In other words, each player can only see what is on his/her table and not what is on the other player’s table. However, the players can see each other, stimulating communication.

Gameplay:
At one table there are pieces to build a city, on the other table there is a picture of what the city should look like. There can only be 1 player at each table and they are not allowed to switch. The players have to take turns asking questions to each other in order to achieve their shared goal: reconstructing the city. The idea is that the both players have to ask questions and give instructions in order to build the city correctly. The players can ask a supervisor (therapist) whether their city is correct. If not, they can continue to improve by communicating and working together.
The difficulty of this game can easily be adjusted through changing a few variables. For example, increasing the number of pieces and increasing the number of maps (with only 1 correct one). This will depend on the child and his/her needed therapy.

Game Components

The game consists of a map with several buildings. For each level there is a different set of buildings used. Furthermore, within each level, several maps can be created, also with varying buildings. Each map will consist of a combination of several of the following buildings. We explicitly choose buildings that are very common in real life situations and hence more recognizable for children.

Figure 1.1

Figure 1.2

• Bakery
• Boucher's
• Church
• Cinema
• Dentist
• Firefighters
• Hairdresser
• Harbor
• Hospital
• Library
• Park
• Police
• School
• Sports fields
• Supermarket
• Swimming pool
• Town Hall
• Trees
  

Variables that influence the difficulty of the game

The difficulty of the game varies per level. In order to do this, we introduced several variables that can increase the difficulty of the game. These variables can be implemented per level and are explained below. Some variables are explained with the help of a sketch. We start with a really simple map with four pieces. To give a general idea of what this looks like, an example of a four-pieces game can be seen in figure 1.1 (field for player 1) and 1.2 (field for player 2).

Variables:
1. The number of pieces to be put in the right place. Increasing the amount of pieces, increases the difficulty of the game. This is illustrated in figure 2 and 3. Figure 2 illustrates the same idea as figure 1.1 and 1.2 but now in a rough sketch. Figure 3 has eight pieces, so the number of pieces is increased.
2. The level of similarity between the pieces. For example, a baker with a red roof and a baker with a blue roof can be used (instead of only one ‘kind’ of baker). Incorporating similar pieces and increasing the number of them, increases the difficulty of the game. This is shown in the sketch in figure 4.

• 

  Figure 2

• 

  Figure 3

• 

  Figure 4

• 

  Figure 5

3. The amount of background information on the map. Less background information, increases the difficulty of the game. It is easier to tell someone where the piece has to be laid if there is a lot backgroud information around it. For example: “The baker is below the 3 trees, and in between the river and the supermarket” is easier than “The baker is somewhere next to the river”.
4. The fact that the pieces player 2 has are highlighted on the map of player 1. In the beginning, the pieces are highlighted. In this way, player 1 knows exactly about which pieces he has to ask questions and which pieces are just ‘background’ information already present on both maps. If this is not the case, it will increase the difficulty of the game because player 1 does not know which pieces on the map are the same as player 2 has.

5. You could also state number 4 the other way around. In the default mode, player 2 (player with map) would see a dotted circle at the place where a piece has to be placed. In this way, he/she knows exactly how many pieces are involved and where they need to be. In a next level you could leave the dotted circles away. In this way player 2 has way less information and more communication is needed. Therefore, communication is stimulated even more. This is shown in the sketch in figure 5.
6. The number of maps provided for player 1 (player 1 is the guider with the map). More maps, increases the difficulty of the game. The idea behind this is that the players have to find out which map of player 1 can be made from the pieces and map that player 2 has. This can be relatively simple when player 1 has completely different maps with different backgrounds. But this can be quite hard when the maps only have subtle changes. In this way the players really have to communicate and work together quite well to achieve the goal. This is shown in the sketches of figure 6.1 (player 1) and 6.2 (player 2).

• 

  Figure 6.1

• 

  Figure 6.2

7. Whether or not the therapist (who explains the game) informs the players about all the possibilities that increase the difficulty of the game. For example, if the therapist does not notify the players of the fact that there can be pieces on the map of player 1 that player 2 does not have (Player 1 = the guider with the map and Player 2 = the builder with the pieces). An example sketch of this can be seen in figure 7. Telling less information, increases the difficulty of the game.

Note: there is also a extra option that can be used within each level to increase the difficulty or just make sure that the levels vary enough. This option is illustrated in figure 8. It is a quite basic map, but the difficulty comes from the fact that player 2 has redundant pieces, i.e. pieces that do not belong to the map that he/she has to rebuild.

• 

  Figure 7

• 

  Figure 8

Division of variables in levels:

For the matter of a clear overview, we decided to split the game into three separate levels for this project. In level 1, only the variable mentioned at point 1 is implemented. In other words, only the number of pieces that have to be placed can vary. You start with for example four pieces and this number can be increased with the skills of the participant.
In level 2, the variables mentioned at point 2, 3, 4 and 5 can be implemented. This seems a lot, but the idea is that from level 1 on, you can add one or more of the options of level 2. So we certainly do not want to add all of the four options at the same time. The nice thing about this is that there is a lot of variety possible. You can do only point 3, in the next came 2 and 4, and then 4 and 3. In this way the game will not become predictable, and therefore we can prevent that participants are going to play the game in a systematic way.
Lastly in level 3, also points 6 and 7 can be added. The step to multiple maps is probably quite hard for the participants and that is why we decided to put this in the last level. You could start for example with level one and use the three maps, and leave the options from level 2 away.
So to make it really clear: not all variables have to be implemented. By implementing some and leaving out others, it allows us to adjust the difficulty within each level. So, each map can contain a combination of the above mentioned variables. And in this way it is really easy to slowly increase the level of difficulty.

The implementation of Nao

In our game we use Nao as a social mediator in the game. As mentioned under Theory the fact that Nao is a robot helps succeed in establishing and, in some cases, maintaining social engagement due to these four factors:

• The simplified social cues that robots present result in less overstimulation of the children
  
• Robots offer more predictable and reliable responses than those from a human partner with ever-changing social needs
  
• Robots trigger social responses without the learned negative associations that some children have with human-human interactions
  
• The exaggerated social prompts that robots provide are better triggers for social behavior than the nuanced and subtle social prompts from a human partner
  

Based on these factors we made sure that Nao has predictable behavior and simplified but exaggerated social cues.

Due to the design of the game and through testing the game we came up with a set of standardized questions that we programmed into Nao. These questions are ready for immediate use in a trial with a child. We also made sure Nao could introduce himself before the start of the game. And lastly, we implemented responses Nao can give to win time in case the child says something unexpected.

Nao is able to speak in several languages. However, we chose to let him speak Dutch because we would like to test the game with Dutch children. In order to do this, we made sure that the pieces of the game were also in Dutch. Suppose that in the future non-Dutch speakers would like to use the game, Nao as well as the game pieces could be changed into the concerned language.

Game development phase

After making a sketch of our game, which you could see in the above pictures, we thought it would be usefull to play the game it ourself. In this way we could see which rules of the game provide the best design to fulfill the requirements. Another asset of this little pre-test was making a list of the possible question that Nao could say during the game.

We followed the recommendation of the tutors, to work out only one of the levels and explore all the facets of that level. So we can optimize the questions of Nao, the design, the rules and so the testing as a whole.

File:Testspel Transcriptie1.pdf

File:Testspel Transcriptie2.pdf

These test games helped us to think about the question that NAO can ask during the game and hints he can give to the child.

We faced that the rule of asking questions alternatively is a bit of a problem. It is hard to keeping track who's turn it is, and asking relevant questions. We also tested if we shoud make a rule that you can only ask questions that can be answered with yes or no. The idea behind this was that when asking open questions, the game could maybe become too easy. Imagine for example someone asking where the bakery is located. The answer could be: "It is located above the roundabout, under de green car and inbetween the five trees and the bank". When answering this extensively, the game could be really easy and of short duration. But we experienced that this 'yes-no' rule did not give us the desired result because it was too hard. And less communication is needed (because of the short answers) which is not desirable in our opinion.

Testing phase

We would like to test the game with Dutch children.

We would like to measure and/or look the following things in order to see to what extent our game is successful.

Quantitative measures

From an engineering perspective, we would like to measure the success of the game in several quantitative measures, namely:

• How many questions does the child ask, before completing the game?
• How many of these questions are redundant?
• How much time does it take to finish the game?
• How many pieces are put in the wrong place, before correctly finishing the game?
• How often does Nao have to remind the child to let him/her ask the question?
• How often does the team deliver a wrong solution?
• How often is Nao not albe to finish its sentences because the child is interrupting Nao?

Qualitative measures

However, the success of the game cannot only be measured through quantitative measures. Because you are working with children and their (non-verbal)behaviors, emotions, and personal development, you also have to look at qualitative measurements. In order to do this skilled observers have to look at the behavior and choices of the children in order to answer questions like:

• What is the child's attitude towards the game?
• What is the child's attitude towards Nao?
• Does the child react appropriately to Nao?
• What emotions does the child show? And when?
• Does the child show non-verbal behavior?
• Does the child show behaviors of joint attention, eye contact, and/or gaze behavior?
• Does the child show appropriate turn taking behavior?
• Does the child recognize Nao's social cues and does he/she respond accordingly?

Realization

It's difficult to conclude the success of our game based on measured quantitative values. Because, we first need a baseline that is based on a range of trials which we can compare our measured values. After such a baseline is found, we can conclude whether the child’s performance is above or below average. However, during this course we will not be able to find the baseline of the quantitative values, because we won’t be able to test our game on a number of different children. We do have a few children to test with, but we cannot conclude anything from these values regarding above or below average. When looking at current studies in this field of study, most studies conduct qualitative research, instead of quantitative. These studies involve skilled professionals that judge the progress of the child. Unfortunately we are not skilled professionals. However, we did inform ourselves of studies and state of the art technology in this field of study and with our Psychology & Technology background we can observe the children to our best ability. We also do not know the exact level difficulty of our different levels. Only by playing the game several times with a sample of children we can obtain information about the average performance and successive levels. However, with our tests we can conclude something about the level of difficulty, assuming a more difficult level takes longer to finish correctly.

Method

Subjects
We tested our game with seven subjects: three children, three adolescent and one adult. The children are 7, 12 and 14 years old, the adolescents are 20, 21 and 21 years old and the adult is 52 years old. Two of the adolescents, aged 20 and 21, were tested in the simulating phase and the remainder of the participants were tested in the stimulating phase (explanation below at design game). Our game was originally designed for autistic children aged 7 to 9. Ideally we would only test participants who meet this criteria, but it was really hard to find participants in an ethical way. We only found one participant within the right age category. We decided to also include the others because for us the most important aspect is collaboration.

Task
The game is played by two players. Both players have a map of a city in front of them. Each player can only see their own map. The maps are almost identical, except for a certain amount of buildings that are missing on one map. The player with the incomplete map (player 2) has pictograms (‘pieces’) with buildings laying next to his/her map.The goal of the game is to reconstruct the city of player 1 on the map of player 2, by laying the ‘pieces’ on the correct place on the map of player 2. This can be achieved by asking questions to each other. They have to take turns in asking questions, one question each time.
When the players think they have built the city correctly, they can ask the supervisor if the map is indeed correct. If not, they can continue asking questions to each other until they finished the map.

Design

• Game
  

For this project, one game variant is made. The corresponding map for this variant can be seen in figure 1 and figure 2. Player 1 is the ‘guider’ with the complete map and player 2 is the ‘builder’ with the pieces. The white spaces in the map of player 2 are potential places to put a piece on. As can be seen, player 2 has more pieces than necessary to complete the map. Also, player 2 has several abundant blank spaces (so spaces where no piece has to be laid on). Each player can only see their own map: there is a board placed in between them. In this way the players are still able to look at each other. This is important since facial expressions are very important social cues.
The person from our team that controls Nao is seated in the lab, but faces the wall. In this way the participant does not feel like this person is involved. On beforehand, we tell the participants that this person is only taking notes (see procedure). In this way it is less presumably that the participants will think Nao is not operating autonomously. As already stated before in this wiki, we work with a simulate-stimulate approach. In the first phase, the simulating phase, a child is going to play the game together with Nao. In this phase, Nao will be player 1 and the child will be player 2, as explained above. When the skills of the child are sufficient he/she can go to the second phase, the stimulating phase. In this phase two children are going to play the game together and Nao will have a stimulating role. This means that he will play the role of a supervisor, i.e. help solve any disagreements or giving compliments.

Figure 1

Figure 2

• Nao
  

When the participant entered the room, Nao waved and gave a short introduction. Then Nao went in the ‘crouch’ position and the game started. Nao was directly controlled by us, known as the wizard of Oz method. During the game Nao made several movements. Nao was ‘breathing’, face tracking and blinking. When Nao breaths, he moves his body slightly to give the impression that he is alive. The face tracking ensures that Nao looks at the other player and that he moves his head naturally. Finally when Nao blinks, the LEDs of his eyes turn out for a short moment. Next to these movements, Nao sometimes moved his head down to look at his map.

Procedure
When the participants arrive, the informed consent has to be signed: the participants can do this themselves when he/she is older than 18 years, but when when the participant is younger a parent/caregiver has to do this. See Appendix B for the informed consent that has been used. When this is done, they are taken to the lab. Here, the game is explained. Note: the participants do not have the map in front of them when the game is explained. As explained above, we tell the participants that the person who is actually controlling Nao is only taking notes. In this way the participant will not suspect that the robot is not autonomous. In figure 3 a picture of the setup can be seen. In figure 4 the simulating phase is shown and in figure 5 the stimulating phase is shown The explanation is in dutch because our participants are dutch. The explanation read to the participants is the following:
“Je gaat zo met Nao een spel spelen. Jullie krijgen zo allebei een plattegrond. Deze plattegronden zijn grotendeels hetzelfde. Maar op de plattegrond die Nao heeft staan gebouwen aangegeven die bij jou zijn weggelaten. Jij hebt witte vakjes op je plattegrond waar de gebouwen moeten komen. Het doel van het spel is dat jullie samen de plattegrond van Nao gaan nabouwen op jouw plattegrond. Dus jouw plattegrond moet er op het einde exact hetzelfde uitzien als die van Nao. Dit doel kan je bereiken door elkaar vragen te stellen. Dit moet omstebeurt, dus eerst stelt Nao een vraag waar jij antwoord op moet geven. En daarna mag jij een vraag stellen waarop Nao dan weer antwoord. Een voorbeeld van een vraag is bijvoorbeeld: “Heb jij een bakker op je plattegrond liggen?”. Als jullie denken dat de plattegrond helemaal goed is nagebouwd, kan je dat aangeven. Dan checken wij de plattegrond. Als alles goed is nagebouwd ben je klaar met het spel en als er nog iets mist mogen jullie doorgaan met vragen stellen. Nog een paar belangrijke dingen zijn: Je mag niet op de plattegrond van Nao kijken. Je moet op je plek blijven zitten totdat het spel is afgelopen. Het is echt de bedoeling dat je vragen stelt, dus niet gaat commanderen. Dus niet: “Leg de bakker eens daar neer”. Er is iemand mee aan het typen aan je linkerkant, daar hoef je geen aandacht aan te schenken. Het kan zijn dat je meer kaartjes hebt dan nodig zijn om de plattegrond compleet te maken. Dus dat je kaartjes overhoud aan het einde. Het kan zijn dat jij meer witte vakjes hebt dan nodig zijn om de plattegrond compleet te maken. Dus misschien heb je witte vlakjes open aan het einde. Het kan zijn dat Nao er soms wat langer over doet om te antwoorden. Als jij een antwoord hebt gegeven op Nao’s vraag, mag je daarna meteen een vraag stellen. Je hoeft niet te wachten totdat Nao iets zegt. Heb je nog vragen voordat we beginnen?... Dan gaan we nu beginnen. Draai je plattegrond maar om en veel succes!”

• 

  Figure 3

• 

  Figure 4

• 

  Figure 5

Test results

When we tested the game, we recorded everything on video. We analysed those videos on some quantitative values. As showed in the table below, these are: The number of questions that the player asked and how much of these were redundant, the number of questions that Nao Asked, the time it took to complete the map, how many mistakes were made, how often Nao had to remind the participant to ask a question and the times that the player interrupted Nao, while it was speaking.

We decided to not put these videos on the wiki visible for everyone, since we do not feel like putting our participants online. However if you would like to see it, you can ask us to show you.

The results of the different participants look quite familiar. The number of questions that were asked by the player (M=12.25 ; SD = 2.5) and the number of questions Nao asked (M=12,75 ; SD=2.99) are quite the same. Only participant 2 had lower values for these two variables. This was due to the fact that this participant did not very easily ask questions from herself, she needed a reminder and after some time, Nao gave more direct instructions to keep the game going. While with the other participants the conversation went more naturally and the question were asked more naturally. Secondly none of the players asked much redundant questions and the time the participants are also not very different.

The biggest difference that we noticed in the quantitative values is in the number of reminders. We noticed that some players only started to talk to Nao, if they were encouraged to do so. Whereas others did not need the extrinsic motivation.

Evaluation

We asked all our participants (7-year old, 12-year old, 14-year old, 20-year old, two 21-year olds, and a 51-year old) to have a short interview with us about their experience with the game and the robot, after their trial. We conducted the interview in Dutch.

Some interesting things from the interviews were that all children felt comfortable in the company of the robot. Only the 20-year old student was a little uncomfortable because Nao was constantly looking at her and the 51-year old was also uncomfortable in the company of the robot. However, everyone liked doing the game with the robot. But after asking whether they would prefer to play the game with a human or the robot, only the 7-, and 12-year old preferred playing the game with the robot, the rest preferred playing it with humans. Everyone though the game was either too easy or about right for children between the ages of 7 and 9. Which was interesting because after talking to a specialist, she mentioned that our game might be too hard for our target group, because it demands skills that are not necessarily needed for collaboration. For example, knowing what is left and right, being able to read, and understanding a 2D representation of a city. These things might be difficult for children in general and especially for autistic children. Furthermore, everyone experienced the game as collaboration. Except for the 14-year old and the 21-year old. The 14-year old said working together on a school project is what she considered as collaborating. And the 21-year old thought the game was too one-sided, because you are the only one doing something, meaning putting pieces on the map. Interestingly, only the 7-year old, 51-year old and 20-year old, noticed that Nao was looking at them. Lastly, the 3 children thought Nao worked autonomously, while the students and 51-year old thought he was being controlled by an experimenter. Some of the students already knew Nao could not work autonomously, because they had worked with him before.

Below a link to the interviews.
File:Evaluation after test trials.pdf

Week 1

Problem definition

In week 1 we got the first lecture of the course. During the lecture an overview of the course was given. Then, groups were formed and during our first meeting we brainstormed about possible robot technologies we could use as our subject. We created a mindmap with several branches of interest. After eliminating most of them, were hesitating between robots for children with autism, robots for rehabilitation purposes and a persuasive dieting robot.
We ended up choosing robots for children with autism. At the end of the meeting we scheduled another meeting for which each of us had to come prepared by looking up literature on our topic.

During the second meeting we further elaborated on the subject. Where we wanted to go and what we already knew about current technologies involving children with autism. We talked about our objectives and the approach.

File:Notulen Week1 9feb.pdf

Week 2

On monday the first presentation took place and we presented our idea.

File:Presentatie1 Week2.pdf

In the second week we continued with the literature study, to get a good view of the state of the art. And we also searched for the answers on the following questions:

• What is autism?
  
• Why can robots help children with autism?
  
• Which social skills are important for teamwork?
  

Via our literature study, we wanted to find out what already exist and what was missing in this research field. We found that there are already several robots that help children to learn to recognize facial expressions or that learn children how to react on different kinds of behavior. We concluded that another important social skill that autistic children find hard to develop is teamwork and collaboration. However, here was little research done. So we wanted to find out in what way robots could help autistic children during the development of this social skill.

Now we will interview specialists in this field to gather more information about the topic and finally we want to create our own game with Nao that can help autistic children to develop their teamwork skill.

File:Notulen Week2 16feb.pdf

Week 3

On Monday of week 3 we presented our progress to the other groups. We did this with more details and showed our planning of upcoming weeks.
File:Presentatie2 Week3.pdf

Interview with Barakova

In week 3 we had a meeting with Emilia Barakova, in which we asked some question about her research. She suggested some possibilities for our game. Thursday we will discuss her suggestions and determine what sort of game we want to develop.

File:Transcriptie.pdf

File:Notulen Week3 23feb.pdf

Week 4

For this week all of us came up with a game idea. We discussed our ideas, but could not make a decision yet. Then we had our first meeting with the tutors. They advised us to set up a list of requirements for the game, so we could see which game satisfied the requirements, making it easier to choose one.

On Thursday we concluded that none of the games completely satisfied the requirements. And so the next task was to think of ways in which we could adjust our game ideas, so they would fit the requirements.

File:Notulen Week4 6maart.pdf
File:Notulen Week4 9maart.pdf

Week 5

Everyone came up with a solution and we discussed the new game ideas. We are still doubting about which one to choose. After the tutormeeting the tutors advised us to make a table with the game ideas against the requirements, so we could checkmark each requirement per game. The game with the most checkmarks will be selected.

File:Notulen Week5 13maart.pdf
File:Notulen Week5 16maart.pdf

Week 6

We elaborated on the game we selected as most promising in week 5. We discussed the sketches of the design with the tutors and we faced some question that still need to be answered before doing the actual test. We are going to work on the following things:

• Further expand the game concept
• More precisely decribe the role of Nao and think of questions he should be able to say and/or answer
• Furhter examine the programming of Nao
• Think of ways to measure and test the success of our game during the trials.

File:Notulen Week6 20maart.pdf
File:Afspraken Week6 20maart.pdf
File:Notulen Week6 23maart.pdf

Week 7

During week 7 we finished our game design and conducted the trials with our participants. We analyzed the video material and evaluated with the participants. We also talked to a specialist (Iris Soute) about the success of our game, how it would perform if it were to be implemented in the real world. After that we thought about future research and we discussed, evaluated, and concluded our project. Lastly, we worked on the final presentation and made sure all our progress was put on our wiki page.

File:Notulen Week7 27maart.pdf
File:Afspraken Week7 27maart.pdf

File:Afspraken Week7 30maart.pdf
File:Aantekeningen Interview Iris Soute.pdf

Week 8

During the last week of this project we held our final presentation, which was a view into our project and all the steps we took the last weeks. We saw this presentation as a last evaluation of the work we did. The crowd was very positive and did not have unexpected comments.
File:Presentatie3 Week8.pdf

There are still some parts that need to be uploaded, to make complete wiki. Then we can review the complete wiki and successfully end this project.

On Thursday we will have a last meeting in which we will discuss our last to-do points and conduct the peer review. The minutes of the peer review meeting will be sent to the tutor.

File:Afspraken Week8 3april.pdf

References

Papers

• Barakova, E.I. & Lourens, T. (2009). Mirror neuron framework yields representations for robot interaction. Neurocomputing, 72(4-6), 895-900.
  
• Breazeal, C.: Sociable machines: Expressive social exchange between humans and robots. PhD thesis, Massachusetts Institute of Technology (2000)
  
• Costa, S., Soares, F., & Santos, C. (2013). Facial expressions and gestures to convey emotions with a humanoid robot. Paper presented at the , 8239 542-551. doi:10.1007/978-3-319-02675-6_54
  
• Costa, S., Lehmann, H., Robins, B., Dautenhahn, K., Soares, F.: Where is your nose?-developing body awareness skills among children with autism using a humanoid robot. In: The Sixth International Conference on Advances in ComputerHuman Interactions, ACHI 2013, pp. 117–122 (2013)
  
• Gillesen, J., Boere, S., & Barakova, E. (2010). wikitherapist. Paper presented at the 373-374. doi:10.1145/1962300.1962390
  
• Giusti, L., Zancanaro, M., Gal, E., & Weiss, P. L. T. (2011, May). Dimensions of collaboration on a tabletop interface for children with autism spectrum disorder. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 3295-3304). ACM.
• Huijnen, Claire A. G. J, Lexis, M. A. S., & de Witte, L. P. (2016). Matching robot KASPAR to autism spectrum disorder (ASD) therapy and educational goals. International Journal of Social Robotics, 8(4), 445-455. doi:10.1007/s12369-016-0369-4
  
• Mazzei, D., Billeci, L., Armato, A., Lazzeri, N., Cisternino, A., Pioggia, G., Igliozzi, R., Muratori, F., Ahluwalia, A., De Rossi, D.: The face of autism. In: The 18th IEEE International Symposium on Robot and Human Interactive Communication, RO-MAN 2010, pp. 791–796 (2010)
  
• Salvador, M. J., Silver, S., & Mahoor, M. H. (2015). An emotion recognition comparative study of autistic and typically-developing children using the zeno robot. Paper presented at the 6128-6133. doi:10.1109/ICRA.2015.7140059
  
• Scassellati, B., Admoni, H., & Matari, M. (2012). Robots for use in autism research. Annual Review of Biomedical Engineering, 14(1), 275-294. doi:10.1146/annurev-bioeng-071811-150036
  
• Tucker, E. (2015, February 01). How robots are helping children with autism. Retrieved February 22, 2017, from https://www.theguardian.com/lifeandstyle/2015/feb/01/how-robots-helping-children-with-autism
  
• Torres, N., Clark, N., Ranatunga, I., & Popa, D. (2012). Implementation of interactive arm playback behaviors of social robot zeno for autism spectrum disorder therapy. Paper presented at the 1-7. doi:10.1145/2413097.2413124
  
• Landa, R.J. (2008). Diagnosis of autism spectrum disorders in the first 3 years of life. Nature Clinical Practice Neurology. doi: 10.1038/ncpneuro0731
  
• Autisme in het kort. (n.d.). In NVA. Retrieved from http://www.autisme.nl/over-autisme/wat-is-autisme-(spectrum-stoornis)/.aspx
  
• Autism Spectrum Disorder. (n.d.) In NIH (National Institute of Mental Health). Retrieved from https://www.nimh.nih.gov/health/topics/autism-spectrum-disorders-asd/index.shtml?utm_source=rss_readersutm_medium=rssutm_campaign=rss_full
  
• Marwecki, S. Radle, R., & Reiterer, H. (2013, April). Encouraging Collaboration in Hybrid Therapy Games for Autistic Children. CHI 2013: Changing perspectives
  
• Goodrich, M.A. et al. (2012, March/April). Incorporating a Robot into an Autism Therapy Team. IEEE Computer Society
  
• Giullian, N. et al., (2012, October). Detailed requirements for robots in autism therapy. doi: 10.1109/ICMC.2010.5641908
  

Websites

• Author Archives: admin. (n.d.). Retrieved February 22, 2017, from http://robotica-autismo.dei.uminho.pt/index.php/author/admin/
  
• Can Robots Feel Your Pain? (n.d.). Retrieved February 22, 2017, from http://greatergood.berkeley.edu/article/item/can_robots_feel_your_pain
  
• Learn More About Milo -. (n.d.). Retrieved February 22, 2017, from http://www.robokindrobots.com/purchase-milo/
  
• Robots4Autism -. (n.d.). Retrieved February 22, 2017, from http://www.robokindrobots.com/robots4autism-home/
  
• Wiki Therapist. (n.d.). Retrieved February 22, 2017, from https://www.tue.nl/en/research/research-institutes/robotics-research/projects/wiki-therapist/
  

Appendices

A: Planning

A1: Sections

To make sure that we manage our project in the best possible way we made a planning in which we incorporated everything we wanted to do. At the end of the eight weeks of the project we want to have two deliverables a literature study and our own game.
Since the knowledge of the literature study is needed to develop a game, we decided that the literature study had to be finished by the end of week 3. This is also our first milestone.
There are a couple of reasons why we first want to do a literature study. Firstly, we want to have a good and complete overview of what is already developed and what researches have already done. Our goal is to develop a new game and not to duplicate or further develop an existing technology. Secondly, this gives us a good idea which niches we have to further investigate for our own game. This is a lot of work but should be feasible within 3 weeks.
The second part of our research will be qualitative research. More specifically: interviews with specialists. The first one to interview is Emilia Barakova. We thought it would be nice if we could speak to her as soon as possible since she has done of research about this topic as well. That is why we planned an appointment with her at the beginning of week 3. Of course we also planned in some time to prepare the interviews and to transcribe and code them. This should be finished by the end of week 5.
With the knowledge of the literature study and the interviews we can develop our own game. This last phase will consist of a design phase and a realization phase. We planned to start this phase already at the beginning of week 3, because by that time we will already have sufficient knowledge to take the first steps in our design. The design should be finished by the end of week 7. This is our second milestone. The last week will be reserved for any delays and the cleaning up of the wiki. So just the finishing touch. The third and last milestone represents the end of our project. This will of course be at the end of week 8.

A2: Gantt chart

To make this planning as clear as possible we made a Gantt chart out of it. For this Gantt chart we divided all the work we had to do, as discussed above, in five sections. These sections are: presentations, literature study, qualitative research, implementation and finishing touch. Out of this basic planning we made a Gantt chart. This was done with Smartsheet (https://www.smartsheet.com/). The sections and its subtasks and the Gantt chart can be seen in the appendix. The grey horizontal bars represent the sections as explained earlier and the green bars below each grey one represent its subtasks. Some bars are connected with arrows. These represent a causal relationship: which task is the follow up task of another, i.e. what has to be done before a task can be started. In week 8, it can be seen that everything comes together in the last milestone: project finished! At this point in time, all the sections are finished which means that all the tasks specified are done. Note: the week of Feb 27 is made red because that is the week of the spring holydays. Since this is not an official school week we decided not to plan any deadlines in that week.

B: Informed Consent

We used two different informed consent forms. The first one is applicable if the participant is under 18. Then a parent/caregiver has to sign the form for them. This form can be found here: File:Informed Consent Form Child.pdf. The second one is applicable if the participant is over 18. In this case they can just sign the form themselves. This form can be found here: File:Informed Consent Form Adult.pdf.