PRE2016 3 Groep18: Difference between revisions

From Control Systems Technology Group
Jump to navigation Jump to search
 
(119 intermediate revisions by 5 users not shown)
Line 23: Line 23:
| 0888603
| 0888603
|}
|}


== Introduction ==
== Introduction ==
Line 31: Line 30:


== Subject ==
== Subject ==
A first step towards this robot-teaching concept is the use of an assistive robot. A robot as an addition to the current teachers. Today the elementary school is all about keeping children on the same level. The current teachers focuses too much on the children that needs more attention or are bad at a subject, but the children that don’t need it and are good at a subject get less attention. A robot can help these ‘better’ students to learn as much as they can. With the robot they can learn on a fun way and on a personal level, this way the children will always be motivated to learn.  
A first step towards this robot-teaching concept is the use of an assistive robot. A robot as an addition to the current teachers. Today the elementary school is all about keeping children on the same level. The current teachers focuses too much on the children that needs more attention or are bad at a subject, but the children that don’t need it and are good at a subject recieve less attention. A robot can help these more advanced students to learn as much as they can. With the robot they can learn more using a fun way and on a personal level, this way the children will always be motivated to learn. Fir this we have set a few goals which will allow us to achieve this goal.


== Objectives ==
== Objectives ==
=== User ===
*''' Fun learning''': To motivate children to learn, we want to make the learning experience more fun. That way, the child will not loathe studying and will enjoy it.
* Fun learning: To motivate children to learn, we want to make the learning experience more fun. That way, the child won't loathe studying and will actually enjoy it.


* Provide personalized learning experience for the students: Every child is different, but currently most of the school system is the same for every student. We want to provide a personalized learning experience for every child. Students who have trouble with the curriculum will get more time and exercise to study, while students who quickly master the material will get more advanced exercises. Students with learning disabilities can be helped easier and more direct using the robot
*''' Provide personalized learning experience for the students''': Every child is different, but currently most of the school system is the same for every student. We want to provide a personalized learning experience for every child. Students who have trouble with the curriculum will get more time and exercises to study, while students who quickly master the material will get more advanced exercises.
 
 
=== Society ===
* Making young students be more comfortable with interaction with robots: It's almost impossible to picture a future where there are no robots. Even now robots are already in the home of people. Our education robot will make the children more comfortable around the robots. The child will have fun when playing (learning) with the robot and thus will feel more comfortable around robots. Because of these happy experiences with a robot, the child will feel more comfortable around robots in general.


== USE aspects ==
== USE aspects ==
=== Users ===
=== Users ===
The primary users for this project are the children of the elementary school who use the robot. The robot has to be able to teach the children something. Children will learn faster when they are motivated, the robot uses social cues and will speak in order to motivate the child. In order to keep the child motivated a personalized learning system is needed. A personalized learning system can help motivate a child by letting him learn on his own level, for instance when the child is bad at an subject the robot helps him with that subject, to better understand it. But also when a child is smarter than what the teacher teaches him, he can learn more with the robot.  
The primary users for this project are the children of the elementary school who use the robot. The robot has to be able to teach the children. Children will learn faster when they are motivated, the robot uses social cues and will speak in order to motivate the child. In order to keep the child motivated a personalized learning system is needed. A personalized learning system can help motivate a child by letting him learn at his/her own level, for instance when the child is bad at a subject the robot helps him with that subject, to better understand it. Even when the student is at a higher level of understanding than most others within the classroom, the robot is albe to push the student further.


The secondary users are the teachers of the elementary school. The robot will be implemented in the back of the classroom, so the teacher has to know how the robot works when something fails for instance.
Elementary school teachers are the main secondary users. The robot will be implemented in the hallway next to the classroom, thus the teacher has to know how the robot works when something fails for instance. Furthermore, the teacher ensures that the progress of the student is inline with the curriculum and that it does not deviate greatly.


A tertiary user can be a parent of the child. A parent wants what is best for their child, and wants that the child learns as most as he can. The robot helps the child to learn on their personal level so the child will learn as much as he is able to learn.
A tertiary user can be a parent of the child. A parent wants what is best for their child, and wants that the child learns as most as they can. The robot helps the child to learn on their personal level so the child will learn as much as he is able to learn.


===Society===
===Society===
The society will benefit from the robot, because children starts using a robot with a young age. This way they know how a robot works and know what to expect from it. In the future more and more robots will be used so a child has to be comfortable around robots.
The society will benefit from the robot, because children come in contact with robots at a young age. This way they know how a robot works and know what to expect from it. In the future more and more robots will be used, so a child has to be comfortable around robots.


=== Enterprise ===
=== Enterprise ===
Line 59: Line 53:


== Approach ==
== Approach ==
The goal of our project is to make young students learn on a fun way and on a personal level, by implementing a teaching robot  
The goal of our project is to make young students are taught on a level that is personal to them in a fun way, by implementing a teaching robot  


We will do this in the following order
We will do this in the following order
Line 68: Line 62:
* Experiment with the robot.
* Experiment with the robot.
* Try to implement an algorithm which will make the robot able to perform the chosen task(s).
* Try to implement an algorithm which will make the robot able to perform the chosen task(s).
* Refine the algorithm/reformulate task robot.  
* Refine the algorithm/reformulate task robot.


== State-of-the-art ==
== State-of-the-art research ==
=== Social cues and gestures ===
=== Social cues and gestures ===
Social cues and gestures are very important in any conversation whether that is human-human interaction or human-robot interaction. For this reason, research has been made for the interaction between humans and robots. It was shown that interactions between robots and humans have resulted in better message retention. In the classroom environment, this is needed. (Chapter 6, Torta, 2014)
*Social cues and gestures are very important in any conversation whether that is human-human interaction or human-robot interaction. For this reason, research has been made for the interaction between humans and robots. It was shown that interactions between robots and humans have resulted in better message retention. In the classroom environment, this is needed.<ref>Torta, E., Juola, J. F., & Cuijpers, R. H. (2014). Approaching independent living with robots</ref>
=== Appearance ===
To ensure that child-robot interaction is successful, it must be ensured that the children are comfortable with interacting with the robot. For that, the appearance of the robot must be carefully designed. Children respond better to human-like features but not to a completely humanoid robot appearance. (Fang-Wu Tung, 2016)
=== Role-switching mechanism ===
‘Role-switching’ as an intrinsic motivational mechanism facilitates the engagement in long-term child–robot interaction. In this study, this was done in the specific learning framework of improving knowledge and habits with regards to healthy-life styles, though the article suggests that this can be applied in other learning frameworks as well (Ros, R., Oleari, E., Pozzi, C. et al. Int J of Soc Robotics (2016)
=== Robots in the classroom ===
Most special needs schools cannot afford the NAO. The paper investigates the non-humanoid Lego Mindstorm robot as an alternative learning method for pupils with intellectual disabilities. An interview at the end concludes that three out of four students were significantly more engaged with the non-humanoid robot than the humanoid robot, whilst one student was found to be equally engaged with both robots. (Sarmad Aslam, Penny J. Standen, Nick Shopland, Andy Burton, David J. Brown. iTAG (2016)


== Interviews ==
*Since in the traditional classroom, the one-on-one interaction between the teacher and the students is decreased due to the format of the classrooms, it is necessary to have the robot be able to make gestures and make use of social cues. Since for human-human interaction, it comes naturally to most people to be able to take turns when a conversation takes place. Similarly, when a physical object is the topic of conversation, joint-attention takes place without the participants conscious decision However, these ideas are not inherent for a robot and must be used to make the conversation more natural.
===Interview elementary teacher in training===
'''With Eveline Engering'''


To gather more information about how teaching is done on elementary schools and whether a robot could be implemented in this kind of education, an interview is conducted. The first person we interviewed was Eveline Engering. Currently she is studying Pabo in the ‘Fontys Hogeschool Den Bosch’ to become an elementary teacher. This is why she had much experience in different elementary classes over the last two years. She worked in the following groups: 1&2, 5, 7 and currently she is teaching in group 3 in a dutch elementary school.
*It has been shown that when conducting a conversation with a robot that does not make use of these ideas, the conversation is less engaging. Subsequently, when the same things were tested with gestures and gaze, more of the information (“messages”) were retained better. In a classroom environment, this is necessary to allow for the person (in this case, the student) to retain more of the information that was discussed.


*Thus, it is necessary to make good use of gestures and gazes when there is interaction between the robot and the child.


The following questions were asked to gain more information about the basics of our project.
=== Appearance ===
To ensure that child-robot interaction is successful, it must be ensured that the children are comfortable with interacting with the robot. For that, the appearance of the robot must be carefully designed. Children respond better to human-like features but not to a completely humanoid robot appearance.<ref>Tung, F. (2016). Child perception of humanoid robot appearance and behavior. International Journal of Human-Computer Interaction, 32(6), 493-502. doi:10.1080/10447318.2016.1172808</ref>


'''When teaching elementary school kids, what are important (speech) patterns?'''
*Robot design is a crucial step in any process of introducing robots for interaction with humans. It was found that applying anthropomorphic designs to robots is not necessary for young children. The reason for this is that most young children are used to the toy-like appearance of the objects they see daily. In the study that was conducted, it seemed that Aldebaran’s Nao robot fared better visually than other robots with less human-like features but also better that very human-like robots.


There are a few qualities that a elementary school teacher needs to have. You need to be deductive, pedagogic sensitive and of course you need to master everything you are teaching. Besides these skills and knowledge it is very important to have a good relating with your students. If you know more about your pupils you can connect the things you are teaching to their direct world, and that is very effective.  
=== Role-switching mechanism ===
‘Role-switching’ as an intrinsic motivational mechanism which facilitates the engagement in long-term child–robot interaction. In this study, this was done in the specific learning framework of improving knowledge and habits with regards to healthy-life styles, though the article suggests that this can be applied in other learning frameworks as well.<ref>Ros, R., Oleari, E., Pozzi, C., Sacchitelli, F., Baranzini, D., Bagherzadhalimi, A., . . . Demiris, Y. (2016). A motivational approach to support healthy habits in long-term Child–Robot interaction. International Journal of Social Robotics, 8(5), 599-617. doi:10.1007/s12369-016-0356-9</ref>
=== Robots in the classroom ===
Most special needs schools cannot afford the NAO. The paper investigates the non-humanoid Lego Mindstorm robot as an alternative learning method for pupils with intellectual disabilities. An interview at the end concludes that three out of four students were significantly more engaged with the non-humanoid robot than the humanoid robot, whilst one student was found to be equally engaged with both robots.
<ref>Aslam, S., Standen, P. J., Shopland, N., Burton, A., & Brown, D. (2016). A comparison of humanoid and non-humanoid robots in supporting the learning of pupils with severe intellectual disabilities. Paper presented at the 7-12. doi:10.1109/iTAG.2016.9</ref>


When approaching pupils it is important that no negativity is used. When for example two children are sitten near to each other and one is really paying attention and the other is doing something else, you better compliment the pupil that is doing well instead of saying the other is doing something wrong. The pupil who wasn’t paying attention will experience the compliment as something he wants too.
== NAO Robot ==


Another thing  that is important in the educations of elementary school kids, is that the process is more important than the result. This means that the result could be worthless while the whole process was good. It is more important that the pupils learn the process to a result instead of focusing on the result. How this is done is by feed up, feedback and feedforward. In elementary schools this means that you start with feed up: telling the goal of the assignment (for example: Today we are going to finish chapter three). Next feedback will be given in the form of telling the pupils where they are at the moment (so for example: You are now at the beginning of chapter two), and finally give feed forward in the form of telling the pupils what they need to do to accomplish the goal.
=== What is NAO and how does it work? ===
[[File:Doc_naoqi_nao.jpg|thumb|200px|right|NAO robot fifth generation]]


'''Is it important to use some kind of body language while teaching elementary school kids.'''
NAO is an autonomous, programmable humanoid robot, which is being used for specific research and education purposes. There have been made multiple generations of the NAO, all with various versions of design, which all feature a platform with either 14, 21 or 25 degrees of freedom. To provide that NAO can position itself within space in a stable manner, all these generations feature an inertial measurement unit (IMU). This IMU is an electronic device that measures the robots angular rate and forces by means of a gyroscope, accelerometer and four ultrasonic sensors. NAO last generation, the ‘NAO Evolution’ in particular consists of a multilingual text-to-speech synthesis, voice recognition and sound localization, shape and facial detection and recognition and an anti-collision system. To control all these features, the robot uses a specialised Linux-based operating system, ‘NAOqi’. This OS powers the robots multimedia system corresponding with the mentioned features. This includes four microphones (voice recognition and sound localization), two speakers (multilingual text-to-speech synthesis) and two HD cameras (computer vision, in particular facial and shape recognition). <ref> https://www.ald.softbankrobotics.com/sites/aldebaran/files/press-releases/cp_nao_evolution_en_def.pdf </ref>


It is very important to have an open attitude. This is because all pupils need to feel fine to ask for help or anything else. Also positive gestures are important to complement what you are saying, these gestures can for example strengthen a compliment. This way they will learn to understand nonverbal gestures. While teaching gestures help to support the learning aspect by adding a visual aspect. Young children really benefit from this addition.
=== Academic & scientific usage NAO ===
As mentioned, the NAO is used in numerous research and educational settings: over 200 academic institutions worldwide have made use of the robot. An example is on a school in Birmingham, United Kingdom, where autistic children were taught by the NAO robots. One of the conclusions was that some of the children found the childlike, expressive robots more relatable than humans. In a more general setting, at various schools in the United Kingdom, the NAO is utilised to familiarise children with robots and the robots industry. Other usages of the NAO include customer service use in (Japanese) bank branches and testing dummy for robotic “autobiographical memory”, which in turn can train International Space Station (ISS) crews and assist elderly patients.


'''Do you have the feeling that you are understaffed at the school?'''
=== NAO & TiViPe ===
By default, the NAO comes with a software suite that includes a simulation software package, a software developers kit and a graphical programming tool called Choregraphe. However, there is alternative (graphical) software available, and for our project we came across TiViPe, which is graphical end user developing environment made by Tino Laurens. The software is ‘end-user’ in the sense that the environment itself is built by Tino, but within this environment users can develop software programs their selves. This is done by connecting ready graphical modules within the environment, so that no hard programming language skills are required. There is a distinction to be made between two types of modules: elementary modules and compound modules. Normally, the elementary modules are the pre-defined available modules made by Tino, the compound modules are created by the user self by merging elementary modules together. However, one can also create elementary graphical modules their selves by writing code and wrap/embed this code into a new module. <ref> http://www.tivipe.com/ </ref>


Yes I do. The increased size of classes today lead to less personal education. With large groups it is hard to focus on individual pupils. Nowadays the focus is to keep level of the group at one level, this means that the education is given at a mean level. In order to get everyone to the mean level of knowledge, an extra focus of the education is on the pupils who are under this level. Because of the large groups and focus on the pupils with a lower level, kids with a higher level receive few or none support in the current elementary school system. Sometimes they need to help the kids with a lower level, or get extra assignments which they need to make without support. In this way, according to Eveline it is possible that these smart kids don’t use their full potential, which is too bad.


Not only in the class rooms it is noticeable, but as well after school. Because of the increase in group sizes and therefore also the increase of paperwork the pressure is high.  
{|style="margin: 0 auto;"
| [[File:Nao_v5_dimension.png|thumb|400px|right|NAO robot technical drawing dimensions]]
| [[File:NaoRobot2merge.jpg|thumb|400px|center|Multiple connected elementary modules (green) are being 'merged' into one compound module (red)]]
|}


'''Do you think that some kids will benefit from personal learning?'''
=== NAO & YOLO ===
In the educational games the robot will play with the student, we will also utilise some objects (see below: for instance the dice with the summing up game). Therefore, our robot of choice should incorporate some sort of object recognition. It just so happens that TiViPE has an option to use such an object recognition on the latest generation NAO, named ‘You Only Look Once’ (abbreviated YOLO).  Stated as a state-of-the-art, real-time object detection system, YOLO uses a single neural network to scan the full image, where prior detection systems repurpose classifiers or localizers to come to a significant detection. The neural network splits the image up into smaller regions and comes with a prediction for bounding boxes and a probability per region, in that sense it is solving a regression problem. A claimed property of the YOLO object recognition by the developers is that it is “extremely fast”. This is relevant for us, since the processing of the NAO in a reaction should not be delaying the reaction to much, so that the interaction of the NAO and the student stays natural and interesting for the student. The basic YOLO database model processes images in real-time at 45 frames per second, which we assume to be fast enough. If not, we can always switch to the smaller database "fast YOLO", which processes a picture in 155 frames per second.  <ref>Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi. You Only Look Once: Unified, Real-Time Object Detection. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 779-788 </ref>


I think that personal learning will be a good thing for all pupils. When pupils are learning at their own level, learning will go most efficient. As said weak learners already get a bit extra attention, so it would be a good thing to introduce personal learning by the smarter kids.


'''Which group of kids will benefit the most of personal learning?'''
[[File:YOLOModel2.png|thumb|1300px|center|Recognising objects in picture using YOLO]]


The smart kids are less helped by today's system! So in Eveline’s view they will benefit the most when adding personal learning in our current system.
== Interviews ==


'''Which subjects are most important for this group of kids to learn, and in your view can be done by a robot?'''
[http://cstwiki.wtb.tue.nl/index.php?title=0LAUK0_PRE2016_3_Groep18_Interview Interview]: An overview of the interviews.


In my view only the theoretical subject can be taught by a robot like math, language and for example geography.  
We have learned several things from the interviews.  
 
Firstly, in the current educational system, the school is concerned with keeping the children on the same level, and the most when someone lacks behind keep them up to the standard level. In this way, the smarter children do not get as much attention, nor challenge, as they need. This is why we decided that our focus group are the more advanced children. The second thing we learned is that when teaching a child a new topic, we cannot us any negativity, but have to use positive feedback and constructive criticism. Furthermore, we need an open attitude, this way the child will be more motivated and positive about learning. Moreover, we learned that it is smart to use fun learning for a child, this is because they will have more intrinsic motivation. For younger students, teachers already use educational games to make learning fun. So a robot can use those games to teach a child something in our case. And it is good to use different tones in a conversation, this is also playable on the robot which can speak louder and softer. The last thing we learned from the interviews is that the process is always more important than the result.
'''Do you think that a robot in the classroom can add something in elementary school classes?'''
 
Yes, I think that a good function robot which is able to add some personal learning really can add something to elementary school classes. This because robots are really interesting for young people. I once made a robot out of cardboard and put it in front of the class and together brainstormed about the function of the robot. The whole group was really excited and enthusiastic about the robot. We called it henk and imagined it doing a lot of stuff. This was in group 7 so you can imagine what a real robot can contribute in elementary school. 
 
'''Do you think that kids will experience learning with a robot as fun learning?'''
 
Like I said in the previous question I definitely think that learning with a robot will be experienced as fun learning.
 
'''Do you think fun learning is an effective way of teaching?'''
 
Yes I really do think that it is an effective way of learning. This I know because we use it all the time to teach children things like math and languages.
 
Stick to their environment.
 
'''Do you use educational games to learn skills to kids?'''
 
We use educational games all the time, especially by the lower classes. One example of a educational game is that we throw two dices and let the kid add it up, or retract it from each other. This really learns the meaning of numbers to children. We use this around class 3 and 4, because around this age kids begin to understand what numbers are.
 
'''Which groups are most suitable for implementing personalised learning with a robot ?'''
 
I think that every kid in every elementary school class will find the robot very interesting. But the simple educational games will be more suitable for the lower classes. The robot could be interesting for higher groups if you implement some kind of programming game to teach children to simple program a robot. You can for example maybe make a game where the kid needs to program the robot by voice. And the robot will act on this just like the kid mentioned.
 
===Interview with occupational therapist===
'''With Fleur Bennenbroek'''
Fleur is an occupational therapist and works with children on the field of academic skills, fine motor skills and concentration. She works with all kind of children of the elementary school. After I explained our idea and vision I asked her some questions to focus on children's behaviour. She was very positive about our ideas and the project itself.  
In the beginning when explaining our idea she said that she thinks the higher lower game is too easy for the 3rd grade children because they can count to 10 already in 3rd grade. For the game with the dices with letters she advises to check out Boggle, because that is a game she plays sometime with children. She thinks that one of the most important things is that the child has to actively participate in a game like throwing the dice it self. Because children like to do things theirself.
 
 
Questions:
 
'''When teaching elementary school kids, what are important (speech) patterns?'''
 
Some kind of incentives come more direct to the children based on noise. Speaking is more direct, different tones helps the child to keep attention to the robot. Strong incentives are  more direct and the child can keep attention better.
'''Is it important to use some kind of body language while teaching elementary school kids.'''
 
Children like to do things or copy things like moving. So it is smart to let the child show something like body language. Or let the child really play with the robot. She suggested something like that the robot has to move their arms 10x in the air and the child has to do it also himself and has to count along.
She also advises that the children has to be as active as possible when working with a robot.
'''Which subjects are most important for this group of kids to learn, and in your view can be done by a robot?'''
 
Writing, math and language. Writing is very hard because some handwritings of children she can’t read herself let alone a robot can do it. Language and math is a good idea to do with a robot.
Recognizing letters is something for group 3. Something like showing a letter and the child has to write is himself on a ipad for instance can maybe be a good idea.
 
'''What is the best way to learn a child something?'''
 
It is per child different but she said that moving helps a child to learn better and that they remember it better, this was scientifically proven she said. A lot of children are  sitting slouched or are sitting on 2 legs from the chair, this is because children seek movement (especially boys). She thinks the robot can help these children get movement.
'''How will a child react to something new?'''
 
It depends on how interesting it is. Every child finds a robot interesting and fun, but maybe a few can be scared by the robot.
'''Do you think a robot can teach a child something/ what?'''
 
Yes, especially math or spelling a robot can teach, but spelling is hard because there are so many rules like Jantje snoeper or using dt etc. She thinks the block game is a good idea to teach the children spelling.
'''Does fun learning really helps a child to learn something?'''
 
Fun learning helps a lot to learn a child something extra. A game helps a child to learn something because the child has more intrinsic motivation. When the motivation only apply to the teacher a child can learn less. When children are coming to Fleur to learn writing or reading, the most doesn't like it in the beginning because they are not good at it. When she tries to do it in a fun way then the child is more motivated, and the child can actually learn a lot then. An unmotivated child is hard to learn something to, it has to come from both sides.
 
'''Do you think we can learn all the classes of the elementary school something with a robot?'''
 
Yes you can use a robot in any case, it is only the skill level what is different to teach a child really something. Even children of group 8 does find a robot interesting. The programming game is a good idea, because the child really has to remember something. The child finds if very fun when the robot is moving based on what they said.
 
'''Is the robot applicable on your job?'''
 
I don’t think so because occupational therapy is especially based on fine motor skills, fingers and writing. It is very hard to apply these to a robot, because the robot has to detect something of the child how it is dealing with something or does something.


== Setting ==
== Setting ==
The setting in which the robot operates is in the hallway next to the classroom. The teacher can semi-supervise the child which is playing with the robot, also other teachers walk by and can also semi-supervise the child. This way it won't be disturbing for the other children of the class and they can focus on what the teacher has to say.  
The setting in which the robot operates is in the hallway next to the classroom. The teacher can semi-supervise the child which is playing with the robot, also other teachers walk by and can also semi-supervise the child. This way it won't be disturbing for the other children of the class and they can focus on what the teacher has to say.  
For now we are only focussing on the smarter children, the most time when a child is sent to the hallway to do other stuff, it always have to obligation to work as hard as it can, because the teacher can show up at any time. And the child wants to prove himself that next time he can go as well. We think that children are very interested in playing with the robot and thereby will behave themself when playing with the robot.
For now we are only focusing on the smarter children, the most time when a child is sent to the hallway to do other stuff, it always have to obligation to work as hard as it can, because the teacher can show up at any time. And the child wants to prove himself that next time he can go as well. We think that children are very interested in playing with the robot and thereby will behave themselves when playing with the robot.


== General scenarios ==
== General scenarios ==
=== Math helping robot by dice game.===
=== Math helping robot by dice game.===
A child throws a dice, the robot can recognizes the pips of the dice and the robot ask for instance what is the total number thrown, or what is that dice multiply by that dice if they are more advanced. This way there will always be a random factor, which is more fun for the child and doesn't get boring.  
A child throws a dice, the robot can recognizes the pips of the dice and the robot ask for instance what is the total number thrown, or what is that dice multiply by that dice if they are more advanced. This way there will always be a random factor, which is more fun for the child and doesn't get boring.  
Various dice games are possible, for instance some harder dice game to keep the student on a higher level. A easy game for the children of group 3 and 4 is higher lower game, and a bit harder game for instance is a game where the child has to add the 2 dices up.
Various dice games are possible, for instance some harder dice game to keep the student on a higher level. A easy game for the children of grade 3 and 4 is higher lower game, and a bit harder game for instance is a game where the child has to add the 2 dices up.
The child will understand math better by playing the game multiple times, it will be a fun way for the child and keep them on a personal level.  
The child will understand math better by playing the game multiple times, it will be a fun way for the child and keep them on a personal level.  


But of course there can be risks in using a robot to teach a child something about math. For instance the robot malfunctions, the software crashes or the robot is broken. These are all situations which are needed to be taken into account.  
But of course there can be risks in using a robot to teach a child something about math. For example, the robot malfunctions, the software crashes or the robot is broken. These are all situations which are needed to be taken into account.  
Also there can be a lot of undesired situations. When the flow is not going as expected and the robot has to hop into a fallback and later go back to the standard flow. For instance:
Also there can be a lot of undesired situations. When the flow is not going as expected and the robot has to hop into a fallback and later go back to the standard flow. For instance:
*The dice can be thrown off the table
*The dice can be thrown off the table
Line 209: Line 137:
These situations are just examples of things that can happen wrong. To make a good scenario most of these situations needed to be taken into account to program a working robot.
These situations are just examples of things that can happen wrong. To make a good scenario most of these situations needed to be taken into account to program a working robot.


 
===Spelling helping robot by making a word ===
===Spelling helping robot by making a word with blocks.===
There are multiple boxes with objects, one box with difficult objects and one with simpler objects. The child has to take an object from the box and show it to the robot, and the child has to make this word with blocks, or pieces of cardboard.
The robot says a word, and the child has to make this word with blocks, or pieces of cardboard.
The robot recognizes these blocks and to make it easier for the robot all the block can be made with using some sort of barcode. The robot scans the barcode and knows which letter is given.   
The robot recognizes these blocks and to make it easier for the robot all the block can be made with using some sort of barcode. The robot scans the barcode and knows which letter is given.   
The robot helps the child with spelling in this way. A database is needed in this case of all the words that can be made with these blocks, and if the barcode is easier recognized also with all the barcodes.To keep students on their personal level easier and more difficult words can be given by the robot.
The robot helps the child with spelling in this way. A database is needed in this case of all the words that can be made with these blocks, and if the barcode is easier recognized also with all the barcodes. To keep students on their personal level easier and more difficult words can be given by the robot.
The child will learn on a fun way to spell different words. Though it is hard for the robot to recognize the letters which are spelled. In this case also the robot can malfunction or crashes. But the undesirable situations are different in relation to the dice game. Some undesirable situations are:
The child will learn on a fun way to spell different words. Though it is hard for the robot to recognize the letters which are spelled. In this case also the robot can malfunction or crashes. But the undesirable situations are different in relation to the dice game. Some undesirable situations are:
*The robot does not recognize the object
*The robot does not recognize a letter
*The robot does not recognize a letter
*The child gives a letter upside down  
*The child gives a letter upside down  
Line 226: Line 154:
A concept educational game is made to challenge the smarter kids of elementary school from classes 7 and 8. This game makes it possible to let the kids program the robot by voice. The idea is that the robot stands on a table with some kind of physical maze which has tiles and walls. The robot will move through the maze. This movement will be decided by the kid. The goal of the game is that the robot reaches the finish tile in the maze.
A concept educational game is made to challenge the smarter kids of elementary school from classes 7 and 8. This game makes it possible to let the kids program the robot by voice. The idea is that the robot stands on a table with some kind of physical maze which has tiles and walls. The robot will move through the maze. This movement will be decided by the kid. The goal of the game is that the robot reaches the finish tile in the maze.


The pupil can program the robot with a secual of actions. If the goal is not reached after this secual another secual of moves can be programmed by the kid. This will continue until the goal is reached.  
The pupil can program the robot with a sequence of actions. If the goal is not reached after this sequence another sequence of moves can be programmed by the child. This will continue until the goal is reached.  
To make this game more advanced and interesting for the child when he plays again, a bigger maze can be used. The normal beginning maze is 8x8 blocks. The child will learn to program a bit and intensively use their memory, because the robot will move after the child is done saying commands. Some example of commands are:
To make this game more advanced and interesting for the child when he plays again, a bigger maze can be used. The normal beginning maze is 8x8 blocks. The child will learn to program a bit and intensively use their memory, because the robot will move after the child is done saying commands. Some example of commands are:
*Move forward
*Move forward
Line 239: Line 167:
*When the robot is moving the child interrupts
*When the robot is moving the child interrupts


== Detailed scenarios ==
[http://cstwiki.wtb.tue.nl/index.php?title=0LAUK0_PRE2016_3_Groep18_DetailedScenarios Detailed Scenarios]: An overview of the detailed scenarios process.




===Practice with speech/pronunciation words ‘repeat the word’:===
===Math helping robot by dice game (higher lower)===
Robot speaks out a certain word and child has to repeat it. Based on voice recognition the robot either accept the word, or declines it. When declined, the robot asks to repeat the word again (“one more time”). When accepted, robot reacts with positive feedback gestures (i.e. thumbs up and a smile on display ‘face’).
The robot will play a dice game with a child of the elementary school grade 3/4. The robot will throw the dice and the child will throw the dice. The child has to repeat the numbers thrown and eventually has to say if he has higher or lower than the robot. When the child is correct he wins the game and can play another game of the same game. The child will learn how numbers work, and a bit of short term memory.
Improvements:
This is the most basic dice game there is, and maybe can be used in grade 2 as well, because we want to focus on the ‘smarter’ children.
Words that a hard for the child (not good recognised) are repeated.
Instead of only declining or accepting, intermediate states ratings are added, where difficulties with particular syllables are identified.
Remarks/critics:
Current level of speech recognition (not perfect) might actually be desirable, since this corresponds with
It is quite plausible that there is a difference in the speech recognition of the words that lie outside the ‘fault’ of the child
For instance, it is assumable that high pitched words are recognized easier by the robot than low pitched words.  
 


===Drawing and recognizing basic geometric forms and basic objects===
[http://cstwiki.wtb.tue.nl/index.php?title=0LAUK0_PRE2016_3_Groep18_higherlower Higher lower scenario]: The final higher lower scenario.
This game can be played in two parts:  
 
Part 1:
Robot names a basic object (or a certain geometric form) and asks the child to draw it on paper. The robot then looks at the paper and checks if the drawing corresponds with the real object.
Here, memorisation words & connection/linking them with their shapes and drawing skills of the child is tested. Distinguishing difficulties in this 2 learning areas might be done as follows: the child is first asked to draw the object himself. If the YOLO dataset doesn’t recognise the drawing (not drawn good enough), the robot gives the child a diagram/table with possible pictures, where one corresponds with the real object.
 
Part 2:
Now, the roles are exchanged in the sense that now, the robot shows a picture of a certain object to the child. Then, the child is asked to name the certain object. If he/she doesn’t know the name, the robot says n words, and the child has to the pick the right one.
 
== Detailed scenarios ==
===Math helping robot by a dice game (higher lower)===
The robot will play a dice game with a child of the elementary school group 3/4. The robot will throw the dice and the child will throw the dice. The child has to repeat the numbers thrown and eventually has to say if he has higher or lower than the robot. When the child is correct he wins the game and can play another game of the same game. The child will learn how numbers work, and a bit of short term memory.
This is the most basic dice game there is, and maybe can be used in group 2 aswell, because we want to focus on the ‘smarter’ children.  (NEW diagram)
 
Gestures used in this scenario:
*Wave: the robot waves, this gesture is used at the start and the end of the conversation
*Correct: the robot's eyes will turn green and the robot will give a thumbs up, the gesture is used in correct answers
*Wrong: the robot's eyes will turn red and the robot will move his head to the left and right. This gesture is used for wrong answers.
*Point: The robot will point at the dice when the child has to throw the dice, this gesture is used when the child has to throw the dice.
 
[[File:Flow_diargram_robot.png]]


===Math helping robot by dice game (summing up) ===
===Math helping robot by dice game (summing up) ===
This variant of the dice game is more advanced. And is also for the elementary school group 3/4 The children have to sum up the dice the robot throws and the child throws. This way they have to understand how numbers work and can use their short term memory. To fully implement the scenario we have to ask ourself a few important questions. For instance what are the risks when a child plays a game with a robot. An important risk is that the child can break the robot or just walks away from the robot, we will try to prevent this with a teacher as a semi supervisor (\ref setting).  
This variant of the dice game is more advanced. And is also for the elementary school grade 3/4 The children have to sum up the dice the robot throws and the child throws. This way they have to understand how numbers work and can use their short term memory. To fully implement the scenario we have to ask ourselves a few important questions. For instance what are the risks when a child plays a game with a robot. An important risk is that the child can break the robot or just walks away from the robot, we will try to prevent this with a teacher as a semi supervisor.
Also the game can be more complicated as well if a dice with more faces is used. This will be harder for the child and more fun. A dice with more faces mostly consists of the numbers rather than the dots of the number. To use those dices the robot needs to recognize the numbers of a small dice, and the robot gives the incorrect number a lot of the times.
The explanation of the summing-up game is in the link below. This includes the strings and actions used in the scenario.
Gestures used in this scenario:
*Wave: the robot waves, this gesture is used at the start and the end of the conversation
*Correct: the robot's eyes will turn green and the robot will give a thumbs up, the gesture is used in correct answers
*Wrong: the robot's eyes will turn red and the robot will move his head to the left and right. This gesture is used for wrong answers.
*Point: The robot will point at the dice when the child has to throw the dice, this gesture is used when the child has to throw the dice.


Summing up strings for the diagram below.
[http://cstwiki.wtb.tue.nl/index.php?title=0LAUK0_PRE2016_3_Groep18_Summingup Summing up scenario]: The final summing up scenario.


SUQ1:
=== Spelling helping robot by making a word===
*Wil je weten hoe het spel gaat?
The robot will play a game with a child, the child has to take out an object of one of the boxes, and the child has to spell this word with the given blocks. Some boxes are have more difficult objects this way we implement personalized learning. The spelling game uses deep object learning, this is a relatively new technique which is not used much in variant scenarios.
*Wil je de regels van het spel weten?
*Zal ik het spel uitleggen?


SUA1:
The blocks contain 6 sides with 6 different letters, like a dice but with letters. If the child spells the correct word the robot continues. When not the correct spelling is given it first checks the length of the word and then it will check the order of the letters. This is done in multiple stages and eventually the correct word is spelled.
*TUTORIAL
The child will learn to spell in a fun way with the blocks. And the robot will give positive feedback


SUQ2:
[http://cstwiki.wtb.tue.nl/index.php?title=0LAUK0_PRE2016_3_Groep18_Spellinggame Spelling scenario]: The final spelling scenario.
*Snap je de regels van het spel?
*Snap je het?
*Heb je het allemaal begrepen?


SUA2:
===Maze programming game===
*Kom, laten we beginnen! Jij begint met het gooien van de dobbelsteen!
A concept educational game is made to challenge the smarter kids of elementary school from classes 7 and 8. This game makes it possible to let the kids program the robot by voice. The idea is that the robot stands on a table with some kind of physical maze which has tiles and walls. The robot will move through the maze. This movement will be decided by the kid. The goal of the game is that the robot reaches the finish tile in the maze.
*We gaan beginnen! Jij gooit de dobbelsteen als eerste!


SUQ3:
The pupil can program the robot with a secual of actions. If the goal is not reached after this secual another secual of moves can be programmed by the kid. This will continue until the goal is reached.
*Welk getal heb jij gegooit?
*Welk getal staat er op de dobbelsteen?


SUA3:
[http://cstwiki.wtb.tue.nl/index.php?title=0LAUK0_PRE2016_3_Groep18_programminggame Programming game scenario]: The final programming game scenario.
*Goed gedaan!
*Goed zo!
*Super!
*Top!
*Hartstikke goed!


SUA4:
== Conclusion ==
*Niet helemaal goed! Probeer het opnieuw.


SU
Elementary school classes become more crowded every day. The overcrowded classes will result in couple of negative effects. The first negative effect is that the education becomes less personal. The teacher does not have the time to pay attention on every individual kid, which could have negative effects on personal growth of kids. From interviews with teachers we know that the current educational system on elementary schools in the Netherlands solely focusses on the kids who perform under the mean level of the class. The higher performing kids get less attention and possibly will not reach their full potential. This is why the target group of this project will be high performing kids on elementary schools. We want to improve their level and try to achieve their maximal potential.


[[File:Summing_up_scenario_-_detailed.png]]
This problem could not simply be tackled by hiring more teachers to make classes smaller, because there aren´t more teachers available to achieve this. This is why a different solutions need to be found. In this project we focus on solving this problem by implementing a robot as addition to the current system . This robot will focus on individual kids and therefor introduce personal learning. The kids will not get lessons from the robot, but will learn by playing educations games. This way the kid not only learns from its experience with the robot, but also have fun while doing so.


=== Spelling helping robot by making a word===
After a clear definition of the objectives fun learning in a personal way, educational games where created. Two of the games focus on learning how to calculate. One game focuses on the spelling together with coupling objects to words, and one game introduces the basics of simple programming. First scenarios where made which later where converted to state diagrams. The information that was gathered from the interviews where used as basis for the games. For example the process of reaching the goal has more priority than really reaching the goal. Also gestures where used to make interacting with the robot more natural. Also the YOLO deep learning algorithm has been implemented in the spelling game to have a more natural interaction and make it more easy for the kid to connect words to objects. More usage of the YOLO deep learning algorithm in the basics of all of our educational games could make the interaction more natural, which will improve the experience of learning with a robot.
The robot will play a game with a child, the robot gives a word and the child has to spell this word with the given blocks. The blocks contain 6 sides with 6 different letters, like a dice but with letters. If the child spells the correct word the robot continous. When not the correct spelling is given it first checks the length of the word and then it will check the order of the letters. This is done in multiple stages and eventually the correct word is spelled.
The child will learn to spell in a fun way with the blocks. And the robot will give positive feedback
[[File:Spelling_game.png]]


===Maze programming game===
The scenarios and state diagrams where ready to be tested at the end of this project, so a beginning of programming the dice game in the TiViPe program was started. Sadly this didn’t work out as planned and little time was left to make it work. We are certain that the state diagrams form a good basis to program the NAO robot for educational games, and further research should begin where this project had ended.
A concept educational game is made to challenge the smarter kids of elementary school from classes 7 and 8. This game makes it possible to let the kids program the robot by voice. The idea is that the robot stands on a table with some kind of physical maze which has tiles and walls. The robot will move through the maze. This movement will be decided by the kid. The goal of the game is that the robot reaches the finish tile in the maze.
 
The pupil can program the robot with a secual of actions. If the goal is not reached after this secual another secual of moves can be programmed by the kid. This will continue until the goal is reached.
 
The simple state diagram of this game is displayed below.
 
[[File:Flow_diargram_robot_programming_game.png]]


== Planning ==
== Planning ==
Line 342: Line 215:
*Interview with elementary school teacher(s)
*Interview with elementary school teacher(s)
*Detailed scenarios
*Detailed scenarios
*Behaviour models
*Final demo
*Final demo


== Planning ==
=== Planning ===
[[File:gantt chart USE.png]]
[[File:gantt chart USE.png]]


Line 352: Line 224:
**Rationale of the project
**Rationale of the project
**Stakeholders & USE aspects
**Stakeholders & USE aspects
**Research
**State-of-the-art
**State-of-the-art
**Interview(s)
**Interview(s)
**Scenarios
**Our process
**Our process
*Scenarios
*Code
*Code
*Demo (simulation/real life)
*Demo (simulation/real life)


== Logbook ==
== Logbook ==
Line 375: Line 244:
* The second presentation was made which holds the milestones, planning and the deliverables.
* The second presentation was made which holds the milestones, planning and the deliverables.
=== Week 4 ===
=== Week 4 ===
* An interview was done with an elementary school teacher in training, who had teached group 1&2, 3 and 7. She was very positive about the our idea.
* An interview was done with an elementary school teacher in training, who had teached grade 1&2, 3 and 7. She was very positive about the our idea.
* Another interview was done.
* Another interview was done.
* We came up with more scenarios which we are going to work out further.
* We came up with more scenarios which we are going to work out further.
* We made 2 detailed scenarios.
* We made 2 detailed scenarios.
* Feedback session:
* Feedback session:
 
=== Week 5 ===
=== Week 5 ===
* Came up with more scenarios
* Came up with more scenarios
* worked out more detailed scenarios
* Worked out more detailed scenarios
* had a interview with a occupation therapist
* An interview was done with an occupational therapist, who works with children. She was very positive about our idea
* weekly feedback session:
* Weekly feedback session: these scenarios are basic try to improve those


=== Week 6 ===
=== Week 6 ===
* Weekly feedback sessions
* Updated the wiki
* Made one final scenario
* Tried this scenario with one playing as the robot and one playing as the child.
* Discussed about the setting
* Feedback session: the final scenario is a flowchart rather than a state-diagram. Make a state diagram next.
 
=== Week 7 ===
=== Week 7 ===
* Weekly feedback sessions
* Updated the wiki
* Programming on the Nao was done
* Final state diagrams were made
* Final strings/gestures were made
* Weekly feedback session: We have to hurry with the programming if we want to demonstrate that next week.
 
=== Week 8 ===
=== Week 8 ===
* Final presentation
* Programming on the Nao was done
* Final state diagrams were updated
* Final strings/gestures were updated
* Final presentation was made
* Wiki was updated


== References ==  
== References ==  
* Torta, E. (2014). Approaching independent living with robots Eindhoven: Technische Universiteit Eindhoven DOI: 10.6100/IR766648
 
* Fang-Wu Tung (2016) Child Perception of Humanoid Robot Appearance and Behavior, International Journal of Human–Computer Interaction, 32:6, 493-502, DOI: 10.1080/10447318.2016.1172808
<references/>
* Ros, R., Oleari, E., Pozzi, C. et al. Int J of Soc Robotics (2016) 8: 599. doi:10.1007/s12369-016-0356-9

Latest revision as of 10:27, 13 April 2017

Group members

Name Student ID
L.H.M.M. Abuerban 0926850
N.J. Bennenbroek 0954352
C.R.C. van der Gracht 0815638
R.M.H van Hout 0814519
S.H.N. van der Linde 0899183
J.R. Michels 0888603

Introduction

It’s the first day after summer vacation. Kids are running excitingly, at least most of them, to their classrooms. The door swings open as they rush to their seats. But instead of a human being in front of the class, the only thing they see is a robot. They are greeted by the robot, combined with a welcoming gesture.

Imagine this futuristic idea of teaching in elementary school. Teachers completely replaced by educational robots. Educational robots that can communicate with either an individual or a complete class by speaking, moving and making gestures. Artificial Intelligent software, like motion tracking and speech recognition assists within this overall communication.

Subject

A first step towards this robot-teaching concept is the use of an assistive robot. A robot as an addition to the current teachers. Today the elementary school is all about keeping children on the same level. The current teachers focuses too much on the children that needs more attention or are bad at a subject, but the children that don’t need it and are good at a subject recieve less attention. A robot can help these more advanced students to learn as much as they can. With the robot they can learn more using a fun way and on a personal level, this way the children will always be motivated to learn. Fir this we have set a few goals which will allow us to achieve this goal.

Objectives

  • Fun learning: To motivate children to learn, we want to make the learning experience more fun. That way, the child will not loathe studying and will enjoy it.
  • Provide personalized learning experience for the students: Every child is different, but currently most of the school system is the same for every student. We want to provide a personalized learning experience for every child. Students who have trouble with the curriculum will get more time and exercises to study, while students who quickly master the material will get more advanced exercises.

USE aspects

Users

The primary users for this project are the children of the elementary school who use the robot. The robot has to be able to teach the children. Children will learn faster when they are motivated, the robot uses social cues and will speak in order to motivate the child. In order to keep the child motivated a personalized learning system is needed. A personalized learning system can help motivate a child by letting him learn at his/her own level, for instance when the child is bad at a subject the robot helps him with that subject, to better understand it. Even when the student is at a higher level of understanding than most others within the classroom, the robot is albe to push the student further.

Elementary school teachers are the main secondary users. The robot will be implemented in the hallway next to the classroom, thus the teacher has to know how the robot works when something fails for instance. Furthermore, the teacher ensures that the progress of the student is inline with the curriculum and that it does not deviate greatly.

A tertiary user can be a parent of the child. A parent wants what is best for their child, and wants that the child learns as most as they can. The robot helps the child to learn on their personal level so the child will learn as much as he is able to learn.

Society

The society will benefit from the robot, because children come in contact with robots at a young age. This way they know how a robot works and know what to expect from it. In the future more and more robots will be used, so a child has to be comfortable around robots.

Enterprise

The enterprise that is going to use the scenarios is TiViPE. TiViPE (Tino’s Visual Programming Environment) is a software solutions provider and offers consultancy, research, processes, documentation, architecture, and implementation. These software solutions are mostly related but not restricted to parallel and real time systems. The scenarios benefit this company because it can implement them in the Nao Robot and use it in real time. Eventually in the long goal for this project a company that makes the Nao robots will benefit of our project because more and more Nao robots are needed in the future.

Approach

The goal of our project is to make young students are taught on a level that is personal to them in a fun way, by implementing a teaching robot

We will do this in the following order

  • Determine the basic objectives of the robot. What should the robot do?
  • What has already been done? Explore state-of-the-art.
  • Choose which robot could be implemented to fit the needs of the project, or design a new robot.
  • Choose one specific task in the form of an educational game that the robot should be able to do.
  • Experiment with the robot.
  • Try to implement an algorithm which will make the robot able to perform the chosen task(s).
  • Refine the algorithm/reformulate task robot.

State-of-the-art research

Social cues and gestures

  • Social cues and gestures are very important in any conversation whether that is human-human interaction or human-robot interaction. For this reason, research has been made for the interaction between humans and robots. It was shown that interactions between robots and humans have resulted in better message retention. In the classroom environment, this is needed.[1]
  • Since in the traditional classroom, the one-on-one interaction between the teacher and the students is decreased due to the format of the classrooms, it is necessary to have the robot be able to make gestures and make use of social cues. Since for human-human interaction, it comes naturally to most people to be able to take turns when a conversation takes place. Similarly, when a physical object is the topic of conversation, joint-attention takes place without the participants conscious decision However, these ideas are not inherent for a robot and must be used to make the conversation more natural.
  • It has been shown that when conducting a conversation with a robot that does not make use of these ideas, the conversation is less engaging. Subsequently, when the same things were tested with gestures and gaze, more of the information (“messages”) were retained better. In a classroom environment, this is necessary to allow for the person (in this case, the student) to retain more of the information that was discussed.
  • Thus, it is necessary to make good use of gestures and gazes when there is interaction between the robot and the child.

Appearance

To ensure that child-robot interaction is successful, it must be ensured that the children are comfortable with interacting with the robot. For that, the appearance of the robot must be carefully designed. Children respond better to human-like features but not to a completely humanoid robot appearance.[2]

  • Robot design is a crucial step in any process of introducing robots for interaction with humans. It was found that applying anthropomorphic designs to robots is not necessary for young children. The reason for this is that most young children are used to the toy-like appearance of the objects they see daily. In the study that was conducted, it seemed that Aldebaran’s Nao robot fared better visually than other robots with less human-like features but also better that very human-like robots.

Role-switching mechanism

‘Role-switching’ as an intrinsic motivational mechanism which facilitates the engagement in long-term child–robot interaction. In this study, this was done in the specific learning framework of improving knowledge and habits with regards to healthy-life styles, though the article suggests that this can be applied in other learning frameworks as well.[3]

Robots in the classroom

Most special needs schools cannot afford the NAO. The paper investigates the non-humanoid Lego Mindstorm robot as an alternative learning method for pupils with intellectual disabilities. An interview at the end concludes that three out of four students were significantly more engaged with the non-humanoid robot than the humanoid robot, whilst one student was found to be equally engaged with both robots. [4]

NAO Robot

What is NAO and how does it work?

NAO robot fifth generation

NAO is an autonomous, programmable humanoid robot, which is being used for specific research and education purposes. There have been made multiple generations of the NAO, all with various versions of design, which all feature a platform with either 14, 21 or 25 degrees of freedom. To provide that NAO can position itself within space in a stable manner, all these generations feature an inertial measurement unit (IMU). This IMU is an electronic device that measures the robots angular rate and forces by means of a gyroscope, accelerometer and four ultrasonic sensors. NAO last generation, the ‘NAO Evolution’ in particular consists of a multilingual text-to-speech synthesis, voice recognition and sound localization, shape and facial detection and recognition and an anti-collision system. To control all these features, the robot uses a specialised Linux-based operating system, ‘NAOqi’. This OS powers the robots multimedia system corresponding with the mentioned features. This includes four microphones (voice recognition and sound localization), two speakers (multilingual text-to-speech synthesis) and two HD cameras (computer vision, in particular facial and shape recognition). [5]

Academic & scientific usage NAO

As mentioned, the NAO is used in numerous research and educational settings: over 200 academic institutions worldwide have made use of the robot. An example is on a school in Birmingham, United Kingdom, where autistic children were taught by the NAO robots. One of the conclusions was that some of the children found the childlike, expressive robots more relatable than humans. In a more general setting, at various schools in the United Kingdom, the NAO is utilised to familiarise children with robots and the robots industry. Other usages of the NAO include customer service use in (Japanese) bank branches and testing dummy for robotic “autobiographical memory”, which in turn can train International Space Station (ISS) crews and assist elderly patients.

NAO & TiViPe

By default, the NAO comes with a software suite that includes a simulation software package, a software developers kit and a graphical programming tool called Choregraphe. However, there is alternative (graphical) software available, and for our project we came across TiViPe, which is graphical end user developing environment made by Tino Laurens. The software is ‘end-user’ in the sense that the environment itself is built by Tino, but within this environment users can develop software programs their selves. This is done by connecting ready graphical modules within the environment, so that no hard programming language skills are required. There is a distinction to be made between two types of modules: elementary modules and compound modules. Normally, the elementary modules are the pre-defined available modules made by Tino, the compound modules are created by the user self by merging elementary modules together. However, one can also create elementary graphical modules their selves by writing code and wrap/embed this code into a new module. [6]


NAO robot technical drawing dimensions
Multiple connected elementary modules (green) are being 'merged' into one compound module (red)

NAO & YOLO

In the educational games the robot will play with the student, we will also utilise some objects (see below: for instance the dice with the summing up game). Therefore, our robot of choice should incorporate some sort of object recognition. It just so happens that TiViPE has an option to use such an object recognition on the latest generation NAO, named ‘You Only Look Once’ (abbreviated YOLO). Stated as a state-of-the-art, real-time object detection system, YOLO uses a single neural network to scan the full image, where prior detection systems repurpose classifiers or localizers to come to a significant detection. The neural network splits the image up into smaller regions and comes with a prediction for bounding boxes and a probability per region, in that sense it is solving a regression problem. A claimed property of the YOLO object recognition by the developers is that it is “extremely fast”. This is relevant for us, since the processing of the NAO in a reaction should not be delaying the reaction to much, so that the interaction of the NAO and the student stays natural and interesting for the student. The basic YOLO database model processes images in real-time at 45 frames per second, which we assume to be fast enough. If not, we can always switch to the smaller database "fast YOLO", which processes a picture in 155 frames per second. [7]


Recognising objects in picture using YOLO

Interviews

Interview: An overview of the interviews.

We have learned several things from the interviews. Firstly, in the current educational system, the school is concerned with keeping the children on the same level, and the most when someone lacks behind keep them up to the standard level. In this way, the smarter children do not get as much attention, nor challenge, as they need. This is why we decided that our focus group are the more advanced children. The second thing we learned is that when teaching a child a new topic, we cannot us any negativity, but have to use positive feedback and constructive criticism. Furthermore, we need an open attitude, this way the child will be more motivated and positive about learning. Moreover, we learned that it is smart to use fun learning for a child, this is because they will have more intrinsic motivation. For younger students, teachers already use educational games to make learning fun. So a robot can use those games to teach a child something in our case. And it is good to use different tones in a conversation, this is also playable on the robot which can speak louder and softer. The last thing we learned from the interviews is that the process is always more important than the result.

Setting

The setting in which the robot operates is in the hallway next to the classroom. The teacher can semi-supervise the child which is playing with the robot, also other teachers walk by and can also semi-supervise the child. This way it won't be disturbing for the other children of the class and they can focus on what the teacher has to say. For now we are only focusing on the smarter children, the most time when a child is sent to the hallway to do other stuff, it always have to obligation to work as hard as it can, because the teacher can show up at any time. And the child wants to prove himself that next time he can go as well. We think that children are very interested in playing with the robot and thereby will behave themselves when playing with the robot.

General scenarios

Math helping robot by dice game.

A child throws a dice, the robot can recognizes the pips of the dice and the robot ask for instance what is the total number thrown, or what is that dice multiply by that dice if they are more advanced. This way there will always be a random factor, which is more fun for the child and doesn't get boring. Various dice games are possible, for instance some harder dice game to keep the student on a higher level. A easy game for the children of grade 3 and 4 is higher lower game, and a bit harder game for instance is a game where the child has to add the 2 dices up. The child will understand math better by playing the game multiple times, it will be a fun way for the child and keep them on a personal level.

But of course there can be risks in using a robot to teach a child something about math. For example, the robot malfunctions, the software crashes or the robot is broken. These are all situations which are needed to be taken into account. Also there can be a lot of undesired situations. When the flow is not going as expected and the robot has to hop into a fallback and later go back to the standard flow. For instance:

  • The dice can be thrown off the table
  • The child does not respond at all
  • The child is scared by the robot and doesn't want to play
  • The robot read the wrong number of dots on the dice
  • The child does not wait for its turn
  • The robot fails to grasp the dice

These situations are just examples of things that can happen wrong. To make a good scenario most of these situations needed to be taken into account to program a working robot.

Spelling helping robot by making a word

There are multiple boxes with objects, one box with difficult objects and one with simpler objects. The child has to take an object from the box and show it to the robot, and the child has to make this word with blocks, or pieces of cardboard. The robot recognizes these blocks and to make it easier for the robot all the block can be made with using some sort of barcode. The robot scans the barcode and knows which letter is given. The robot helps the child with spelling in this way. A database is needed in this case of all the words that can be made with these blocks, and if the barcode is easier recognized also with all the barcodes. To keep students on their personal level easier and more difficult words can be given by the robot. The child will learn on a fun way to spell different words. Though it is hard for the robot to recognize the letters which are spelled. In this case also the robot can malfunction or crashes. But the undesirable situations are different in relation to the dice game. Some undesirable situations are:

  • The robot does not recognize the object
  • The robot does not recognize a letter
  • The child gives a letter upside down
  • Something in the database is wrong and a correct word is given as false.
  • The child does not respond
  • The child is scared
  • The robot does not pronounce the word the right way

These are examples of what can go wrong/ is an undesirable situation in the spelling game with a robot.

Maze programming game

A concept educational game is made to challenge the smarter kids of elementary school from classes 7 and 8. This game makes it possible to let the kids program the robot by voice. The idea is that the robot stands on a table with some kind of physical maze which has tiles and walls. The robot will move through the maze. This movement will be decided by the kid. The goal of the game is that the robot reaches the finish tile in the maze.

The pupil can program the robot with a sequence of actions. If the goal is not reached after this sequence another sequence of moves can be programmed by the child. This will continue until the goal is reached. To make this game more advanced and interesting for the child when he plays again, a bigger maze can be used. The normal beginning maze is 8x8 blocks. The child will learn to program a bit and intensively use their memory, because the robot will move after the child is done saying commands. Some example of commands are:

  • Move forward
  • Turn left
  • Turn right
  • Move back

When the child makes a mistake for instance it also needs to correct itself by saying go back. To fully use this scenario where no extra assistance from a teacher is necessary, the robot should make the maze itself or the teacher makes the maze in the beginning of the day. This scenario is harder to implement, because more objects are needed. In this scenario also a lot of undesirable situations can occur:

  • The robot slips and does not execute a command
  • The robot hits a wall
  • The robot does not turn the full 90 degrees
  • When the robot is moving the child interrupts

Detailed scenarios

Detailed Scenarios: An overview of the detailed scenarios process.


Math helping robot by dice game (higher lower)

The robot will play a dice game with a child of the elementary school grade 3/4. The robot will throw the dice and the child will throw the dice. The child has to repeat the numbers thrown and eventually has to say if he has higher or lower than the robot. When the child is correct he wins the game and can play another game of the same game. The child will learn how numbers work, and a bit of short term memory. This is the most basic dice game there is, and maybe can be used in grade 2 as well, because we want to focus on the ‘smarter’ children.

Higher lower scenario: The final higher lower scenario.

Math helping robot by dice game (summing up)

This variant of the dice game is more advanced. And is also for the elementary school grade 3/4 The children have to sum up the dice the robot throws and the child throws. This way they have to understand how numbers work and can use their short term memory. To fully implement the scenario we have to ask ourselves a few important questions. For instance what are the risks when a child plays a game with a robot. An important risk is that the child can break the robot or just walks away from the robot, we will try to prevent this with a teacher as a semi supervisor. The explanation of the summing-up game is in the link below. This includes the strings and actions used in the scenario.

Summing up scenario: The final summing up scenario.

Spelling helping robot by making a word

The robot will play a game with a child, the child has to take out an object of one of the boxes, and the child has to spell this word with the given blocks. Some boxes are have more difficult objects this way we implement personalized learning. The spelling game uses deep object learning, this is a relatively new technique which is not used much in variant scenarios.

The blocks contain 6 sides with 6 different letters, like a dice but with letters. If the child spells the correct word the robot continues. When not the correct spelling is given it first checks the length of the word and then it will check the order of the letters. This is done in multiple stages and eventually the correct word is spelled. The child will learn to spell in a fun way with the blocks. And the robot will give positive feedback

Spelling scenario: The final spelling scenario.

Maze programming game

A concept educational game is made to challenge the smarter kids of elementary school from classes 7 and 8. This game makes it possible to let the kids program the robot by voice. The idea is that the robot stands on a table with some kind of physical maze which has tiles and walls. The robot will move through the maze. This movement will be decided by the kid. The goal of the game is that the robot reaches the finish tile in the maze.

The pupil can program the robot with a secual of actions. If the goal is not reached after this secual another secual of moves can be programmed by the kid. This will continue until the goal is reached.

Programming game scenario: The final programming game scenario.

Conclusion

Elementary school classes become more crowded every day. The overcrowded classes will result in couple of negative effects. The first negative effect is that the education becomes less personal. The teacher does not have the time to pay attention on every individual kid, which could have negative effects on personal growth of kids. From interviews with teachers we know that the current educational system on elementary schools in the Netherlands solely focusses on the kids who perform under the mean level of the class. The higher performing kids get less attention and possibly will not reach their full potential. This is why the target group of this project will be high performing kids on elementary schools. We want to improve their level and try to achieve their maximal potential.

This problem could not simply be tackled by hiring more teachers to make classes smaller, because there aren´t more teachers available to achieve this. This is why a different solutions need to be found. In this project we focus on solving this problem by implementing a robot as addition to the current system . This robot will focus on individual kids and therefor introduce personal learning. The kids will not get lessons from the robot, but will learn by playing educations games. This way the kid not only learns from its experience with the robot, but also have fun while doing so.

After a clear definition of the objectives fun learning in a personal way, educational games where created. Two of the games focus on learning how to calculate. One game focuses on the spelling together with coupling objects to words, and one game introduces the basics of simple programming. First scenarios where made which later where converted to state diagrams. The information that was gathered from the interviews where used as basis for the games. For example the process of reaching the goal has more priority than really reaching the goal. Also gestures where used to make interacting with the robot more natural. Also the YOLO deep learning algorithm has been implemented in the spelling game to have a more natural interaction and make it more easy for the kid to connect words to objects. More usage of the YOLO deep learning algorithm in the basics of all of our educational games could make the interaction more natural, which will improve the experience of learning with a robot.

The scenarios and state diagrams where ready to be tested at the end of this project, so a beginning of programming the dice game in the TiViPe program was started. Sadly this didn’t work out as planned and little time was left to make it work. We are certain that the state diagrams form a good basis to program the NAO robot for educational games, and further research should begin where this project had ended.

Planning

Milestones

  • Planning
  • State-of-the-art research
  • General scenarios
  • Interview with elementary school teacher(s)
  • Detailed scenarios
  • Final demo

Planning

Gantt chart USE.png

Deliverables

  • Wiki:
    • Rationale of the project
    • Stakeholders & USE aspects
    • State-of-the-art
    • Interview(s)
    • Scenarios
    • Our process
  • Code
  • Demo (simulation/real life)

Logbook

Week 1

  • Brainstorm session
  • Discussion about the possible topics for this course
  • Come to a decision about the topic
  • Make the idea more concrete such that everyone in the team knows what we will be doing further on. We have made an initial list of objectives and approach.

Week 2

  • Had a meeting with one of the professors of the course and two of the team members. This was to ensure that the idea was sufficient for this course and that it was also feasible. Within this meeting it was discussed whether we are able to use the Nao robot for the purposes of our project. Furthermore, some extensions were discussed. A follow-up meeting will be scheduled to further discuss this.
  • Preparations for the presentation were made and the decision of who will be presenting for the first, and the later, presentations were also made.

Week 3

  • We redefined our goals and our subject after the meeting with Tino Lourens. And we set up the milestones and made the planning
  • The second presentation was made which holds the milestones, planning and the deliverables.

Week 4

  • An interview was done with an elementary school teacher in training, who had teached grade 1&2, 3 and 7. She was very positive about the our idea.
  • Another interview was done.
  • We came up with more scenarios which we are going to work out further.
  • We made 2 detailed scenarios.
  • Feedback session:

Week 5

  • Came up with more scenarios
  • Worked out more detailed scenarios
  • An interview was done with an occupational therapist, who works with children. She was very positive about our idea
  • Weekly feedback session: these scenarios are basic try to improve those

Week 6

  • Updated the wiki
  • Made one final scenario
  • Tried this scenario with one playing as the robot and one playing as the child.
  • Discussed about the setting
  • Feedback session: the final scenario is a flowchart rather than a state-diagram. Make a state diagram next.

Week 7

  • Updated the wiki
  • Programming on the Nao was done
  • Final state diagrams were made
  • Final strings/gestures were made
  • Weekly feedback session: We have to hurry with the programming if we want to demonstrate that next week.

Week 8

  • Programming on the Nao was done
  • Final state diagrams were updated
  • Final strings/gestures were updated
  • Final presentation was made
  • Wiki was updated

References

  1. ↑ Torta, E., Juola, J. F., & Cuijpers, R. H. (2014). Approaching independent living with robots
  2. ↑ Tung, F. (2016). Child perception of humanoid robot appearance and behavior. International Journal of Human-Computer Interaction, 32(6), 493-502. doi:10.1080/10447318.2016.1172808
  3. ↑ Ros, R., Oleari, E., Pozzi, C., Sacchitelli, F., Baranzini, D., Bagherzadhalimi, A., . . . Demiris, Y. (2016). A motivational approach to support healthy habits in long-term Child–Robot interaction. International Journal of Social Robotics, 8(5), 599-617. doi:10.1007/s12369-016-0356-9
  4. ↑ Aslam, S., Standen, P. J., Shopland, N., Burton, A., & Brown, D. (2016). A comparison of humanoid and non-humanoid robots in supporting the learning of pupils with severe intellectual disabilities. Paper presented at the 7-12. doi:10.1109/iTAG.2016.9
  5. ↑ https://www.ald.softbankrobotics.com/sites/aldebaran/files/press-releases/cp_nao_evolution_en_def.pdf
  6. ↑ http://www.tivipe.com/
  7. ↑ Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi. You Only Look Once: Unified, Real-Time Object Detection. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 779-788
Retrieved from "https://dsdwiki.wtb.tue.nl/index.php?title=PRE2016_3_Groep18&oldid=38770"