PRE2020 4 Group3: Difference between revisions

From Control Systems Technology Group
Jump to navigation Jump to search
Line 365: Line 365:


This week I was still mostly struggling with tensorflow but the hard work finally payed off. The model that I have constructed with my current dataset has an accuracy rating of 97%! Now the only thing that still needs to be done in order for the image recognition to be usable is to create a function that has the ability to input images into the model and output a result on the website. In order to do so the image should first be cut into the seperate images each containing one shape which is grayscaled and downsized to a 28x28 image. This seems like the next logical step in order to complete the image recognition part. Afterwards some testing can be done to see if accuracy is still retained with our own input. Although I have a high confidence that it will work now that I have seen the images in the training data and the accuracy that it has on those.
This week I was still mostly struggling with tensorflow but the hard work finally payed off. The model that I have constructed with my current dataset has an accuracy rating of 97%! Now the only thing that still needs to be done in order for the image recognition to be usable is to create a function that has the ability to input images into the model and output a result on the website. In order to do so the image should first be cut into the seperate images each containing one shape which is grayscaled and downsized to a 28x28 image. This seems like the next logical step in order to complete the image recognition part. Afterwards some testing can be done to see if accuracy is still retained with our own input. Although I have a high confidence that it will work now that I have seen the images in the training data and the accuracy that it has on those.
---------UPDATE 30/06
This week I worked on getting the AI implemented on the webpage which was more challenging than originally expected. There exist some Javascript modules that allowed for a easy implementation of a neural network inside the web browser (e.g. Brain.js seemed promising at first) but they all suffer from the same problem. The problem is that these modules are either not very compatible with the tensorflow module I used in python or they are not compatible at all. So it took me some time to search for one that works but I managed to discover that the tensorflow module I used in python actually has a javascript variant which could be used (which is really convenient if you ask me). In order to use a pretrained tensorflow model from python in javascript we only need to convert the .H5 file in the python script to a .JSON file. Luckily for us there is a module for python that does just that which was also made by the tensorflow team although for some reason python did not like me installing this module much like the actual tensorflow module (so this took quite some time troubleshooting). In the end I managed to get it working which means we now have a .JSON file, now we should test whether it works with the webpage. I am however a absolute novice at HTML and webdesign so if possible I would like to work with some of the people who also worked on the actual webpage to try and get it working together next week.


=Platform=
=Platform=

Revision as of 22:40, 30 May 2021

Group members

Name Student number
Eline Boom 1465872
Luuk van Dorst 1469789
Robin van de Hoef 1479679
Kyra Moviat 1349171
Jeroen Pullen 1477730
Veerle Uhl 1462229

Abstract

With the rise of technology comes new ways to implement technology into our daily lives. One of these implementations is technology in education. More specifically, we wanted to develop a technological tool that will help young children in their learning process of number sense. Your knowledge of counting and quantities is tested daily throughout your life. Not a day goes by where you do not have to count or decide the quantity of something, which it is such an important skill to learn from an early age. We wanted to make an application that teaches children number sense in a fun and easy way.

Introduction

Teaching children to count is very important. It helps with their development as well as being used in day-to-day life later on. The current curriculum used in most schools works well, but we feel that implementing technology into this curriculum would improve the learning of small children, both in school and at home. According to Hsin(bron), who studied the influence of children's use of technology on their learning, this technology does have a positive effect on learning. Hsin does provide us with a few conditions in which learning with technology is optimal. When children are allowed to work together and communicate with each other, technologies may not only improve knowledge, but social skills as well. These skills are most important for children, since interaction with others is needed for humans to survive. The second condition that needs to be met, is that adults need to be present to provide a safe climate and to encourage participation. This makes sure children stay motivated to learn and the adult can give guidance when instructions might be unclear. Our product is heavily influenced by these two conditions. What we designed is a web-based application that teaches children how to count and recognize shapes and color. It is targeted at children aged 3-5 that are at an average level of counting and shape / color recognition. It has a level-based design with three main levels. These levels then have several exercises each. The difficulty increases for each level, but stays the same within levels. OUr product is designed to be used in a classroom setting. It can be used on a whiteboard with a stylus or on a computer or laptop. When it is used on a whiteboard, children can interact and discuss with each other about the correct answer to the questions. This way, interaction and collaboration is promoted. When it is used in a classroom setting, the teacher can give extra help and information where needed, although the product is designed to need very little to none additional explanation.

Project Plan

Subjects: Child psychology, learning, image processing, software development.

Objectives: Create software to help children learn

Users: Children (of parents that want to provide a playful learning app)

State-of-the art: Alternative apps that help children either count, draw or learn colors individually. Not in a combined way, we feel the combination might improve the way to learn these things. Additionally, we hypothesize that explaining problems via a childs own drawn elements could improve their understanding.

Approach: Make up a model for how the software should work, make software in which one can draw, program software that can recognize multiple drawn objects/shapes, implement everything in a working framework according to the model.

Milestones: Drawing software, working framework, Working object recognition, implementation in task framework

Deliverables: Peer review, Wiki, Final Presentation, The software (?)


Planning

Task Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8
1. Brainstorm Project ideas X
2. Investigate Literature X X
3. Create Software Model X X
4. Create Software Framework X X
5. Create Drawing Software X X
6. Create Object Recognition Software X X
7. Implement Recognition Software in Framework X X
8. Work on Wiki X X X X X X X X
9. Work on Presentation X X X

Subtasks

subtask Done / to do by who
1.1 Find topics that are of interest and comply with technical skills of the group Everyone
2.1 Investigate state of the art Kyra
2.2 Investigate how children learn to count and learn colors and shapes Veerle/Robin
2.3 Investigate current models or curricula that are used for learning Veerle
2.4 Investigate the influence of online or "games" learning Veerle/Robin
3.1 Meet with stakeholders to see what they want to see in an excercise Jeroen
3.2 Figure out what kind of levels we want to make Jeroen
3.3 Implement levels in the website Jeroen / Eline
3.4 Make an instruction video for the levels Kyra / Eline
4.1 Set up a website Eline
4.2 Code and make a website layout, make it suitable for young children Eline
5.1 Set up a canvas where users can draw Eline
5.2 Find a way to use colors, a drawing tool and an eraser in the canvas Eline
6.1 Find databases to train a neural network with Luuk
6.2 Write code for a neural network Luuk
6.3 Write code that can read a database and train a neural network Luuk
6.4 Work on improving the accuracy of the neural network Luuk
7.1 Make the drawing made convert into a png Eline
7.2 Make the neural network compatible with a png Luuk
7.3 Convert the drawing png so it cuts out every individual shape and recognizes color Robin
8.1 Write down as much as we can on the wiki Everyone
8.2 Transform the summary on the wiki into a good text with logical structure -

Research

State Of The Art

There are already a lot of apps on the market that are geared towards educating children. Certainly apps that teach children how to count or how to draw separately. Combining the two is also done, but less frequently. One app and one website that combine learning to count and to draw are found on the internet. The first one is an app made by Bini Bambini and is called “123 tekenen tellen van nummers voor kinderen”. This game gives children very simple descriptions along with visual guides for drawing complex shapes. It then tells the child how many objects there are and lets the child draw the number. It does not have any AI to see what the user draws, it only gives an outline and a field in which to draw. Once the user stops drawing for a few seconds, the app goes on to the next step. The second one, a website, is called Kids Academy. This has the same general layout of the bambini app, only the Kids Academy focuses on more than just learning math. What stands out quite prominently in both apps is the design layout. It is clearly targeted towards kids with simple shapes, bright colors and audio directions instead of text. Our product keeps these general GUI attributes, but it is different in the level design. We combine drawing and counting in a different way. First, we start off with a level where children do not even need to draw and just need to recognize amounts, colors and shapes. Then, children need to draw (simple) shapes themselves, without guidelines, and also need to draw the correct amount eventually. This makes it so the child is more actively working on the amounts of objects, which will promote their learning.

How children learn

In order to develop an application that supports the learning of children aged 3-5, we first need to know how children's learning develops during their early years and what knowledge they already possess or still need to learn. From when they are born to around the age of 5, children develop “everyday mathematics”. These are informal ideas of more and less, taking away, shapes and sizes. These everyday mathematics can be surprisingly broad, complex and sophisticated. Apart from the everyday mathematics that children learn on their own, they also start learning mathematics in school. From the age of 2, children learn the language and grammar of counting. They memorize the first ten or so counting words, and then learn a set of rules to generate the higher numbers. When children are around 4 or 5 years old, they begin to develop metacognitive skills: They become increasingly aware of their own thinking and begin to express it in words. The hardest form of language for children to learn, which they thus only learn at a later date, is the special written symbolism of mathematics, like 5, +, - or =.

Understanding numbers involves more than saying a few counting words. It involves reasoning about numbers, making inferences and developing a mental number line. Children also need to mathematize: to conceive of problems in explicitly mathematical terms. They need to understand that the action of combining one bear with two others can be meaningfully interpreted in the terms of the mathematical principles of addition and the symbolism 1 + 2. This mathematizing is not very important for our product, as this only happens at a later age than the target group we are designing for. This means that we do not yet have to include problems of mathematization in our level design

The Number Worlds Curriculum

This curriculum is one of many out there that learns children what numbers are and how to use them. It provides a good understanding of the process a child goes to in order to learn how to count. Looking at 5-year-olds, there are several points of knowledge they need to know, relating to number sense. Five year olds know:

  • that numbers indicate quantity and therefore, that numbers themselves have magnitude
  • that the word “bigger” or “more” is sensible in this context
  • that the numbers 7 and 9, like every other number from 1 to 10, occupy fixed positions in the counting sequence
  • that 7 comes before 9 when you are counting up
  • that numbers that come later in the sequence
  • that are higher up - indicate larger quantities and therefore, that 9 is bigger (or more) than 7.
  • that each counting number up in the sequence corresponds precisely to an increase of one unit in the size of a set. This enables children to use the counting numbers alone, without the need for real objects, to solve quantitative problems involving the joining of two sets. In doing so, it transforms mathematics from something that can only be done out there (e.g. by manipulating real objects) to something that can be done in their own heads, and under their own control.


As children get older, they progress through different knowledge they know: By the age of 4, children have constructed knowledge of counting and quantity. Sometime in kindergarten, children become able to integrate these knowledge networks. Around the age of 6-7, children connect this integrated knowledge network to the world of formal symbols. By the age of 8 or 9, most children become capable of expanding this knowledge network to deal with double-digit numbers and the base-ten system.


In the number worlds curriculum, there are 5 instructional principles that lie at the heart of the program:

1. Build upon children's current knowledge

2. Follow the natural developmental progression when selecting new knowledge to be taught

3. Teach computational fluency as well as conceptual understanding

4. Provide plenty of opportunity for hands-on exploration, problem-solving and communication

5. expose children to the major ways number is represented and talked about in developed societies


Lastly, the core of number sense forms a knowledge network that is called the central conceptual structure for number. This core is important for children for two reasons: 1) it enables children to make sense of quantitative problems across context, 2) it provides the foundation on which children's learning of more complex number concepts is built.

Stakeholders And OSL

3-4 years old: Easy levels, can count to 10, can manage color comparison, learning pen grip important, learn to count via touch, “which object is blue”? appropriate question, combination of tasks as described in goal level still to hard.

4-5 yeas old: Goal level, know all colors, can count, learn shapes important, number recognition important.


*What makes the software good?

- Rising level of difficulty (3-4 to 4-5 level ?)

- Possible on digital school board

- Possible with pen (learn pen grip)

- Returns performance data to authorized figure

- Test mode

- Feels like a game (reward system)


*What can kids learn from the software?

- Colors, Shapes, Counting & Drawing

- Pen grip

- Vocabulary (if drawing objects)

- Listening skills (if tasks given in audio)

- Number recognition (number on screen)


*Useful purposes

- Being able to use the software collaboratively (class setting, performing tasks together)

- Being able to use the software to test how far children are.

- Being able to have the kid perform tasks on their own, behind computer or tablet.

OSL learning goals:

Colors: “Knows all colors”

Stakeholders


Shapes: “Be able to construct shapes”

https://www.slo.nl/thema/meer/jonge-kind/doelen-jonge-kind/


Counting goals: “Learn to count to 20, can recognize amounts up to 12”

https://www.slo.nl/thema/meer/jonge-kind/doelen-jonge-kind/


Drawing/Motor skills: “Learn pen grip”

Stakeholders


Vocabulary: “Understand instructions, learn new words”

https://www.slo.nl/sectoren/po/inhoudslijnen-po/inhoudslijnen-nederlands/


Number Recognition: “Know, read and write numbers”

https://www.slo.nl/sectoren/po/inhoudslijnen-po/inhoudslijnen-rekenen-wiskunde/


Listening skills: “Learn how to follow a task description”

https://lesintaal.nl/platform_taaldidactiek/1_mondelinge_taalvaardigheid/kennisbasis.htm


Digital skills: “Playfully get acquainted with technology and how to operate them, partially via educative games”

https://www.slo.nl/thema/meer/jonge-kind/doelen-jonge-kind/

How We Use The Research

As a basis, we will use Number worlds curriculum and OSL curriculum. These have been tested and used by many schools and education institutes around the world and are proven to support learning of children.

Number words has 5 principles for learning, we used them as a framework to develop good exercises that cater to the needs of the children that want to learn. Principle 1 talks about building upon children’s current knowledge. We do this by slowly incrementing the difficulty of our levels. First, we ask children to simply move an object into a box, then when they have mastered this, they need to move a specific amount or specific color or shape of object into a box. This way, we slowly add more variables while building onto the knowledge children already have. Principle 2 is about following the natural developmental progression when selecting new knowledge to be taught. With this, we looked at what knowledge young children typically learn at what age. We also took into account the desired age for users and combined this to find what lessons work best. For example, we use squares and circles for objects instead of cars and dogs, since the latter is too far ahead in the developmental progress of knowledge for the small children. Principle 4 talks about providing opportunity for hands- on exploration and communication. We hope to do this by making our product work as well on a digiboard. This way, children and teachers can work together to solve problems and do exercises, promoting communication between them.

The OSL curriculum is one that does not just focus on math, or on younger children. It focuses on all subjects and children of all ages. For our application, we look at their learning materials for young children (ages 4-7) and specifically the math and number sense material. SLO works with "inhoudslijnen" that give an overview of everything the young child (until age 7) needs to be able to do and what knowledge they need to posess. Even though our app focuses on children ages 3-5, we can still use these "inhoudslijnen" to shape our excercises. Learning never stops, and everything we teach children at a young age, they do not need to learn at a higher age still. If we for example look at some of the learning goals of SLO, we can see that, at age 7, they want children to learn how to count to 20 and to also be able to count quantities of at least 20. At age 4 or 5, children using our app probably would not already be able to have knowledge on all these topics, as they are aimed at children aged 7, but we can give them a good push in the right direction. Since we know that children aged 7 need to know at least how to count to 20, then children aged 5 should not only be able to count to 5, but probably to 10 already. In conclusion, using the learning goals of SLO, we can modify them to set our own learning goals for users of the application. Based on these learning goals, we can then shape the levels of the game to teach the children just what we want to know.

OSL has learning goals for children. The learning goals important for our level design are on the drive file. Here is a summary of important findings: What we have now is doable for children aged 3-5 yeras old. What we might need to test is that it will not be too easy for 5 year olds, seeing as according to OSL, at that age they already learn additional things as well. What we have now as exercises follows the learning lines of children and will support their learning process. Nothing from the levels is too difficult, so it wont be the case that the level difficulty is above the level of the users. Related to language (listening/reading): reading is indeed too difficult for younger children. When you look at phase 1 of OSL, you can see it is mostly simple words and very very short sentences as well as looking at pictures. There is not a whole lot about listening, but according to the prephase (for ages 2-4) then children should be able to listen to and interpret explenations of exercises. We do have to keep in mind to only use very simple words

Test plan

In order to make sure our product works as intended, user tests were run on children aged 5-6. Since there was limited time available for the study and design of the product (only 8 weeks), there was no possibility to test whether the product actually supports and improves learning for children. What was able to be tested, however, was whether children engage with the product and whether the product can keep their attention. In order to test this, a test plan was written out:

At the start, when the whole class is present, a shorter version of our test plan has to be given to the children. The observer will explain that the children will be divided up into groups of ~5 / 6 to do an experiment. The way these groups are made does not matter that much. The children can either choose the groups on their own or the teacher can make the groups. The most important part is that all groups are approximately the same size and not too big. After the groups are made, the obesrver will continue the explanation. They will show the website and say that in the groups, the children will do some games on the website. Only one group at a time will be interacting with the website, the other groups will (preferably) be outside in another room. When the groups are divided and only one group is present in the room, the experiment can start.

One child in the group is given the pen. The website is started up and then the children watch the introductory video. After this video, the teacher (or presenter) asks whether they understand what they need to do. Then, the website opens up into the level design and the children need to select the first level. This level, along with the others, is played as normal. When the children cannot figure out together what the solution to a problem is, the experimenter can step in and help. Then, the second, third and fourth level are done in a similar fashion. After all the levels are done, ask the children a few questions:

  • Did you like using the website?
  • Do you think you learned something from the exercises?
  • How did you like the videos with explanations?
  • How did you like the layout of the website?

After this is done with the first group, the group switches for the next group and the experiment repeats. Important is to reload the page so the second and third group will also get the introductory video.

During the whole experiment, it would be ideal to film everything if possible. If not, the experimenter needs to take notes on what the children find difficult. If the experimenter ever has to help or the teacher has to step in, it needs to be noted down at what point this is and what extra information is given. Also a few other points need to be paid attention to and written down. First, whether the children ever lose attention. THis is important to see whether the content is engaging enough. When a child looks away or starts talking with a groupmate about a different topic, note it down. Second, the experimenter also needs to write down whether the children work together. This means writing down when the children talk to each other about the level or exercise, as well as correcting each other on solutions.

Software Model

Our website combines learning of shapes and numbers into a fun experience for young children. It is targeted at children aged 3-5 that are starting to learn number sense and shape recognition. It is a web-based design that is intended for use in the classroom, on digiboards or school computers. It is designed to follow the natural learning curve of children and to promote communication between users.

The levels are designed in a way to reflect the natural learning of children. There are 3 main levels, each with sub-levels that vary slightly. The first level is mostly about shapes. In this level, users will have to move the correct colored and shaped object into a square box. This will make users get used to the layout of the website as well as teach them about shapes and colors. The shapes will be quite simple, circles, squares and triangles will be used. When this first part is done, users will then get an instruction such as “move the 2 red triangles and 1 blue square into the box”. Here, shapes and colors are combined with a small amount of counting. Level 2 starts to get a bit more difficult. Here, users need to draw the shapes in the instructions. The kinds of shapes and colors are still the same as in level 1, but instead of moving them, they have to draw the shapes on their own. Level 3 then gets a bit more difficult still. Users now have to draw a specific amount of shapes, instead of just 1. This final level will integrate their knowledge learned throughout the earlier 2 levels.

The website itself is designed with the users in mind. Since the main users will be children aged 3-5, the website needs to be exciting and fun for them. This means the use of lots of bright colors and fun shapes or cartoons. The website also has little to no text, with instructions being given through audio. During the levels there is also a story element with small videos to keep the attention of the children. The buttons for the levels as well as those used for the drawing tool are all quite large. This makes it so even children that cannot move the mouse with a lot of accuracy can still click on the interactables with ease.

Level design

Level Design

We separate two age groups, 3-4 years and 4-5 years old.

Within these groups, task complexity increases by means of two variables 'Level' and 'Difficulty'.

After completing age group 3-4, the child can be tested for the age group 4-5.

Level

The age group 3-4 will have 6 levels where the goal is to move certain figure(s) to the middle of the screen.

The age group 4-5 will have 6 levels where the goal is to draw certain figure(s) on the screen.

Levels age 3-4:

1 - All the same shapes, child must move 1 correct color

2 - All the same color, child must move 1 correct shape

3 - Altering shapes and colors, child must move 1 correct figure with corresponding color & shape

4 - All the same shapes, child must move # figures with corresponding color

5 - All the same color, child must move # figures with corresponding shape

6 - Altering shapes and colors, Child must move # figures with corresponding color & shape

Levels age 4-5

...

Difficulty

The child will have to succesfully move through all difficulties of a level before moving on to the next level.

The amount of difficulty has influence on the range of colors and amount of shapes that are available in the exercize.

The difficulty will go up or down depending on the childs success rate. This is an example of how a child can move trough the stages:

1.1 -> succes // 1.2 -> success // 1.3 -> failed // 1.2 -> failed // 1.1 -> success // 1.2 -> success // 1.3 -> success // 1.4 -> failed // 1.3 -> success // 1.4 -> success //

Level complete, move on to level 2.1

Whether the difficulty should go up or down after 1, 2 or 3 successive performances is yet to be determined.

Image Recognition

In order to recognise images drawn by children we have decided to use a neural network. In order to train this network we have settled upon a couple of usefull databases that contain drawings and letters, I will list the found databases below:

-MNIST dataset (http://yann.lecun.com/exdb/mnist/)

-Google quick draw dataset (https://github.com/googlecreativelab/quickdraw-dataset)

-(One other dataset that I decided to use but can't find the source for anymore as of this moment :( )


We have also created some code which can read a database, build a neural network, train a network and test the resulting network. However thus far the results from the neural network have been dissapointing, in total the succes rate of the neural network has been 0.0036 which is worse than random guessing (with 250 labels that would be 1/250 = 0.004) so optimisation is still necessary. The low accuracy rate can either be the result of using a too small dataset (each of the 250 labels have only 50 or so images in them) or because of wrong parameters used in the neural network or because we have not trained it long enough. Our current guess is that it's a mix of the first two and we should use less labels and more images to increase accuracy and add more internal nodes and layers to our neural network.


In order to solve these problems and move to a succesfull working prototype I suggest the following goals for next week:

1. Fix the accuracy rate and try to get a neural network to at least work with one of these datasets.

2. Try to find a way to extract the complete neural network from our code and find a way to implement it on our website.

3. Decide which labels are usefull and chose a final dataset which we train our network on.


Of these three the biggest challenge will probably be problem 1 since training the network takes a lot of time so we cannot really bruteforce our way to succes which means we should probably look at what previous work has been done in this subject.



UPDATE 16/06

The code is now adjusted so that it works with the google draw dataset. The reason for the switch to the google draw set is because it has a better support for simple shapes like squares and circles which we have decided to use for our final product. The downside of this is that the google dataset came in 4 different formats each of which has a different amount of preprocessing done to it which made it hard to make a informed decision about it. To make matters worse each dataset is stored in a different format which means that the part of the code that reads the input needs to be updated if we decide to switch again in the future. So I took some time to research the pros and cons of each dataset and I decided upon the 38x38 grayscale bitmap version of the original images. The reason for this choice is that the simple shapes will hopefully be easy to recognize even with this heavy downscaling and that the low image resolution will resolution in shorter training times and a less complex neural network. The sad part is that this dataset came in the .npy format which meant that it was completely incompatibly with 50% of the code that I already wrote which made me have to update that part again which took quite some time.



UPDATE 25/06

This week I was still mostly struggling with tensorflow but the hard work finally payed off. The model that I have constructed with my current dataset has an accuracy rating of 97%! Now the only thing that still needs to be done in order for the image recognition to be usable is to create a function that has the ability to input images into the model and output a result on the website. In order to do so the image should first be cut into the seperate images each containing one shape which is grayscaled and downsized to a 28x28 image. This seems like the next logical step in order to complete the image recognition part. Afterwards some testing can be done to see if accuracy is still retained with our own input. Although I have a high confidence that it will work now that I have seen the images in the training data and the accuracy that it has on those.



UPDATE 30/06

This week I worked on getting the AI implemented on the webpage which was more challenging than originally expected. There exist some Javascript modules that allowed for a easy implementation of a neural network inside the web browser (e.g. Brain.js seemed promising at first) but they all suffer from the same problem. The problem is that these modules are either not very compatible with the tensorflow module I used in python or they are not compatible at all. So it took me some time to search for one that works but I managed to discover that the tensorflow module I used in python actually has a javascript variant which could be used (which is really convenient if you ask me). In order to use a pretrained tensorflow model from python in javascript we only need to convert the .H5 file in the python script to a .JSON file. Luckily for us there is a module for python that does just that which was also made by the tensorflow team although for some reason python did not like me installing this module much like the actual tensorflow module (so this took quite some time troubleshooting). In the end I managed to get it working which means we now have a .JSON file, now we should test whether it works with the webpage. I am however a absolute novice at HTML and webdesign so if possible I would like to work with some of the people who also worked on the actual webpage to try and get it working together next week.

Platform

Designing the website

Link to website: https://group3use.glitch.me/

Link to code: https://glitch.com/edit/#!/group3use

Storyline throughout the levels

To make completing all the levels more appealing to children, a storyline will be shown through visuals in the levels and animated videos before them. The subject of helping animals was chosen because we have observed that it is a timeless subject. Things like tv-shows and apps geared towards children use animals very often.

The overarching storyline is about a group of animals (a horse, a monkey, a bunny and a tiger) that is moving from the city to the forest. The levels designed for 3-4 year olds are preceded by a video about the animals that explains the move and tells the children that they need help moving because there is a broad river around the forest without bridges. The levels itself are about building a bridge for each animal by moving the right shapes into the water. Upon completing the level the children are rewarded with a visual of the animal crossing the bridge, if possible.

The levels designed for 4-5 year olds will depend on the development of the program. If the levels will be randomized the storyline will continue with the animals wanting to decorate their houses with colorful shapes. If the levels will not be random the storyline will continue with the animals having to find food in the forest. The shapes the children draw will transform into food if they complete the level correctly.

Animating the videos

All the videos needed for the reward system of the drag and drop levels have been animated and added to the site. PowerPoint was used to make the animation. The storyline has been changed and is still changing to fit the game as it is now. Since we are still developing it, the story is also still changing and evolving.

Time Log

Eline Boom

Date Description Hours
22/04/21 Brainstorm meeting 1
29/04/21 Meeting 1
29/04/21 Setting up the website 3
30/04/21 Design of website 2
03/05/21 Meeting 1
03/05/21 Creating canvas function on website 1
03/05/21 Converting canvas to image 2
03/05/21 Colour pens and eraser 3
06/05/21 Colour pens and eraser 2
06/05/21 Switching between pen colours through button 2
06/05/21 switching between draw and erase through button 4
10/05/21 meeting 1
13/05/21 Researching possible ways to code Jeroen's game ideas 2
13/05/21 Coding different geometric figures in css 0.5
14/05/21 Start working on the actual game 2
17/05/21 Meeting 1
17/05/21 Difficulty level 3, randomization of figures 3
19/05/21 Finishing all level difficulties + reward when correct 7
20/05/21 working together with kyra on animations 8
25/05/21 changing button text to images 1
25/05/21 adding audio files and play buttons 3
25/05/21 Overall layout 1.5
26/05/21 revisiting code for reward system 4
27/05/21 audio, creating levels, reward system 8
Total Hours: 64

Luuk van Dorst

Date Description Hours
22/04/21 Brainstorm meeting 1
29/04/21 Meeting 0.5
29/04/21 Searching for viable datasets 2
30/04/21 Researching neural networks 3
02/05/21 Making a prototype neural network 4
03/05/21 Meeting 1
08/05/21 install and get tenserflow working (took longer than expected) 1
08/05/21 Found some additional datasets 1
08/05/21 Finished neural network code 4
09/05/21 Did some research into how much nodes the neural network should have to increase performance 2
09/05/21 Tested some parameters to increase performance 4
10/05/21 edited wiki 1
14/05/21 Did a couple of tensorflow tutorials to understand the library better 2
14/05/21 Modified neural network to work with a different dataset 3
15/05/21 Adjusted parameters and tested the neural network to see if performance improved 5
16/05/21 Did some research to see how performance can be improved in the neural network 4
21/05/21 Fixed input problems in neural network training code 3
22/05/21 Tried fixing neural network model code 3
23/05/21 Tried fixing neural network model code again 5
24/05/21 Finally got the neural network model code working 5
25/05/21 messed with some parameters to increase accuracy of model 2
25/05/21 Edited the wiki 1
27/05/21 Worked on implementing the AI in the webpage with Eline and Kyra 5
28/05/21 Worked on implementing the AI in the webpage at home 4
30/05/21 Tried converting the current image recognition model to a .JSON file 3
30/05/21 edited the wiki 1
Total Hours: 10.5 + 12 + 15 + 16 + 13 = 66.5

Kyra Moviat

Date Description Hours
22/04/21 Research topics 5
22/04/21 Brainstorm meeting 1
29/04/21 Meeting 0.5
02/05/21 Add tables to wiki and update layout 1.5
02/05/21 Research state of the art 1.5
03/05/21 Meeting 1
09/05/21 Writing about state of the art 1.5
09/05/21 Updating wiki 0.5
09/05/21 Researching methods children learn counting 2
10/05/21 Meeting 1
11/05/21 Updating the wiki to only have 1 logging system instead of 2 0.5
11/05/21 Working on script videos that will accompany the levels 2
11/05/21 Thinking of ideas for name of the app 0.5
12/05/21 Working on script and asking for feedback on it 1
12/05/21 Updating wiki 1
17/05/21 Meeting 1
20/05/21 Working together with Eline, animating videos to fit the levels 8
20/05/21 Updating wiki 1
22/05/21 Animating introduction video drag and drop levels 8
26/05/21 Meeting and processing minutes 1.5
27/05/21 Working together with Eline and Luuk, animating videos 8
28/05/21 Animating reward videos 7
29/05/21 Animating reward videos 5
Total Hours: 60

Robin van de Hoef

Date Description Hours
22/04/21 Brainstorm meeting 1
29/04/21 Meeting 0.5
03/05/21 Meeting 1
07/05/21 Searching for suitable literature 2
09/05/21 Searching for and researching literature 4
10/05/21 Meeting 1
14/05/21 Installing and testing python image libraries 1
15/05/21 Color recognition 2
16/05/21 Image separation 4
17/05/21 Meeting 1
17/05/21 Increasing code performance 2
21/05/21 Testing code with front-end output 3
22/05/21 Debugging 4
23/05/21 Rewriting and optimizing code 3
24/05/21 Trying to fix black pixel issue 4
11 .
Total Hours: 33.5

Jeroen Pullen

Date Description Hours
21/04/21 Research project ideas 1
22/04/21 Brainstorm meeting 1
26/04/21 Prepare poject planning 1
27/04/21 Research state of the art 1
28/04/21 Updating Wiki 1
29/04/21 Contacting stakeholders 1
29/04/21 Meeting 0.5
29/04/21 Preparing stakeholder questions 1
30/04/21 Interview first stakeholder 1.5
03/05/21 Meeting 1
03/05/21 Prepare second stakeholder meeting 1
03/05/21 Interview second stakeholder 1.5
04/05/21 Work out model using interview data 2
05/05/21 Work out model by researching learning paths 3
05/05/21 Work on Wiki 1
08/05/21 Work on Wiki 1
09/05/21 Work out complete model based on combined information 2
10/05/21 Meeting 1
11/05/21 Create Level Generator 3
12/05/21 Create Level Generator 2
13/05/21 Create Level Generator 3
16/05/21 Discuss Level Design with new stakeholder 1
16/05/21 Redefine level design according to advice & SLO 2
17/05/21 Finalize Level Generator according to redefined levels 3
19/05/21 Watch HTML tutorials 2
20/05/21 Watch Python to HTML tutorials 2
22/05/21 Turn Python Level Generator into Javascript 2
22/05/21 Update wiki on level design 1
24/05/21 Finish Level 1 & 2 in Javascript 3
25/05/21 Finish Level 3 in Javascript 1
26/05/21 Meeting 1
26/05/21 Contact with stakeholder for possible testing 1
27/05/21 Implement Javascript levels in website 3
Total Hours: 52.5

Veerle Uhl

Date Description Hours
22/04/21 Brainstorm meeting 1
29/04/21 Meeting 1
29/04/21 Literary research 1.5
30/04/21 Literary research 1
03/05/21 Meeting 1
03/05/21 Literary research on how children learn 2
09/05/21 Literary research on the Number World model 2
09/05/21 Work on wiki, update the research and timetables 1
10/05/21 Meeting 1
10/05/21 Write a more detailed planning on the wiki 1
12/05/21 Write something about the use of research 1.5
16/05/21 Write something about our use of OSL and number worlds 2
17/05/21 Meeting 1
17/05/21 Looking into "inhoudslijnen" of OSL and level design 2
19/05/21 Editing wiki texts 5
19/05/21 writing wiki texts: level and website design 2
24/05/21 Editing wiki texts 1.5
27/05/21 reading through website code / work done by others 1
27/05/21 Begin work on project plan 3
27/05/21 work out testplan 2
Total Hours: 32.5