PRE2020 4 Group3

Group members

Name	Student number
Eline Boom	1465872
Luuk van Dorst	1469789
Robin van de Hoef	1479679
Kyra Moviat	1349171
Jeroen Pullen	1477730
Veerle Uhl	1462229

Introduction

We want to create an interactive software learning children how to draw, count and recognize colors. Children will get small tasks like "draw 3 red circles", the software will recognize the extent to which this task is done correctly and provide compliments and or suggested improvements.

Project Plan

Subjects: Child psychology, learning, image processing, software development.

Objectives: Create software to help children learn

Users: Children (of parents that want to provide a playful learning app)

State-of-the art: Alternative apps that help children either count, draw or learn colors individually. Not in a combined way, we feel the combination might improve the way to learn these things. Additionally, we hypothesize that explaining problems via a childs own drawn elements could improve their understanding.

Approach: Make up a model for how the software should work, make software in which one can draw, program software that can recognize multiple drawn objects/shapes, implement everything in a working framework according to the model.

Milestones: Drawing software, working framework, Working object recognition, implementation in task framework

Deliverables: Peer review, Wiki, Final Presentation, The software (?)

Planning

Task	Week 1	Week 2	Week 3	Week 4	Week 5	Week 6	Week 7	Week 8
1. Brainstorm Project ideas	X
2. Investigate Literature		X	X
3. Create Software Model		X	X
4. Create Software Framework			X	X
5. Create Drawing Software			X	X
6. Create Object Recognition Software					X	X
7. Implement Recognition Software in Framework							X	X
8. Work on Wiki	X	X	X	X	X	X	X	X
9. Work on Presentation						X	X	X

Subtasks

subtask	Done / to do by who
1.1 Find topics that are of interest and comply with technical skills of the group	Everyone
2.1 Investigate state of the art	Kyra
2.2 Investigate how children learn to count and learn colors and shapes	Veerle/Robin
2.3 Investigate current models or curricula that are used for learning	Veerle
2.4 Investigate the influence of online or "games" learning	Veerle/Robin
3.1 Meet with stakeholders to see what they want to see in an excercise	Jeroen
3.2 Figure out what kind of levels we want to make	Jeroen
3.3 Implement levels in the website	Jeroen / Eline
3.4 Make an instruction video for the levels	Kyra / Eline
4.1 Set up a website	Eline
4.2 Code and make a website layout, make it suitable for young children	Eline
5.1 Set up a canvas where users can draw	Eline
5.2 Find a way to use colors, a drawing tool and an eraser in the canvas	Eline
6.1 Find databases to train a neural network with	Luuk
6.2 Write code for a neural network	Luuk
6.3 Write code that can read a database and train a neural network	Luuk
6.4 Work on improving the accuracy of the neural network	Luuk
7.1 Make the drawing made convert into a png	Eline
7.2 Make the neural network compatible with a png	Luuk
7.3 Convert the drawing png so it cuts out every individual shape and recognizes color	Robin
8.1 Write down as much as we can on the wiki	Everyone
8.2 Transform the summary on the wiki into a good text with logical structure	-

Research

How children learn

From birth to around age 5, children develop "everyday mathematics". These are informal ideas of more and less, taking away, shapes and sizes. These everyday mathematics can be surprisingly broad, complex and sophisticated. The everyday mathematics can develop in interesting ways, without adult assistance.

From the age of 2 or so, children learn the language and grammar of counting. They memorize the first ten or so counting words, and then learn a set of rules to generate the higher numbers. When children are around 4 or 5 years old, they begin to develop metacognitive skills: They become increasingly aware of their own thinking and begin to express it in words. The hardest form of language for children to learn is the special written symbolism of mathematics, like 5, +, - or =.

Understanding numbers involves more than saying a few counting words. It involves reasoning about numbers, making inferences and developing a mental number line. Children also need to mathematize; to conceive of problems in explicitly mathematical terms. They need to understand that the action of combining one bear with two others can be meaningfully interpreted in the terms of the mathematical principles of addition and the symbolism 1 + 2.

There are different curricula possible with different ways to learn children of all ages to count. Examples are in the drive.

The number worlds curriculum

This curriculum is one of many to learn children how to use numbers and count and such. It provides us with a good understanding of how children learn to count. There is different knowledge that underlies number sense for 5-year olds. This knowledge is important for them to learn how to use numbers. Five year olds know:

- that numbers indicate quantity and therefore, that numbers themselves have magnitude

- that the word “bigger” or “more” is sensible in this context

- that the numbers 7 and 9, like every other number from 1 to 10, occupy fixed positions in the counting sequence

- that 7 comes before 9 when you are counting up

- that numbers that come later in the sequence

- that are higher up - indicate larger quantities and therefore, that 9 is bigger (or more) than 7.

- that each counting number up in the sequence corresponds precisely to an increase of one unit in the size of a set. This last one enables children to use the counting numbers alone, without the need for real objects, to solve quantitative problems involving the joining of two sets. In doing so, it transforms mathematics from something that can only be done out there (e.g. by manipulating real objects) to something that can be done in their own heads, and under their own control. It might be interesting for us to see which of these goals we want children to know at least. That way we can accurately set a goal for how much we need to teach the children.

As children get older, they progress through different knowledge they know: By the age of 4, children have constructed knowledge of counting and quantity. Sometime in kindergarten, children become able to integrate these knowledge networks. Around the age of 6-7, children connect this integrated knowledge network to the world of formal symbols. By the age of 8 or 9, most children become capable of expanding this knowledge network to deal with double-digit numbers and the base-ten system.

In the number worlds curriculum, there are 5 instructional principles that lie at the heart of the program: 1. Build upon children's current knowledge 2. Follow the natural developmental progression when selecting new knowledge to be taught 3. Teach computational fluency as well as conceptual understanding 4. Provide plenty of opportunity for hands-on exploration, problem-solving and communication 5. expose children to the major ways number is represented and talked about in developed societies

Lastly, the core of number sense forms a knowledge network that is called central conceptual structure for number. This core is important for children for two reasons: 1) it enables children to make sense of quantitative problems across context, 2) it provides the foundation on which children's learning of more complex number concepts is built. This core is something we can use to argument why our application will be very important. Our app will teach children the basics of this core, which they can use to build further knowledge on and they can use it in different ways in the real world.

In the drive there is a file with worked out summaries of articles read and articles to (possibly) read

State of the art

There are already a lot of apps on the market that are geared towards educating children. Certainly apps that teach children how to count or how to draw separately. Combining the two is also used, but less frequently. Two apps that use both are easily found, one made by Bini Bambini and the other by Kids Academy, and both games work the same. The child makes a single drawing that is quite extensive, by tracing over lines that the game puts down for them in small steps. When the drawing is done, it multiplies and the child needs to trace over the corresponding number the game offers them. These games give the child auditory feedback, by for example saying the color of the pencil they have selected out loud. This means in those games both the drawing and number are given to the child.

Our product is different because we combine drawing and counting to give the child a different type of challenge. The game will give the kid(s) a prompt, for example: “Draw three red circles”. Since this asks kids to reproduce knowledge it could work better when they can work together, for example in a classroom setting. Or they have a little more knowledge of counting than the children using the two aforementioned apps. Different levels could also involve more complicated drawings.

How we use the research

As a basis, we will use Number worlds curriculum and OSL curriculum. These have been tested and used by many schools and education institutes around the world and are proven to support learning of children.

Number words has 5 principles for learning, we used them as a framework to develop good exercises that cater to the needs of the children that want to learn. Principle 1 talks about building upon children’s current knowledge. We do this by slowly incrementing the difficulty of our levels. First, we ask children to simply move an object into a box, then when they have mastered this, they need to move a specific amount or specific color or shape of object into a box. This way, we slowly add more variables while building onto the knowledge children already have. Principle 2 is about following the natural developmental progression when selecting new knowledge to be taught. With this, we looked at what knowledge young children typically learn at what age. We also took into account the desired age for users and combined this to find what lessons work best. For example, we use squares and circles for objects instead of cars and dogs, since the latter is too far ahead in the developmental progress of knowledge for the small children. Principle 4 talks about providing opportunity for hands- on exploration and communication. We hope to do this by making our product work as well on a digiboard. This way, children and teachers can work together to solve problems and do exercises, promoting communication between them.

Software Model

Important Features by Stakeholders

3-4 years old: Easy levels, can count to 10, can manage color comparison, learning pen grip important, learn to count via touch, “which object is blue”? appropriate question, combination of tasks as described in goal level still to hard.

4-5 yeas old: Goal level, know all colors, can count, learn shapes important, number recognition important.

*What makes the software good?

- Rising level of difficulty (3-4 to 4-5 level ?)

- Possible on digital school board

- Possible with pen (learn pen grip)

- Returns performance data to authorized figure

- Test mode

- Feels like a game (reward system)

*What can kids learn from the software?

- Colors, Shapes, Counting & Drawing

- Pen grip

- Vocabulary (if drawing objects)

- Listening skills (if tasks given in audio)

- Number recognition (number on screen)

*Useful purposes

- Being able to use the software collaboratively (class setting, performing tasks together)

- Being able to use the software to test how far children are.

- Being able to have the kid perform tasks on their own, behind computer or tablet.

Learning Goals - End Second Grade

Colors: “Knows all colors”

Stakeholders

Shapes: “Be able to construct shapes”

https://www.slo.nl/thema/meer/jonge-kind/doelen-jonge-kind/

Counting goals: “Learn to count to 20, can recognize amounts up to 12”

https://www.slo.nl/thema/meer/jonge-kind/doelen-jonge-kind/

Drawing/Motor skills: “Learn pen grip”

Stakeholders

Vocabulary: “Understand instructions, learn new words”

https://www.slo.nl/sectoren/po/inhoudslijnen-po/inhoudslijnen-nederlands/

Number Recognition: “Know, read and write numbers”

https://www.slo.nl/sectoren/po/inhoudslijnen-po/inhoudslijnen-rekenen-wiskunde/

Listening skills: “Learn how to follow a task description”

https://lesintaal.nl/platform_taaldidactiek/1_mondelinge_taalvaardigheid/kennisbasis.htm

Digital skills: “Playfully get acquainted with technology and how to operate them, partially via educative games”

https://www.slo.nl/thema/meer/jonge-kind/doelen-jonge-kind/

Image recognition

In order to recognise images drawn by children we have decided to use a neural network. In order to train this network we have settled upon a couple of usefull databases that contain drawings and letters, I will list the found databases below:

-MNIST dataset (http://yann.lecun.com/exdb/mnist/)

-Google quick draw dataset (https://github.com/googlecreativelab/quickdraw-dataset)

-(One other dataset that I decided to use but can't find the source for anymore as of this moment :( )

We have also created some code which can read a database, build a neural network, train a network and test the resulting network. However thus far the results from the neural network have been dissapointing, in total the succes rate of the neural network has been 0.0036 which is worse than random guessing (with 250 labels that would be 1/250 = 0.004) so optimisation is still necessary. The low accuracy rate can either be the result of using a too small dataset (each of the 250 labels have only 50 or so images in them) or because of wrong parameters used in the neural network or because we have not trained it long enough. Our current guess is that it's a mix of the first two and we should use less labels and more images to increase accuracy and add more internal nodes and layers to our neural network.

In order to solve these problems and move to a succesfull working prototype I suggest the following goals for next week:

1. Fix the accuracy rate and try to get a neural network to at least work with one of these datasets.

2. Try to find a way to extract the complete neural network from our code and find a way to implement it on our website.

3. Decide which labels are usefull and chose a final dataset which we train our network on.

Of these three the biggest challenge will probably be problem 1 since training the network takes a lot of time so we cannot really bruteforce our way to succes which means we should probably look at what previous work has been done in this subject.

Platform

Designing the website

Link to website: https://group3use.glitch.me/

Link to code: https://glitch.com/edit/#!/group3use

Storyline throughout the levels

To make completing all the levels more appealing to children, a storyline will be shown through visuals in the levels and animated videos before them. The overarching storyline is about a group of animals (a horse, a monkey, a bunny and a tiger) that is moving from the city to the forest. The levels designed for 3-4 year olds are preceded by a video about the animals that explains the move and tells the children that they need help moving because there is a broad river around the forest without bridges. The levels itself are about building a bridge for each animal by moving the right shapes into the water. Upon completing the level the children are rewarded with a visual of the animal crossing the bridge. The levels designed for 4-5 year olds are preceded by a video about the animals having to find food now they live in the forest. In the levels they draw shapes in certain colors, when they get it right the shapes will transform into foods.

Time Log

Eline Boom

Date	Description	Hours
22/04/21	Brainstorm meeting	1
29/04/21	Meeting	0.5
29/04/21	Setting up the website	3
30/04/21	Design of website	2
03/05/21	Meeting	1
03/05/21	Creating canvas function on website	1
03/05/21	Converting canvas to image	2
03/05/21	Colour pens and eraser	3
06/05/21	Colour pens and eraser	2
06/05/21	Switching between pen colours through button	2
06/05/21	switching between draw and erase through button	4
	Total Hours:	21.5

Luuk van Dorst

Date	Description	Hours
22/04/21	Brainstorm meeting	1
29/04/21	Meeting	0.5
29/04/21	Searching for viable datasets	2
30/04/21	Researching neural networks	3
02/05/21	Making a prototype neural network	4
03/05/21	Meeting	1
08/05/21	install and get tenserflow working (took longer than expected)	1
08/05/21	Found some additional datasets	1
08/05/21	Finished neural network code	4
09/05/21	Did some research into how much nodes the neural network should have to increase performance	2
09/05/21	Tested some parameters to increase performance	4
10/05/21	edited wiki	1
	Total Hours:	10.5 + 12 = 22.5

Kyra Moviat

Date	Description	Hours
22/04/21	Research topics	4
22/04/21	Brainstorm meeting	1
29/04/21	Meeting	0.5
02/05/21	Add tables to wiki and update layout	1.5
02/05/21	Research state of the art	1.5
03/05/21	Meeting	1
09/05/21	Writing about state of the art	1.5
09/05/21	Updating wiki	0.5
09/05/21	Researching methods children learn counting	2
10/05/21	Meeting	1
11/05/21	Updating the wiki to only have 1 logging system instead of 2	0.5
11/05/21	Working on script videos that will accompany the levels	2
11/05/21	Thinking of ideas for name of the app	0.5
12/05/21	Working on script and asking for feedback on it	1
12/05/21	Updating wiki	0.5
10	.
11	.
	Total Hours:	19

Robin van de Hoef

Date	Description	Hours
22/04/21	Brainstorm meeting	1
29/04/21	Meeting	0.5
03/05/21	Meeting	1
10/05/21	Meeting	1
5	.
6	.
7	.
8	.
9	.
10	.
11	.
	Total Hours:	x

Jeroen Pullen

Date	Description	Hours
21/04/21	Research project ideas	1
22/04/21	Brainstorm meeting	1
26/04/21	Prepare poject planning	1
27/04/21	Research state of the art	1
28/04/21	Updating Wiki	1
29/04/21	Contacting stakeholders	1
29/04/21	Meeting	0.5
29/04/21	Preparing stakeholder questions	1
30/04/21	Interview first stakeholder	1.5
03/05/21	Meeting	1
03/05/21	Prepare second stakeholder meeting	1
03/05/21	Interview second stakeholder	1.5
04/05/21	Work out model using interview data	2
05/05/21	Work out model by researching learning paths	3
05/05/21	Work on Wiki	1
08/05/21	Work on Wiki	1
09/05/21	Work out complete model based on combined information	2
10/05/21	Meeting	1
11/05/21	Create Level Generator	3
12/05/21	Create Level Generator	2
	Total Hours:	27.5

Veerle Uhl

Date	Description	Hours
22/04/21	Brainstorm meeting	1
29/04/21	Meeting	1
29/04/21	Literary research	1.5
30/04/21	Literary research	1
03/05/21	Meeting	1
03/05/21	Literary research on how children learn	2
09/05/21	Literary research on the Number World model	2
09/05/21	Work on wiki, update the research and timetables	1
10/05/21	Meeting	1
10/05/21	Write a more detailed planning on the wiki	1
12/05/21	Write something about the use of research	1.5
	Total Hours:	13