PRE2020 4 Group3

Group members

Name	Student number
Eline Boom	1465872
Luuk van Dorst	1469789
Robin van de Hoef	1479679
Kyra Moviat	1349171
Jeroen Pullen	1477730
Veerle Uhl	1462229

Introduction

We want to create an interactive software learning children how to draw, count and recognize colors. Children will get small tasks like "draw 3 red circles", the software will recognize the extent to which this task is done correctly and provide compliments and or suggested improvements.

Project Plan

Subjects: Child psychology, learning, image processing, software development.

Objectives: Create software to help children learn

Users: Children (of parents that want to provide a playful learning app)

State-of-the art: Alternative apps that help children either count, draw or learn colors individually. Not in a combined way, we feel the combination might improve the way to learn these things. Additionally, we hypothesize that explaining problems via a childs own drawn elements could improve their understanding.

Approach: Make up a model for how the software should work, make software in which one can draw, program software that can recognize multiple drawn objects/shapes, implement everything in a working framework according to the model.

Milestones: Drawing software, working framework, Working object recognition, implementation in task framework

Deliverables: Peer review, Wiki, Final Presentation, The software (?)

Planning

Task	Week 1	Week 2	Week 3	Week 4	Week 5	Week 6	Week 7	Week 8
Brainstorm Project ideas	X
Investigate Literature		X	X
Create Software Model		X	X
Create Software Framework			X	X
Create Drawing Software			X	X
Create Object Recognition Software					X	X
Implement Recognition Software in Framework							X	X
Work on Wiki	X	X	X	X	X	X	X	X
Work on Presentation						X	X	X

Research

How children learn

From birth to around age 5, children develop "everyday mathematics". These are informal ideas of more and less, taking away, shapes and sizes. These everyday mathematics can be surprisingly broad, complex and sophisticated. The everyday mathematics can develop in interesting ways, without adult assistance.

From the age of 2 or so, children learn the language and grammar of counting. They memorize the first ten or so counting words, and then learn a set of rules to generate the higher numbers. When children are around 4 or 5 years old, they begin to develop metacognitive skills: They become increasingly aware of their own thinking and begin to express it in words. The hardest form of language for children to learn is the special written symbolism of mathematics, like 5, +, - or =.

Understanding numbers involves more than saying a few counting words. It involves reasoning about numbers, making inferences and developing a mental number line. Children also need to mathematize; to conceive of problems in explicitly mathematical terms. They need to understand that the action of combining one bear with two others can be meaningfully interpreted in the terms of the mathematical principles of addition and the symbolism 1 + 2.

There are different curricula possible with different ways to learn children of all ages to count. Examples are in the drive.

The number worlds curriculum

This curriculum is one of many to learn children how to use numbers and count and such. It provides us with a good understanding of how children learn to count. There is different knowledge that underlies number sense for 5-year olds. This knowledge is important for them to learn how to use numbers. Five year olds know:

- that numbers indicate quantity and therefore, that numbers themselves have magnitude

- that the word “bigger” or “more” is sensible in this context

- that the numbers 7 and 9, like every other number from 1 to 10, occupy fixed positions in the counting sequence

- that 7 comes before 9 when you are counting up

- that numbers that come later in the sequence

- that are higher up - indicate larger quantities and therefore, that 9 is bigger (or more) than 7.

- that each counting number up in the sequence corresponds precisely to an increase of one unit in the size of a set. This last one enables children to use the counting numbers alone, without the need for real objects, to solve quantitative problems involving the joining of two sets. In doing so, it transforms mathematics from something that can only be done out there (e.g. by manipulating real objects) to something that can be done in their own heads, and under their own control. It might be interesting for us to see which of these goals we want children to know at least. That way we can accurately set a goal for how much we need to teach the children.

As children get older, they progress through different knowledge they know: By the age of 4, children have constructed knowledge of counting and quantity. Sometime in kindergarten, children become able to integrate these knowledge networks. Around the age of 6-7, children connect this integrated knowledge network to the world of formal symbols. By the age of 8 or 9, most children become capable of expanding this knowledge network to deal with double-digit numbers and the base-ten system.

In the number worlds curriculum, there are 5 instructional principles that lie at the heart of the program: 1. Build upon children's current knowledge 2. Follow the natural developmental progression when selecting new knowledge to be taught 3. Teach computational fluency as well as conceptual understanding 4. Provide plenty of opportunity for hands-on exploration, problem-solving and communication 5. expose children to the major ways number is represented and talked about in developed societies

Lastly, the core of number sense forms a knowledge network that is called central conceptual structure for number. This core is important for children for two reasons: 1) it enables children to make sense of quantitative problems across context, 2) it provides the foundation on which children's learning of more complex number concepts is built. This core is something we can use to argument why our application will be very important. Our app will teach children the basics of this core, which they can use to build further knowledge on and they can use it in different ways in the real world.

In the drive there is a file with worked out summaries of articles read and articles to (possibly) read

State of the Art

There are already a lot of apps on the market that are geared towards educating children. Certainly apps that teach children how to count or how to draw separately. Combining the two is also used, but less frequently. Two apps that use both are easily found, one made by Bini Bambini and the other by Kids Academy, and both games work the same. The child makes a single drawing that is quite extensive, by tracing over lines that the game puts down for them in small steps. When the drawing is done, it multiplies and the child needs to trace over the corresponding number the game offers them. These games give the child auditory feedback, by for example saying the color of the pencil they have selected out loud. This means in those games both the drawing and number are given to the child.

Our product is different because we combine drawing and counting to give the child a different type of challenge. The game will give the kid(s) a prompt, for example: “Draw three red circles”. Since this asks kids to reproduce knowledge it could work better when they can work together, for example in a classroom setting. Or they have a little more knowledge of counting than the children using the two aforementioned apps. Different levels could also involve more complicated drawings.

Software Model

Important Features by Stakeholders

3-4 years old: Easy levels, can count to 10, can manage color comparison, learning pen grip important, learn to count via touch, “which object is blue”? appropriate question, combination of tasks as described in goal level still to hard.

4-5 yeas old: Goal level, know all colors, can count, learn shapes important, number recognition important.

*What makes the software good?

- Rising level of difficulty (3-4 to 4-5 level ?)

- Possible on digital school board

- Possible with pen (learn pen grip)

- Returns performance data to authorized figure

- Test mode

- Feels like a game (reward system)

*What can kids learn from the software?

- Colors, Shapes, Counting & Drawing

- Pen grip

- Vocabulary (if drawing objects)

- Listening skills (if tasks given in audio)

- Number recognition (number on screen)

*Useful purposes

- Being able to use the software collaboratively (class setting, performing tasks together)

- Being able to use the software to test how far children are.

- Being able to have the kid perform tasks on their own, behind computer or tablet.

Learning Goals - End Second Grade

Colors: “Knows all colors”

Stakeholders

Shapes: “Be able to construct shapes”

https://www.slo.nl/thema/meer/jonge-kind/doelen-jonge-kind/

Counting goals: “Learn to count to 20, can recognize amounts up to 12”

https://www.slo.nl/thema/meer/jonge-kind/doelen-jonge-kind/

Drawing/Motor skills: “Learn pen grip”

Stakeholders

Vocabulary: “Understand instructions, learn new words”

https://www.slo.nl/sectoren/po/inhoudslijnen-po/inhoudslijnen-nederlands/

Number Recognition: “Know, read and write numbers”

https://www.slo.nl/sectoren/po/inhoudslijnen-po/inhoudslijnen-rekenen-wiskunde/

Listening skills: “Learn how to follow a task description”

https://lesintaal.nl/platform_taaldidactiek/1_mondelinge_taalvaardigheid/kennisbasis.htm

Digital skills: “Playfully get acquainted with technology and how to operate them, partially via educative games”

https://www.slo.nl/thema/meer/jonge-kind/doelen-jonge-kind/

Image recognition

In order to recognise images drawn by children we have decided to use a neural network. In order to train this network we have settled upon a couple of usefull databases that contain drawings and letters, I will list the found databases below:

-MNIST dataset (http://yann.lecun.com/exdb/mnist/)

-Google quick draw dataset (https://github.com/googlecreativelab/quickdraw-dataset)

-(One other dataset that I decided to use but can't find the source for anymore as of this moment :( )

We have also created some code which can read a database, build a neural network, train a network and test the resulting network. However thus far the results from the neural network have been dissapointing, in total the succes rate of the neural network has been 0.0036 which is worse than random guessing (with 250 labels that would be 1/250 = 0.004) so optimisation is still necessary. The low accuracy rate can either be the result of using a too small dataset (each of the 250 labels have only 50 or so images in them) or because of wrong parameters used in the neural network or because we have not trained it long enough. Our current guess is that it's a mix of the first two and we should use less labels and more images to increase accuracy and add more internal nodes and layers to our neural network.

In order to solve these problems and move to a succesfull working prototype I suggest the following goals for next week:

1. Fix the accuracy rate and try to get a neural network to at least work with one of these datasets.

2. Try to find a way to extract the complete neural network from our code and find a way to implement it on our website.

3. Decide which labels are usefull and chose a final dataset which we train our network on.

Of these three the biggest challenge will probably be problem 1 since training the network takes a lot of time so we cannot really bruteforce our way to succes which means we should probably look at what previous work has been done in this subject.

Platform

Link to website: https://group3use.glitch.me/

Link to code: https://glitch.com/edit/#!/group3use

Tasks

Eline Boom

Week	Tasks	Number of hours
1	Research project ideas and brainstorm	2
2	Setting up a website as platform for the project	5
3	Creating a basis for drawing on the website and converting drawings to png's	14
4	Example	Example
5	Example	Example
6	Example	Example
7	Example	Example
8	Example	Example

Luuk van Dorst

Week	Tasks	Number of hours
1	Example	Example
2	Example	Example
3	Example	Example
4	Example	Example
5	Example	Example
6	Example	Example
7	Example	Example
8	Example	Example

Robin van de Hoef

Week	Tasks	Number of hours
1	Example	Example
2	Example	Example
3	Example	Example
4	Example	Example
5	Example	Example
6	Example	Example
7	Example	Example
8	Example	Example

Kyra Moviat

Week	Tasks	Number of hours
1	Researching topics and brainstorm meeting	5
2	Two meetings, researching state of the art, updating wiki	4
3	Meeting, updating wiki, writing about state of the art, literature research	5
4	Example	Example
5	Example	Example
6	Example	Example
7	Example	Example
8	Example	Example

Jeroen Pullen

Week	Tasks	Number of hours
1	Research ideas and brainstorm for project	2
2	Prepare project planning, contact and meet with stakeholders	7
3	Meet with stakeholders, plan out product model and update wiki	12.5
4	Create Level Generator function in Python	Example
5	Example	Example
6	Example	Example
7	Example	Example
8	Example	Example

Veerle Uhl

Week	Tasks	Number of hours
1	Brainstorming project ideas, meeting	2
2	broad research on children's learning	3,5
3	research on learnign of children and specific learning models	6
4	Example	Example
5	Example	Example
6	Example	Example
7	Example	Example
8	Example	Example

Time Log

Name, Total, Breakdown

Eline Boom

Date	Description	Hours
22/04/21	Brainstorm meeting	1
29/04/21	Meeting	0.5
29/04/21	Setting up the website	3
30/04/21	Design of website	2
03/05/21	Meeting	1
03/05/21	Creating canvas function on website	1
03/05/21	Converting canvas to image	2
03/05/21	Colour pens and eraser	3
06/05/21	Colour pens and eraser	2
06/05/21	Switching between pen colours through button	2
06/05/21	switching between draw and erase through button	4
	Total Hours:	21.5

Luuk van Dorst

Date	Description	Hours
22/04/21	Brainstorm meeting	1
29/04/21	Meeting	0.5
29/04/21	Searching for viable datasets	2
30/04/21	Researching neural networks	3
02/05/21	Making a prototype neural network	4
03/05/21	Meeting	1
08/05/21	install and get tenserflow working (took longer than expected)	1
08/05/21	Found some additional datasets	1
08/05/21	Finished neural network code	4
09/05/21	Did some research into how much nodes the neural network should have to increase performance	2
09/05/21	Tested some parameters to increase performance	4
10/05/21	edited wiki	1
	Total Hours:	10.5 + 12 = 22.5

Kyra Moviat

Date	Description	Hours
22/04/21	Brainstorm meeting	1
29/04/21	Meeting	0.5
03/05/21	Meeting	1
10/05/21	Meeting	1
5	.
6	.
7	.
8	.
9	.
10	.
11	.
	Total Hours:	x

Robin van de Hoef

Date	Description	Hours
22/04/21	Brainstorm meeting	1
29/04/21	Meeting	0.5
03/05/21	Meeting	1
10/05/21	Meeting	1
5	.
6	.
7	.
8	.
9	.
10	.
11	.
	Total Hours:	x

Jeroen Pullen

Date	Description	Hours
21/04/21	Research project ideas	1
22/04/21	Brainstorm meeting	1
26/04/21	Prepare poject planning	1
27/04/21	Research state of the art	1
28/04/21	Updating Wiki	1
29/04/21	Contacting stakeholders	1
29/04/21	Meeting	0.5
29/04/21	Preparing stakeholder questions	1
30/04/21	Interview first stakeholder	1.5
03/05/21	Meeting	1
03/05/21	Prepare second stakeholder meeting	1
03/05/21	Interview second stakeholder	1.5
04/05/21	Work out model using interview data	2
05/05/21	Work out model by researching learning paths	3
05/05/21	Work on Wiki	1
08/05/21	Work on Wiki	1
09/05/21	Work out complete model based on combined information	2
10/05/21	Meeting	1
	Total Hours:	22.5

Veerle Uhl

Date	Description	Hours
22/04/21	Brainstorm meeting	1
29/04/21	Meeting	1
29/04/21	Literary research	1.5
30/04/21	Literary research	1
03/05/21	Meeting	1
03/05/21	Literary research on how children learn	2
09/05/21	Literary research on the Number World model	2
09/05/21	Work on wiki, update the research and timetables	1
10/05/21	Meeting	1
10	.
11	.
	Total Hours:	10,5