Verslag group 14

Return PRE2017 3 Groep14

Introduction

Slim In Rekenen

For the course Robots Everywhere from the Technical University of Eindhoven, a project about some form of robotics needed to be made. The only explanation given was that robotics has to be part of the project in some way and that a product needed to be delivered at the end. Because of the huge amount of freedom in this course, for this particular project it was decided to create a smart quizzing program for in class. The program can be used in groups 3 and 4 from schools with a traditional educational tactics, all research is focused on these schools in the Netherlands. Simple mathematics is the subject that this quiz will cover, so addition, subtraction, multiplying, and division. The purpose of the program is easily identifying students that need some extra teacher attention, while stimulating all students on an individual level. The way in which this is done is by finding the child’s knowledge boundary based on his or her answers to the previous questions. This way, the child will get questions that are still challenging enough to be interesting, but not too hard that the pupil will give up. As a quizzing program will be started at the same time for a whole class, but all pupils soon have an individual test, it is a perfect combination of individualism while working as a class so nobody feels left out. All questions in the test are multiple choice to make it easier for the children to input their answers. The teachers will be able to see from the results which children have mastered the subjects sufficiently and are ahead of the program, but also which pupils lack behind. Because these results will be visible very quickly, the teacher knows sooner which children will need some extra attention to catch up. This extra attention can then be given when the teacher sees fit. To test whether or not teachers think the program could make an actual difference in their teaching programs, an questionnaire was send to 50? elementary school teachers. The questionnaire focused on their opinions and whether they had tips to keep in mind while designing the program. An important question is whether or not they think the program will help them plan their time better, as this is our main goal?. The following report is an elaborate documentation about the program, the questionnaire and the results from this.

Problem Description

The purpose of our program is to quickly recognize which children need extra help on a specific topic and to customize the test to each child, such that they can train specific topics on their own level. Quickly recoginizing the children will help the teacher to immediately step in. For example at the start of the year teachers do not know the weaknesses of the children, but our program can find them immediately after 1 test. Furthermore the personalised test ensures that the children will not lose interest, due to the questions being to easy, and ensures that they will not feel left out, due to always getting the questions wrong. These personalised questions are determined on the answers of previous questions.

Our program is now focussed on children in the third and fourth grade of the elementary school in the Netherlands using a tradional education system. The test consists of mathematical questions, that are divided by subject, for example multiplication tables or adding numbers up to 20. These choices were made to ensure an easy starting point for the program.

Objective

Develop a smart quiz program for a computer that can assess knowledge levels of the pupils, to give the teacher a better view on his pupils, and ask questions to the pupils on their personal boundary, so they learn effectively.

Users

The users of the program will be the pupils of the third and fourth grade of the elementary school in the Netherlands using a tradional education system, the parents and teacher of these pupils. The users will have the following requirements:

• The pupils want:
  • A program that stimulates their imagination
  • A test that remains interesting throughout the whole session
  • A test that does not make them feel dumb or left out

• The parents want:
  • A test that stimulates their children to learn and to perform the best they can.
  • Their children to be happy
  • Check their children's progress
  • The privacy of their children to be guaranteed

• The teacher wants:
  • A test that motivates the pupils to learn and to perform the best they can
  • The system to match the curriculum, meaning the teacher has to change the settings of each test
  • To check the progress of each pupil in an easy way
  • To divide his or her time in the best way over all the pupils

For the program we did not put our focus on privacy, because this is only a small part of our program. The program will not contain an interface for the parents, due to fact that the goal only incorporates the pupils and the teachers. However this could be added later on if desired. Almost all of the other the requirements will be implemented in our program by the use of the traditional didactic system.

Intelligent Quiz System

Given a set of questions, the quiz master will test the knowledge of a child, and help the child improve by asking the right questions at the right time. We will build an application that selects the next question to ask the child, based on the previous answers the child gave to previous questions. The quiz will find the knowledge level of the child and ask questions at the child's knowledge boundary so he can still learn from the question but will not be overwhelmed.

The quiz master has to:

• Find out the level of knowledge the child has, and ask questions that are on the 'edge' of a childs knowledge in order to improve their knowledge.
• Generate new questions.

In order to do so, we must:

• Define distance (or question similarity) between questions, which questions are of similar difficulty. So cluster questions based on their difficulty. Note that this will vary per child.
• Simulate the (increasing/decreasing) knowledge of different children. (To be able to train our app.)
• Construct a (large enough) data set to use parts of it for training and validation.
• Find out what the next right question would be. Our app should do this, based on the question similarity for a certain child.

Desk Research

Didactic Systems

In the book, het didactische werkvormenboek , a few important aspects of teaching are found.

• The best way for pupils to take in information is when they read and/or see it.
• When asking questions, there need to be both open questions and closed questions.
• Make sure pupils work with the whole class, but also occasionally in smaller groups or on their own.
• Switching between teaching tactics stimulates learning. An example could be switching between a game and simple sums.

For our project we chose to focus on a traditional education system instead of a constructive education system. This means that we will keep in mind that

• The teacher largely decided on the curriculum and order in which subjects are taught.
• The focus lies on teaching the whole class at once.
• There is a curriculum in which some subjects are more central than others.
• Learning is an individual activity.
• Pupil’s progress is checked by means of tests.

The Use of Computer Programs in Dutch Education

From a government study , the following things became clear:

• Separating the class in smaller groups in year 3 has a positive effect on the pupils’ performances as it causes more interaction.
• More than 90 percent of the teachers in primary education use computers to teach. According to them using ICT can contribute to a more efficient, effective and likeable education.

The next article shows us that education often does not seem to fit the natural, experimental learning process that children have. In games it is possible to naturally lean, which is why using games can be relevant for primary education. There would have to be a good balance between playing and effectively learning, to make sure that the advantages of learning in a fun way are not lost. This means the following things need to be taken into account when asking questions:

• The information should not be presented in a way that is too abstract, make sure it speaks to the children’s imagination.
• There needs to be a lot of repetition.
• Important information should be taught in more than one way.
• Take some time, it should not be taught too fast.
• Use realistic characters.
• Use animation, childlike dialogues, interaction and direct feedback.
• Do not punish children for making mistakes.

The following factors influence the quality of a course.

• From the international and Dutch literature it follows that all researched interventions that focused on the methods used during instructions and lectures were effective. It is nearly impossible to identify the elements that make a certain intervention work effectively, as several effective interventions might have opposite starting points. Furthermore there are not a lot studies that directly compare different methods. The resulting idea is that `it works to manipulate the methods used'. This is in line with the report of the KNAW (2009) that the Dutch research available does not give a clear view on the relation between the didactics and the skills when it comes to teaching arithmetics.
• Using technology for teaching arithmetics has a positive influence on the prestations of children, just as using real three dimensional objects. This follows from both international and Dutch research, for example the use of the programs Snappet and Rekentuin. It is not clear how much the use of the programs increased the available time to teach arithmetics, and thus this will need some further investigation.
• The use test results to improve the learning process has a positive relation with the arithmetic prestations. This can either be feedback to the teacher, for example by means of a digital system to track students, or direct feedback to the students per question that is done.
• Dividing students in different groups which each have their own level has a positive influence on the prestations of the individual students. Note that further investigation is required with respect to the implementation of this division. In The Netherlands, it is common to divide students in three different groups.

Individual or in the Class

• As we can see in the following article , the program is part of groupdifferentiation with individual learling lines. This means that there is quite some form of external regie but with focus on the individuals. Consequently, the teacher has some set goals and subjects on which the pupils need to work, and they will work on these subjects when the teacher tells them to. However, as the program will ask questions on their personal knowledge levels, the individuals are also taken into account. This is a perfect combination of working with the class while working on one’s own.

Working on Paper vs Working on an Electronic Device

• As indicated in the following atricle using computers instead of paper to administrate achievement tests in elementary schools is supported. Using computers has multiple advantages such as being able to use an adaptive technology, which is the goal for the finished program. On the other hand, it is possible that children experience computer anxiety resulting in lower achievements, but for this project it is assumed that this will not be the case as most children these days work with computers from very early on.

Articles About the Layout of the Test

How to implement feedback

• In this article it becomes clear that attributional feedback to children is useful as it is an effective way to promote rapid problem solving, self-efficacy, and achievement. This is probably because children have a sense of how well they are doing and attributional feedback helps to support these self-perceptions and validates their sense of efficacy. Because of this, the children will stay motivated to work on leading to a better performance in the end. The best way to do this is by giving them ability attributional feedback, so telling them they are either very smart, or not that good at math.
• Feedback in between The major findings evidenced in this research were the following:
  • Feedback was generally effective for learning, but more so on the lower level (identical) questions than on the higher level (reworded) ones;
  • Feedback information had greater impact in the absence of supporting text than with supporting text;

In this article they used several feedback methods such as knowledge of correct response (KCR), which identifies the correct response, and answer until correct (AUC). These two were shown to be the most effective.

When implementing these articles about feedback in our program the following decisions were made the KCR method with almost no supporting text except for the attributional feedback will be used.

How to Construct Arithmetical Questions

When constructing arithmetic questions it is important to know how students put numbers and operations on numbers into context. In the article it is explained what should be taken into consideration.

Mental Arithmetic

Mental arithmetic is insightfully doing calculations (arithmetic) with numbers, while the value of the numbers is kept in mind. While doing mental arithmetic one also makes use of ready knowledge, and properties of numbers, operations, and the underlying relations. There are three main methods for doing mental arithmetic:

• Step-by-step mental arithmetic: [math]\displaystyle{ 36 + 12 \rightarrow 36 + 10 \rightarrow 46 + 2 = 48 }[/math].
• Mental arithmetic by dividing the digits: [math]\displaystyle{ 36 + 12 \rightarrow 30 + 10 = 40 \rightarrow 6 + 2 = 8 \rightarrow 40 + 8 = 48 }[/math].
• Handy mental arithmetic, for example by compensation: [math]\displaystyle{ 73 + 29 \rightarrow 74 + 30 }[/math].

"Use of symbols in mathematical questions" In an article about the mathematical development of children it is mentioned that in order to get a correct numerical magnitude representation, children need to practice with symbolic numbers, so digits, but also with non-symbolic formats, like the herd questions in our program. Therefore we have to keep this in mind when implementing this in the layout of the questions our program.

From these articles we can conclude that we want to focus on learning in a likeable way, using a computer program. We will do this by asking simple maths questions using fun pictures of animals. This way we hope to intrinsically motivate the children to play with our program. Other things we will implement is that we want to make sure it will not go too fast and we can use child-friendly language.

Program

Score

In the program we use a score function to determine which level a pupil is. There are 3 levels, easy, average or hard. If the child is in an easy level, the questions will become simpler by narrowing the range set by the teacher by... . If the child is in the hard level, it will broaden the range by ... . The score of a child is determined by the following function:

[math]\displaystyle{ score = score+\frac{The Current Time}{The Total Time} }[/math]

here the total time and the current time are the times of the question. This score is than compared with the borders of the three groups. These borders use the following function:

Child Interface

Description with pictures of all the things the children can adjust and see

Start

Waiting screen for children.

When the program is started for a child, they will first see a waiting screen, as is shown on the right. The children do not have to do anything as long as they see this screen, but wait. While the children are waiting, the teacher will generate a quiz. After the teacher has generated a quiz, the children will be able to join.

Screen for children where they enter their name to joing a quiz.

The children get to see a screen where they can enter their name and click "Start!" By clicking start they will join the quiz and their name will appear on the leaderboard which only the teacher can see. After joining a quiz, a child will see an empty question screen. Now the child has to wait until the teacher starts the quiz (for everyone at once).

Questions

When the teacher starts the quiz, the children will see a question. We have 2 types of questions:

• Flock questions, where children can add or subtract two flocks of pigs, cows, or chickens.
• Animals with small blackboards which have simple additions or subtractions on them.

By giving these two different kinds of questions, basic equations to make sure the symbolic numbers are learned and on the other hand the herd questions which combine the non-symbolic format with a digit, the previously found desk research is accounted for. Here the non-symbolic format is very much needed to get a correct numerical magnitude representation, but on its own it will not have a big effect on the school-relevant mathematics. To make sure this is the case we combine the two, as the children can count the herds (non-symbolic) and have to click the correct number (symbolic) of the amount of animals. All questions are multiple choice to make it easier for the young children to reply.

The two type of questions. Left. A flock question. Right. A regular question.

Feedback

The children will get feedback on every question they answer, as supported by our literature study. Because the children are most likely to be young and are only just learning to read, we use smiley faces to indicate to the children how they are doing. A happy face means that they answered the question correctly, a sad face means they had the question wrong.

Feedback for the children. Left. This question was answered correctly. Right. This question was answered wrong.

End

After the last question has been answered, the program is automatically closed.

Teacher interface

According to this website students from our target audience should be able to do the following things at the end of the school year:

• Group 3 can add and subtract from 0 to 20.
• Group 4 can multiply and add and subtract from 0 to 100.

For this reason exactly, we made sure the teacher can change the settings of the program to fit the needs of the current group. The teacher can choose between addition, subtraction, multiplication, and division on the range [x, y] or a combination of these operations. Furthermore, the teacher can input the maximal number of arguments each questions can have, how many questions there are in a session, whether the questions will be done with integers or decimals, and whether the program will go through a session automatically or manually.

Before starting a quiz

Interface of the teacher before the quiz starts

When the teacher starts a new quiz they will see a form on which they can adjust the settings for the quiz. When clicking the 'start' button, the program will generate exactly as many unique questions as the teacher requested in the category they provided. The form on the right will generate 20 questions. The numbers in each question (both the numbers in the question itself and its answer) will be within the range [1, 20], and this while be integers. The operations in this question set will only be addition and subtraction. The nubmer of questions for each operation will be random, so there might not be 10 addition and 10 subtraction questions. The maximum number of arguments of each question in this quiz will be two, which means that each question will be of the form x + y. When the children answer the questions in this quiz, they will have 60 seconds for each question. When these 60 seconds are over and a child has not answered the question, the question will be handled as answered uncorrectly. The upper of the last two checkboxes controls if the quiz automatically continues to the next question after all children have answered or time is up, or if the teacher manually has to start the next question. The last checkbox controls whehter the children get to see the timer for each question.

Results

Survey

The purpose of the survey will be the following:

• look if there is demand for our program
• look if the program can be implemented
• look if teacher will find it a fun new addition to their class
• important features the program should have according to teachers

Conclusion

Part 1

• relatie tussen onderzoek en hoe het eruit ziet en hoe het werkt( wat komt er uit het literatuur onderzoek en hebben we gebruikt in de praktijk)

Part 2

• conclusion of the enquete

Discusion

The following is summation of topics that would be of interest in further research.

• Implementation and---more importantly---the comparison of different types of questions, e.g. reading the clock. This can be done in terms of plug-in forms in the program.
• Machine learning classification of pupils in (possibly more than three) categories.
• Changing the question difficulty based on more question properties (e.g. operations or 'tricky' multiple choice questions) and measuring the effect of these changes.
• Implementation of the quiz on different platforms, as well as surveying the need for this.