PRE2017 3 Group8 Test plan

Main wiki page

Online Questionnaire

Due to issues in organizing the focus group, the team decided to also look into creating an online questionnaire. This questionnaire is similar in content to the focus group, although it will of course not be possible to record a discussion between participants. The questionnaire is currently in development, but its contents will be posted here when it is finished.

0LAUK0 - Focus group scheme draft

Participants: ideally 1 teacher of every course, or most math teachers.

Time: TBA

Location: TBA

Date: 19-03-2018 - 23-03-2018

Introduction

[Welcome participants]

Let us first introduce ourselves, we are students from Eindhoven University of Technology. We are currently working on the project "Robots everywhere". Within this project we attempt to do something new and interesting with the research done on robotics and artificial intelligence. Our team has chosen to research an interactive online learning system that can adapt the homework based on the strengths and weaknesses of students.

In this focus group we want to discuss the possibilities of an adaptive online learning system that can aid your students in making their homework. This system will use some form of artificial intelligence to measure the performance of students, and to decide what topics need the most attention in their homework. You, as teachers, can use these performance metrics to gain detailed insight in the performance of every student. This can also allow shy students to more easily show that they need attention when it comes to certain topics that are taught in your courses. The aim of this system is to motivate students to make efficient use of their time when making homework, the system will supplement traditional lessons.

Moving on to the planning of today. In this focus group you will discuss topics that are relevant to this system. We are interested in a discussion amongst our participants, to find out what you really find important, so we can use that information to improve our design. Our role as researchers is to moderate this discussion.

Before we continue we would like you to sign these informed consent forms. Notes will be made of this discussion, as well as an audio recording. During analysis of this data your personal information will be removed. Everything you say here will be treated confidentially. On your request we will destroy the data after the research is completed.

We cannot stress enough that your participation in this focus group is completely voluntary. You have the right to withdraw from the focus group at any time. We will gladly answer your questions regarding the research. There may be certain questions which we cannot answer before the main discussion starts, but we will come back to them during the debriefing. To confirm that you agree to these conditions we kindly ask you to sign these forms. [hand over informed consent forms]

[Misschien moeten we ook nog kijken naar een manier om makkelijk de deelnemers voor ons zelf te identificeren, met naambordjes bijvoorbeeld.

Topics to discuss

[Begin showing the system, what it looks like for students and teachers, show where a few of the primary functions can be found.]

“In what sense do you think the system will be an improvement of the studying process? In other words, what do you think is good about the system?”

• [Discussion]
  • [If the discussion staggers, “Think of other online learning systems that you may know, and how this system compares to those, or think of what you think the current school system lacks and what gaps this may fill in.”]

“What do you think the system lacks?”

• [Discussion]
  • [Same stagger solution as before]

“How would you like this system to work?”

• [Discussion]
  • [“What functionalities does the system need to have? In other words, what should you be able to do with it?”]

“What kind of output do you want the system to give you, as teachers?”

• [Discussion]
  • [“Do you want to see students’ grades or learning curves, how they are doing in other subjects, etc.”]

“Are there any additional features you think the system must have? Anything you haven’t gotten the chance to mention yet?”

• [Discussion]

“Any remaining comments?”

• [Discussion]

Debriefing

[Answer remaining questions]

Test plan

Research question

Does the use of an adaptive learning agent affect students’ enjoyment and learning efficiency when compared to the use of a non-adaptive learning agent?

Hypothesis

The adaptive learning agent (ALA) is more pleasant for students to work with and improves the students’ learning efficiency when compared to a non-adaptive learning agent (NLA).

Research objective

To investigate the effects of using an adaptive learning agent — that is designed according to current research on flow theory, teaching models, gamification and e-learning — on teaching high-school students.

Method

Required is a self-study period of at least 35 minutes. The experiments start off with a short 5-minute test to assess the students’ initial level. This test contains more questions than the students can possibly finish in the five minutes. The tests will be scored based on how many answers the students gave and how many of those were correct. After that, the students will be given time to practice the subject matter. A third of the class will be given the ALA, another third the NLA. Both will be programmed with the subject matter relating to the last instruction. Neither of these groups will know to which group they belong. The remaining third will simply be given the book. After a fixed time, all students will be given another test, similar to the one they were given in the beginning. At the end, all students that used the learning agents will be given a short questionnaire about their experience and where they can indicate what they think the system lacks.

Schedule

• 5-minute instruction
• 5-minute test on the subject matter
• Practicing (total study time minus 20 minutes)
• 5-minute test on the subject matter
• 5-minute reflection

When testing is done using an online questionnaire, the participants will be separated in two groups, one with the ALA, one with the NLA. The schedule remains the same, except the instruction will simply be written in text, and no fixed time interval has to be taken into account. The same goes for the reflection. Participants ‘are done when they are done’.