Second project: Educative game for blind children

Group members

• Yngwie Baron (0936539)
• Axel Deenen (0947031)
• Moos Müller (0936214)
• Dimitar Nikolov (1000095)
• Wybe van Vlokhoven (0914565)

Project definition

Problem statement

There are not many resources or convenient ways to practice math and mathematics related skills for blind people. Overall, they have less reading resources that provide an adequate explanation. This is not necessarily a problem of the books themselves, but the fact that without a way to visually represent math and observe the whole exercise it becomes very tedious for them to do problem solving effectively. Take for example geometry or graphs of functions. The lack of visual aid makes the task much more harder and with most people impossible as it requires you to remember information you would otherwise be able to look back at. Those reasons make it less appealing to young children to learn math and math related sciences. This leads to unfamiliarity with math among blind people.

User analysis

The aimed users are fully blind children in the age of 6-14 years old. Also children with a major visual impairment, such that they cannot read visually-based text, of 6-14 years old are in the group of aimed users. Since the aimed users are fairly young it is very important that the educational game is enjoyable, since the motivation to learn about mathematics will most likely not come from the child itself. A good combination of gameplay and mathematics must be found to make it so that the user does not have the feeling it is learning or practicing while playing the game. Due to the popularity of online mobile games with a fictional reward points system for kids this enjoyment could be created by making a reward system in the game as well as a competitive element.

End product idea

The End product idea that we envision is an app which engages blind children in a light-hearted manner with the visualization of graphs.
The app should provide children with a brief explanation of a mathematical function. This explanation can be read out loud by the app. This is to give the child the necessary conceptual knowledge. The core of the app will be a game in which children are challenged to apply their creativity and knowledge of functions to try to figure out which graph it is they need to identify. We expect one level of the app to function by the end of this project on a phone or pc. In short, we aim to make an app which turns what is ordinarily a tedious and slow process into something enjoyable for blind children.

Requirements

Note: These are the requirements that we envision for a proper end product version of our app. These do not fully correspond to what we aim to deliver at the end of the course. The requirements we set for the deliverables can be seen further below.

• A tutorial needs to be present that introduces and teaches various functions and their corresponding graphs.
• A game which challenges the player to visualize functions and understand their shape.
• The option to play with other players online.
• Incentives to keep the player engaged with the game on a long term (gain points for playing and more if one does well, online ranking so one can compare himself/herself with other players).
• Game needs various functions such as audio and vibration.
• A practice mode that is single player and can be played offline.
• Multiple difficulties where more complex functions are used on a higher difficulty.
• Input from the user is obtained through various ways such as a touch display, microphone and accelerometer.

Due to lack of time, the actual app will not fulfill these requirements. Instead, we aim to create an app which incorporates some key elements of the above mentioned properties. This will allow for a proof of concept, i.e. see to what extent an app focussing on learning the behavior of functions will be helpful for and appreciated by blind children. The requirements that are needed for such research are listed below.

• The app needs to be navigable without any visual information.
• The app must contain a visual menu, so that it will be accessible to sighted people as well.
• The app needs to contain one fully functional level, including the depiction of a specific function, and the option to answer which function it is. This also encloses the possibility of giving a wrong or wright answer, and the app needs to indicate whether a wrong or correct answer is given.
• The app must have a general user interface, which includes a main menu from where the user can choose to play a level.

Approach

The way the final product is made consists of several stages. In the first stage knowledge is gathered. During this stage sources will be found on fields of educational benefits of games, games for blind people but also the way mathematics is taught to the blind. Interviews with an expert on the field of educating blind people will give insight into the possible requirements for the device. During this stage the things that the device should be able to do are defined. After all the requirements are quantified and the desired abilities are listed the second stage can begin. The second stage will be focused on writing the software necessary to fit the requirements. The final product would be an app that will run on a smartphone. During this stage it is checked if all the requirements on what we aim for the end of the course are met and everything works.

Results from literature research

After conducting a vast literature research, a brief summary of all articles and patents deemed relevant has been made. These summaries are listed below.

Preparation in and use of the Nemeth braille code for mathematics by teachers of students with visual impairments
This paper describes a study about the use of the Nemeth braille code. 135 teachers, that have followed a course in Nemeth code and teach visually impaired students, were observed. Also a survey was conducted to learn about the current state of Nemeth code usage in the United States.

Digital Games in Education: The Design of Games-Based Learning Environments
This paper examines the evolution of how videogames are designed. From that the characteristics of game-based learning are analyzed. Remaining obstacles and challenges concerning the use of games for learning are discussed. Several types benefits that videogames can offer are listed as well as the use of game-based learning in school.

AudioMath: blind children learning mathematics through audio
In this paper the design, development and usability of AudioMath are presented. AudioMath is a virtual environment that communicates to the user via sound. The goal is to help blind children develop a better short-term memory and to assist them in learning mathematics. The software was tested for its usability and the results are presented.

Science Learning by Blind Children through Audio-Based Interactive Software
Initiated due to the lack of science-oriented software for the blind, this paper presents software designed to teach blind children about science-oriented subjects. The software is heavily based on audio to communicate to the children. The learning of science and the impact on cognitive skills of using such software was researched and its results are presented.

Teaching science to visually impaired students
This paper examines how blind people are currently being thought about science and what needs to be improved to allow blind people to learn better about scientific subjects. Interviews and observations were conducted and from this data conclusions were presented and implications were discussed.

Development of navigation skills through audio haptic videogaming in learners who are blind
This study discusses the implementation of an audio haptic maze game, in which children aged 10 to 15 are tasked with navigating through a maze. The study discussed the combination of audio and haptic interfaces, and concluded that they together more than complemented each other. This is something that we should make note of, if we want to create a game to which blind children can easily adapt.

VBGhost: a Braille-Based Educational Smartphone Game for Children
The authors discuss their newly developed game for smartphones, VBGhost, bases on the game ghost in which players take turns adding letters to a word fragment. The letters are entered in braille, with a 3 by 2 braille cell presented on the screen. Players can raise or lower these dots by double tapping on them. If a dot is raised and a player taps on it, the phone vibrates. The app also includes a high contrast menu, meaning that people with low vision can also read the interface.

Virtual Mobile Science Learning for Blind People
This paper discusses AudioNature, an audio-based interface for pocketPC. It is designed to assist in science learning. The device uses an audio interface to transmit information to the user, and takes input in the form of buttons and a touch screen. One striking feature of this method is that it enables the user to use while moving. Many devices for blind people, the author notes, are made to be used in a static position. Focusing on mobility will help blind people to freer in their movements, and will ultimately lead to better integration with society.

MathMelodies: Inclusive Design of a Didactic Game to Practice Mathematics
With the introduction of tablets in an educational environment, a gaming or entertaining atmosphere can be created in which children learn new concepts. However, these games are mostly visually oriented. This paper describes a game which teaches basic arithmetic through sound. Most of the exercises are read out loud, and also how to answer them is explained. To keep the user entertained, a variety of sounds are added, a rating system based on the number of errors made and a storyline is added. The authors describe that teachers have reacted enthusiastically to this game.

Game-based Learning: Latest Evidence and Future Directions
This article describes the exact nature of game-based learning, and lists the evidence to support commonly made claims about it. It describes in a more abstract manner the principles which go behind game-based learning, and the mechanics with which it tries to adhere to these. Interesting is the following conclusion: “Don’t try to divorce decontextualized components of a game (such as badges, scores or leaderboards) from the fictional context and rules of the game (the ‘mechanics’). Using badges and medals can work for certain simple tasks, but actual game-based learning will require using those techniques in the context of rule-sets and role-playing.”

Auditory Augmentations of Haptic Graphs: Developing a Graphic Tool for Teaching Precalculus Skill to Blind Students
This paper discusses the development of a graphic tool to assist in the teaching of pre-calculus skills to blind people. It looks at existing and on-going developments of instruments to assist blind students with basic mathematics, i.e. examine and explore data and abstract graphs. The paper also looks at auditory and haptic stimuli to present mathematical information. The end goal is to provide a readily usable tool for blind students to learn mathematics.

Issues and Aids for Teaching Mathematics to the Blind
This article looks at the difficulties for blind people to learn mathematics. It also discusses some tools that are available to help these students in their efforts to learn mathematics. It gives a list of resources that help the blind and visually impaired. The article also briefly looks at what might be available in the future to assist the students.

Methods for Presenting Braille Characters on a Mobile Device with a Touchscreen and Tactile Feedback
In this paper three interaction methods were designed for reading six-dot Braille on a mobile device. To do this a prototype device with a piezoelectric actuator embedded under the touchscreen was used to create tactile feedback. The three methods were scan, sweep and rhythm. All of these methods proved successful to convey information.

The Effects of Modern Math Computer Games on Learners’ Math Achievement and Math Course Motivation in a Public High School Setting
This paper looks at the effects of mathematical video games to assist the teaching of mathematics to students. The results show that students who played the video game scored significantly higher of a math benchmark exam. Teachers and students supported the results in interviews.

From dots to shapes: An auditory haptic game platform for teaching geometry to blind pupils
This paper describes an auditory platform based on three classic games, Simon, Point Connecting and concentration game, for blind and visually impaired students. The tool is based on sonic and haptic interaction, and therefore could be used by special educators as a help for teaching planar geometry.

"Learn Braille": A Serious Game Mobile App for sighted Braille Learners
This article describes a mobile learning tool designed to learn sighted people braille. It also contains the game ‘hangman’ to provide a competitive way to keep learning. The teacher is able to select the vowels and words that are taught during the lesson. The app supports both Greek and English braille.

The Today and Tomorrow of Braille Learning
This article mainly elaborates on the decreasing literacy for people using braille due to the use of modern technology. The motivation to learn braille has decreased. It highlighted the problems with existing technology and provided ways in which the situation can be improved. The article was mainly based on interviews with blind people.

Students' attitude towards the use of educational video games to develop competencies
As the study suggests, described in this paper, students' positive attitude towards the use of educative games cannot be taken for granted. Four students' characteristics (perceived relevance, perceived confidence, media affinity, and perceived self-efficacy) influence their attitude towards the games. Relevance is not related only to the content being learned but also to the way the content is taught. Confidence is an important motivational driver which can influence learners' persistence and accomplishment. Media affinity is how, in this case, important games are for the students. Self-efficacy refers to an individual's belief on his/her ability to achieve a desired outcome.

BraillePlay: Educational Smartphone Games for Blind Children
The conclusion of this article was that children were not easily motivated in playing a simple game for an extended amount of time. The article described four different games like hangman and ghost. For children learning braille these games proved difficult due to their limited vocabulary. The games where apps running on a smartphone. The article also names a few ways in which braille can be displayed on a smartphone.

Design and Usability of a Braille-based Mobile Audio game Environment
This article describes multiple games that could be implemented for educational purposes. The games use a GBraille keyboard, which is a way to allow a user to type in braille on their smartphone. The interviews with teachers expressed a concern with the keyboard. Using the keyboard gives a wrong impression of braille since real braille uses raised dots instead of vibrations.

TDraw: A Computer-Based Tactile Drawing Tool for Blind People
The article provides a lot of information on the theory that blind people have the ability to create a mental 3D model of the world around them. This leads to the belief that blind people can draw provided the tools for it and for that reason the research is about a computer tool that would allow blind people to do drawings and graphical work.

Blind Hero: Enabling Guitar Hero for the Visually Impaired
This article explains how certain games can be very effectively played by blind people. Games such as Guitar Hero which include rhythm, patterns, sound and vibration in their core system can be adjusted to be suitable for blind people. The article explains about how that is accomplished.

Designing Haptic Computer Interfaces for Blind People
This article gives information about how interfaces for computer programs are made, when intended for blind people. The article provides information on how interfaces for normal people differ from interfaces for blind people and focuses on haptic computer interfaces.

JustSpeak: Enabling Universal Voice Control on Android
This article provides information on voice command control for smartphones, more specifically about Android platforms. It talks more specifically about the JustSpeak, a solution that makes everything on an Android platform operable by voice commands, by automatically constructing the set of available voice commands based on application context. These commands are directly synthesized from on-screen labels and accessibility metadata, and require no further intervention from the application developer. Second, it provides more efficient and natural interaction with support of multiple voice commands in the same utterance.

Designing Mobile Apps for Visually Impaired and Blind Users
This article provides information on the different ways and methods with which applications that are made specifically or with the addition of blind people controls and assistance. overall it points out that iOS is the most commonly used as it makes use of the many features such as voice navigation that iPhones and other apple products provide.

Planning

Below is a shown the planning. If a cell is colored green, it means that the task presented in the same row should be done in the week of the same column. In red are shown the milestones (Please click on the image to view in the highest resolution).

Research

Possible collaboration with Visio

We contacted an employee of Visio, Don van Dijk, who has experience with teaching math to blind children. When we explained our idea, he said he was working on something similar, and suggested we might help each other out. We have decided to follow up on his proposal, and have arranged for us to call him next thursday.

Don gave us a lot of useful information. He told us that the biggest problem in the education of math to the blind is currently that graphs and images are really hard to depict in an understandable way. The company Dedicon can make tangible drawings for them on request, but this takes a long time and could be improved.

He made clear the the Nemeth code is not used in the Netherlands, because another system is developed. There is a person currently doing a PhD research on this topic. Her name is Annemiek and Don can bring us in contact with her.

Don also offered us to visit him at a school for the blind in Grave. Here we can get a better understanding of how they are currently teaching mathematics to the blind. He also offered us to try the current devices in use there so that we can get familiar with them. We will visit him tuesday 06-03.

Visit to Visio Onderwijs Grave

To get more information about the current systems on schools for the blind and visually impaired, a visit to Visio's school in Grave was planned. The tour was given by Don van Dijk as stated above. It was a very educational excursion where we were shown all the devices available to them. A student taught us how he operated a computer using the 'Brailleregel' for example. Besides this Don explained to us that they used some really old devices that still worked well for them, e.g. a Braille typewriter, and also some newer once.

One of the devices that worked really well for them was a device that helped them explain graphs to their students. Through heating a special piece of paper the black ink on it could create a ridge on the paper. To perfect this system Don gave us the idea of that he would like a system in which he could take a picture of some shape he found interesting and the system could increase the contrast and make it black and white. Due to these adjustmensts the picture could either be printed with their device or a 3D-printer. It was elected not to follow up on his idea, because it would not fulfill the original goal of the project.

Don was also interested in the idea of making a game. He explained to us that the games that would last and work best were the games that were made for more users than just the blind. It should have either a function for the blind to be able to play it or not depend on sight at all. The game that was currently popular at their school was a game for iOS called BlackBox.

The effectiveness of educative games

The effectiveness of educative games is largely based on two factors: the educational value of the game and the attitude of students towards the game.

Educational value

The educational value of a game is based on how much knowledge is packed into the game and the skills that might be taught by playing the game. “What are the students going to learn from the game?” Is an important question to answer. The game should teach topics which are not easily taught in another form and the time spend on playing the game should be proportional to how much the students learn. It is important to notice that the learning content not only consists of the knowledge taught, several skills are also developed. For instance, gaming could improve problem solving, collaboration, communication and social skills.

Students’ attitude

Students have varying types of personalities and therefore all respond differently towards the use of educational games. The seemingly positive effects of games may not be the same for every student. The attitude of the students towards the games is a deciding factor in their behavior. A study of the attitude of students towards an educational game is crucial to its success. A student’s attitude is influenced by four factors: Relevance, Confidence, Media affinity and Self-efficacy. These four factors will now be explained in more detail.

The relevance of a game should be clear to the student. The relevance is related to the content learned and to the way in which it is taught. If, for instance, a game has a high educational content but the way it is taught is very awkward the perceived relevance is low. On the other hand, if the game is very well designed but it has little to no educational content it would no longer qualify as an educational game. A student should believe the content learned with the game is best learned using the game. The game developed in this context might have relevance to the student because it will help them visualize a graph or shape in different ways. By using various sensors in their phones this way of visualization may only be achieved by the described game.

The students’ confidence in succeeding will influence the students’ persistence. They should not be worried about their inability to properly use a game to learn. Frequent feedback in the form of rewards during the game will increase their confidence and various game levels (easy, medium, hard) will allow the student to learn at their own pace. The graph game will increase the user’s confidence by giving of both visual as audible rewards after completing a level. It will also remember the amount of levels that were completed to stimulate a sense of achievement.

Media affinity is the importance that a medium has in the lives of the user. Research suggested that, for example, the affinity towards the use of a mobile phone had a positive influence on mobile shopping. The iPhone and other Apple products are of major importance to visually impaired and blind people, as was told to us by Don van Dijk. These products help them to function as normally as possible by the use of several features such as voice assist. This results in a high media affinity and therefore the main reason for making an app build for IOS.

Self-efficacy is the belief of an individual in their own abilities to achieve a desired outcome. The student has to believe he/she has the ability to finish the game and to gain the desired knowledge to play the game. Self-efficacy might, in this case, be influenced by eliminating the possibility of failure and letting the student get acquainted to the game by the use of a simple tutorial.

Considerations on entertainment

As explained in the section “on the effectiveness of educative games”, the amount of new things that a student learns from an educative game should be proportional to the time spent playing the game. Clearly then, it is desirable that the game is played extensively by the student.
This desire can be fulfilled by exploiting one property of games: their goal is not only to educate, but also to entertain. If this last characteristic is implemented properly, the students will voluntarily occupy themselves with the game.

Mechanisms of entertainment
The question then becomes: how can an educational game be made entertaining, without comprising its educational core? Perrotta et al have done extensive research into videogame based learning, and have listed a number of principles which should go into the design of educational game, as well as the mechanisms by which they can be implemented in a game. Below are listed the mechanisms which can provide entertaining value.

Challenging goals

Providing a clear goal will show the users the result of their effort. It should be clear towards what the users are working. The goals should be quite challenging, or else the users won’t take the game seriously or they will get bored.

Our game could implement this by having a number of levels, each pertaining to a different function and corresponding graph. It should be made clear that the goal is to determine the shape of each of these graphs.

A fictional setting to provide a compelling background

Perrotta notes that this feature should not be used as a means of escapism for the users, but should be used to provide them with an environment in which they feel free to experiment with different methods of approaching a problem, without risking actual failure as in real life.

Any type of fictional setting can be adapted to suit our game. E.g. some sort of magic-themed game where you have to swing your wand a specific way to cast spells.

Difficulty levels

Using different difficulty levels through which the users can advance progressively will stimulate and challenge the users to improve their performance. Again, the criteria for advancing or “leveling-up” should be made explicit. Advancing to a next level can be combined with a sort of reward, thereby acknowledging the user’s mastery of the material. This also increases the feeling for the users that they are in control, and are directly responsible for their performance.

For our game, users could tackle different graphs, where things like y = x are of a lower difficulty than y = x^3.

Uncertainty

Adding a non-linear element, such as the ability for the users to choose different tasks themselves instead of always being presented with a task, also increases the user’s feeling of control. Doing so will encourage the user’s to think independently and with that, increase their capability of learning new things.

Users should be able to select themselves different levels in our game. They won’t have preliminary knowledge about the graph, but it would be possible for them to know something about the relative difficulty of a given level.

A social element which allows the users to share experiences

Another advantage of games is they allow for a strong social experience. If users are encouraged to share their findings and knowledge about the game so that others can advance, users may form bonds. This will increase the enjoyment users experience from the game, besides allowing them to develop additional social skills.

An online ranking system could be added, so that users can compare their performance with their friends.

Solution

The Solution section is split in two parts the "Solution Idea" (how we envision the solution being done given we had enough time, knowledge and resources to work with), and "Achieved Solution" which represents what we were able to achieve during the span of the course. The "Solution Idea" section will represent what we had in mind during the course of the project and what we would like to make in the ideal situation. The "Achieved Solution" section will cover what we actualy were able to make.

Solution Idea

Our initial idea for a solution to the problem was a game in the form of an iOS app. This platform, instead of Android for example, is chosen because it was found that the majority of blind people own an due to it have very user friendly capabilities for blind users. However, what we will actually be delivering at the end of this project is an Android app. The reason for this is purely practical, in the sense that none of the group members possesses or has any means of acquiring a device which runs iOS. This doesn't detract from the value of our project, since we aim only to build a single level to test the feasability of an educative math game which is meant specifically for gaining intuition for the shape and general behavior of graphs. Also we believe that the platform is of little importance as we can achieve the desired effects on Android devices as well. All this will be discussed in further detail below. The game is also designed such that it can be played both by seeing, visually impaired and blind people. This is done to create broad support for the app, as advised by an expert from Visio.

Description of the educative game

The game consists of a collection of levels, which do not necessarily have to be completed in order. The ultimate goal of the game is to complete all the levels. By completing levels the users gains points which the user can trade for hints, new levels or new sounds. There are two game modes that will be imlemented in the game. The goal of the first game mode is to figure out what function it is that a level contains by tracing a representation of a graph of that function. The goal of the other mode is to draw a graph corresponding to a specified function. However, each level represents the graph or requires its input in a completely different way, but it is certain that in none of the levels the graph will be visual on the screen. The users will need to use various senses and methods to try to visualize the graph. They will also need mathematical intuition and knowledge to connect the shape they have in mind to a function, or the other way around.

The core learning part of the game is to gain “a feeling” for the behavior of functions and shapes. As stated, users will receive no information whatsoever on how to solve a level. This is much the same as with the app Blackbox, which was according to an expert from Visio a big success among the blind and visually impaired children (as well as seeing children). Since users do not even know how to describe the graph, they will have to use all possible ways of inputting information they can think of. It is this that will challenge the creativity of the user, and also be the most enjoyable part of the game. Herein lies also the biggest design challenge, since all levels need to be more or less unique to keep the game challenging and surprising.

The game will also contain an online ranking as to encourage users to play it with friends and help each other solve problems. This can be done as follows. When users complete levels, they can gain points. The amount of points they get will be proportional to their performance, measured by a number of parameters such as speed and accuracy. For example, users could get upon completion of a level either one, two or three points. If they complete it without mistakes, i.e. correctly following the shape of graph at all times, and within a certain time limit, they get full points. A point can be subtracted if between one and three mistakes are made, and only one point is awarded when more than three mistakes are made.

Users can see a ranking of their performance based on the total number of points they have, compared with other players. In addition, if a user completes a level with three stars, that user gains a point. This point can be, as described above, used to obtain ornaments such as new sounds, or it can be used to enable more functional features such as new levels or hints on how to complete an existing level.

In addition, users can assist their friends not only verbally, but also by donating them points, so their friends can unlock hints for difficult levels.

Example of a level

An example of a level could be a level in which the users has to move its phone through the air, trying to follow the shape of the unknown graph. The more the shape traced with the phone resembles the graph the longer and higher a pitch will sound. When the path traced through the air with the phone resembles the graph enough (correct amount of maxima and minima, correct qualitative increase or decrease in slope) the level is completed. It could take quite a while before the user figures out how to "see" the graph. Before the user knows the movement of the phone is the input, the user might experience moments in which the game notifies the users that the answer is wrong, while the user does not know why he receives that message. In this exploration aspect lies the fun of the game. Trying to think of functions to solve the level, while not having all possible obtainable information could also learn the user a lot about the graphs of functions.

Possible ways a phone can be used to sense a graph

Line to formula

• The user traces the graph by moving his/her finger along the line. If the user’s finger diverts to much from the line a vibration will be produced. The further away once finger from the line the harder the phone will vibrate or the shorter the time between vibrations.
• The user follows the x-axis and the higher the y-value the higher the pitch of the sound that is being produced (DESMOS). Or the user traces the graph and the further away the finger is the higher the pitch.

The levels below have to be performed in a controlled environment. The user gets the formula of a graph through audio and visual signals. It uses a combination of all the sensors in a phone.

Formula to line

• The user has to move the phone according to the shape of the graph.
• The GPS is used to calculate the speed at which the user is moving. This allows the user to move in a straight line while accelerating and decelerating according to the shape of a graph. A good way to learn derivatives.
• The user has to move in the shape of the graph. The movement is traced by the GPS.
• The user draws a graph on a piece of paper and takes a photograph of it. Recognition software will check if the right graph is drawn.
• The user sings through the microphone at the right pitch to represent the height of the graph.

Achieved Solution

Deliverables

Since time is limited the goal of our project is to create an Android game app as described above containing at least one working level. Below is a short list of some of the other parts of the software which will not be developed in this course:

• Settings
• Many different levels
• Online ranking
• Point system
• Shop

These will not be developed, but since navigating through the app is important (think of choosing levels, using hints or changing settings) it will be discussed here.
Navigation through the app will make use of the 'voiceover' function. This voiceover function has the ability to read the text written on a button on the screen of the phone when the button is clicked. Only when the button is clicked again will it trigger the button. If you were looking for something else, click somewhere else and repeat the process.
The app will be made such that the "double-click" feature of the voiceover function is used in all menu's and buttons. The buttons will all be fairly big to ensure that clicking the right thing is easy. Moreover, the buttons will also contain written text for the sighted users. When a level is clicked on it will not only state the name/number of the level but also if it has been completed already by the user, to help the user keep track of the progress made (of course the user is free to complete a level multiple times).
Finally, everything clickable in the game will have sound, and buttons with similar functions, such as a button for going back to level select and a button for going back to the main menu will have similar sounds, but with different lengths, pitches and such. This will blind users can navigate the game easily, but the sounds also serve an important role in creating an enjoyable and immersive game enviroment.

Level to be delivered

Again the goal is to find the function belonging certain graph, where the graph can be represented through various ways which are unknown to the player. In this level the graph will be communicated to the player through the touchscreen and vibration function of the phone. The player can move its finger across the screen of the phone and if the player is within a certain distance to the graph then the phone will vibrate. Otherwise it will not. Now that the player can trace the curve he can visualize the function. When ready the player can click on a button in the bottom of the screen. When clicked the player can choose from 4 multiple-choice answers which will be located in the four vertical quadrants of the screen of the phone. The answer will be read out loud when clicked on, and will be chosen when clicked on again (as is with all buttons). When the player chooses the wrong answer the level needs to be solved again, but with a different graph. It is to be noted that in our case we only have one graph hence selecting a wrong answer will not result in a new graph rather the same one.

Structure of level selection

The image below shows the structure the level selection menu will have. There are three different main pathways. These pathways entail the different options the app offers. One pathway will have levels based on drawing shapes and functions, one pathway will have the image mathematical functions that the user either needs to trace or has to answer the formula and the last pathway has shapes, for example a circle or square, were the users needs to complete the same actions as for the functions. The tracing and answering part of the image have a structure for themselves. Answering starts of with a multiple choice question followed by an open question. Tracing always starts of with tracing the function or shape on the screen. After this two out of 5 ways of answering are pulled from the database. These ways of answering are described in the 'Example of a level' part of this wiki.

Code structure

The code below shows the main concept of how our program will be structured. The idea is to have a simplistic and not deep infrastructure of our game. It will consist of a main menu at opening of the app. and a Game mode in which you play the game. The game will have a level menu in which you choose which level to play. Then the level starts and it consists of the Active Section in which the player is given the question and the Answer Section button. When the user has solved the question and selects the Answer Section button it opens up the Answer List in which the the answers will be listed. After the choice has been made there are 2 options:

1. If the answer is wrong the user is given the option to try the previous level. (Considering the levels in the list will be in difficulty order) 2. If the answer is correct the next level is loaded.

Load mainMenu;

//Main menu section
while (in mainMenu){
	//Music would indicate the game is running for blind people
	play soundEffect on loop;

	//Bonus feature to change soundEffect
	if (input is detected on specialSection){
		change soundEffect;	//*maybe 2 sound effects are enough (and easy to implement)
	}

	//entering level
	if (input is detected on screen){
		stop soundEffect loop;	//stops mainMenu soundEffect loop to indicate that the level starts 
		play loadingSound;
		switch to gameMode; 	//start playing
		select level from levelGrid which draws from the levelDatabase;
		load selected level;
	}
}

Game portion section 
 
{constant elements section}
@param questionSection represents the workspace (roughly 4/5 of the screen counting vertical from the top)
questionSection contains all of the shapes regarding the question (taskShape, emptySpace1, emptySpace2, emptySpace3)
 
@param answerButton a button located on the bottom 1/5 of the screen which opens up answerSection
@param answerSection represents the answer list (a list of answers)

{elements dependant on the level section}
answerSection contain multiple choices of answers (2 to 4 possible answers ordered in a vertical list)

@param taskShape the shape/curve/figure the user is trying to guess/answer represented by a field that can be activated
@param emptySpace1 All of the other space
@param emptySpace2 in case it is needed other empty spaces might be encorporated to achieve the wanted taskShape 
(need of a more efficient way for coordinate navigation may be needed after testing or more shapes depending on ease of use)
(the implementation may also be made with only taskShape without the use of emptySpace)

@param flagSpace1 subshape inside taskShape that is a key part of the shape, provide additional information by voice
(the player is alerted of their presence when he navigates near one of them)
(more than 1 can be used)

selecting any of the shapes/spaces by entering their boundries plays a soundEffect indicating if the person is in the shape or not
selecting any flagSpace provides additional info in terms of voice 
selecting one of the answers in answerSection plays a sound explaining the answer, selecting the same answer again confirms the choice

The developed app

Main menu

The first screen the user will encounter when opening the app is the main menu as seen in the screenshot below. The main menu contains five large buttons labeled: “level selectie”, “uitleg”, “online ranking”, “winkel” & “opties”. When one of these buttons is tapped the text on the button will be read out loud by the phone. Only when a button is clicked twice without clicking on another button in between will it perform an action and play a clicking sound similar to that of a chest being opened. The only button in the main menu that leads to another screen is “level select” and because of this it is the only button that will play the clinking sound when clicked twice in a row.

Level selection

The level select screen contains twelve square buttons in a grid all labeled with a level number as seen in the left screenshot below. When first opening the app only level 1 will be the same bright blue as the rest of the app to indicate to the sighted users that this is the only unlocked level. When level 1-3 are clicked once the number of the level (as well as the word level) will be read out loud. When the user clicks again on level 2-3 the app will tell the player that this level has not been unlocked yet. When the user clicks again on level 1 it will enter level 1 and play the clicking sound. If the user clicks on level 4-12 the app will tell the user that “this level is not available in the demo”. Only level 1-3 are functional and each time a user completes a level the text of the next one will be turned to the bright blue and it will be unlocked as can be seen in the right screenshot below.

The level

The level screen contains only one button named “antwoorden” as seen in the screenshot below. This button is to be clicked (again twice) by the player when he/she thinks to know the answer. When the user touches the (black) part above the button of the screen the phone will vibrate unless the finger of the user is within a certain (shortest) distance to the graph. The user will need to figure this mechanic out for themselves, as it is part of the fun exploration in the game the blind children enjoyed. The app contains three working levels with the functions y = 0, y = x & y = x^2 (in that order).

Answering

The answering screen contains four button labelled each with a different function as seen in the screenshot below. Again the voice-over double click function is implemented. When the user clicks the wrong answer a loud buzzer will sound indicating that it was wrong and the user will be redirected back to the level the user was trying to answer so he/she can try to visualize the graph better. When the correct answer is selected a cheerful harp will sound and the user will be redirected back to the main menu.

App download

Please feel free to test our android app on your own phone! Below is a .zip file containing the .apk file that you can install on your android phone. Do not forget to turn on “Allow installation of apps from unknown sources” in the security tab within the settings. And please note that the app might not function 100% correctly on every phone, since it only was tested on a couple of different phones.

File:Learning Curve.zip

Explanation of the code

The code works by defining two arrays corresponding to the x- and y-values of the graph. The number of elements in the x- and y- array are both equal to the width of the screen in pixels. The standard coordinate system on the phone (which has the origin in the left corner, the positive y-axis pointing downward and the positive x-axis pointing to the right) is converted to a coordinate system with the origin in the center of the screen. All the x-array elements are filled in in such a way that the first element corresponds to minus half the width of the screen and the last element is plus half the width of the screen. These x values are then used to the define the y values in the y-array according to a desired function.

The position of the user’s finger is then measured. A circle can now be drawn around this position with its radius corresponding to a desired margin. If the graph is not within this circle and the user moves his/her finger, the telephone vibrates. The vibration last for 5 seconds unless something else happens that will automatically terminate the vibration (i.e. if the user lifts his/her finger). The vibration will not restart every cycle because the program remembers whether it is on or not. This is achieved by the use of an array consisting of two elements, one for the current loop and one for the loop that was done before. If there is no change between the current loop and the loop before it, there will be no change. If they differ the vibration will be started or cancelled accordingly.

Discussion

Referring back to the section on requirements, it is clear that our app fulfills all the requirements that had been set. A menu has been created that is navigable by both blind and sighted users. Three distinct levels have been created and the user can navigate between them. Results from the user testcase showed that Piece on the feedback by the blind children.

As is mentioned multiple times already, the app that has been delivered at the end of this project is not the endstation of our conception of this game. The app as it is now serves only as a proof of concept, an indication of whether this conception of how the game should be organized would work. This it has done quite well. By no means, however, is it a finished product. Several functions such as online ranking, a reward system with a shop where points can be traded for sounds, a tutorial etc. still need to be implemented. A large number of different levels, using far more exotic means of input or output than multiple choice answers and the vibration function, are required to keep the game exciting and enjoyable. Using the app as it currently is as a foundation for that should make that task a lot easier.

References

Coaching Questions Group 2