PRE2019 3 Group4

Group 4

Group member	Student number	E-mail	Study
Tom Janssen	1233021	t.j.a.janssen@student.tue.nl	Chemical Engineering and Chemistry
Ivo Kersten	1233717	i.p.c.kersten@student.tue.nl	Electrical Engineering
Sander van Bommel	1017917	s.p.h.a.v.bommel@student.tue.nl	Psychology & Technology
Tim Driessen	1006903	t.driessen@student.tue.nl	Software Science
Rob Vissers	1244863	r.t.w.a.vissers@student.tue.nl	Electrical Engineering

Introduction

In Europe only, there are already an estimated 30 million people that are either blind or visually impaired. Furthermore it is found that on average 1 in 30 Europeans experience sight loss. This imposes a huge challenge on the society in general, since these people cannot function as properly as intended in the complex society of today. Of all the blind or visually impaired people, 75% is rendered unemployed, while these people could possibly participate in certain jobs if they would receive the necessary education. Now a huge issue arises, since there is a lack of proper braille-teaching material available, which is holding the blind and visually impaired people back.

Also loss of sight can be linked to people getting older, whereas the retinitis pigmentosa deteriorates with increasing age. By looking at the European statistics, one in three seniors with an age over 65 years old struggles with visual impairment or even blindness. Due to these large numbers, it can be seen that the problem of visual impairment and blindness has to be tackled, such that these people can stay active in the society . Since these elderly people are not able to read anymore, they might want to adapt to learning braille to increase their independence.

Problem statement

According to the World Health organization , the estimated number of people that are suffering from visual impairment in the world is 285 million. These people experience difficulties with daily activities that require vision. Vision is considered as an extremely vital sensory modality in humans. The loss of vision affects the performance of almost all activities of daily living (ADL) and instrumental activities of daily living (IADLs); thereby hampering an individuals’ quality of life (QoL), general lifestyle, personal relationships and career . Due to these limitations, these people have a greater probability of experiencing social exclusion, depression and loneliness .

However due to increased knowledge and assistive technologies, there are many new applications and learning systems created to support visually impairment people with their daily activities and make life much easier. Braille learning is considered as one of the most well-known methods that is used to support the visually impaired with reading. In this method, visual impaired people are basically reading text with their fingers by identifying several patterns of raised bumps or dots. Even though it offers these people the opportunity to actually read a book, not many of these people use Braille. According to the National Federation of the Blind , only one in 10 blind people can read Braille, which is dramatically drown from the early 1900s. Furthermore a great proportion of blind children experience considerable difficulties learning to read braille and some never master the skill . Therefore they are more likely to lose interest in learning Braille and search for alternatives.

Even though the interest in Learning Braille had decreased over time, this does not mean that it is outdated or irrelevant. In fact, Braille represents information and education - the currency and the future – for blind people . By learning Braille, blind people will be capable to get access to relevant information, develop high-level skills in reading and writing. Therefore it should be understood, however the current methods of how Braille is taught, might be outdated. Assistive technology should open new ways for Braille to make it more interesting among visually impaired people and improving their well-being.

Objectives

The central objective of the project is:

Main objective: Realize a device that helps an inexperienced person to learn the basics of reading braille. The device will make the Braille literacy more accessible to visually impaired people and can be seen as the first introduction to the Braille language.

From this main objective, a number of smaller objectives can be deduced, namely:

Objective: A visually-impaired-user-friendly interface

As the target group is visually impaired, an interface will be developed that provides clear communication in both ways between user and device without requiring the ability to see.

Objective: Facilitate learning

A number of learning modes will be created that provides the user with a fun/interactive way of learning Braille. These learning modes will use inputs from the user in the form of pushing braille pins and/or outputs from the system by means of sound or moving braille pins.

Requirements

Overall system must:

• be able to be set up in less than a minute.
• be able to be activated by a blind person.
• be built with high contrast between colors of the box the system is in and the buttons.
• have functional buttons with clearly recognizable shapes to ease operation.
• be affordable to as many people as possible (preferably < €200).

Physical braille display must:

• be capable of correctly displaying all basic braille symbols: letters, numbers, and punctuation, one at a time.
• be able to be reset and rewritten in less than 0.5 seconds.
• provide enough force to each braille dot such that they can be read easily.

Physical braille input must:

• have braille dots that are easily pressed down with light force, and then kept in downward position.
• be able to be reset in less than 0.5 seconds.
• reliably capture presses (>99%).

Expected Impact

Currently about 10% of the blind people worldwide is actually able to read Braille. This means that 90% of these people is not capable or is not willing to learn Braille. However, our product is meant to increase this interest and provide a new type of Braille learning that is understandable, educational and interesting. If blind people would experience the benefits in terms of improvements in their daily life experience, then there is evidence found that our product actually has a positive effect on the experience of blind people and it can be implemented in a real-life setting. Therefore there is expected that a great number of this 90% blind people will regain interest in Braille and thereby also generate more interest in our product.

Our product also allows independent use, which means that a constant accompaniment is not necessary any longer. Blind people are capable to activate our product by themselves and can use it to learn Braille. In this way, caregivers of these blind people can spend their time to other individuals that actually need guidance with reading. This is not only less costly, but caregivers will also be able to work more efficiently. For this reason, cost-efficiency will be maintained. Furthermore it expected that engineers or developers of our product will experience a financial gain due to the increased demand on the market. If governments would acknowledge the benefits of the product as well, they can provide support to these developers/engineers in terms of scientific funds. Scientific funds will contribute to further development in research and knowledge of our product, which can lead to even new applications or breakthroughs.

Approach, Planning + Milestones & Deliverables

Approach

The different aspects of the approach can be subdivided into milestones. Consequently, these can be distributed over a planning that fits the time span of this project.

In order to obtain the optimal and most adequate results on the topic, several different methods can be used to obtain information:

• Literature research into the different aspects related to this topic:
• Collaboration with visually impaired people to incorporate the advice of the primary users.
• Development of a small prototype and applying a series of tests to check whether the device meets the previously set requirements.
• Comparative tests with current state-of-the-art devices to discover points of improvement, as well as points of superiority of our product.

Planning & Milestones

Below the global planning of this project is displayed.

Week	Tasks & milestones for the report	Tasks & milestones for the prototype
1	Choose the subject Write the start of the sections for the problem statement, objectives, users, requirements, approach, planning & milestones, deliverables, logbook Investigate and summarize current state-of-the-art techniques and devices on this topic
2	Continue writing on the different sections of the report Write the start of the sections for current state-of-the-art and our own research	Approximated design List of needed hardware components
3	Continue writing on the different sections of the report Gantt chart Manual Contact Visio or other instances Decide on focus project	Design SolidWorks Purchase hardware components Software concepts
4	Test plan Working out Hable interview	Developing software Creating prototype
5	Further work report	Developing software Creating prototype
6	Create questions/exercises for testing of week 7	Finishing software Finishing prototype
7	Create presentation Work out test results	Test prototype with visually impaired people
8	Presentation Finish report Work out test results	Incorporate feedback tests week 7 Bug fixing

Deliverables

• A small prototype of a device that helps an inexperienced user with learning braille.
• A report of all our literature research, analysis and results on this topic. This will be documented on this wiki page.
• A presentation at the end of this course to share our findings.

Gantt Chart

USE aspects

Users

When designing a product, it is important to keep the users (actively or passively) involved in the design process as soon as possible. Therefore it is important to take into account the values and needs of all involved users. With proper participation and empirical research, the design process can be centered around the user for the best final result. The users involved within the subject of learning braille can be categorized in primary and secondary users as described below.

Primary Users

The primary users are the people that will actively use the product, namely being the visually impaired and blind people that do not yet know the braille language. This is caused due to the lack of braille learning material that is generally available for consumers and organizations. By designing a system that teaches braille with one letter or one word at a time, with the help of a braille example or pronounced words, braille can be learnt by a much broader public. Since these visually impaired and blind people cannot get the necessary education they need from reading literature or browsing the internet, they will get a learning disadvantage. Therefore with a system that teaches the braille language, the independence of the users will grow. The independence of the user is an important aspect, since blind and visually impaired people strive for a certain amount of independence, which is lost due to their disability. With multiple levels of difficulty, this braille-teaching device can be used by a widespread public, i.e. by children that are learning letters, or by adolescents that want to learn braille letters and words (including pronunciation).

Secondary Users

Regarding the secondary users, there are multiple organizations that would want to use a device to teach braille to people.

• At first, educational institutes, such as kindergartens, schools, and universities, would want to use the braille-teaching device to make learning braille for blind and visually impaired people more attractive at their institution. By showing that their needs are taken into account, the blind or visually impaired people will be triggered more to go to a certain institution that respects their disabilities. This will generally lead to an increased number of students at certain institutions. It will also help to reduce the workload that is imposed on the braille teachers at a certain institution.

• Secondly, non-profit organizations that want to help children with certain disabilities might adapt to this design. By investing in a braille-teaching device for children that cannot afford it, the literacy for these children will greatly increase. Since the main goal of these non-profit organizations is to help disabled children, this device would be a proper addition to their functional capabilities.

Society

In the society section the impact of the Braillearn will be assessed based on two important target groups, namely the companies and consumers. For both target groups the importance of the design phases sourcing, manufacturing, using, and end-of-life are described in detail.

Companies

• Sourcing phase: For companies it is important to not simply distribute a product on the market. It is important to gather enough empirical information and to work closely with the users of the product to incorporate possible feedback from these people. Thus with a design process centered around the users (being the blind and visually impaired), workers can create a product that takes into account the needs and values of this group. The design should also be made measureable to check whether the product will eventually suffice the needs of the blind and visually impaired people. By having measureable guidelines, it is more practical to check where the flaws of a product are present.
• Manufacturing phase: In the manufacturing phase it is important to create a low-cost product that takes into account all specifications of the design. With a new product on the market, a new team is needed to control the manufacturing process, which in turn creates jobs. Most of the research has already been done in the sourcing phase, but component-wise the manufacturing phase is important to be able to create a reliable product for a proper price. With pre-established target prices, weights, functionalities, dimensions, etcetera, it is possible to create an appealing product that satisfies the consumers. However at first it is important to test a crude prototype to simply check the functionalities of a design. For this a test group is needed that will assess the product based on questionnaires about and interactions with the product.
• Using phase: From the using phase feedback will be given back to the designers of the product to refine their design. Since the sourcing phase is already centered around the user, mostly practicalities will come to light to increase the ease of use of the product. This feedback will be incorporated by a design team and the product will be improved for the future.
• End-of-life phase: In this phase, jobs are created for the handling of the lifespan of the product coming to an end. All the components will have to be disposed in an environmental friendly way or they might be re-used. It is important to establish a team of engineers that can assess the quality of components and to check whether it might be re-used in the process.

Consumers

• Sourcing phase: In the sourcing phase the consumers will have the need for a product, which is a device that can teach braille. This will decrease the need for a long trajectory of one-on-one lessons with the lack of braille teachers. Consumers will work closely with the workers to ensure that their needs and values are taken into account in the sourcing phase, and their feedback will be implemented to create a product that appeals to the consumers.
• Manufacturing phase: Consumers and stakeholders will have to invest money in a start-up company to ensure that the manufacturing process will run as smooth as possible. With pre-orders placed on the product, there will be a budget available for the workers to create the product in practice.
• Using phase: The using phase is most important to the consumers. The main societal impact will be from people that are able to learn braille through the process of using the Braillearn. As mentioned earlier it is estimated that 285 million people are either blind or suffering from visual impairment, whereas only an estimated 10% of these people can read braille. If at least a part of this group would be able to learn braille, this would increase their overall independence and it would lead to more chances on the job market.
• End-of-life phase: To recycle parts of the Braillearn, it could be possible to make certain recycling posts that are handled by a small team of engineers. Recycling products is important to decrease the amount of new components being made and therefore to prevent extra pollution.

Overall it can be seen that mostly the societal impact comes from the fact that more blind or visually impaired people are able to learn the braille language, increasing their independence and educational level. Another impact on society is that jobs will be created for the design, manufacturing, and recycling process, which will boost the economy.

Enterprise

Looking at the enterprise aspects there are two main stakeholders, which are respectively the manufacturer of the Braillearn and educational institutes.

• Manufacturer of the Braillearn: The Braillearn will have to be manufactured to be as low-cost, but reliable, as possible such that it can be adapted by a large portion of the blind or visually impaired people. With the total cost of the components being €145,33 and an estimated production cost of €20,00 per Braillearn, the total cost of the system would be €165,33. If it were to be introduced on the market the rule of thumb for good profit is considered to be 20% on top of the purchase price, which would mean that the Braillearn would cost €194,80 which is much lower than all other existing products on the market. When the price of a product is low and the product satisfies the needs of the users, the Braillearn would become the standard on the market for people willing to learn the braille language and could be sold in bulk, reducing the price even further.
• Educational institutes: If educational institutes were to adapt to the Braillearn, a lot of profit can be obtained from new students willing to learn braille. There would still be a need for skilled braille teachers, but they would infer a more passive role in the process of learning braille. This would be in the form of psychological help and encouragement. Also if a student cannot get the results it wants, they can choose for face-to-face braille training with a braille teacher.

State of the art: Existing Devices

Looking at the State-of-the-Art regarding devices made for learning devices we have come across a number of products. Looking at these products we can see how they work, what are the advantages and disadvantages and therefore be able to find a place to fit our concept.

LEGO Braille Bricks

Recently LEGO unveiled a new project aiming to help blind and visually impaired children to learn Braille. The Braille bricks are similar to the common 2x4 blocks, except they don’t have eight “studs”, but use a 2x3 array of studs to represent a braille cell. At the bottom of the brick there is room for a visual indicator of the letter or symbol for the supervisors. The LEGO Braille bricks should fully launch in 2020.

LEGO Braille Bricks

The combination of LEGO and Braille looks like a perfect match. Each original LEGO block already contains “studs”, and more importantly LEGOs is meant to be a toy for children. Children associate pleasant thoughts with the bricks. Introducing the Braille language here is a perfect way to make learning fun. Because of the freedom of placement people have with LEGOs, sentences and words can be easily constructed using the braille blocks. An obvious disadvantage is the need for a supervisor. As stated in [BRON HIER], a supervisor can only effectively help one person at a time. Therefore teaching Braille to a class of visually impaired people is still quite difficult.

Annie

Tinkerbell Labs have created a Braille literacy device to help the visually impaired to learn Braille on their own, through audio-guided gamified content. It consists of two large Braille cells which are used to introduce braille to an inexperienced Braille reader. It also includes six standard-sized braille cell to cover all primary learning needs. The input of the user is giving through a large Braille keyboard placed at the center of the device.

Annie

A big advantage of this device is that it enables one teacher to teach more than one student simultaneously. This can be seen as an obvious advantage, as opposed to the more traditional methods for learning Braille [BRON HIER]. A large disadvantage though, is the price point. As of now, the device costs $949 with access to the companion app and analytics program for $149 per year. This makes the device mostly accessible to institutional organizations and less accessible for individual use.

Taptilo

Taptilo is an innovative braille education machine without the need of a professional teacher. The top of the device contains nine removable, magnetic Braille cells which users can use to create their own letter (they can push in “studs”). These cells interact with the permanent row of nine refreshable cells placed at the bottom. These cells are ‘jumbo-sized’ meaning they are bigger than usual, making it easier for inexperienced users to learn.

The device has implemented five teaching modes:

1. READ: Select a word which will be displayed and read out.
2. TRACE & WRITE: Select a word which will be displayed and read out. Trace this Braille word (on the permanent row) and try to match the blocks with the removable braille cells.
3. DICTATION: Select a word which will be read out. Then try to spell the word using the removable braille blocks.
4. WRITE: Make your own word using the blocks. Then Taptilo will read the word.
5. GAME: The permanent row will display the letters of a scrambled word. Try to put the letters in the right order using the removable cells.

They have utilized an artificial intelligence (AI) speaker to be able to output, not only predefined letters/words, but also words the system has never pronounced.

Taptilo

The advantages/disadvantages are very similar to that of Annie. A big advantage is again that the devices enables one teacher to teach a full class of students instead of only one student at a time. The disadvantage is the price point. Taptilo, which costs $1,349.00, is even more expensive than Annie making it again less accessible for individual use.

Hable

Visually impaired users often use speech-recognition to operate their phones. The problem with that is inaccuracy and a lack of privacy (imagine sitting in a train). This is where Hable steps in. They are developing a device which presents visually impaired users with a way to enter text onto their phone. Work is also being done on ways to make the device also able to navigate the phone and open apps.

The device consists of six braille buttons which are combined with two functions buttons for commands such as spacebar, enter and backspace, but also to navigate through your phone. It can be attached to the back of your smartphone, or be used freely. Bluetooth is used to connect to the phone.

While Hable is not exactly a braille learning device, it still is a company with a lot of experience and knowledge surrounding Braille. Hable having its roots in the TU/e, we managed to get into contact with them, of which the findings are shown in [Verwijzing hier].

Hable

Research into braille

Fundamentals of braille

Braille cell

Braille is a tactile writing system for people that are blind or to some degree visually impaired. The system has many variations, but the most commonly used is the 6-dot braille. This type makes use of a 2 by 3 dot cell that can represent a letter, digit, punctuation mark and even certain contractions. These different elements are represented by combinations of raised dots within a cell. Since there are 6 dots that can potentially be raised, there are 64 (2^6) different combinations possible. The 6 dots are numbered in a downward fashion in the consecutive columns, where the top left dot has number 1 and the bottom right dot has number 6.

The different literary elements are grouped in certain decades. A decade is a group of 10 different combinations that only make use of a specific subsection of the 6 dots. Literary elements that are alphabetically adjacent or just similar in use are placed in the same decade to improve the efficiency and ease of reading braille.

• The first decade is made up of the upper four dots (numbers 1,2,4 and 5), and represents the letters 'a' to 'j' as well as digits 0 to 9.
• The second decade makes use of dot 3 in addition to the upper four dots, and represents the letters 'k' to 't'.
• The third decade makes use of both dots 3 and 6 in addition to the upper four dots. The first half of this decade makes up the remaining letters of the alphabet 'u' to 'z' with the exception of 'w'. The other half is used for the commonly used words 'and','for','of','the' and 'with'.
• The fourth decade makes use of dot 6 in addition to the upper fout dots, and represents some common 2-letter combinations in print as well as the letter 'w'.
• The fifth decade is the same as the first but then all dots are shifted one down. These represent the punctuation marks.
• The sixth decade makes use of dots 3,4,5 and 6, and represent some of the remaining punctuation marks and common 2-letter combinations.
• The seventh decade only makes use of the right column, and is used to obtain a certain effect such as two-celled contractions, italic letters, capital letters, etc.
• Last, there is the completely empty cell (no dots raised), which represents a space.

Below you can find a simplified overview of the literary elements per decade on the left. This overview does not contain all the different braille symbols and is thus incomplete. Therefore, a complete overview is displayed on the right. However, this overview is not organized per decade.

Simplified overview braille symbols per decade

Complete overview braille

Main issues in learning braille

It is estimated that 285 million people are visually impaired and 39 million people are completely blind. Only a small percentage of approximately 10% of this group can read braille. This is a relatively small number and greatly limits the freedom and capabilities of blind individuals and the complete blind community. There are multiple factors that complicate or eliminate the possibility to learn braille. These factors can lie with the subject but also with the current means available for braille. Therefore, some factors can be controlled and some cannot. Below the different factors will be discussed.

• To start, only 80% of all visually impaired people are potentially able to read braille, because they can experience the required haptic feelings. The remaining 20% can not and is therefore unable to learn braille even if they would experience no further issues.
• People above 40 years of age that have not yet learned braille in a previous stage of their life will have an increasingly more difficult task to learn it. Until 40 years of age there is no drastic increase in how hard it is to learn braille. Moreover, if one has already learned braille when they were younger, then no major issues should occur if they try to continue in a later stage in life.
• The phonological deficit theory states that reading retardation is caused by the deficit of representation, processing and storing speech sounds. It has been shown that this also holds for blind subject and can indeed cause issues in one's ability to read braille. Additionally, for people who just started learning braille the process can be slowed down drastically by a deficit in processing and storing speech sounds, since audio will be used to provide the definition and meaning to the tactile sensory input from the fingers.
• The magnocellular theoery states that difficulty with reading (braille) can be caused by damaged sensory pathways that are responsible for rapidly varying streams of input.
• In some poorer countries in the world, many institutions for blind people simply don't have the means and knowledge to properly help their students.
• The current method to learn braille mainly exists out of 1-to-1 sessions between a tutor and a student. Learning in a group is inefficient, since it causes a lot of distractions for the students and makes it impossible for a tutor to give every student the required attention. Since it takes 150 hours on average to obtain a decent level of braille, these private lessons can get rather expensive.
• The sensory tactile input that braille relies on is in many aspects inferior to the visual input from reading print text. The stimuli are processed much slower and especially while learning braille many people first have to convert the braille to normal letters in their head, which takes up even more time. Furthermore, distinguishing so many different characters is much harder by touch due to the lack of detail. This also makes it easier to become confused.
• There are some inherent difficulties to braille itself. First of all, there are a lot of different combinations that one has to memorize just to learn individual letters, contractions and words. For children that were blind at birth, this is an extra challenge on top of learning letter combinations to form words. Some of these specific combinations can also easily be forgotten after one hasn't seen it for a long time. Second, there are many different characters that have the same braille code. The specific meaning of the 6-dot cell is then dependent on context, which takes a lot of practice to master. Additionally, there are sometimes multiple different codes to write the same thing, which can also cause confusion. Last, every language or sometimes even every country (even when two countries speak the same language) differs slightly in what code represents a letter, contraction or word. This gives an extra dimension of difficulty for learning different languages in braille, since one must first learn the new code before even starting to learn the new words and letter combinations.
• Many visually impaired people simply do not want to start with learning braille, because this would mean to them that they are officially blind. So, the stigma on being blind often delays when people start to learn braille if they start at all. This also greatly impacts the number of blind people that can actually read braille.
• Since learning braille is a slow process, many people get discouraged along the way. Making sure that people stay positive during the learning process is actually one of the most important tasks of the 1-to-1 tutors nowadays. If more entertaining methods for learning braille or more positive feedbacks systems could be implemented this issue could be helped significantly.
• Current methods to learn braille often do not match the most efficient ways to learn braille.

Strategies for learning braille & Gamification

As mentioned in the previous section, one of the major issues in learning braille is to stay motivated in the slow process. Not that much is known on the best way to learn the tactile writing system efficiently and most of the current method rely on simple repitition. Granted that this is what learning most languages comes down to, it is not always stimulating student engagement. One method that could help in this aspect is Gamification, where different game elements are incorporated in the learning exercises to make them more exciting and engaging. Below, different elements are explained that incorporate gamification into the prototype of this project.

• Setting a time clock on different difficulty levels for users to type a certain number of letters or words. This gives a score for both the taken time and number of correct letters/words.
• The scores set in different learning modes can be used to unlock different accomplishments, keep track of performance over time and compare oneself to others using the device.
• The device can also be made to pair up with other devices, which makes it possible to compete against friends.

Another method to reduce the percentage of illiterate blind people, it is important to get rid of the stigma around blindness and braille. This can be done by making braille cooler and more of an accomplishment. Hopefully, the prototype in this project combined with the elements of gamification can help with this.

Prototype Braillearn

Design layout: hardware & software

The product that will be created throughout the project will have multiple functionalities. Therefore it is important to get an overview of all these functionalities to find a proper way of implementing each subpart in the final design. At first the RaspberryPi (RPi) has been chosen, due to the better performance than the Arduino (1 GB RAM Memory, 40 IO Pins, and an 1.2 GHz micropocessor). This will be the core of the design, since the RPi will direct all the different signals to the different subparts of the final design. The different subparts that will be implemented are:

• A switch to turn the device on and off (an on/off switch);
• A braille example keyboard that will show the letters that are under consideration (6 solenoids);
• An input braille keyboard where the user has to press the button of the example keyboard to generate the required letter (6 solenoids);
• A button to reset the current input on the braille keyboard (a reset button);
• Voice-controlled user encouragement via a speaker or via an Aux input (speaker and Aux port);
• Two buttons to control the mode that is tested on the user, whereas the modes are letter_in/letter_out, voice_letter_in/letter_out, word_in/word_out, and voice_word_in/word_out. All of these modes will be deliberated upon more further into this section (two button for mode control);
• A next button to confirm the input of the user (one button for next letter or to complete input);
• A power input port for the RPi (power adapter regarding the RPi).

A crude approximation of the final product that will be manufactured throughout the project is shown in the picture to the right.

A crude approximation of the final product that will be made throughout the project

Regarding the different input modes that can be controlled by the user, there was:

• letter_in/letter_out: In this mode the user will learn the letters of the alphabet via the example keyboard and the letter under consideration will also be voice-controlled. Then the user presses the next button and presses the solenoid buttons that represent the shown example letter. By pressing the next button again, the input of the user is validated, and if this input is correct the user will be commended. If the input would be wrong, the user can try again with the example letter for a second try.
• voice_letter_in/letter_out: In this mode the user will learn the letters of the alphabet via voice-control. Then the user presses the solenoid buttons that represent the spoken letter under consideration. By pressing the next button to confirm the input, the input of the user is validated. If it is correct the user will be commended. If the input is incorrect, the user can try again with the voice-controlled letter under consideration.
• word_in/word_out: In this mode the user can learn words, whereas the example keyboard will show single letters of the word one by one, while the word under consideration will also be voice-controlled. The user presses next after consecutive letters of the word, and after the whole word has been shown as an example, the user can press the solenoid buttons that represent the letters of the word under consideration by pressing next after each input letter. If the final letter inputs for the word are correct the user will be commended. If one or more of the letter inputs are incorrect, the user can simply try again by repeating the example braille letters.
• voice_word_in/word_out: In this mode the user can learn words solely based on their pronunciation, whereas the word will be voice-controlled. Now the user will press the buttons for the consecutive letters of the word on the input keyboard, where each letter is followed by the next button. If the word has been completed, it will be checked. If the final letter inputs for the word are correct the user will be commended. If one or more letter inputs are incorrect, the user can simply try again by hearing the pronunciation of the word once more.

With all of these different modes, multiple levels regarding the learning of braille can be achieved. In the process an inexperienced person may start with letters and finally learn words, while a more experienced person may already start with words. This will increase the range of the population that can use this product.

Since the product has to be for a broad educational public, the specifications have to be well-defined. For example the cost of the product should not exceed €200,00 such that pre-schools can invest in this product. Also the weight of the product should not be more than 500 grams, since the device has to be portable for mainly younger children. The dimensions of the device should also be kept in its perks, which are now defined as 12x4x5 cm. This property, however, is still subject to change regarding the material that are implemented in the final design.

Case in SolidWorks

The casing for the final product called the 'Braillearn' is given on this separate wiki page. On this page the design decisions regarding the casing and the SolidWorks design files can be found.

Electrical design

The details of the electrical design of the components, their assembly and testing process can be found here.

Software

The software is available on Github and will be kept up to date during the project.

---

Structure

The code for Braillearn was written in Python on the Raspberry Pi 3 Model B. The choice for using a Raspberry Pi as compared to using an Arduino was made following a number of reasons. First of all, the Raspberry Pi is a lot more powerful than an Arduino. Our Raspberry Pi has 1.2Ghz quad-core processor while a normal Arduino board contains often an ATmega processor with a CPU speed in the range of 32 MHz. So power wise there is very large difference. And since we planned on using multiple threads and also a built-in text-to-speech engine (this was later changed) we really required the extra power.

Another advantage is the number of available libraries. Python, which is supported by the Raspberry Pi, brings with it a lot of internal libraries and external libraries we could make use of. This makes the programming job easier and more efficient since these libraries are used by a lot of people by which they get improved and fine-tuned until they are nearly perfect. This way we are a lot better of than trying to program all these low-end features ourselves. In total we are making use of the following libraries:

• importlib : Using this library we can dynamically load tasks into our program. This makes it possible for user-created tasks to be imported into our program.
• time : Used to halt the program for a number of milliseconds. By using this we can add small breaks between audio fragments to give the user time to take in the information heard.
• RPi.GPIO : Used to be able to control the 40 GPIO pins located on the Raspberry PI. This library makes it possible to easily set up inputs and outputs for our device.
• random : Used for making random choices. Using this the questions of a task can be shuffled or random audio fragments can be chosen.
• mixer (from pygame) : Pygame itself is a cross-platform set of Python modules designed for writing video games. While we are not doing that, we make use of their mixer module. This makes dealing with sounds quite easy for us.

Our plan initially was to use the pyttsx3 text-to-speech conversion library to read out texts to the user. This library works offline, so it doesn’t require an internet connection to work. On Windows the results of the library were very positive, but when we tried to run it on the Raspberry Pi the audio quality appeared to be very low. It is possible to distinguish words if listening carefully, but the amount of effort to understand what is being spoken is way to high and distracts the user from learning Braille. On top of that we can imagine that such a robotic voice could become irritating very easily when working with the device for some time.

An audio fragment using the pyttsx3 library can be heard here TODO: Record again in English

To cope with this setback we decided to record the voices ourselves. The advantage is that it won’t sound robotic and therefore is a lot easier to understand. To make the audio as pleasant as possible we’ve decided to use a female voice because we think this sounds more friendly and encouraging.

A big disadvantage though of using recorded voices is the decreased modularity. All texts being spoken must be recorded before-hand so new words can’t be spoken out. Therefore for future a main task is to create / find a library to reproduce high quality spoken text on a Raspberry Pi. (AS CAN BE FOUND IN SECTION: FUTURE WORK)

MP3 versus WAV

The mixer library supports both WAV files and MP3 files. Both formats have their advantages. WAV files are uncompressed, meaning that recording is reproduced without any loss in audio quality. But the big downside is the resulting large file sizes.

The MP3 format is way smaller. This is due to that the MP3 format compresses their files. This results in small files but also the overall sound quality is decreased as compared to WAV files. The sound quality is decreased but usually good enough for average listeners. Using the ‘cheap’ speaker we are planning to use, it’s unlikely that any difference between WAV and MP3 could even be heard.

Since we are recording quite a number of audio files we’ve therefore chosen to store them in MP3 format because of their smaller file sizes.

https://www.arduino.cc/en/products/compare https://www.tinytronics.nl/shop/nl/raspberry-pi/main-boards/raspberry-pi-3-model-b-1gb

Tutorial

In an ideal situation a supervisor could help the visually impaired user to set up Braillearn by reading the manual mentioned here. This manual tells the user everything he/she would need to know about the device. But we want to make the user as independent as possible when using our device. One of the disadvantages of the contemporary braille learning techniques is the need of 1-to-1 sessions between a tutor and a student as mentioned in [LINK HIER]. To prevent this issue of occurring here we designed a tutorial in which the user is audio-guided through the interface and needs to participate actively.

The tutorial consist of 5 parts. Each part tackles a different module of the device. We understand that a user who has used the device frequently won’t need to listen to the complete tutorial. Therefore the user can skip parts by pressing the ‘next’ button when the specific part is playing.

The parts discussed are the following:

• Part 1: Introduction of the device
• Part 2: Explanation of buttons
• Part 3: Explanation of Braille cell usage
• Part 4: The learning modes
• Part 5: Conclusion and how to proceed

As already mentioned the tutorial (and the rest of the device) uses recorded voices. For this we needed to create a script of everything that needs to be recorded. The script can be found here.

Manual

The manual to use the product can be found here.

Usability testing

In this section, tests are mentioned in order to measure user-friendliness and drawbacks of current prototype. After each task, user experience will be asked and assessed. At the end of all tasks, a general user experience can be generated and used as feedback in order to improve the current prototype.

The usability tests can be found here

Bill of materials

The bill of materials includes all the required items for the initial prototype that will be created.

Item Name/Number	Supplier	Cost	Amount	Total Cost Item	Link
Solenoid Push-Pull 6V 300mA - JF-0530B	TinyTronics	€3,50	12	€42,00	https://www.tinytronics.nl/shop/nl/robotica/toebehoren/solenoid-push-pull-6v-300ma-jf-0530b
TIP31C Transistor 100V 3A	TinyTronics	€0,50	12	€6,00	https://www.tinytronics.nl/shop/nl/componenten/transistor-fet/tip31c-transistor-100v-3a
Diode 1N4007	TinyTronics	€0,10	12	€1,20	https://www.tinytronics.nl/shop/nl/componenten/diode/diode-1n4007
Tactile Pushbutton Switch Momentary 4pin 6*6*5mm	TinyTronics	€0,10	8	€0,80	https://www.tinytronics.nl/shop/nl/componenten/schakelaars/tactile-pushbutton-switch-momentary-4pin-6*6*5mm
Alpha Wire Draad - Enkeladerig - Solide - Ø1.5mm 0.33mm2 - Rood - 1m	TinyTronics	€1,00	4	€4,00	https://www.tinytronics.nl/shop/nl/kabels/prototype-draden/alpha-wire-draad-enkeladerig-solide-%C3%B81.5mm-0.33mm2-rood-1m
Alpha Wire Draad - Enkeladerig - Solide - Ø1.5mm 0.33mm2 - Zwart - 1m	TinyTronics	€1,00	4	€4,00	https://www.tinytronics.nl/shop/nl/kabels/prototype-draden/alpha-wire-draad-enkeladerig-solide-%C3%B81.5mm-0.33mm2-zwart-1m
Experimenteer-printplaat 7cm*9cm	TinyTronics	€1,00	3	€3,00	https://www.tinytronics.nl/shop/nl/prototyping/printplaten/experimenteer-printplaat-7cm*9cm
Witte Drukknop 12mm - Reset - PBS-33B	TinyTronics	€0,75	2	€1,50	https://www.tinytronics.nl/shop/nl/componenten/schakelaars/witte-drukknop-12mm-reset-pbs-33b
Raspberry Pi 3 Model B 1GB	TinyTronics	€37,50	1	€37,50	https://www.tinytronics.nl/shop/nl/raspberry-pi/main-boards/raspberry-pi-3-model-b-1gb
Mean Well Voeding - 5V 7A - Switching Power Supply - LRS-35-5	TinyTronics	€13,00	1	€13,00	https://www.tinytronics.nl/shop/nl/voedingen/5v/mean-well-voeding-5v-7a-switching-power-supply-lrs-35-5
Standaard 230V Voedingskabel - 1.8m	TinyTronics	€4,00	1	€4,00	https://www.tinytronics.nl/shop/nl/voedingen/accessoires/standaard-230v-voedingskabel-1.8m
Raspberry Pi 40 pins GPIO Extension kit	TinyTronics	€4,50	1	€4,50	https://www.tinytronics.nl/shop/nl/raspberry-pi/accessoires/raspberry-pi-40-pins-gpio-extension-kit
Devil Design PLA Filament 1.75mm - 1kg - Donker Blauw	TinyTronics	€18,00	1	€18,00	https://www.tinytronics.nl/shop/nl/3d-printen/filament/1.75mm-pla/devil-design-pla-filament-1.75mm-1kg-donker-blauw
Standaard Inbouw Wipschakelaar - Klein	TinyTronics	€0,45	1	€0,45	https://www.tinytronics.nl/shop/nl/componenten/schakelaars/standaard-inbouw-wipschakelaar-klein
Broadband speaker 8 ? 3 W	AlleKabels	€4,99	1	€4,99	https://www.allekabels.nl/luidspreker-zelfbouw/237/1370071/broadband-speaker-8-3-w.html
VD draad - VD H07V-U - 1.5 mm2	AlleKabels	€0,39	1	€0,39	https://www.allekabels.nl/vd-draad/7119/1300302/vd-draad-vd-h07v-u-15-mm2.html
Total	Various	Various	65	€145.33	Not Applicable

Discussion & Outlook

ONLY THE THINGS ARE NOT WRITTEN YET IN THIS SECTION:

• Evaluation of product / current state-of-art
• Potential additions to the current design (one addition already done: microphone)

The main limitations of our product concerns the Braille cells that are implemented. Currently there are only two six-point braille cells that are implemented on the top of our product. According to the experiences of blind people, they request at least a 40-point Braille cells on the device. Otherwise they cannot read words or sentences properly. Furthermore it will take some effort and time on their behalf. Due to financial reasons as such a Braille cell (40-point) will take about thousands of euros, this is not implemented on our product. However this is at the expense of their experience which should be valued more than financial aspects.

Another limitation concerns the current design of our product. Blind people are very sensitive in terms of accepting that they are blind. For this reason, by giving them something that is modern or in line with their personality, there is expected that this feeling will be reduced and be turned in something positive. However our current design does not meet the expectations in terms of modernity and therefore blind people are less likely to use our product, even if it is beneficial for them. This is basically the principle of disuse of technologies which is not the direction the project hopes to take.

The audio is also a limitation of the product as it sounds very unnatural. Blind people experience the sound of our product as ‘robotic’ and not pleasant. Therefore learning Braille might also feel unpleasant. By using an other type of speaker which produces better sounds, this might be solved. However this is more expensive.

In terms of audio, there are also additions that have been thought of in future design. By adding a microphone to our product, a new type of modus can be introduced: letter_in/voice_letter_out. In this modus, a blind person has to feel the combination of Braille cells and then have to pronounce the correct letter that corresponds to this combination. By checking the pronounced letter, our product could indicate if this is correct or not. This principle might also work for pronounced words and even sentences. So there are many new options available with this new addition.

Even though our current design does not meet the main requirements of Blind people, it is still a great attempt in terms of contributing to Braille learning. In particular how to make it more enjoyable to learn Braille. The project started was basically a process of acquiring knowledge of Braille learning and designing and re-designing the product based on evaluations, and by ending with an end-product that works in terms of functionality is a great attempt in the right direction of Braille learning and how it should be: enjoyable and useful. By improving the current design in terms of limitations as mentioned before and by adding additions, there is expected that more blind people are willing to use technological solutions our product. In this way, more blind people are hoped to be reached and showed the importance of Braille. Therefore the expected impact is that the number of people that can read Braille will be increased, at least more than 10% as research has shown.

Meeting with Hable

A full overview of Questions & Answers can be found here: Hable Q&A

New ideas & sketches

Approaches for wrong input answers

• Option 1: the user gets a second and third chance to give the correct input. If the user fails to give the correct answer, the device shows the correct answer on the sample cell and moves on to the next exercise. At the end of the modus, all the incorrect answers are asked once again to ensure the user knows the correct answer. After the modus, the device informs the user how many exercises were answered correctly in the first, second, third or fourth instance.

• Option 2: after a wrong input, the user is shown the correct answer on the sample cell. Afterwards, the incorrect exercise is repeated a couple of time throughout the rest of the modus. In case of an incorrect answer, the procedure is repeated.

• Option 3: the incorrect exercise is shown on the sample cell after the modus is done. The user has to repeat the code on the input cell.

Logbook

Week 1

Name	Student number	Time spent	Break-down
Rob Vissers	1244863	10 hours	Group discussion (1.5 hours), finding proper state-of-the-art literature [13]-[17] (2 hours), reading and summarizing the state-of-the-art literature [13]-[17] (4 hours), writing the introduction and users section with relevant literature (2.5 hours).
Ivo Kersten	1233717	9.5 hours	Group discussion (1.5 hours), formatted the wiki page (1.5 hours), found papers [1], [7]-[12] (2 hours), read and summarized papers [1], [7]-[12] (3 hours), wrote requirements (1.5 hours)
Tim Driessen	1006903	7 hours	Opening lecture (2 hours), group discussion (1.5 hours), found papers [18]-[22] (2 hours), summarized papers [18]-[22] (0.75 hours), finding objectives (0.75 hours)
Tom Janssen	1233021	10 hours	Opening lecture (2 hours), group discussion (1.5 hours), learning how to use wikitext (0.5 hours), writing the sections for approach, milestones and deliverables (3 hours), literature research and summarizing artiles on current state-of-the-art devices [23] - [27] (3 hours),
Sander van Bommel	1017917	7 hours	Group discussion (1.5 hours), found papers [2]-[6] (2 hours), summarized papers [2]-[6] (1 hours), writing problem-statement (2.5 hours)

Week 2

Name	Student number	Time spent	Break-down
Rob Vissers	1244863	9.7 hours	Group discussion (3 hours), filled in BOM (0.5 hours), worked out the approximated design and started with SolidWorks (5.5 hours), rewritten user section (0.5 hours), wrote down two questions for Hable (0.2 hours).
Ivo Kersten	1233717	8.5 hours	Group discussion (3 hours), rewritten requirements (1 hour), looked into working of solenoids (1 hour), worked out design for controlling solenoids (1.5 hours), described working of design (2 hours)
Tim Driessen	1006903	10.5 hours	Group discussion (3 hours), contact hable (0.4 hours), found papers [28]-[30] / existing devices(1.5 hours), summarized papers [28] - [30] (1.2 hours), rewriting objectives (0.8 hours), technical possibilities (0.2 hours), wrote existing devices (3.2 hours), questions halbe (0.2) hours
Tom Janssen	1233021	10,5 hours	Group discussion (3 hours), Finding & summarizing research papers on braille, issues in braille learning and strategies for braille learning + Writing the Research section (7,5 hours)
Sander van Bommel	1017917	7 hours	Group discussion (3 hours), rewrote problem-statement and added new references [3] - [7] (2 hours), wrote expected impact (1.5 hours), created Hable label on Wiki, added questions, created new labels: Implementation, Vision and Usability Testing (0.5 hours)

Week 3

Name	Student number	Time spent	Break-down
Rob Vissers	1244863	17.5 hours	Group discussion (3 hours), meeting Hable (1 hours), meeting approval hardware purchase (0.5 hours), getting acquainted with SolidWorks (2 hours), made SolidWorks casing design (11 hours).
Ivo Kersten	1233717	18.5 hours	Group discussion (3 hours), meeting Hable (1 hours), meeting approval hardware purchase (0.5 hours), looking into raspberry pi (1.5 hours), looking into combining Python files (2 hours), looking into dynamically accessing files (2 hours), writing and testing demo program (8 hours), writing readme on Github (0.5 hours)
Tim Driessen	1006903	8.5 hours	Group discussion (3 hours), planning/gantt chart (1 hours), looking into raspberry pi (2 hours), meeting Hable (1 hours), meeting approval hardware purchase (0.5 hours), looking into code (1 hours)
Tom Janssen	1233021	8 hours	Group discussion (3 hours), meeting Hable (1 hours), meeting approval hardware purchase (0.5 hours), watching youtube tutorials on coding in python and getting familiar with the language (3,5 hours)
Sander van Bommel	1017917	11.5 hours	Group discussion (3 hours), meeting Hable (1 hours), meeting approval hardware purchase (0.5 hours), writing manual device (2 hours), transcribing and writing Q&A Hable conversation (5 hours)

Week 4

Name	Student number	Time spent	Break-down
Rob Vissers	1244863	16.5 hours	Group discussion (1.5 hours), added drilling holes to SolidWorks casing (2.5 hours), made and edited SolidWorks casing page (2 hours), soldering and testing the design regarding the solenoids (10.5 hours).
Ivo Kersten	1233717	11.75 hours	Group discussion (1.5 hours), soldering and testing the solenoid design (9.5 hours), edit the electrical design page (0.75 hour)
Tim Driessen	1006903	12 hours	Group discussion (1.5 hours), contact Hable (0.25 hours), setting up raspberry pi + making code run on raspberry pi (7.75 hours), software (2.5 hours)
Tom Janssen	1233021	9.5 hours	Group discussion (1.5 hours), contact with Visio (2 hours), research into hardware and software implementation Raspberry Pi (6 hours)
Sander van Bommel	1017917	7.5 hours	Group discussion (1.5 hours), writing Manual part 2 (3 hours), writing Usability Tests (3 hours)

Week 5

Name	Student number	Time spent	Break-down
Rob Vissers	1244863	9 hours	Group discussion (1 hours), change SolidWorks design after deliberation with 3D printing contact (2.5 hours), editing SolidWorks design wiki page (0.5 hours), worked on new State Of The Art page (5 hours).
Ivo Kersten	1233717	10.5 hours	Group discussion (1 hours), setting up raspberry pi + making code run on raspberry pi (5 hours), adding repeat feature (2 hours), adding feature to quiz whole words (2.5 hours)
Tim Driessen	1006903	13 hours	Group discussion (1 hours), Contact 3D print person (2.5 hours), Software (9.5 hours)
Tom Janssen	1233021	9 hours	Group discussion (1 hours), making exercises (6 hours), restructure wikipage completely (2 hours)
Sander van Bommel	1017917	4 hours	Group discussion (1 hours), Finishing the Manual part 3 (2 hours), creating pictures for the manual (1 hour)

Week 6

Name	Student number	Time spent	Break-down
Rob Vissers	1244863	7.75 hours	Reordered and refined wiki page state of the art (0.3 hours), continued working on new State Of The Art page (3 hours), wrote the society section (2.2 hours), soldering RPi extension kit (1.5 hours), wrote the enterprise section (0.75 hours)
Ivo Kersten	1233717	6 hours	Implementing the modi described in the implementation section (2 hours), Fixing bug that crashed the program when invalid braille input was entered (1 hour), Added feature to have each single letter of a word read out loud (1 hour), Testing all currently implemented features and combinations (2 hours)
Tim Driessen	1006903	6.5 hours	Working on script for sproken text (3 hours), Contact 3D person (0.5 hours), Writing about software structure (3 hours)
Tom Janssen	1233021	3 hours	Work on Manual (x hours), Work on Usability tests (x hours), Sketch GUI for teachers (x hours), Looking into literature and state-of-the-art for strategies that help remembering correct answers & construct a strategy for wrong answers for Braillearn (3 hours)
Sander van Bommel	1017917	4 hours	Improving top view picture of the Manual (0.5 hours), rewriting text corresponding to this picture (1 hour) in the Manual, writing part 1 of Vision/Future (2.5 hours)

Literature summaries

The reviewed state of the art references are given on a separate wiki page: State Of The Art. These references are about braille teaching devices and practicalities regarding learning braille in practice.

The old state of the art references are also given on a separate wiki page: State Of The Art (Old). These references are about the text-to-braille conversion, which was the first idea for the project. Eventually it was decided to make a braille-teaching device.