PRE2019 3 Group4

Group 4

Group member	Student number	E-mail	Study
Tom Janssen	1233021	t.j.a.janssen@student.tue.nl	Chemical Engineering and Chemistry
Ivo Kersten	1233717	i.p.c.kersten@student.tue.nl	Electrical Engineering
Sander van Bommel	1017917	s.p.h.a.v.bommel@student.tue.nl	Psychology & Technology
Tim Driessen	1006903	t.driessen@student.tue.nl	Software Science
Rob Vissers	1244863	r.t.w.a.vissers@student.tue.nl	Electrical Engineering

Introduction

In Europe only, there are already an estimated 30 million people that are either blind or visually impaired. Furthermore it is found that on average 1 in 30 Europeans experience sight loss. This imposes a huge challenge on the society in general, since these people cannot function as properly as intended in the complex society of today. Of all the blind or visually impaired people, 75% is rendered unemployed, while these people could possibly participate in certain jobs if they would receive the necessary education. Now a huge issue arises, since there is a lack of proper braille-teaching material available, which is holding the blind and visually impaired people back.

Also loss of sight can be linked to people getting older, whereas the retinitis pigmentosa deteriorates with increasing age. By looking at the European statistics, one in three seniors with an age over 65 years old struggles with visual impairment or even blindness. Due to these large numbers, it can be seen that the problem of visual impairment and blindness has to be tackled, such that these people can stay active in the society . Since these elderly people are not able to read anymore, they might want to adapt to learning braille to increase their independence.

Problem statement

According to the World Health organization , the estimated number of people that are suffering from visual impairment in the world is 285 million. These people experience difficulties with daily activities that require vision. Vision is considered as an extremely vital sensory modality in humans. The loss of vision affects the performance of almost all activities of daily living (ADL) and instrumental activities of daily living (IADLs); thereby hampering an individuals’ quality of life (QoL), general lifestyle, personal relationships and career . Due to these limitations, these people have a greater probability of experiencing social exclusion, depression and loneliness .

However due to increased knowledge and assistive technologies, there are many new applications and learning systems created to support visually impairment people with their daily activities and make life much easier. Braille learning is considered as one of the most well-known methods that is used to support the visually impaired with reading. In this method, visual impaired people are basically reading text with their fingers by identifying several patterns of raised bumps or dots. Even though it offers these people the opportunity to actually read a book, not many of these people use Braille. According to the National Federation of the Blind , only one in 10 blind people can read Braille, which is dramatically drown from the early 1900s. Furthermore a great proportion of blind children experience considerable difficulties learning to read braille and some never master the skill . Therefore they are more likely to lose interest in learning Braille and search for alternatives.

Even though the interest in Learning Braille had decreased over time, this does not mean that it is outdated or irrelevant. In fact, Braille represents information and education - the currency and the future – for blind people . By learning Braille, blind people will be capable to get access to relevant information, develop high-level skills in reading and writing. Therefore it should be understood, however the current methods of how Braille is taught, might be outdated. Assistive technology should open new ways for Braille to make it more interesting among visually impaired people and improving their well-being.

Objectives

The central objective of the project is:

Main objective: Realize a device that helps an inexperienced person to learn the basics of reading braille. The device will make the Braille literacy more accessible to visually impaired people and can be seen as the first introduction to the Braille language.

From this main objective, a number of smaller objectives can be deduced, namely:

Objective: A visually-impaired-user-friendly interface

As the target group is visually impaired, an interface will be developed that provides clear communication in both ways between user and device without requiring the ability to see.

Objective: Facilitate learning

A number of learning modes will be created that provides the user with a fun/interactive way of learning Braille. These learning modes will use inputs from the user in the form of pushing braille pins and/or outputs from the system by means of sound or moving braille pins.

Users

When designing a product, it is important to keep the users (actively or passively) involved in the design process as soon as possible. Therefore it is important to take into account the values and needs of all involved users. With proper participation and empirical research, the design process can be centered around the user for the best final result. The users involved within the subject of learning braille can be categorized in primary and secondary users as described below.

Primary Users

The primary users are the people that will actively use the product, namely being the visually impaired and blind people that do not yet know the braille language. This is caused due to the lack of braille learning material that is generally available for consumers and organizations. By designing a system that teaches braille with one letter or one word at a time, with the help of a braille example or pronounced words, braille can be learnt by a much broader public. These visually impaired and blind people cannot get the necessary education they need from reading literature or browsing the internet, they will get a learning disadvantage. Therefore with a system that teaches the braille language, the independence of the users will grow. The independence of the user is an important aspect, since blind and visually impaired people strive for a certain amount of independence, which is lost due to their disability. With multiple levels of difficulty, this braille-teaching device can be used by a widespread public, i.e. by children that are learning letters, or by adolescents that want to learn braille letters and words (including pronunciation).

Secondary Users

Regarding the secondary users, there are multiple organizations that would want to use a device to teach braille to people.

• At first, educational institutes, such as kindergartens, schools, and universities, would want to use the braille-teaching device to make learning braille for blind and visually impaired people more attractive at their institution. By showing that their needs are taken into account, the blind or visually impaired people will be triggered more to go to a certain institution that respects their disabilities. This will generally lead to an increased number of students at certain institutions. It will also help to reduce the workload that is imposed on the braille teachers at a certain institution.

• Secondly, non-profit organizations that want to help children with certain disabilities might adapt to this design. By investing in a braille-teaching device for children that cannot afford it, the literacy for these children will greatly increase. Since the main goal of these non-profit organizations is to help disabled children, this device would be a proper addition to their functional capabilities.

Requirements

Overall system must:

• be able to be set up in less than a minute.
• be able to be activated by a blind person.
• be built with high contrast between colors of the box the system is in and the buttons.
• have functional buttons with clearly recognizable shapes to ease operation.
• be affordable to as many people as possible (preferably < €200).

Physical braille display must:

• be capable of correctly displaying all basic braille symbols: letters, numbers, and punctuation, one at a time.
• be able to be reset and rewritten in less than 0.5 seconds.
• provide enough force to each braille dot such that they can be read easily.

Physical braille input must:

• have braille dots that are easily pressed down with light force, and then kept in downward position.
• be able to be reset in less than 0.5 seconds.
• reliably capture presses (>99%).

Approach, Milestones & Deliverables

Approach

The different aspects of the approach can be subdivided into milestones. Consequently, these can be distributed over a planning that fits the time span of this project.

In order to obtain the optimal and most adequate results on the topic, several different methods can be used to obtain information:

• Literature research into the different aspects related to this topic:

  - Current state-of-the-art devices for this purpose
  - Scanning technology for written/typed text
  - Conversion technology of scanned text to braille
  - Dynamic braille surfaces
  - Design considerations for optimal user experience and versatility for different formats of clustered text

• Collaboration with visually impaired people to incorporate the advice of the primary users.
• Development of a small prototype and applying a series of tests to check whether the device meets the previously set requirements.
• Comparative tests with current state-of-the-art devices to discover points of improvement, as well as points of superiority of our product.

Planning & Milestones

Below the global planning of this project is displayed.

Week	Tasks & milestones for the report	Tasks & milestones for the prototype
1	Choose the subject Write the start of the sections for the problem statement, objectives, users, requirements, approach, planning & milestones, deliverables, logbook Investigate and summarize current state-of-the-art techniques and devices on this topic
2	Continue writing on the different sections of the report Write the start of the sections for current state-of-the-art and our own research	Approximated design List of needed hardware components
3	Continue writing on the different sections of the report Gantt chart Manual Contact Visio or other instances Decide on focus project	Design SolidWorks Purchase hardware components Software concepts
4	Test plan Working out Hable interview	Developing software Creating prototype
5	Further work report	Developing software Creating prototype
6	Create questions/exercises for testing of week 7	Finishing software Finishing prototype
7	Create presentation Work out test results	Test prototype with visually impaired people
8	Presentation Finish report Work out test results	Incorporate feedback tests week 7 Bug fixing

Deliverables

• A small prototype of a device that helps an inexperienced user with learning braille.
• A report of all our literature research, analysis and results on this topic. This will be documented on this wiki page.
• A presentation at the end of this course to share our findings.

Gantt Chart

State of the art

Existing Devices

Looking at the State-of-the-Art regarding devices made for learning devices we have come across a number of products. Looking at these products we can see how they work, what are the advantages and disadvantages and therefore be able to find a place to fit our concept.

LEGO Braille Bricks

Recently LEGO unveiled a new project aiming to help blind and visually impaired children to learn Braille. The Braille bricks are similar to the common 2x4 blocks, except they don’t have eight “studs”, but use a 2x3 array of studs to represent a braille cell. At the bottom of the brick there is room for a visual indicator of the letter or symbol for the supervisors. The LEGO Braille bricks should fully launch in 2020.

LEGO Braille Bricks

The combination of LEGO and Braille looks like a perfect match. Each original LEGO block already contains “studs”, and more importantly LEGOs is meant to be a toy for children. Children associate pleasant thoughts with the bricks. Introducing the Braille language here is a perfect way to make learning fun. Because of the freedom of placement people have with LEGOs, sentences and words can be easily constructed using the braille blocks. An obvious disadvantage is the need for a supervisor. As stated in [BRON HIER], a supervisor can only effectively help one person at a time. Therefore teaching Braille to a class of visually impaired people is still quite difficult.

Annie

Tinkerbell Labs have created a Braille literacy device to help the visually impaired to learn Braille on their own, through audio-guided gamified content. It consists of two large Braille cells which are used to introduce braille to an inexperienced Braille reader. It also includes six standard-sized braille cell to cover all primary learning needs. The input of the user is giving through a large Braille keyboard placed at the center of the device.

Annie

A big advantage of this device is that it enables one teacher to teach more than one student simultaneously. This can be seen as an obvious advantage, as opposed to the more traditional methods for learning Braille [BRON HIER]. A large disadvantage though, is the price point. As of now, the device costs $949 with access to the companion app and analytics program for $149 per year. This makes the device mostly accessible to institutional organizations and less accessible for individual use.

Taptilo

Taptilo is an innovative braille education machine without the need of a professional teacher. The top of the device contains nine removable, magnetic Braille cells which users can use to create their own letter (they can push in “studs”). These cells interact with the permanent row of nine refreshable cells placed at the bottom. These cells are ‘jumbo-sized’ meaning they are bigger than usual, making it easier for inexperienced users to learn.

The device has implemented five teaching modes:

1. READ: Select a word which will be displayed and read out.
2. TRACE & WRITE: Select a word which will be displayed and read out. Trace this Braille word (on the permanent row) and try to match the blocks with the removable braille cells.
3. DICTATION: Select a word which will be read out. Then try to spell the word using the removable braille blocks.
4. WRITE: Make your own word using the blocks. Then Taptilo will read the word.
5. GAME: The permanent row will display the letters of a scrambled word. Try to put the letters in the right order using the removable cells.

They have utilized an artificial intelligence (AI) speaker to be able to output, not only predefined letters/words, but also words the system has never pronounced.

Taptilo

The advantages/disadvantages are very similar to that of Annie. A big advantage is again that the devices enables one teacher to teach a full class of students instead of only one student at a time. The disadvantage is the price point. Taptilo, which costs $1,349.00, is even more expensive than Annie making it again less accessible for individual use.

Hable

Visually impaired users often use speech-recognition to operate their phones. The problem with that is inaccuracy and a lack of privacy (imagine sitting in a train). This is where Hable steps in. They are developing a device which presents visually impaired users with a way to enter text onto their phone. Work is also being done on ways to make the device also able to navigate the phone and open apps.

The device consists of six braille buttons which are combined with two functions buttons for commands such as spacebar, enter and backspace, but also to navigate through your phone. It can be attached to the back of your smartphone, or be used freely. Bluetooth is used to connect to the phone.

While Hable is not exactly a braille learning device, it still is a company with a lot of experience and knowledge surrounding Braille. Hable having its roots in the TU/e, we managed to get into contact with them, of which the findings are shown in [Verwijzing hier].

Hable

Research

Fundamentals of braille

Braille cell

Braille is a tactile writing system for people that are blind or to some degree visually impaired. The system has many variations, but the most commonly used is the 6-dot braille. This type makes use of a 2 by 3 dot cell that can represent a letter, digit, punctuation mark and even certain contractions. These different elements are represented by combinations of raised dots within a cell. Since there are 6 dots that can potentially be raised, there are 64 (2^6) different combinations possible. The 6 dots are numbered in a downward fashion in the consecutive columns, where the top left dot has number 1 and the bottom right dot has number 6.

The different literary elements are grouped in certain decades. A decade is a group of 10 different combinations that only make use of a specific subsection of the 6 dots. Literary elements that are alphabetically adjacent or just similar in use are placed in the same decade to improve the efficiency and ease of reading braille.

• The first decade is made up of the upper four dots (numbers 1,2,4 and 5), and represents the letters 'a' to 'j' as well as digits 0 to 9.
• The second decade makes use of dot 3 in addition to the upper four dots, and represents the letters 'k' to 't'.
• The third decade makes use of both dots 3 and 6 in addition to the upper four dots. The first half of this decade makes up the remaining letters of the alphabet 'u' to 'z' with the exception of 'w'. The other half is used for the commonly used words 'and','for','of','the' and 'with'.
• The fourth decade makes use of dot 6 in addition to the upper fout dots, and represents some common 2-letter combinations in print as well as the letter 'w'.
• The fifth decade is the same as the first but then all dots are shifted one down. These represent the punctuation marks.
• The sixth decade makes use of dots 3,4,5 and 6, and represent some of the remaining punctuation marks and common 2-letter combinations.
• The seventh decade only makes use of the right column, and is used to obtain a certain effect such as two-celled contractions, italic letters, capital letters, etc.
• Last, there is the completely empty cell (no dots raised), which represents a space.

Below you can find a simplified overview of the literary elements per decade on the left. This overview does not contain all the different braille symbols and is thus incomplete. Therefore, a complete overview is displayed on the right. However, this overview is not organized per decade.

Simplified overview braille symbols per decade

Complete overview braille

Main issues in learning braille

It is estimated that 285 million people are visually impaired and 39 million people are completely blind. Only a small percentage of approximately 10% of this group can read braille. This is a relatively small number and greatly limits the freedom and capabilities of blind individuals and the complete blind community. There are multiple factors that complicate or eliminate the possibility to learn braille. These factors can lie with the subject but also with the current means available for braille. Therefore, some factors can be controlled and some cannot. Below the different factors will be discussed.

• To start, only 80% of all visually impaired people are potentially able to read braille, because they can experience the required haptic feelings. The remaining 20% can not and is therefore unable to learn braille even if they would experience no further issues.
• People above 40 years of age that have not yet learned braille in a previous stage of their life will have an increasingly more difficult task to learn it. Until 40 years of age there is no drastic increase in how hard it is to learn braille. Moreover, if one has already learned braille when they were younger, then no major issues should occur if they try to continue in a later stage in life.
• The phonological deficit theory states that reading retardation is caused by the deficit of representation, processing and storing speech sounds. It has been shown that this also holds for blind subject and can indeed cause issues in one's ability to read braille. Additionally, for people who just started learning braille the process can be slowed down drastically by a deficit in processing and storing speech sounds, since audio will be used to provide the definition and meaning to the tactile sensory input from the fingers.
• The magnocellular theoery states that difficulty with reading (braille) can be caused by damaged sensory pathways that are responsible for rapidly varying streams of input.
• In some poorer countries in the world, many institutions for blind people simply don't have the means and knowledge to properly help their students.
• The current method to learn braille mainly exists out of 1-to-1 sessions between a tutor and a student. Learning in a group is inefficient, since it causes a lot of distractions for the students and makes it impossible for a tutor to give every student the required attention. Since it takes 150 hours on average to obtain a decent level of braille, these private lessons can get rather expensive.
• The sensory tactile input that braille relies on is in many aspects inferior to the visual input from reading print text. The stimuli are processed much slower and especially while learning braille many people first have to convert the braille to normal letters in their head, which takes up even more time. Furthermore, distinguishing so many different characters is much harder by touch due to the lack of detail. This also makes it easier to become confused.
• There are some inherent difficulties to braille itself. First of all, there are a lot of different combinations that one has to memorize just to learn individual letters, contractions and words. For children that were blind at birth, this is an extra challenge on top of learning letter combinations to form words. Some of these specific combinations can also easily be forgotten after one hasn't seen it for a long time. Second, there are many different characters that have the same braille code. The specific meaning of the 6-dot cell is then dependent on context, which takes a lot of practice to master. Additionally, there are sometimes multiple different codes to write the same thing, which can also cause confusion. Last, every language or sometimes even every country (even when two countries speak the same language) differs slightly in what code represents a letter, contraction or word. This gives an extra dimension of difficulty for learning different languages in braille, since one must first learn the new code before even starting to learn the new words and letter combinations.
• Many visually impaired people simply do not want to start with learning braille, because this would mean to them that they are officially blind. So, the stigma on being blind often delays when people start to learn braille if they start at all. This also greatly impacts the number of blind people that can actually read braille.
• Since learning braille is a slow process, many people get discouraged along the way. Making sure that people stay positive during the learning process is actually one of the most important tasks of the 1-to-1 tutors nowadays. If more entertaining methods for learning braille or more positive feedbacks systems could be implemented this issue could be helped significantly.
• Current methods to learn braille often do not match the most efficient ways to learn braille.

Strategies for learning braille & Gamification

As mentioned in the previous section, one of the major issues in learning braille is to stay motivated in the slow process. Not that much is known on the best way to learn the tactile writing system efficiently and most of the current method rely on simple repitition. Granted that this is what learning most languages comes down to, it is not always stimulating student engagement. One method that could help in this aspect is Gamification, where different game elements are incorporated in the learning exercises to make them more exciting and engaging. Below, different elements are explained that incorporate gamification into the prototype of this project.

• Setting a time clock on different difficulty levels for users to type a certain number of letters or words. This gives a score for both the taken time and number of correct letters/words.

Expected Impact

Currently about 10% of the blind people worldwide is actually able to read Braille. This means that 90% of these people is not capable or is not willing to learn Braille. However, our product is meant to increase this interest and provide a new type of Braille learning that is understandable, educational and interesting. If blind people would experience the benefits in terms of improvements in their daily life experience, then there is evidence found that our product actually has a positive effect on the experience of blind people and it can be implemented in a real-life setting. Therefore there is expected that a great number of this 90% blind people will regain interest in Braille and thereby also generate more interest in our product.

Our product also allows independent use, which means that a constant accompaniment is not necessary any longer. Blind people are capable to activate our product by themselves and can use it to learn Braille. In this way, caregivers of these blind people can spend their time to other individuals that actually need guidance with reading. This is not only less costly, but caregivers will also be able to work more efficiently. For this reason, cost-efficiency will be maintained. Furthermore it expected that engineers or developers of our product will experience a financial gain due to the increased demand on the market. If governments would acknowledge the benefits of the product as well, they can provide support to these developers/engineers in terms of scientific funds. Scientific funds will contribute to further development in research and knowledge of our product, which can lead to even new applications or breakthroughs.

Bill of Materials

The bill of materials includes all the required items for the initial prototype that will be created.

Item Name/Number	Supplier	Cost	Amount	Total Cost Item	Link
Solenoid Push-Pull 6V 300mA - JF-0530B	TinyTronics	€3,50	12	€42,00	https://www.tinytronics.nl/shop/nl/robotica/toebehoren/solenoid-push-pull-6v-300ma-jf-0530b
TIP31C Transistor 100V 3A	TinyTronics	€0,50	12	€6,00	https://www.tinytronics.nl/shop/nl/componenten/transistor-fet/tip31c-transistor-100v-3a
Diode 1N4007	TinyTronics	€0,10	12	€1,20	https://www.tinytronics.nl/shop/nl/componenten/diode/diode-1n4007
Tactile Pushbutton Switch Momentary 4pin 6*6*5mm	TinyTronics	€0,10	8	€0,80	https://www.tinytronics.nl/shop/nl/componenten/schakelaars/tactile-pushbutton-switch-momentary-4pin-6*6*5mm
Alpha Wire Draad - Enkeladerig - Solide - Ø1.5mm 0.33mm2 - Rood - 1m	TinyTronics	€1,00	4	€4,00	https://www.tinytronics.nl/shop/nl/kabels/prototype-draden/alpha-wire-draad-enkeladerig-solide-%C3%B81.5mm-0.33mm2-rood-1m
Alpha Wire Draad - Enkeladerig - Solide - Ø1.5mm 0.33mm2 - Zwart - 1m	TinyTronics	€1,00	4	€4,00	https://www.tinytronics.nl/shop/nl/kabels/prototype-draden/alpha-wire-draad-enkeladerig-solide-%C3%B81.5mm-0.33mm2-zwart-1m
Experimenteer-printplaat 7cm*9cm	TinyTronics	€1,00	3	€3,00	https://www.tinytronics.nl/shop/nl/prototyping/printplaten/experimenteer-printplaat-7cm*9cm
Witte Drukknop 12mm - Reset - PBS-33B	TinyTronics	€0,75	2	€1,50	https://www.tinytronics.nl/shop/nl/componenten/schakelaars/witte-drukknop-12mm-reset-pbs-33b
Raspberry Pi 3 Model B 1GB	TinyTronics	€37,50	1	€37,50	https://www.tinytronics.nl/shop/nl/raspberry-pi/main-boards/raspberry-pi-3-model-b-1gb
Mean Well Voeding - 5V 7A - Switching Power Supply - LRS-35-5	TinyTronics	€13,00	1	€13,00	https://www.tinytronics.nl/shop/nl/voedingen/5v/mean-well-voeding-5v-7a-switching-power-supply-lrs-35-5
Standaard 230V Voedingskabel - 1.8m	TinyTronics	€4,00	1	€4,00	https://www.tinytronics.nl/shop/nl/voedingen/accessoires/standaard-230v-voedingskabel-1.8m
Raspberry Pi 40 pins GPIO Extension kit	TinyTronics	€4,50	1	€4,50	https://www.tinytronics.nl/shop/nl/raspberry-pi/accessoires/raspberry-pi-40-pins-gpio-extension-kit
Devil Design PLA Filament 1.75mm - 1kg - Donker Blauw	TinyTronics	€18,00	1	€18,00	https://www.tinytronics.nl/shop/nl/3d-printen/filament/1.75mm-pla/devil-design-pla-filament-1.75mm-1kg-donker-blauw
Standaard Inbouw Wipschakelaar - Klein	TinyTronics	€0,45	1	€0,45	https://www.tinytronics.nl/shop/nl/componenten/schakelaars/standaard-inbouw-wipschakelaar-klein
Broadband speaker 8 ? 3 W	AlleKabels	€4,99	1	€4,99	https://www.allekabels.nl/luidspreker-zelfbouw/237/1370071/broadband-speaker-8-3-w.html
VD draad - VD H07V-U - 1.5 mm2	AlleKabels	€0,39	1	€0,39	https://www.allekabels.nl/vd-draad/7119/1300302/vd-draad-vd-h07v-u-15-mm2.html
Total	Various	Various	65	€145.33	Not Applicable

Usability Testing

• Factors to examine usability (user-friendly) according to :
  • ease of use - the system should be easy to learn so that the user can rapidly start getting some work done
  • learnability - the system should be efficient to use so that once the user has learned the system, a high level of productivity is possible
  • memorability - the system should be easy to remember so that the causal user is able to return to the system after some period of not using it, without having to learn it all over again
  • errors - the system should have a low error rate and if errors are made, people should easily recover from it
  • satisfaction - the system should be pleasant to use so that users are subjectively satisfied when using it

• Other measurements:
  • emotions / feelings
  • unforeseen risks / difficulties

Implementation

The product that will be created throughout the project will have multiple functionalities. Therefore it is important to get an overview of all these functionalities to find a proper way of implementing each subpart in the final design. At first the RaspberryPi (RPi) has been chosen, due to the better performance than the Arduino (1 GB RAM Memory, 40 IO Pins, and an 1.2 GHz micropocessor). This will be the core of the design, since the RPi will direct all the different signals to the different subparts of the final design. The different subparts that will be implemented are:

• A switch to turn the device on and off (an on/off switch);
• A braille example keyboard that will show the letters that are under consideration (6 solenoids);
• An input braille keyboard where the user has to press the button of the example keyboard to generate the required letter (6 solenoids);
• A button to reset the current input on the braille keyboard (a reset button);
• Voice-controlled user encouragement via a speaker or via an Aux input (speaker and Aux port);
• Two buttons to control the mode that is tested on the user, whereas the modes are letter_in/letter_out, voice_letter_in/letter_out, word_in/word_out, and voice_word_in/word_out. All of these modes will be deliberated upon more further into this section (two button for mode control);
• A next button to confirm the input of the user (one button for next letter or to complete input);
• A power input port for the RPi (power adapter regarding the RPi).

A crude approximation of the final product that will be manufactured throughout the project is shown in the picture to the right.

A crude approximation of the final product that will be made throughout the project

Regarding the different input modes that can be controlled by the user, there was:

• letter_in/letter_out: In this mode the user will learn the letters of the alphabet via the example keyboard and the letter under consideration will also be voice-controlled. Then the user presses the next button and presses the solenoid buttons that represent the shown example letter. By pressing the next button again, the input of the user is validated, and if this input is correct the user will be commended. If the input would be wrong, the user can try again with the example letter for a second try.
• voice_letter_in/letter_out: In this mode the user will learn the letters of the alphabet via voice-control. Then the user presses the solenoid buttons that represent the spoken letter under consideration. By pressing the next button to confirm the input, the input of the user is validated. If it is correct the user will be commended. If the input is incorrect, the user can try again with the voice-controlled letter under consideration.
• word_in/word_out: In this mode the user can learn words, whereas the example keyboard will show single letters of the word one by one, while the word under consideration will also be voice-controlled. The user presses next after consecutive letters of the word, and after the whole word has been shown as an example, the user can press the solenoid buttons that represent the letters of the word under consideration by pressing next after each input letter. If the final letter inputs for the word are correct the user will be commended. If one or more of the letter inputs are incorrect, the user can simply try again by repeating the example braille letters.
• voice_word_in/word_out: In this mode the user can learn words solely based on their pronunciation, whereas the word will be voice-controlled. Now the user will press the buttons for the consecutive letters of the word on the input keyboard, where each letter is followed by the next button. If the word has been completed, it will be checked. If the final letter inputs for the word are correct the user will be commended. If one or more letter inputs are incorrect, the user can simply try again by hearing the pronunciation of the word once more.

With all of these different modes, multiple levels regarding the learning of braille can be achieved. In the process an inexperienced person may start with letters and finally learn words, while a more experienced person may already start with words. This will increase the range of the population that can use this product.

Since the product has to be for a broad educational public, the specifications have to be well-defined. For example the cost of the product should not exceed €200,00 such that pre-schools can invest in this product. Also the weight of the product should not be more than 500 grams, since the device has to be portable for mainly younger children. The dimensions of the device should also be kept in its perks, which are now defined as 12x4x5 cm. This property, however, is still subject to change regarding the material that are implemented in the final design.

Design

• How are buttons coordinated
• Sizes
• Weight
• Prototype

How is the braille cell actuated

Wiring schematic for the braille pad.

The braille dots in the reading cell will be actuated by solenoids, one solenoid for each dot. The solenoids will be controlled by the RaspberryPi that controls the whole system. However, since the solenoids require a large current to maintain their position, the control signal form the RPi cannot be used to directly supply power to the solenoids. Instead, the control signal is applied to the gate of a power MOSFET, in this case the TIP132, to allow current to flow from the main 5V bus to ground, through the solenoid. This energizes the solenoid, pushing a metal pin upwards. When the control signal to the gate of the MOSFET goes low again, current can no longer flow to ground. However, an inductor such as a solenoid will in this case act as a voltage source temporarily, as it takes time for the current through an inductor to change. To prevent sparks, a flyback diode is put in parallel to the solenoid in opposite direction, to provide a path for the current to flow when the transistor is closed, but the solenoid is still energized. This circuit is depicted in the figure to the right.

An important thing to keep in mind is that the solenoid is not intended to be constantly powered at its maximum force. This would cause the coil to heat up and be damaged fairly quickly. To prevent this, several approaches can be taken. The first of which is to add a resistor in series with the solenoid as this would limit the current through the solenoid. A downside of this approach is that the maximum force exerted by the solenoid will be limited. Another way of reducing the (average) current through the solenoid is to apply a Pulse-Width Modulated (PWM) signal to the gate of the transistor. This would allow current to flow through the solenoid for an adjustable portion of the time, reducing the force generated and current required as well. The advantage of this approach as opposed to adding a resistor is that it is adjustable in code.

How are button presses registered

Wiring schematic for a basic pull-up switch.

Button presses of either the control buttons that advance the system to another level, the control buttons that confirm or reset the given input and the buttons that register braille dot presses will be implemented by using a basic switch. The switch will be normally open, so nu current can flow through it. This causes the 10k resistor to have no voltage drop, letting a voltage of 0V appear at the RaspberryPi GPIO pin. When the button is pressed, the switch will close connecting the resistor directly between 5V and ground. Now the voltage at the GPIO pin is pulled up to 5V, which can be registered by the RaspberryPi and depending on which GPIO pin goes high, the appropriate action can be taken.

Functionality

• Functions of buttons
• Audio (volume)
• Several levels of difficulty/modus
• How can a user operate the device / interaction

SolidWorks Casing

The casing that will be used for the final design has been made in SolidWorks. Due to the broad available literature regarding SolidWorks, this was the most viable option. The final design, and its subparts (case and lid), are depicted in the images to the right.

In the final design, the dimensions L x W x H are much larger than the estimated dimensions for the crude approximation, namely being 220 x 110 x 75 mm. This is due to the fact that an external power supply has to be added that can deliver 7 A. Also the solenoids take much more place than estimated to reduce the total cost, since the tiny braille cells are quite expensive.

Another thing that might be noticeable is that the holes have not been added for the solenoids and the headphone input. This has been done deliberately, since these dimensions might be subject to change throughout the design. Therefore it is better to drill these holes ourselves when all the component placements are known inside the casing.

The left side of the casing that will be used for the final product

The right side of the casing that will be used for the final product

The top of the casing that will be used for the final product

The top of the lid that will be used for the final product

The casing in total that will be used for the final product

Software

The software is available on Github and will be kept up to date during the project.

Vision/Future

• Evaluation of product / current state-of-art
• Potential additions to the current design
  • Microphone
• limitations
• Impact

Manual

The manual to use the product can be found here

Hable

• Questions:
  • How does the Hable system work in its entirety, i.e. how is it connected to the cellphone and how are the braille cells read by the system?
    • Is the Bluetooth protocol similar to that of for example a wireless Bluetooth keyboard?
  • What were your expectations before testing your keyboard on blind people and how much were they in line with the evaluation by the blind people themselves?
  • Were there any unforeseen reactions or particularities investigated during their evaluation which you have used to improve your prototype?
  • How much time did it take for blind people to get used to the keyboard/ were there any difficulties?
  • What materials did you use to build your keyboard and can you explain why you used certain materials over other materials?
  • Did you test the user acceptability for your design, and if so, how did you properly test it?
  • How would navigation on your phone using the Hable system work?
  • How would you explain the functionalities of the system to a blind person, i.e. how do they know which button represents which action?

A full overview of Questions & Answers can be found here: Hable Q&A

State of the art - abstracts (temporary title)

Summary: In this paper a low-cost, low-power portable system is described that serves as a braille reading and writing system. It can use a braille keyboard to display the written text in braille, such that visually impaired people can practice their braille reading and writing skill. It also has the capability of converting a text document to braille.

Summary: The outcomes of a national survey in Greece are presented in this article. In the survey the preferences and choices of students with vision impairments on literacy medium for studying are examined. The study shows that braille and large print are the preferred mediums for studying. However, the majority uses the medium of listening as the best performance medium for studying.

Summary: This paper describes a system that aims to help the visually impaired by recognizing written text around them and converting it to spoken text, which is played to the user. This system responds to all written text around the user and also notifies the user of the distance to the nearest object. It achieves an accuracy of 84%.

Summary: This paper describes a way to create braille dots that can be used on a braille display by using an electrothermal design where the dots can be displaced out of the plane by 250 microns and a temperature difference of 58°C with the environment can be achieved. This way, the dots can be small and cheap, requiring only an input voltage of 1.36 V.

Summary: Electronically refreshable braille displays have been around for some time, but they have been very expensive. This paper presents a low cost refreshable braille display that uses very little power. It also describes open source text to braille scanner using Google’s open source optical character recognition (OCR) engine.

Summary: In general with the production of text entries or smartphones little attention is paid to people with no or impaired vision. Finding the keys for voice control is also an issue for blind people. The newly introduced BrailleEnter will support non-visual interaction with a touchscreen device, since users can tap the screen to raise Braille dots based on Braille coding. When the screen is gently touched, the Braille dots will not rise.

Summary: From the visually impaired or blind people, being namely 161 million people, only 3% are able to read, write, or count. This is due to the fact that there is lack of Braille reading material in schools. A solution is proposed, where 3D printed visual representations of books are manufactured to improve the learning potential of the target users.

Summary: Visually impaired individuals are limited in terms of communication, interaction, and personal autonomy due to the lack of Braille literature linked to economic reasons. A portable device is introduced as a reading system for visually impaired individuals, which is based on segmentation, feature extraction, and machine learning for improved accuracy.

Summary: With the introduction of touchscreens, the accessibility for blind people decreased significantly. Due to the high demands for mobile phones, it is important to also take into account the accessibility for blind people. Research is done to obtain a new way of implementing Braille text in smartphones. From the data of 5 databases, the research is performed.

Summary: A novel approach to converting Chinese text to Chinese Braille is proposed. A Braille word segmentation model, based on statistical machine leaning, is trained on a Braille corpus, and also on Chinese word segmentation. This will avoid the establishment of syntactic and semantic information rules. Furthermore a statistical model will learn these rules automatically in the background.

Summary: Design, Prototype and implementation of a Sign Language (ASL) to Braille Converter as well as an English Language to Braille Converter. The article proposes a simple and affordable device which was experimentally verified to give accurate outputs.

Summary: A comparison between visually impaired (visually impaired print reader: PR, braille reader: BR) and normally sighted (normal vision: NV) school children was performed based on reading rate and comprehension. BR had the lowest reading rate compared to other groups. The findings suggest that visually impaired students required a longer time to read and understand a text.

Summary: Although several text to braille converters are available, the cost is a factor which prevents this technology to reach all people of society. Therefore a low cost gesture controlled text to braille converter was developed.

Summary: A system for converting written text into braille was developed. It uses optical character recognition to translate written text into digitized texts, which are then transferred electronically in a braille haptic device. The overall system reliability is 95.68% and the system can process 1 word in 2 seconds.

Summary: To aid the blind and visually impaired (BVI), a portable text reading system, called Finger-eye was developed. This system uses a small camera placed on the blind person’s finger to continuously process images using the optical character recognition (OCR) method, which are then translated into a refreshable mechanical braille display.

Summary: This model can translate different types of text files in English to braille. Both the input and output are in text format. It uses six point cells to display braille characters obtained from converting eglish text. It does this using Matlab.

Summary: This method describes the translation of many different languages into braille, and saving it on a computer. It uses a table-driven method for this, and it should be relatively easy to adjust the method for a related purpose.

Summary: This portable device includes a scanner of text, converting algorithm for text-to-braille, braille display and braille keyboard for annotation features.

Summary:A cursive handwriting recognition system using artificial neural networks is applied. The features of each written character in the input is extracted and passed on to the neural network. This network uses data sets of handwriting from many different people to convert the handwriting to text, making it very accurate.

Summary: An Optical Character Recognition (OCR) program is developed using the Hidden Markov Model. The input is an image of written english text that is converted to printed text, with said model.

After First Feedback

Summary: Since the literacy rate among visually impaired people in many countries is very low, a braille learning device was developed that uses a braille keypad and microphone as input and produces speech and pins of a single braille cell as output.

Summary: Braille literacy has been declining mostly due to the use of electronic text and assistive software, such as screen readers. However Braille literacy is still the most empowering form of literacy for blind people. Therefore the research goal is to provide new tools to improve Braille literacy. First the problems with the nowadays used methods for learning Braille are stated. Next hardware and software tools for alternative Braille-based applications are shown/discussed.

Summary: Learning Braille requires the assistance of another person to help identify the correspondence between the Braille pattern and the character. To eliminate the need for assistance, a spoken dialogue system was developed that allows visually impaired individuals to self learn Braille.

Logbook

Week 1

Name	Student number	Time spent	Break-down
Rob Vissers	1244863	10 hours	Group discussion (1.5 hours), finding proper state-of-the-art literature [13]-[17] (2 hours), reading and summarizing the state-of-the-art literature [13]-[17] (4 hours), writing the introduction and users section with relevant literature (2.5 hours).
Ivo Kersten	1233717	9.5 hours	Group discussion (1.5 hours), formatted the wiki page (1.5 hours), found papers [1], [7]-[12] (2 hours), read and summarized papers [1], [7]-[12] (3 hours), wrote requirements (1.5 hours)
Tim Driessen	1006903	7 hours	Opening lecture (2 hours), group discussion (1.5 hours), found papers [18]-[22] (2 hours), summarized papers [18]-[22] (0.75 hours), finding objectives (0.75 hours)
Tom Janssen	1233021	10 hours	Opening lecture (2 hours), group discussion (1.5 hours), learning how to use wikitext (0.5 hours), writing the sections for approach, milestones and deliverables (3 hours), literature research and summarizing artiles on current state-of-the-art devices [23] - [27] (3 hours),
Sander van Bommel	1017917	7 hours	Group discussion (1.5 hours), found papers [2]-[6] (2 hours), summarized papers [2]-[6] (1 hours), writing problem-statement (2.5 hours)

Week 2

Name	Student number	Time spent	Break-down
Rob Vissers	1244863	9.7 hours	Group discussion (3 hours), filled in BOM (0.5 hours), worked out the approximated design and started with SolidWorks (5.5 hours), rewritten user section (0.5 hours), wrote down two questions for Hable (0.2 hours).
Ivo Kersten	1233717	8.5 hours	Group discussion (3 hours), rewritten requirements (1 hour), looked into working of solenoids (1 hour), worked out design for controlling solenoids (1.5 hours), described working of design (2 hours)
Tim Driessen	1006903	10.5 hours	Group discussion (3 hours), contact hable (0.4 hours), found papers [28]-[30] / existing devices(1.5 hours), summarized papers [28] - [30] (1.2 hours), rewriting objectives (0.8 hours), technical possibilities (0.2 hours), wrote existing devices (3.2 hours), questions halbe (0.2) hours
Tom Janssen	1233021	10,5 hours	Group discussion (3 hours), Finding & summarizing research papers on braille, issues in braille learning and strategies for braille learning + Writing the Research section (7,5 hours)
Sander van Bommel	1017917	7 hours	Group discussion (3 hours), rewrote problem-statement and added new references [3] - [7] (2 hours), wrote expected impact (1.5 hours), created Hable label on Wiki, added questions, created new labels: Implementation, Vision and Usability Testing (0.5 hours)

Week 3

Name	Student number	Time spent	Break-down
Rob Vissers	1244863	17.5 hours	Group discussion (3 hours), meeting Hable (1 hours), meeting approval hardware purchase (0.5 hours), getting acquainted with SolidWorks (2 hours), made SolidWorks casing design (11 hours).
Ivo Kersten	1233717	18.5 hours	Group discussion (3 hours), meeting Hable (1 hours), meeting approval hardware purchase (0.5 hours), looking into raspberry pi (1.5 hours), looking into combining Python files (2 hours), looking into dynamically accessing files (2 hours), writing and testing demo program (8 hours), writing readme on Github (0.5 hours)
Tim Driessen	1006903	7.5 hours	Group discussion (3 hours), planning/gantt chart (1 hours), looking into raspberry pi (1 hours), meeting Hable (1 hours), meeting approval hardware purchase (0.5 hours), looking into code (1 hours)
Tom Janssen	1233021	8 hours	Group discussion (3 hours), meeting Hable (1 hours), meeting approval hardware purchase (0.5 hours), watching youtube tutorials on coding in python and getting familiar with the language (3,5 hours)
Sander van Bommel	1017917	11.5 hours	Group discussion (3 hours), meeting Hable (1 hours), meeting approval hardware purchase (0.5 hours), writing manual device (2 hours), transcribing and writing Q&A Hable conversation (5 hours)