PRE2019 3 Group4

Group 4

Group member	Student number	E-mail	Study
Tom Janssen	1233021	t.j.a.janssen@student.tue.nl	Chemical Engineering and Chemistry
Ivo Kersten	1233717	i.p.c.kersten@student.tue.nl	Electrical Engineering
Sander van Bommel	1017917	s.p.h.a.v.bommel@student.tue.nl	Psychology & Technology
Tim Driessen	1006903	t.driessen@student.tue.nl	Software Science
Rob Vissers	1244863	r.t.w.a.vissers@student.tue.nl	Electrical Engineering

Introduction

In Europe only, there are already an estimated 30 million people that are either blind or visually impaired. Furthermore it is found that on average 1 in 30 Europeans experience sight loss. This imposes a huge challenge on the society in general, since these people cannot function as properly as intended in the complex society of today. Of all the blind or visually impaired people, 75% is rendered unemployed, while these people could possibly participate in certain jobs if they would receive the necessary education. Now a huge issue arises, since there is a lack of proper Braille reading and study material available, which is holding the blind and visually impaired people back.

Also loss of sight can be linked to people getting older, whereas the retinitis pigmentosa deteriorates with increasing age. By looking at the European statistics, one in three seniors with an age over 65 years old struggles with visual impairment or even blindness. Due to these large numbers, it can be seen that the problem of visual impairment and blindness has to be tackled, such that these people can stay active in the society.

Problem statement

According to the World Health organization , the estimated number of people that are suffering from visual impairment in the world is 285 million. These people experience difficulties with daily activities that require vision. Leisure is considered as one of the main problems that visually impaired people experience, in particular reading . With regard to reading, these people experience a lack of accessibility. Namely, there is only a limited number of inclusive or accessible books available for the visually impaired. More than 90% of all published material is not accessible to the blind or partially sighted . The internet in particular provides multiple information sources, such as e-books or websites, that are either amusing or informative. However these sources are still not accessible for people that are suffering from visual impairment, since numerous websites disregard their visually impaired visitors and do not provide curate content which is actually accessible for low visual or blind vision individuals. Due to this lack of accessibility, these people have a greater probability of experiencing social exclusion, depression and loneliness . Therefore, they are unable to live a pleasant life like any other individual. Thus the limited leisure options are considered as a main problem for visually impaired people.

Currently, there are multiple methods provided to people with visual impairments in order to support them in performing certain daily activities. With regard to reading, braille learning is a well-known method that is used to support the visually impaired with reading. In this method, visual impaired people are basically reading text with their fingers by identifying several patterns of raised bumps or dots . Even though it offers these people the opportunity to actually read a book, the main disadvantage of this method is that is not applicable for any situation. Digital information devices, such as tablets or computer, are currently inaccessible for visually impairment people, because patterns of dots cannot be implemented on these types of devices. Furthermore there is only a small percentage of books that is translated into braille and is accessible for the visually impairment. Therefore there is still a great lack of accessibility in terms of reading.

Objectives

The main objective of this project is to:

• Realize a device that helps an inexperienced person to learn the basics of reading braille.

From this rather abstract main objective, a number of sub-objectives can be deduced that need to be accomplished, namely:

• Find out more details about the problems visually impaired people face in daily life;
• Learn more about the nowadays used methods for learning Braille;
• Built braille pins that can be pushed in by the user (input) and lower and raise themselves on command of the processor (output);
• Be able to recreate the braille pin configuration of the complete braille alphabet (including the numbers 0 until 9 and the dot and comma);
• Incorporate a speaker that is able to read out all letters of braille alphabet (including the numbers 0 until 9 and the dot and comma);

Users

When designing a product, it is important to keep the users (actively or passively) involved in the design process as soon as possible. Therefore it is important to take into account the values and needs of all involved users. With proper participation and empirical research, the design process can be centered around the user for the best final result. The users involved within the subject of text-to-braille conversion can be categorized in primary and secondary users as described below.

Primary Users

The primary users are the people that will actively use the product, namely being the visually impaired and blind people that have a major disadvantage in learning study material, reading books, or using smartphones. This is caused due to the lack of braille material that is generally available for consumers and organizations. Since these visually impaired and blind people cannot get the necessary education they need from reading literature or browsing the internet, they will get a learning disadvantage and cannot function to their full extent. This is holding these people back in striving for their life goals and they will not be able to develop their cognitive skills as intended. Next to that, blind and visually impaired people strive for a certain amount of independence, which is lost due to their disability. By introducing the text-to-braille converter, they can actively read lots of material without constantly being dependent on other people.

Secondary Users

Regarding the secondary users, there are multiple organizations that would want to use the text-to-braille converter.

• At first, educational institutes, such as kindergartens, schools, and universities, would want to use the text-to-braille converter to make learning for blind and visually impaired people more attractive at their institution. By showing that their needs are taken into account, the blind or visually impaired people will be triggered more to go to a certain institution that respects their disabilities. This will generally lead to an increased number of students at certain institutions.

• Secondly, libraries would want to adapt to the needs of blind or visually impaired people. In general, libraries strive for proper literary development for all people. By investing in a text-to-braille converter, it is possible to make written literature available to a much broader public, increasing overall education and happiness.

• Finally, book stores could implement a text-to-braille converter to increase overall sales numbers under the blind and visually impaired. With a text-to-braille converter on-site, blind and visually impaired people are able to read for example the sides, cover, back cover, or summary of a book, due to which they are able to choose between a huge variety of available books.

Requirements

Now that the users and their needs for the text-to-braille system are described, we can move on to describing the impact these needs have on the system and what the system must be capable of to meet these needs.

Basic needs

The described user needs are of course under the assumption that a working text-to-braille system is present. In order to achieve this working system certain basic requirements must be met, assuring that the system is actually able to help the intended group of primary users. The basic requirements are that the text-to-braille system must:

• Recognize written text by means of computer vision, with a probability of error that is less than one percent.

• Have a physical component capable of displaying at least 20 braille symbols.

• Let the physical braille display be reset and rewritten in less than one second.

• Be affordable to as many people as possible.

Additional needs

Furthermore, the users described above have additional needs and requirements for the system. First of all, the visually impaired people who will use the system to be able to read a printed document will want to be able to control the speed the processed text is displayed on the braille display. Furthermore it should not be too much work to use the system for a visually impaired person. So additional requirements are that:

• There is a way to indicate to the system that the next set of braille symbols can be displayed.
• The system is easy to set up and use by a visually impaired person.

Approach, Milestones & Deliverables

Approach

The different aspects of the approach can be subdivided into milestones. Consequently, these can be distributed over a planning that fits the time span of this project.

In order to obtain the optimal and most adequate results on the topic, several different methods can be used to obtain information:

• Literature research into the different aspects related to this topic:

  - Current state-of-the-art devices for this purpose
  - Scanning technology for written/typed text
  - Conversion technology of scanned text to braille
  - Dynamic braille surfaces
  - Design considerations for optimal user experience and versatility for different formats of clustered text

• Collaboration with visually impaired people to incorporate the advice of the primary users.
• Development of a small prototype and applying a series of tests to check whether the device meets the previously set requirements.
• Comparative tests with current state-of-the-art devices to discover points of improvement, as well as points of superiority of our product.

Planning & Milestones

Below the global planning of this project is displayed.

Week	Tasks & milestones for the report	Tasks & milestones for the prototype	Other tasks & milestones
1	Choose the subject Write the start of the sections for the problem statement, objectives, users, requirements, approach, planning & milestones, deliverables, logbook Investigate and summarize current state-of-the-art techniques and devices on this topic
2	Continue writing on the different sections of the report Write the start of the sections for current state-of-the-art and our own research	Investigate and start creating software and hardware for scanning written/typed text Investigate and start creating software for converting regular text into braille
3	Continue writing on the different sections of the report	Investigate and start creating dynamic 3D surface for braille tablets Investigate and start creating a design
4	Continue writing on the different sections of the report	Start to combine all the different elements into one working prototype
5	Continue writing on the different sections of the report Set up a list of questions and test for the collaboration with visually impaired people in week 6	Finish a preliminary working prototype and check whether it meets the requirements
6	Continue writing on the different sections of the report Write the start of the sections for the discussion and conclusion	Schedule tests & information sessions with visually impaired people
7	Continue writing on the different sections of the report Incorporate the results from the questionaires of week 6 into the report	Incorporate the results from the tests of week 6 into the prototype
8	Finish the report on wiki	Finalize the prototype	Make and practice the final presentation Peer review

Deliverables

• A small prototype of a device that scans and converts written/typed text into braille (if this is feasible).
• A report of all our literature research, analysis and results on this topic. This will be documented on this wiki page.
• A presentation at the end of this course to share our findings.

State of the art - abstracts (temporary title)

Summary: In this paper a low-cost, low-power portable system is described that serves as a braille reading and writing system. It can use a braille keyboard to display the written text in braille, such that visually impaired people can practice their braille reading and writing skill. It also has the capability of converting a text document to braille.

Summary: The outcomes of a national survey in Greece are presented in this article. In the survey the preferences and choices of students with vision impairments on literacy medium for studying are examined. The study shows that braille and large print are the preferred mediums for studying. However, the majority uses the medium of listening as the best performance medium for studying.

Summary: This paper describes a system that aims to help the visually impaired by recognizing written text around them and converting it to spoken text, which is played to the user. This system responds to all written text around the user and also notifies the user of the distance to the nearest object. It achieves an accuracy of 84%.

Summary: This paper describes a way to create braille dots that can be used on a braille display by using an electrothermal design where the dots can be displaced out of the plane by 250 microns and a temperature difference of 58°C with the environment can be achieved. This way, the dots can be small and cheap, requiring only an input voltage of 1.36 V.

Summary: Electronically refreshable braille displays have been around for some time, but they have been very expensive. This paper presents a low cost refreshable braille display that uses very little power. It also describes open source text to braille scanner using Google’s open source optical character recognition (OCR) engine.

Summary: In general with the production of text entries or smartphones little attention is paid to people with no or impaired vision. Finding the keys for voice control is also an issue for blind people. The newly introduced BrailleEnter will support non-visual interaction with a touchscreen device, since users can tap the screen to raise Braille dots based on Braille coding. When the screen is gently touched, the Braille dots will not rise.

Summary: From the visually impaired or blind people, being namely 161 million people, only 3% are able to read, write, or count. This is due to the fact that there is lack of Braille reading material in schools. A solution is proposed, where 3D printed visual representations of books are manufactured to improve the learning potential of the target users.

Summary: Visually impaired individuals are limited in terms of communication, interaction, and personal autonomy due to the lack of Braille literature linked to economic reasons. A portable device is introduced as a reading system for visually impaired individuals, which is based on segmentation, feature extraction, and machine learning for improved accuracy.

Summary: With the introduction of touchscreens, the accessibility for blind people decreased significantly. Due to the high demands for mobile phones, it is important to also take into account the accessibility for blind people. Research is done to obtain a new way of implementing Braille text in smartphones. From the data of 5 databases, the research is performed.

Summary: A novel approach to converting Chinese text to Chinese Braille is proposed. A Braille word segmentation model, based on statistical machine leaning, is trained on a Braille corpus, and also on Chinese word segmentation. This will avoid the establishment of syntactic and semantic information rules. Furthermore a statistical model will learn these rules automatically in the background.

Summary: Design, Prototype and implementation of a Sign Language (ASL) to Braille Converter as well as an English Language to Braille Converter. The article proposes a simple and affordable device which was experimentally verified to give accurate outputs.

Summary: A comparison between visually impaired (visually impaired print reader: PR, braille reader: BR) and normally sighted (normal vision: NV) school children was performed based on reading rate and comprehension. BR had the lowest reading rate compared to other groups. The findings suggest that visually impaired students required a longer time to read and understand a text.

Summary: Although several text to braille converters are available, the cost is a factor which prevents this technology to reach all people of society. Therefore a low cost gesture controlled text to braille converter was developed.

Summary: A system for converting written text into braille was developed. It uses optical character recognition to translate written text into digitized texts, which are then transferred electronically in a braille haptic device. The overall system reliability is 95.68% and the system can process 1 word in 2 seconds.

Summary: To aid the blind and visually impaired (BVI), a portable text reading system, called Finger-eye was developed. This system uses a small camera placed on the blind person’s finger to continuously process images using the optical character recognition (OCR) method, which are then translated into a refreshable mechanical braille display.

Summary: This model can translate different types of text files in English to braille. Both the input and output are in text format. It uses six point cells to display braille characters obtained from converting eglish text. It does this using Matlab.

Summary: This method describes the translation of many different languages into braille, and saving it on a computer. It uses a table-driven method for this, and it should be relatively easy to adjust the method for a related purpose.

Summary: This portable device includes a scanner of text, converting algorithm for text-to-braille, braille display and braille keyboard for annotation features.

Summary:A cursive handwriting recognition system using artificial neural networks is applied. The features of each written character in the input is extracted and passed on to the neural network. This network uses data sets of handwriting from many different people to convert the handwriting to text, making it very accurate.

Summary: An Optical Character Recognition (OCR) program is developed using the Hidden Markov Model. The input is an image of written english text that is converted to printed text, with said model.

Bill of Materials

Name	Student number	Time spent	Break-down

Logbook

Week 1

Name	Student number	Time spent	Break-down
Rob Vissers	1244863	10 hours	Group discussion (1.5 hours), finding proper state-of-the-art literature [13]-[17] (2 hours), reading and summarizing the state-of-the-art literature [13]-[17] (4 hours), writing the introduction and users section with relevant literature (2.5 hours).
Ivo Kersten	1233717	9.5 hours	Group discussion (1.5 hours), formatted the wiki page (1.5 hours), found papers [1], [7]-[12] (2 hours), read and summarized papers [1], [7]-[12] (3 hours), wrote requirements (1.5 hours)
Tim Driessen	1006903	5 hours	Group discussion (1.5 hours), found papers [18]-[22] (2 hours), summarized papers [18]-[22] (0.75 hours), finding objectives (0.75 hours)
Tom Janssen	1233021	10 hours	Opening lecture (2 hours), group discussion (1.5 hours), learning how to use wikitext (0.5 hours), writing the sections for approach, milestones and deliverables (3 hours), literature research and summarizing artiles on current state-of-the-art devices [23] - [27] (3 hours),
Sander van Bommel	1017917	7 hours	Group discussion (1.5 hours), found papers [2]-[6] (2 hours), summarized papers [2]-[6] (1 hours), writing problem-statement (2.5 hours)

Week 2

Name	Student number	Time spent	Break-down
Rob Vissers	1244863	3 hours	Group discussion (3 hours)
Ivo Kersten	1233717	3 hours	Group discussion (3 hours)
Tim Driessen	1006903	5.4 hours	Group discussion (3 hours), Contact halbe (0.4 hours), found papers/information (specifiy which) (1.5 hours), rewriting objectives (0.3 hours), technical possibilities (0.2 hours)
Tom Janssen	1233021	3 hours	Group discussion (3 hours)
Sander van Bommel	1017917	3 hours	Group discussion (3 hours)