PRE2018 3 Group13

Abstract

A robot which is able to read a book/form of text and is able to translate it into braille so that deaf-blind people can read it.

Problem statement

In this world, there is an estimate of around 356.000 deafblind people internationally. Only around 1% of all books are translated in braille which means that people who are deaf-blind have small amount of available reading material. Even the blind people will most often need to resort to audio books if they want to read a certain novel.

Objective

Make it possible for people whose only choice to read books is using braille to read books which are not translated in braille and give a choice to blind people who prefer to read in silence to do so.

Target group

The users of our device will be people with visual and/or hearing impairments who know how to read braille. For the people with both visual and hearing impairments, they will use this product in order to read novels. For the people with just visual impairments, they will use the product incase they prefer to read in silence. Most visual impaired people have to resort to use audio books because of limited availability braille books.

Approach

If we build a prototype of the mechanical part: Firstly, we will research braille on which we can base our user requirements on. We then look for existing patents and their mechanics on a Braille output system and look for possible trade-offs. Once we have detailed insight on the possible solutions, we devise our own solution based on our design decisions that correspond to the user requirements. We make a concept to visualize our idea and eventually we will make a (physical) prototype that can be tested with our target users.

If we only focus on user-friendliness
Firstly, we will research braille on which we can base our user requirements on. We then look for the most user-friendly materials and user-interface so we can improve the user-friendliness of the machine.

Milestones

Have digital text be the input of a reusable interface where the text is converted to braille. The braille can be projected per word or group of words, depending on their length. Choose the research topic Summarize a State-of-the-Art by performing a literature study Find a contact person that fits our target user group to gather additional information regarding our case Create user requirements for the robot Validate the requirements with the contact person Design a concept build corresponding to the requirements Validate the concept with the contact person Build a prototype of our concept and test it with the contact person Produce a final presentation in which we discuss the process, design and prototype

Deliverables

Compact, user-friendly and cost-efficient physical braille interface. The Braille interface should accept digital text and convert it to several Braille characters at a time. User-friendliness consists of: Type of material to interact with Speed of the machine Possible physical interaction with buttons/triggers (functionality)

Possibly a working prototype of one or several letters, that can be reset and configures itself to a given input.

Role division

• Material choice
  • Dennis
• User interface (layout, buttons etc.)
  • Thomas
• Converting feedback of test person to useful information
  • Luc
• Hardware internals stuff
  • Luc
• Builder
  • Simon
• Programmer if we build a 6-segment letter prototype
  • Dirk

Planning

The planning of the project can be found here.

User requirements

We will need a way to convert (digital) text to braille and we can do it as follows:

Create a robot which is able to translate e-books into braille so that the users can read it (with translate we do not mean to write a new book with the translation but instead some other form of reusable interface).

In order to determine fitting user requirements, we have broken down our research into different sections in which we look at general literature studies, user focused research and materials for the braille interaction. From these sections we draw our overall user requirements, which can be found at: https://drive.google.com/open?id=1D0T3C2wYtNyWK7ZE4wtTIpXq9gRA280T4eBAQZk00WI

State-of-the-Art literature study

We have a seperate page for the literature study which contains summaries of knowledge relevant to braille and its users. From systems, mechanisms to its perception and standards. The page can be found here.

Mechanisms

Here, existing solutions (mechanisms) can be described. Other subdivisions can be made separately in this chapter.

Material study

For materials we’ll first look into materials which can act as a cover over the pins of the robot. For this to be possible, the material would need to be sufficiently elastic. With this approach we aim to provide a good user experience which might be hindered by the spaces and inconsistencies that a non-elastic braille display would provide. This inconsistencies might be created because of faults in the connection between the braille display and the pins or unintended deformation of the braille display. Based on the feedback we’ll receive from our interviews, we might look into the non-elastic materials. Further in depth information can be found here.

Material sources

[1]https://www.the-warren.org/ALevelRevision/engineering/Materialclasses.html

[2]https://www.engineeringtoolbox.com/young-modulus-d_773.html

[3]http://burncentrecare.co.uk/about_burned_skin.html https://www.orfit.com/faq/what-are-low-temperature-thermoplastic-materials-lttps/

[4]https://agmetalminer.com/metal-prices/carbon-steel/

http://www.campaignforwool.org/about-wool/

http://bieap.gov.in/Pdf/CGTPaperII.pdf

https://global.kyocera.com/fcworld/charact/strong/rigidity.html

https://www.engineeringtoolbox.com/ceramics-properties-d_1227.html

http://www.dynacer.com/properties/

http://www.newworldencyclopedia.org/entry/Elastomer

https://www.britannica.com/science/elastomer

https://en.wikipedia.org/wiki/Thermosetting_polymer

https://en.wikipedia.org/wiki/Thermoplastic

https://www.onealsteel.com/carbon-and-alloy-steel.html

https://www.metalsupermarkets.com/what-is-chromoly/

https://www.economicshelp.org/blog/301/concepts/understanding-elasticity/

User research

Besides the literature study, we would like more direct involvement from our target group as well. To involve them in the design and development process and in order to do so, we have attempted to contact several institutes and communities for blind people both through phone calls and online posts. Unfortunately, the amount of responses we got is low and we only managed to get a few of our survey filled in. Therefore we have decided to look for existing surveys that relate to our users and questions. The results of all user surveys are on this page.

Final results

In this chapter, the final results will be described, as well as the process of getting to these results.

Sentence (dummy)

The wiki section on the dummy sentence can be found here.

Letter (solenoids)

Here, the prototype of the working letter is explained.

Casing

The second part of the dummy will be the casing. This casing refers to the outside of the dummy which is used to provide an estimate of the size of our robot and a base on which the other parts will be attached to. Since we focus more on the reading aspect, for the base a simple box would be sufficient. The dimensions of this box will most likely depend on the needed estimated space for the hardware and the size of the braille display. An important thing to take into consideration when creating the casing, will be to consider interaction between the braille display dummy and the casing itself. For example, in case you want to change the letters of braille on the display, the casing should allow for a display to be removed and reattached.

Software

Here, the software of the letter is elaborated on.

Braille to Arduino

We're using an Arduino Uno with 6 outputs which represents a braille character. For testing purposes, we'll be using LEDs for these 6 outputs. Serial Monitor is used for input. Enter the numbers of the LEDs you want to enable (eg. 1256). After you press “send” on the interface or “enter” on your keyboard, the lights corresponding to the number input will light up. We've also added a button and by pressing it, you can reset all lights (the Braille output).

Defining the output of Arduino to Braille

An electro magnet (suited for Arduino) can be turned on and off via a digital pin. The current through the Arduino itself is too weak for it to operate, so we need an external power supply (battery). Then, to regulate the voltage we’ll need a transistor.

Alternatively, we can also use relays. The Arduino would power the On/Off action of the relay and an external power supply can power the magnet.

We’ll need 6 electro magnets, 6 transistors (or relays if we go for that option), and one battery.

To reverse polarity of an electro magnet we can use the transistor, simply switch the output of the two transistor pins.

Serial communication between Windows and Arduino

We’ll be using Serial communication to connect our Windows software (PDF to binary output) to our Arduino software (binary output to solenoid). And in order to do that, we’ll be using the following library. https://sourceforge.net/projects/javaarduinolibrary/ As we do not have all the parts yet for the solenoid, we’ll be using LEDs for now. It’s the same principle.

We can for instance send an output of 111000, it will be communicated to the Arduino via the Serial port, and the Arduino will enable the first 3 of the 6 LEDs.

Solenoid & Serial communication testing

We have acquired a single solenoid for testing purposes. Our main goal was to find out if the solenoid we got was of good quality and if the strength of the solenoid was sufficient (can we push it down “accidentally” while reading?), before we invest in 6 of them. We did at first get a faulty solenoid, but the second one we got is perfect. After a lot of testing we could conclude that the solenoid is powerful enough and works reliably.

We have made a program to test both our solenoid and serial communication, while also gaining some more experience with GUIs (which will be used in our final product).

To connect to the Arduino from Windows, we have made a port selection GUI. Select the COM port, and click on connect.

We have added two simple functions: “Pin up” and “Pin down”. These buttons control the state of the solenoid.

The single solenoid testing setup.

We’ll be buying 5 more solenoids next. (for complete braille input we require 6 pins)

Finalizing the functional prototype (hardware)

We now have 6 solenoids in total, to display a single braille character. Every single solenoid is connected to a transistor to regulate the current and voltage, a diode to prevent current from flowing back, and a 220 Ohm resistor. We have a button to continue to the next character (from the loaded text file).

To power this all, we’re using a 9V battery connected to a 5V converter (because our solenoids can only handle 5V), connected directly to the solenoids. The reason we’re connecting the battery directly to the solenoids is because we ran into an issue earlier when connecting them to the 5V from the Arduino. The Arduino would crash, trying to supply too much current to power the 6 solenoids.

Functional prototype finished

We have now finished the functional prototype. We encountered another power issue, namely that the solenoids would be getting a bit inconsistent after the 9V battery has drained a little. They weren’t getting enough current. We solved this issue by replacing the 9V battery with a 6X AA battery pack. After the circuit was perfectly working, we finished the software for the PC. We extended the software such that you first have to select the COM port (to connect to the Arduino), then you select the PDF file that you would like to read, and finally a window will pop up informing you about the state of the Arduino and the letters (and Braille output) sent.

3D modelling the mechanism

For the 3D modelling of the mechanism, several iterations will be made of different ideas. The model will keep changing because used methods or components will change. First a single dot of a braille letter was modelled, using some assumed dimensions for the parts. The technical drawing of this model can be seen below as a section of the complete pin.

Then, six of these dots were used to model one complete braille letter, as is one of the goals for the end of the project. However, the model changed because we chose to use a different mechanism. The whole model was made from scratch, to include a solenoid mechanism with a cam and follower. This model can be seen below.

Also, a model was made of a pad of arbitrary length with several letters next to each other. This could be used for the dummy, but is also a help to visualize how much space we will have. The eventual length of the pad will be decided upon when the user-requirement list is set up.

Braille dot distance research

The braille standards are different all around the world. The exact reasoning for these standards are nowhere to be found, however there was an article about how much you can feel with your body, which found out that the human finger is able to detect objects of a size of 2.3mm. Most values of the braille standards had a distance of between dots for around this value. While I did not find any further research to support my claim, I believe that the dot distance is based on this found value. Because I can not find what the exact reasonings for the differences between the values are, I suggest that we go for the electronic braille standards since we our robot falls within this category.

https://www.pharmabraille.com/pharmaceutical-braille/marburg-medium-font-standard/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3928539/ https://cdn.discordapp.com/attachments/140133975606689793/557703767332618260/Meier_FootFeedback_iWOAR2015.pdf

Converting a PDF file to Braille code

Using http://braillebug.afb.org/braille_deciphering.asp for general braille rules.

We can describe a 6-pin braille symbol by the position of the ‘active’ pins. In order from top to bottom and then left to right. We can represent this as a boolean vector or equivalent binary number. The symbol ‘a’ for example becomes 100000.

Conversion rules:

• Each letter in the alphabet and some symbols are converted to the corresponding braille code. (using a table)
• A capital letter is made by adding 000001 in front of the normal code.
• A capitalised word is made by adding the 0000001 symbol twice before it. (what if the only non-capitalised symbols in a word are not letters?)
• A number is made by placing 001111 in front of the symbols for the letters that correspond to the digits. For example a=1,b=2 so 12 = 001111 100000 110000
• 000011 must be placed between a number and a letter if the letter directly follows the number
• Some groups of letters get contracted to a single symbol or multiple (but less) symbols. (using a table)

Currently we have written a program in java that accepts a pdf file and outputs a txt file with the text translated symbol by symbol to braille. The braille is encoded as described above.

TODO: Apply word context dependent rules including the number and letter prefixes. Add a way for the user to supply character translations so that things like language specific translation can be added when needed. Pass the translation to the arduino.

Converting a PDF file to Braille code V2

DONE: Segregation into words. Word specific number and letter symbols. Contractions for full words, prefixes and suffixes. A full list of these is not implemented and has to be provided, what kind of contractions people want to read also differs from person to person (grade 2 and grade 3 braille).

PROBLEMS:

• Complicated grammar rules like mentioned in https://en.wikipedia.org/wiki/English_Braille :

“When there are several ways to write a word, the shortest one is chosen, and when they are of equal length, the one without (two-cell) abbreviations is chosen. So, thence is written th-ence (3 cells) rather than the-n-c-e (4 cells). However, with the sequences -anced, -ancer, -enced, -encer, the form with -ance/-ence is used even if not shorter.” “In general, these are acceptable as parts of longer words as long as they retain their pronunciation”

Some of these could be practically impossible to implement, for example when pronunciation is mentioned.

• Currently capitalized parts words are not contracted unless this part is also capitalised in the list of contractions used by the program. (this could actually be the desired behaviour)

• Contractions containing punctuations symbols cannot be recognized right now because they are not considered part of the word. (luckily I don't think these exist)

We are choosing to ignore the more complicated situations since we do not even know if most people can read these kind of contractions.

Description of the algorithm

• The algorithm looks for the next word in the text. A word is a sequence of symbols which is between 2 symbols that are not alphanumeric (e.g. punctuation and whitespace)
• All symbols that are between this word and the previous word are translated to the corresponding braille symbol (punctuation and whitespace)
• A table is used to look for possible contractions for the whole word, a prefix or a suffix. The contracted part of the word is then removed an*d the contraction is saved.
• The rest of the word is translated symbol for symbol while also adding the symbols indicating capitalization and numbers.

Final results

In the next section, we will split the final results were split into two products: a dummy sentence and a working refreshable letter. The main reason is to reduce costs, since a fully working sentence would be very costly and not on a true scale.

Reflection (OPTIONAL)

If we have time left, we can add a Reflection chapter to evaluate the project.