0LAUK0 PRE2018 3 Group 13 SotA Literature Study

Introduction

This page contains the State-of-the-Art literature study performed by group 13 for the course Project Robots Everywhere (0LAUK0). Here we have listed relevant topics to our main research subject. Each topic has its own summary and list of references.

Summaries

Braille system

In 1821, Barbier de la Serre invented Braille. Which got a then twelve-year-old blind student Louis Braille very excited. The system wasn’t like the Braille we know today, it used 12 dots and and every character described sounds. Louis Braille worked on la Serre’s system for several years and improved it to how we know it today. He changed the characters to 6 dots and instead of sounds, it describes glyphs (eg. letters, numbers) [1].

The number of people that use Braille has been a long-term decline. In the UK, there are two million visually impaired people, of which the majority is over 65 years old [2]. Fewer than 1% of the two million visually impaired people are users of Braille (18-20,000). Braille users typically are those who have not been able to see from an early age. Only 2,000 people regularly order books from the Braille library, which suggests most people only use it for practical reasons rather than entertainment.

The braille system consists of evenly arranged dots in quadrangular letter spaces, called cells. A full cell is three dots high and two dots wide. Only 63 different characters can be formed. Braille uses a fixed-width font, which means that every character occupies the same amount of space regardless of the amount of dots in the cell. Although the standards for creating braille differ per country, the American National Standard: Accessible and Usable Buildings and Facilities: 2003: Standard and Commentary, Section 703.4 details a range of standards in an effort to improve readability [3].

Braille Signage Standards

Measurement Range	Minimum - Maximum in Millimeters
Dot Base Diameter	1.5 - 1.6
Distance between two dots in the same cell	2.3 - 2.5
Distance between corresponding dots in adjacent cells	6.1 - 7.6
Dot height	0.6 - 0.9
Distance between corresponding dots from one cell directly below	10.0 - 10.2

Braille users

People who are blind from birth or in their early childhood are more likely to be able to read Braille fluently unlike those who are taught Braille in a later stage in life. The methods of communication strongly vary between deaf-blind people, which has to do with the level of sensory impairments, extent, type, history, personal characteristics and their developed skills. Deaf-blind people have significantly higher rates of depression compared to those with no sensory impairments [1][2]. This is mainly due to communication difficulties and the lack of social support, which consequently lead to social isolation, loss of independence and the feeling of reduced self-confidence and security.

Blindness and deafness cause sensory deprivation which consequently create neuroplastic changes in the brain [3]. In case of blindness, people become more sensitive to touch, hearing and verbal memory. While deafness improves processing of tactile information. People who are deaf-blind from birth can only interact with those in close proximity with touch and require reliable routines to allow communication. By having predictability, blind-deaf people develop expectations which ease the anxiety caused by the lack of sensory information [4]. Blind people are not necessarily better than sighted people in terms of measuring tactile feedback. Once the sighted people have practiced, their performance does not differ significantly to those of blind people [5].

Blind braille readers who use three fingers often misperceive which of their fingers is being touched [6]. As each Braille reader has their own preferences of which hand they read with, it is also common to read using both hands [7][8] and this happens to be the most efficient as well in terms of speed. Children are found to be faster and more accurate when reading with their left hand fingers, while adults seem to have no difference in speed but fewer mistakes are made when reading with their left hand [9]. Restricting tactile cues during braille reading leads to poor performance in letter recognition tasks. Particularly, the lack of sliding contact between the fingertip and the braille surface results in more errors and that number increases as a function of the presentation speed. It would be better to have the user move his/her fingers over the braille than having moving braille under a static finger [10].

Braille mechanisms

For the braille mechanisms, we have a seperate page as it also consists of a lot of research along with our own comparisons and choices on the final selection. It can be found here.

Perception of reading braille

The human sensory neural system is spread throughout our bodies and its sensitivity to perception varies in each body part. The sensory neural pathways in the human lip, tongue and fingers are specialized for spatial information processing [1], which trivially explains why braille reading is best suited by using the fingers instead of for example using the arm to read as it has a larger surface.

First, we look at how the reading speed is affected by the hand usage such as using either the main hand or both hands to read. Overall there is no evident superiority when reading with only one hand, however in general the blind people read faster with two hands compared to one hand [2]. As the relative gain of two-handed reading negatively correlated with the absolute speed difference between the hands in one-handed reading, we can see that both hands are actively used in the collection of textual information.

This reading speed is not only affected by physical factors, but also psychological ones such as the interpretation of context. Trivially, depending on the degree of contextual constraint, readers are able to read faster. Just like with sighted reading, the process of reading is much smoother if one has a grasp of the full context. Here, reading speed can possibly be affected due to breaking up a sentence such that a contextual part starts on a new line. Similarly we can conclude the same effect in braille, where some words are separated on another line.

The right-to-left movement of the finger is known as “regression” or “reversal”, in which it allows one to re-read letters that were missed in the first pass and it can be performed as often as in every sentence at any location in a sentence. While no reading actually takes place during reversal, it is one of the main characteristics of Braille reading and is performed by even the most experienced Braille readers [3]. More often, reversals take place due to language-processing demands and not due to a missed physical perception of a braille letter. Therefore it is to be expected that reversal actions always have a possibility of taking place.

Conclusively, we see that reading braille with both hands would increase the reading speed, however it is up to the user whether he/she want to read like that. The reading speed strongly varies in each person and is mostly dependent on their preference and interpretation. Coding the shape of Braille words is unlikely to be a major factor in producing faster Braille reading [4]. Therefore we will use the standard braille coding.

Tactile Resolution

As mentioned previously, blind people have a heightened tactile spatial acuity due to the loss of their sensory vision. This has been proved by several studies that use magnetic source imaging and somatosensory evoked potentials to show that the cortical representation of the reading fingers of blind Braille readers is expanded compared to that of sighted readers. Furthermore, the visual cortex in blind people is activated during tactile tasks, but not in that of sighted readers. This reflects an enhanced fidelity in neural transmission of spatial details of a stimulus. A mean grating orientation discrimination method has been applied to confirm this, here it is determined which threshold performance is accounted for by the spatial resolution limits of the neural image evoked by a stimulus. The results have shown clear superiority of blind people (1.04 mm) compared to that of sighted people (1.46 mm) [1].

Another experiment that also investigated the spatial neural mechanisms underlying tactile sensation have shown similar conclusions, this example used a two-point test to see whether subjects could reliably discriminate two 0.5 mm points even when there was no seperation between those points. This was mostly possibly at a gap size of 0.87 mm [2]. However, this two-point experiment has been a controversial topic due to the extreme variability within and between subjects [3].

The use of textured braille dots greatly increases the contrast of the dot sensation and improves legibility, but at the cost of some realism [4]. This use is often not considered as it strays from the standard Braille sensations and at the scale of which the braille dots are made, it requires incredible accuracy and effort to be made. However, it is interesting to note and could be improved on in the future.

Finally, we see that a 1.5 mm of distance between dots forms a fitting lower bound as it is readable for the sighted and therefore it is trivial that the blind are more than capable of perceiving it as well.

Standards

Reading speed was tested for 3 different values for each of the 3 spacing variables[1]. The choices were based on the standards used for braille. Only with the cell spacing the fasted value was not the standard one. 0.09 inch dot spacing (0,002286 meters) 0.123 inch cell spacing (0.140 was the standard) (0,0031242 and 0,003556 meters) 0.220 inch line spacing (0,005588 meters) These results suggest that the used standards are more or less optimal. We do from other research also see that beginning braille readers might profit from extra space between the braille characters comprising words [2] so having the spacing be slightly larger will benifit beginning readers. We should also definitely let them use their fingertips to read as oppposed to alternative methods since the sensitivity of the fingertip helps a lot with reading[2].

Nearly all people won’t notice the subtle differences between the different normal braille standards so exactly what numbers we use isn’t that important[3]. Using the jumbo braille [3] (just increase sized by 25%) would make it more readable for people with reduced tactile sensitivity so this could be a consideration.

Patents

The Automated Braille Production from Word-Processed Documents paper [1] describes a novel method for automatically generating Braille documents from word-processed documents. It contains a documentation on how the system would work as well as how it can be developed.

Pantobraille [2] is a design for a bi-dimensional single cell braille display in which they combine a standard braille cell with a force feedback device. The reading performance and comfort with this design is inferior to standard braille displays.

Some patents [3][4] describe a device that allows blind people to read certain text through the use of an optical sensor array which scans the text. It will then use an electromagnetic unit to display the characters in braille. This device is attached to the end of the user’s index finger. Although the paper gives a detailed description of the invention, there is no testing reports with the user group. One can also argue that the usability is flawed, due to the fact that blind people cannot follow a line of text with their index finger.

UbiBraille [5] is a device that is attached to the user’s index, ring and middle finger on both hands. These six devices shall vibrate to represent a certain braille character. Based on their testing, deaf-blind users are able to read one character per second.

A prototype capable of deploying physical Braille characters from digital text that is send from a computer program [6][7].

Refreshable Braille displays have been worked out many times, one example is the transparent resinous ultraviolet-curing type (TRUCT) Braille signs [8]. A finger cover made of soft, thin polyester non-woven fabric to reduce friction during Braille reading has proved to result in significantly faster and more accurate reading of the TRUCT Braille, regardless of the base material and dot height. Additionally, the finger cover keeps the forefinger clean.

Most common methods of making refreshable Braille displays is through the use of piezo-electric benders that regulate each dot of a letter seperately [9]. Piezo-electric benders do not require much volume, which complies with the Braille standards.

There are existing design patents that identify emotion in the form of facial expressions and voice frequencies and then translate it into Braille [10]. This would be an alternative to translating text only.

Another example of a Braille mechanism is the slot machine [11]. Here braille pins move vertically independent of each other using electromagnetism according to a read only memory (ROM) that sends the information to the braille pin cluster.

A user interface system is specified for the use of Braille [12]. All letters of the six dot Braille alphabet, with exceptions of ‘X’ and ‘Y’, when seperate from other letters represent a whole word. The traditional universal dimensions of six dot braille is as follows: dot heights of approximately 0.5 mm, horizontal and vertical spacing between dotcenters is approximately 2.5 mm and the black space between dots and adjacent cells is 3.75 mm horizontally and 5.0 mm vertically.

Examples [13][14] of refreshable braille display unit, where each dot is powered by a DC motor.

Braille printing machine using six linear actuators, set-up box containing a printed circuit board (PCB), a channel made of two parallel iron rods, a wheel to turn pages forward and another two wheels for the consistent horizontal movement of the actuators along the channel. The linear actuators are made using small gear motors rated 9V, 200rpm base speed. The system uses Bluetooth communication to control the motion of the actuators. A sonar sensor is used to detect the distance of the actuator box while printing Braille alphabets. Production cost: $150 [15].

The idea is to create a portable, inexpensive device that could scan text and convert it to Braille in real time [16]. The latest prototype is about the size of a candy bar and can display six characters at a time. It has a built-in camera. Users can place it down on a line of text and with a push of a button, the device takes an image. Using Microsoft’s Computer Vision API, the characters are identified and with the team’s software translates each character into Braille by triggering the mechanical system in the box to raise and lower the pins. One of the many challenges in development is figuring out the best way to raise and lower the pins. The team hopes to use microfluidics (differences in liquid or air pressure) or electromagnetism. They are testing both to figure out which is the least expensive but most responsive and shrinkable for their final prototype.

The Braille interface by Yoseph Bar-Cohen [17] uses an EAP (electroactive polymers) actuator array made of a field-activated type material. Rows of electrodes on the top surface and columns on the bottom of the EAP film allow activating the individual elements in the array. Each of the elements is mounted with a Braille dot and is lowered by applying voltage across the thickness of the selected element.

A physical interface which is to be connected to a smartphone (or tablet) that allows you to type one braille character at a time and convert it to digital text [18].

Patent of a system that allows deaf-blind people to communicate through an instrumented glove [19], such that their hand movements get translated to sound, display or Braille depending on the individuals.

Patent of a portable device used for communication between blind-deafs and non-handicapped persons who know Braille [20]. It’s a very simple device where the transmitter person presses six key combinations that produce a single Braille cell for the receiving person to touch.

References

Braille system

1. History of Braille. Available at: https://typeculture.com/academic-resource/articles-essays/braille-tactile-writing-system/
2. Popularity of Braille in the U.K. Available at: https://www.bbc.com/news/magazine-16984742
3. National Library Service for the Blind and Physically Handicapped, Library of Congress. Specification 800: Braille Books and Pamphlets. Available at: http://www.loc.gov/nlsold/specs/800_march5_2008.pdf

Braille users

1. Marion Hersh; Deafblind People, Communication, Independence, and Isolation, The Journal of Deaf Studies and Deaf Education, Volume 18, Issue 4, 1 October 2013, Pages 446–463. Available at: https://academic.oup.com/jdsde/article/18/4/446/560048
2. Golob, Gorazd & Gregor-Svetec, Diana & Leskovšek, Ana & Marija Turnšek, Ana & Majnaric, Igor & Dudok, Taras & Mayik, Volodymyr & Urbas, Raša. (2014). BRAILLE TEXT AND RAISED IMAGES USED IN BOOKS FOR CHILDREN WHO ARE BLIND OR VISUALLY IMPAIRED. Available at: https://www.researchgate.net/publication/309292310_BRAILLE_TEXT_AND_RAISED_IMAGES_USED_IN_BOOKS_FOR_CHILDREN_WHO_ARE_BLIND_OR_VISUALLY_IMPAIRED
3. Merabet, Lotfi B., Pascual-Leone, Alvaro, Neural reorganization following sensory loss: the opportunity of change, Nature Reviews Neuroscience 11, pp 44-52 (2010). Available at: https://www.nature.com/articles/nrn2758
4. Miles B., Ed M. (2008), Overview on Deaf-Blindness, Available at: http://documents.nationaldb.org/products/Overview.pdf
5. Grant A.C., Thiagarajah M.C., Sathian K. (2000), Tactile perception in blind Braille readers: A psychophysical study of acuity and hyperacuity using gratings and dot patterns, Perception & Psychophysics 2000, 62 (2), pp 301-312. Available at: https://link.springer.com/content/pdf/10.3758%2FBF03205550.pdf
6. Sterr, Annette & M. Müller, Matthias & Elbert, Thomas & Rockstroh, Brigitte & Pantev, Christo & Taub, Edward. (1998). Changed perceptions in Braille readers. Nature. 391. 134-5. 10.1038/34322. Available at: https://www.researchgate.net/publication/13801187_Changed_perceptions_in_Braille_readers
7. Susanna Millar, The Perceptual "Window" in Two-Handed Braille: Do the Left and Right Hands Process Text Simultaneously?, Volume 23, Issue 1, 1987, Pages 111-122, ISSN 0010-9452. Available at: https://www.sciencedirect.com/science/article/pii/S0010945287800230?via%3Dihub
8. Fertsch, P. (1947). Hand Dominance in Reading Braille. The American Journal of Psychology, 60(3), 335-349. Available at: https://www.jstor.org/stable/1416915?origin=crossref&seq=1#page_scan_tab_contents
9. Hermelin B., N. O'Connor, Functional asymmetry in the reading of Braille, Volume 9, Issue 4, 1971, Pages 431-435, ISSN 0028-3932. Available at: https://www.sciencedirect.com/science/article/pii/0028393271900078?via%3Dihub
10. A. Russomanno, S. O’Modhrain, R. B. Gillespie and M. W. M. Rodger, "Refreshing Refreshable Braille Displays," in IEEE Transactions on Haptics, vol. 8, no. 3, pp. 287-297, 1 July-Sept. 2015. Available at: https://ieeexplore.ieee.org/document/7086320

Perception of reading braille

1. https://n.neurology.org/content/44/12/2361.short
2. https://journals.sagepub.com/doi/abs/10.1080/14640748508400931
3. Hughes B, McClelland A, Henare D. On the nonsmooth, nonconstant velocity of braille reading and reversals. Scientific Studies of Reading. 2014;18(2):94–113. 10.1080/10888438.2013.802203.
4. https://journals.sagepub.com/doi/abs/10.1068/p130567

Tactile Resolution

1. https://n.neurology.org/content/54/12/2230.short
2. https://doi.org/10.1152/jn.1981.46.6.1177
3. https://journals.sagepub.com/doi/10.1111/1467-8721.00054
4. Lévesque V, Pasquero J, Hayward V. Braille Display by Lateral Skin Deformation with the STReSS 2 Tactile Transducer. In: Proc. World Haptics 2007 (Second Joint Eurohaptics Conference And Symposium On Haptic Interfaces For Virtual Environment And Teleoperator Systems); 2007. p. 115–120. Available at: https://www.academia.edu/29862504/Braille_Display_by_Lateral_Skin_Deformation_with_the_STReSS2_Tactile_Transducer

Standards

1. https://psycnet.apa.org/record/1959-09173-001
2. https://journals.sagepub.com/doi/abs/10.2466/pms.1985.61.2.363?casa_token=kN1xpVIJY2gAAAAA:4tmmmahKcYNHNu-eYyz1hQX5rp_sYcngG4KzcA0HpzvndnzTo_Vi-Qh4Zq0EQbenz-VK0bqV8kP5
3. https://www.researchgate.net/profile/David_Fourney/publication/236660376_Research_based_tactile_and_haptic_interaction_guidelines/links/54613b740cf27487b452706f/Research-based-tactile-and-haptic-interaction-guidelines.pdf#page=27

Patents

1. Blenkhorn, P. and Evans, G. (2001). Automated Braille Production from Word-Processed Documents. Available at: http://www.duxburysystems.org/downloads/library/history/blenkhorn3.pdf
2. Ramstein, Christophe. (1996). Combining Haptic and Braille Technologies: Design Issues and Pilot Study.. Annual ACM Conference on Assistive Technologies, Proceedings. 37-44. 10.1145/228347.228355. Available at: https://dl.acm.org/citation.cfm?id=228355
3. Parienti R. (1999). U.S. Patent No. US6159013A. Available at: https://patents.google.com/patent/US6159013
4. Leonardis D., Loconsole C. (2019) Braille Cursor: An Innovative and Affordable Refreshable Braille Display Designed for Inclusion. In: Di Bucchianico G. (eds) Advances in Design for Inclusion. AHFE 2018. Advances in Intelligent Systems and Computing, vol 776. Springer, Cham. Available at: https://link.springer.com/chapter/10.1007%2F978-3-319-94622-1_29#citeas
5. Nicolau H., Guerreiro J., Guerreiro T., Carriço L. UbiBraille: Designing and Evaluating a Vibrotactile Braille-Reading Device. Available at: http://www.academia.edu/4301322/UbiBraille_Designing_and_Evaluating_a_Vibrotactile_Braille-_Reading_Device
6. Başçiftçi F., Eldem A., An interactive and multi-functional refreshable Braille device for the visually impaired, Volume 41, 2016, Pages 33-41, ISSN 0141-9382. Available at: http://daneshyari.com/article/preview/538370.pdf
7. C. Guerra, D. Novillo, D. Alulema, H. Ortiz, D. Morocho and A. Ibarra, "Electromechanical prototype used for physical deployment of Braille characters for digital documents," 2015 CHILEAN Conference on Electrical, Electronics Engineering, Information and Communication Technologies (CHILECON), Santiago, 2015, pp. 191-198. Available at: https://ieeexplore.ieee.org/document/7400374/citations#citations
8. Doi, K. & Fujimoto, H. Med Bio Eng Comput (2007) 45: 1153, Polyester non-woven fabric finger cover as a TRUCT Braille reading assistance tool for Braille learners, Volume 45, Issue 11, pp 1153-1159. Available at: https://link.springer.com/article/10.1007%2Fs11517-007-0250-6#citeas
9. Zagler W.L., Treml M., Busse D., Busboom M., Deák I. (2018) BrailleRing: The Shortest Long Braille-Display in the World – A Review of the State-of-the-Art and a New Approach. In: Miesenberger K., Kouroupetroglou G. (eds) Computers Helping People with Special Needs. ICCHP 2018. Lecture Notes in Computer Science, vol 10897. Springer, Cham. Available at: https://link.springer.com/chapter/10.1007%2F978-3-319-94274-2_43#citeas
10. Janakiraman J., Ullmannn C.N., (2003) Translating emotion to braille, emoticons and other special symbols, U.S. Patent No. US7607097B2. Available at: https://patents.google.com/patent/US7607097B2/en
11. Kaplan E.B. (1995), Braille slot machine, U.S. Patent No. US5429507A. Available at: https://patents.google.com/patent/US5429507A/en
12. Rea R.M., Morgan D. (2012), Electronic braille typing interface, U.S. Patent No. US20110020771A1. Available at: https://patents.google.com/patent/US20110020771A1/en
13. Souluer F. (2003), Refreshable Braille display unit, U.S. Patent No. US6827512B1. Available at: https://patents.google.com/patent/US6827512B1/en
14. Thompson J.M. (1993), Braille board with movable dot pins, U.S. Patent No. US5466154A. Available at: https://patents.google.com/patent/US5466154A/en
15. Chowdhury, Dhiman & Haider, Md & Sarkar, Mrinmoy & Refat, Mustakim & Datta, Kanak & Anowarul Fattah, Shaikh. (2018). An intuitive approach to innovate a low cost Braille embosser. International Journal of Instrumentation Technology. 2. 1. 10.1504/IJIT.2018.090858. Available at: https://www.researchgate.net/publication/324134963_An_intuitive_approach_to_innovate_a_low_cost_Braille_embosser
16. Text to Braille translator. Available at: https://www.smithsonianmag.com/innovation/device-translates-text-braille-real-time-180963171/
17. Reading Braille text using refreshable displays. Available at: https://spie.org/membership/spie-professional-magazine/archives/oct2009-spie-professional/dynamic-braille?SSO=1
18. Morgan D. (2012), Electronic braille typing interface, U.S. Patent No. US20130157230A1. Available at: https://patents.google.com/patent/US20130157230
19. Kramer J.P., Lindener P, George W.R. (1998), Communication system for deaf, deaf-blind and non-vocal individuals using instrumented glove, U.S. Patent No. US5047952A. Available at: https://patents.google.com/patent/US5047952A/en
20. Fewell W.B. (1997), Braille deaf-blind communicators, U.S. Patent No. US4215490A. Available at: https://patents.google.com/patent/US4215490A/en