PRE2022 3 Group8
Dear team,
Group 8:
Fee Noordermeer (1587404) - [[1]]
Matthieu den Otter (0333626) – [[2]]
Merel Nienhuis (1721763) - [[3]]
Wilco van der Veen (1021299) - [[5]] Automotive Technology
Dave Hannet (1715666)
Subject: A wearable device for people with problems remembering names (mild dementia, temporal or permanent memory loss or just bad memory) that recognizes a face and gives the user a corresponding name via a Bluetooth connect earplug (possibly hearing device).
Special request: one of our team members has classes during hr 1-2 on monday. Is it possible to arrange a meeting during hr 3 or 4?
Kind regards, group 8
Fee Noordermeer, Matthieu den Otter, Merel Nienhuis, Sam Lancee and Wilco van der Veen
Problem statement
Objective
Who are the user?
The user consists of people that have problems with memory loss. This is typically a group of people of older age. The group consists of for example people with dementia.
What do they require?
Approach
Milestones
Deliverables
Who’s doing what?
- hardware (Wilco)
- software (Mathieu, Merel, Sam)
- facial recognition
- Audio
- Data base
- User writing part (Merel, Sam)
-
State of the Art (25 articles)
Sam
1 https://dl.acm.org/doi/abs/10.1145/1090785.1090806?casa_token=M2Z7T1CRUDAAAAAA:OTdONvxGmFEzED-EVnPLnwj3W6lwWpdxyHNIHEX9YsEzU3mSZbWlq99lttpeHd4piY9kanAwNNJ4 --> pretty much what we plan on doing, but for people with visual impairments.
2 https://dl.acm.org/doi/abs/10.1145/2459236.2459262?casa_token=ujZ8wpZTorkAAAAA:WDsDXJZZCHIuySAv53DFSyvW5xGCtSdaq8riNw07sxXSfXgo8CjtEjNuURSYpaDu6vGnEVOlw4I8 --> what we are doing but for a different user group
3 https://link.springer.com/chapter/10.1007/978-3-319-16634-6_31 --> facial recognition in your glasses, with user interface desplay
4 https://link.springer.com/chapter/10.1007/978-3-319-92384-0_11 --> wearables and security/privacy problems
5 https://journals.sagepub.com/doi/pdf/10.1177/1533317519883493 --> looking at the acceptence of using wearables for person with dementia, from caregiver view
Wilco
1 https://ojs.wiserpub.com/index.php/AIE/article/view/62/328 --> An Innovative Approach for Face Recognition Using Raspberry Pi
2 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3513528 --> Data Protection Challenges in the Internet of Things Era: An Assessment of Protection Offered by PDPA 2010
3 https://www.sciencedirect.com/science/article/pii/S0747563217304284 --> Consumer adaptation and infusion of wearable devices for healthcare
4 https://academic.oup.com/ageing/article/30/6/455/39799?login=true --> Assistive technology in elderly care
5 https://www.mdpi.com/1648-9144/56/6/257 --> Stress in the Volunteer Caregiver: Human-Centric Technology Can Support Both Caregivers and People with Dementia
Merel
1 https://www.faceblind.org/research/ --> what is prosopagnosia (official name for having problems with recognizing faces) + interesting sentence for our project: " It is important to note that prosopagnosia is defined by problems recognizing faces, not recalling names. "
2 https://www.cbc.ca/documentaries/the-nature-of-things/this-woman-can-t-recognize-her-own-daughter-s-face-but-technology-is-helping-her-1.6312174 --> zelfde maar dan met app op telefoon
3 https://www.researchgate.net/publication/263091789_A_room_full_of_strangers_every_day_The_psychosocial_impact_of_developmental_prosopagnosia_on_children_and_their_families --> impact on families, important to manage to help them with face recognition difficulties
4 https://courses.engr.illinois.edu/ece445/getfile.asp?id=12216 --> requirements for device that is similar to our idea
5 http://wearcam.org/smc2020/IEEE_SMC2020_0700_prosopagnosia.pdf --> glasses, but also proof that participants have shown improvements after using glasses
Matthieu
1 https://www.orcam.com/en/myeye2/#:~:text=OrCam%20MyEye%20is%20a%20revolutionary,live%20a%20more%20independent%20life! Currently existing device that does all and more, but high costs
2 https://nitratine.net/blog/post/python-face-recognition-tutorial/ Face recognition with python
3 https://www.sciencedirect.com/science/article/pii/S0197457214001360?via%3Dihub Maybe useful to set gradations in dementia
4 https://dl.acm.org/doi/pdf/10.1145/3351248 Face Recognition Assistant for People with Visual Impairments
5 https://aging.jmir.org/2019/1/e13378/PDFA Web-Based Mobile App With a Smartwatch to Support Social Engagement in Persons With Memory Loss: Pilot Randomized Controlled Trial
Fee
1 Support vector machines for face recognition
2 Training Support Vector Machines: an Application to Face Detection
3 Robust face detection using local CNN and SVM based on kernel combination
4 Computationally Efficient Face Detection
5 Support vector machine based multi-view face detection and recognition
Summaries articles 0LAUK0 Merel
Article 1
https://www.faceblind.org/research/
Understanding Prosopagnosia
People who suffer from prosopagnosia have difficulties with the recognition of facial identity. This can create social problems. Prosopagnosics do not have problems with recalling names.
It is possible that prosopagnosia occurs after brain damage due to trauma, stroke or a disease. Sometimes, it is the case that this impairment has always been there for someone.
Article 2
https://www.cbc.ca/documentaries/the-nature-of-things/this-woman-can-t-recognize-her-own-daughter-s-face-but-technology-is-helping-her-1.6312174
This woman can't recognize her own daughter's face, but technology is helping her
This article is about a woman that is not able to recognize faces, but technology does help her.
The woman tells that her inability to recognize faces can cause issues. She feels insecure and people are hurt that she does not recognize them. She does recognize voices and other things like birthmarks. Sometimes she does not even recognize herself.
A personalize app was developed for her to help her recognize people. People stand on the opposite of the phone, the camera scans the face and gives the name which can be seen by the woman.
The reactions are positive, for the woman with prosopagnosia but also for her close ones. They are delighted that the woman did recognize her with the help of the phone. The woman felt sure which felt good for her, it was not just a guess. However, the woman was not pleased about the fact that she had to look at the phone the whole time to be able to recognize someone.
Article 3
https://www.researchgate.net/publication/263091789_A_room_full_of_strangers_every_day_The_psychosocial_impact_of_developmental_prosopagnosia_on_children_and_their_families
“A room full of strangers every day”: The psychosocial impact of developmental prosopagnosia on children and their families
Individuals with developmental prosopagnosia have severe face recognition difficulties due to a failure to develop the necessary visual mechanisms for recognizing faces. These difficulties occur in the absence of brain damage and despite normal low-level vision and intellect. Adults with developmental prosopagnosia report serious personal and emotional consequences from their inability to recognize faces, but little is known about the psychosocial consequences in childhood. Given the importance of face recognition in daily life, and the potential for unique social consequences of impaired face recognition in childhood, we sought to evaluate the impact of developmental prosopagnosia on children and their families. We conducted interviews with 8 children with developmental prosopagnosia and their parents. A battery of face recognition tests was used to confirm the face recognition impairment reported by the parents of each child. We used thematic analysis to develop common themes among the psychosocial experiences of children with developmental prosopagnosia and their parents. Our findings indicate a need for increased awareness and treatment of developmental prosopagnosia to help these children manage their face recognition difficulties and to promote their social and emotional wellbeing.
Article 4
https://courses.engr.illinois.edu/ece445/getfile.asp?id=12216
Assistive Technology for Patients with Medical Face Blindness
This article gives an overview of requirements and verification for a device that assists social interaction for individuals with prosopagnosia. It also states that it is important to follow the IEEE Code of ethics and to develop something that is aware of potential safety-related issues.
Article 5
http://wearcam.org/smc2020/IEEE_SMC2020_0700_prosopagnosia.pdf
Face Recognition and Rehabilitation: A Wearable Assistive and Training System for Prosopagnosia
The design and implementation of an integrated wearable face recognition and training system for prosopagnosia patients are presented. The purpose of this assistive technology is to provide real-time memory assistance and long-term rehabilitation. The real-time face recognition mode provides audio and visual notification of people who interact with the subject, while the at-home training mode combines features of mnemonic and perceptual training to help with prosopagnosia rehabilitation. In addition, a custom eye tracker is developed to determine the person whom the subject is making eye contact with within a crowd. Using the inverted face effect to mimic the difficulties of prosopagnosia patients, clinically healthy participants have shown improvements in their face-naming abilities. Early results indicate the system’s potential to enrich the well-being of prosopagnosia patients.
Summaries articles 0LAUK0 Matthieu
OrCam MyEye is a voice activated device that attaches to virtually any glasses. It can instantly read to you text from a book, smartphone screen or any other surface, recognize faces, help you shop on your own, work more efficiently, and live a more independent life! OrCam MyEye conveys visual information audibly, in real-time and offline. COST: >3000 euros at lowvisionshop.nl
Article 2 https://nitratine.net/blog/post/python-face-recognition-tutorial/
Using the face_recognition for detecting and matching a face
Article 3 https://www.sciencedirect.com/science/article/pii/S0197457214001360?via%3Dihub
Dementia can be divided in stages. These can be useful to specify target group for device
Article 4 https://dl.acm.org/doi/pdf/10.1145/3351248 Face Recognition Assistant for People with Visual Impairments
Gives interesting background for investigating the requirements and challenges that must be addressed in the design of a face recognition system for helping VIPs recognize people with whom they have weak-ties
Article 5 https://aging.jmir.org/2019/1/e13378/PDFA Web-Based Mobile App With a Smartwatch to Support Social Engagement in Persons With Memory Loss: Pilot Randomized Controlled Trial
The Social Support Aid (SSA) App on mobile phone: Reasons why presented technology was not usefull: (1) complexity of the SSA, (2) enrollment process, (3) impracticality, (4) stigma, and (5) functionality of the SSA.
Summary articles Sam
Article 1
A wearable face recognition system for individuals with visual impairments
Icare interaction assistant which is there for helping with social interactions for people who are visually impaired. Talks about the problems with using real-time facial recognition for a wearable device. This device is their to see what the other person for facial expressions has.
Article 2
Who are you?: A wearable face recognition system to support human memory
In this paper they talk about the use of a wearable that is there to aid human memory. This can be for remembering names of other people. They first deal with different design ideas and after that a face recognition approach.
Article 3
A Wearable Face Recognition System on Google Glass for Assisting Social Interactions
They present a face recognition on Google Glass that is there to help with face detection, eye localization, face recognition and comes with a user interface for information display. Multiple methods are used to see which works better for localizing the eyes and different lighting conditions.
Article 4
Wearables Security and Privacy
Wearable devices are currently in many different applications in health care. Since these applications are relatively new there is little experience with security and privacy. It ends with the conclusion that before these wearables are for open market the security and privacy issues should be worked out.
Article 5
Facotrs Affecting Caregivers' Acceptance of the use of wearable devices by patiens with dementia: an extension of the unified theory of acceptance and use of technology model
Using a structered online questionnaire it was researched if caregivers will expect the use of wearables for patients with dementia. They constructed a model to see which factors had an influence. The conclusion is that social influence, effeort expectancy, facilitating conditions and behavioral intention all had a great impact on this model. But it was also seen that the technology anxiety and that people tend to have a resistance to chance were both not a predictor for the model.
MoSCoW Analysis
MUST
- have a camera
- be able to be connected to an audio device (ear, headphones, connected by bluetooth, hearingaid)
- maintain a database with Images ('faces') and Audio fragments ('names') [maximum number of entries?]
- operations on database: add / delete (not edit)
- have a user button1 to make an image of the person to be recognized, which triggers the recognition and plays the corresponding audio fragment on the ear
- have a button2 to switch on/off
- have a button3 to add an entry to the database (photo + audio)
- be very, very easy to use
SHOULD
- be closed system to protect data
- have a rechargeable battery and a device to charge
- have an indicator for low battery
- a housing to contain the hardware (easy to wear for instance around the neck)
COULD
- have a possibility of changing the audio volume (button4)
- send audio output to hearing device
WONT
- have a display
WEEKLY REPORTS?
| Matthieu | Group discussion (2), Study papers (3) | 5 |
|---|---|---|
| Merel | ||
| Sam | ||
| Fee | ||
| Wilco | ||
| Dave |
Hardware and CAD
- Raspberry pi 3B+
- Picam V1.3 (5 MP It is to be determined still if the resolution is sufficient)
- Charge controller with battery board is ordered will arrive some time next week
Pictures CAD
Pictures Test print
