PRE2018 4 Group8

Project Plan

Members

Name	Student ID	Email
Rik Hoekstra	1262076	r.hoekstra@student.tue.nl
Wietske Blijjenberg	1025111
Kilian Cozijnsen	1004704	k.d.t.cozijnsen@student.tue.nl
Arthur Nijdam	1000327	c.e.nijdam@student.tue.nl
Selina Janssen	1233328	s.a.j.janssen@student.tue.nl

Ideas

Surgery robots (Autonomous robots), Elderly care robots, New technology robot, Facial recognition (Just like Facebook) (happy/not happy)

Subject

Facial recognition (Just like Facebook) (happy/not happy)

The use of Convolutional Neural Networks (CNNs) for the purposes of emotion recognition.

Sub-subjects

What is a possible application of our CNN emotion recognition technology?

What is a suitable deep learning method to analyse dynamic facial expressions? (transfer learning)

What are the requirements for the dataset that will be used?

What are the best features to use for facial expression analysis?

Is the use of facial recognition in conflict with privacy?

What are the consequences of false-positives versus false-negatives?

Which users and enterprises would benefit from our software?

Are there legal issues conserning facial recogition?

Training set

- How large does it need to be
- where do we get it
- How can we garantee that the set is not biased

Test

- use a test set and a validation set (separate from the training set)
- real live 

Plan

contains a subject (Problem statement and objectives), What do they require?, objectives, users, state-of-the-art, approach, planning, milestones, deliverables, who will do what, SotA: literature study, at least 25 relevant scientific papers and/or patents studied, summary on the wiki!

Interesting persons

Emilia Barakova

weriak@iti.uio.no

Sources

1) https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8039024

2) https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=956083

3) https://link.springer.com/content/pdf/10.1007%2Fs00521-018-3358-8.pdf

4) https://link.springer.com/content/pdf/10.1007%2F978-94-007-3892-8.pdf

5)Extended deep neural network for facial emotion recognition, with extensive input data manipulation: https://reader.elsevier.com/reader/sd/pii/S016786551930008X?token=3E015F2B3E9E6290D0EA5A3C8CA42C6F7198698E6A17043ADA159C2A5106C4053CBDEE27E39196AE6C415A0DDAF711F4

6) https://ieeexplore.ieee.org/abstract/document/1556608

7) https://pdfs.semanticscholar.org/e97f/4151b67e0569df7e54063d7c198c911edbdc.pdf

8) Bayesian face recognition https://www.sciencedirect.com/science/article/pii/S003132039900179X

Kalman filters for emotion recognition:

9) https://link.springer.com/chapter/10.1007/978-3-642-24600-5_53: Kalman Filter-Based Facial Emotional Expression Recognition This article uses a 3D candide face model, that describes features of face movement, such as 'brow raiser' and they have selected the most important ones according to them. The joint probability describes the similarity between the image and the emotion described by the parameters of the Kalman filter of the emotional expression as described by the features, and it is maximised to find the emotion corresponding to the picture. The system is more effective than other Bayesian methods like Hidden Markov Models and Principle Component Analysis.

10) https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=4658455: Kalman Filter Tracking for Facial Expression Recognition using Noticeable Feature Selection. This paper used conventional CNNs to recognise the facial expression, but the tracking of the features was carried out with a Kalman Filter.

11) Facial recognition with the Google glass, for children with Autism Spectrum Disorder (ASD): https://humanfactors.jmir.org/2018/1/e1/ Second Version of Google Glass as a Wearable Socio-Affective Aid: Positive School Desirability, High Usability, and Theoretical Framework in a Sample of Children with Autism. See also https://www.youtube.com/watch?v=_kzfuXy1yMI for a demonstration of Stanford's autismglass.

12) But autistic children are not the only target group that has difficulty recognising emotions in others. This is also the case for elderly: https://www.tandfonline.com/doi/pdf/10.1080/00207450490270901. EMOTION RECOGNITION DEFICITS IN THE ELDERLY. This is problematic, as primary care facilities for the elderly try to care using their emotions, e.g. to cheer the elderly person up by smiling.

13) https://www.researchgate.net/profile/Antonio_Fernandez-Caballero/publication/278707087_Improvement_of_the_Elderly_Quality_of_Life_and_Care_through_Smart_Emotion_Regulation/links/562e0bc808ae518e34825f40/Improvement-of-the-Elderly-Quality-of-Life-and-Care-through-Smart-Emotion-Regulation.pdf. This paper proposes that the quality of life of the elderly improves if smart sensors that recognise their emotions are installed in their environment. DOI: 10.1007/978-3-319-13105-4_50

14) https://tue.on.worldcat.org/oclc/4798799506 "Face recognition technology: security versus privacy" good source for arguments about face recognition. Keep in mind that we plan on developing software to recognice emotions not to identify faces. Also gives current (2004) state of face recognition technlogy.

15) https://www.mdpi.com/2073-8994/10/9/387/htm "Smart Doll: Emotion Recognition Using Embedded Deep Learning" This article describes a doll which uses local emotion recognition software. Exactly the kind of software we want to develop. Cohn Kanade Extended is used as database for facial expressions.

16) Neural network for emotion recogition, used because it can adapt to the user and context of the situation. (User and context adaptive neural networks for emotion recognition) https://www.sciencedirect.com/science/article/pii/S092523120800218X

17) By using a technique called "transfer learning", neural networks can be trained on a certain set of images, unrelated to the goal of the neural network. After this, the network can be trained on a small data set, so it can implement the needed functionality. https://www.researchgate.net/profile/Vassilios_Vonikakis/publication/298281143_Deep_Learning_for_Emotion_Recognition_on_Small_Datasets_Using_Transfer_Learning/links/56e7b18408ae4c354b1bc8d8/Deep-Learning-for-Emotion-Recognition-on-Small-Datasets-Using-Transfer-Learning.pdf