PRE2020 3 Group6

Sign to text software

Group Members

Name	Student ID	Department	Email address
Ruben Wolters	1342355	Computer Science	r.wolters@student.tue.nl
Pim Rietjens	1321617	Computer Science	p.g.e.rietjens@student.tue.nl
Pieter Michels	1307789	Computer Science	p.michels@student.tue.nl
Sterre van der Horst	1227255	Psychology and Technology	s.a.m.v.d.horst1@student.tue.nl
Sven Bierenbroodspot	1334859	Automotive Technology	s.a.k.bierenbroodspot@student.tue.nl

Problem Statement and Objective

At the moment 466 million people suffer from hearing loss, it has been predicted that this number will increase to 900 million by 2050. Hearing loss has, among other things, a social and emotional impact on ones life. The inability to communicate easily with others can cause an array of negative emotions such as loneliness, feeling of isolation and sometimes also frustration !!!!!!!!!!!("Deafness and hearing loss", 2021)!!!!!!!! . Although there are many different types of speech recognition technologies for live subtitling that can help people that are deaf or hard of hearing (DHH), these feelings can still be exacerbated during online meetings. DHH individuals must concentrate on the person talking, the interpretation, and of any potential interruptions that can occur !!!!!!!!(Peruma & El-Glaly, 2017)!!!!!!!!!!. Furthermore, to be able to take part in the discussion, they must be able to spontaneously react in conversation. However, not everyone understands sign language which makes communicating even more difficult. Nowadays, especially due to the COVID-19 pandemic, it is becoming more normal to work from home and therefore the number of online meetings is increasing quickly !!!!!!!("Microsoft Teams reaches 115 million DAU—plus, a new daily collaboration minutes metric for Microsoft 365 - Microsoft 365 Blog", 2021)!!!!!!!!. This leads us to our objective: to deveop software that translates Sign Language to text to help DHH individuals communicate in an online environment.

The project

This project is part of the course 0LAUK0 Robots Everywhere. As mentioned before, we have decided to focus on creating a system that DHH individuals can use to communicate during online meetings.

Deliverables

Week 1

Week 1 mostly consisted of putting together a group and decide upon a topic. We settled on the topic of emotion recognition on children with ASD. Research has been done on this topic and references to similar projects have been gathered. The focus for next week is to explore what is possible to achieve within this topic.

Week 2

In week 2 we decided after discussing the possible deliverables and came to the conclusion that it is difficult to find a dataset which we could use. The creation of a dataset is nearly impossible due to the slim target group and the current corona measures. for these reasons we abandoned the subject and discussed a new topic. The selected topic is to develop software which can convert sign language into text using video as an input.

Week 3

Research into the functioning of people suffering for hearing loss: http://www.werkenmeteenbeperking.nl/downloads/werkgeversboekje-dove-werknemer-pdf.pdf https://www.tolkcontact.nl/tolkcontact-app/wat-kun-je-met-de-app/ https://dl.acm.org/doi/abs/10.1145/3373625.3418032?casa_token=qBFFHzT_TIYAAAAA:5J8VJu4e5kn85EA339sWPTJ5NPrzG3dfX6VY5-DBertEQeKUk9xDEnSorUJ8S6svFsBRnxgdOI23M9U