PRE2024 3 Group3

From Control Systems Technology Group
Jump to navigation Jump to search

Members

Name Student number Study E-mail
Andreas Sinharoy 1804987 Computer Science and Engineering a.sinharoy@student.tue.nl
Luis Fernandez Gu
Alex Gavriliu 1785060 Computer Science and Engineering a.m.gavriliu@student.tue.nl
Theophile Guillet 1787039 Computer Science and Engineering t.p.p.m.guillet@student.tue.nl
Petar Rustić 1747924 Applied Physics p.rustic@student.tue.nl
Floris Bruin 1849662 Computer Science and Engineering f.bruin@student.tue.nl

Planning

Roadmap

Week 1: Problem ideation and specification

Week 2: Robot design and specifications

Week 3: Begin construction of prototype of the robot

Week 4: Conduct interviews with relevant user groups

Week 5: Finish prototype of robot

Week 6: Gather feedback for the prototype

Week 7: Finalize the prototype after having taken feedback into account

Milestones

  1. Selecting who and what problem we are going to address
  2. Selecting how our robot shall address our problem
  3. Conducting research and a literature review on our topic
  4. Creating a design for the robot
  5. Conducting interviews to gauge the receptiveness of the robot
  6. Speaking with our primary user group to obtain their feedback on our proposed solution
  7. Creating a prototype of the robot

Introduction

Problem Statement

Speech impairment diagnosis in children is a flawed area of current day healthcare systems. According to research done by (citation), there is an existence of a large body of misdiagnosed children with speech impediments. According to interviews with a proffessional speech therapist, one of the reasons for this issue lies within the testing procedure for these diagnoses. For example, the current tests for children involve up to three hours of rigorous questioning, exhausting and inadequate for not only the children involved but also for the therapists. The interviews revealed the existance of bias within the therapist - as their analysis may succumb to understandable human error after these hours of questioning. They also revealed the potential for an inadequate environment for the child when it comes to the test. The format in which the tests take place may not provide the effective medium for the child's responses to to be fully accurate and undeterred. For example, current tests are administered in a therapist's office where questioning may be harsh or uncomfortable for the child to effectively respond. It also places a lot of emphasis on how well the child interacts and views the therapist performing the tests. Furthermore, as with the therapist, after hours of questioning, the children's answers may start succcumbing to biases and lower quality responses.

Therefore, for our project, we wish to address these issue by creating a robot which do the following. Firstly, it questions these children consistently, not suffering from its questioning decreasing in quality as the tests progress. Secondly, it records the child's responses, and provides an adequate level of preprocessing to allow the results of the test to be easily and conveniently analyzed by the therapist, allowing the therapist to be able to listen to the recording multiple times, as well as when they have the energy and state of mind to conveniently do so. Finally, it partitions the exam into smaller, gamified chunks - while still ensuring the test provides the same effectiveness as a full three hour session - which offers the children breaks and a more adequate interface which can improve the quality of their responses. Also its friendly appearing, non-threatening interface may provide the child with a more comfortable adn engaging exam experience - although this will need to be tested empirically to analyze its effectiveness.

Other benefits such a robot provides include a separation between the therapist's office and the child's location. This allows for diagnoses to be taken in locations where access to therapists is limited or nonexistent as the responses from the robot can be sent digitally to therapists abroad. It also allows for a diagnosis from multiple therapists, as with consent from the child's guardians, multiple therapists can have access to the recordings and results from the tests allowing for a more robust and certified diagnosis from the therapists as well as reducing bias present in any one therapist.   

Objectives

  1. Create a robot which can ask questions, record answers, provide some basic preprocessing to answers, and possess suitable paths of executions for unexpected use cases - for example children crying, incoherent responses, etc.
  2. Perform multiple tests to understand the effectiveness of the robot. The first being a test analyzing how well a child interacts with the robot. The second test being a comparison between an analysis from a therapist of the results from the device and the standard results from a current test.
  3. Test how well the robot performs when encountered with unexpected use cases.

Hypothesis

If the robot

USE Analysis

Users

Personas

Speech-Language Therapist (Primary User)  

Clinicians that are dedicated professionals responsible for diagnosing and treating children with speech, language, and communication challenges. They are often under high workloads and constant pressure to deliver accurate assessments quickly.  

Needs:
  •  Efficient Diagnostic Processes: Therapists require a tool that transforms traditional 2–3 hour assessments into shorter, engaging sessions that maintain clinical rigor. By reducing patient fatigue and maintaining diagnostic quality, this approach aims to enable therapists to focus on other patients and improve overall efficiency [citation].  
  •   Comprehensive Data Capture: The therapists need the capability to capture high-quality audio and sensor data during assessments so that each session can then be reviewed later. This comprehensive data collection supports detailed analysis and would facilitate collaboration among specialists to enhance diagnostic precision [citation].
  •   Intuitive Digital Tools: The development of a secure, user-friendly dashboard is helpful, as it allows therapists to manage patient records, annotate sessions, and access data remotely and efficiently. Naturally, such tools should comply with healthcare standards such as GDPR, to ensure data integrity and patient confidentiality.
Child Patient (Secondary User)  

Young children aged 5–10, who are the focus of speech assessments, often experience anxiety, discomfort or boredom in traditional clinical environments. Their engagement in therapy sessions is crucial for accurate assessment and treatment outcomes.  

Needs:
  •  Interactive, Game-Like Experience: The device must offer a playful and interactive interface that transforms the clinical assessment into “playtime”, effectively encouraging natural participation. Such an approach has been shown to improve engagement and make the therapeutic process feel more like play ([citation]).  
  •   Immediate, Clear Feedback: Children benefit from receiving real-time visual and auditory cues that help guide them through the session [citation]. The integration of LED indicators and sound effects aims to keep the patient engaged and focused, and indicating progress during exercies. 
Parent/Caregiver (Support User)

Parents or caregivers play an essential role in supporting the child’s therapy and need to feel confident that the process is both secure and effective. To ensure the therapy is reinforced outside the clinical setting, the parents need to be fully on-board.

Needs:
  •  Data Security and Transparency: They require (re-)assurance that all recordings and sensitive data are stored securely and handled in strict compliance with healthcare regulations. Furthermore, that the collected data is used solely for clinical purposes particular to their child. This aims to build trust in the technology and to guarantee that their child’s information remains confidential.
  •   Accessible Progress Monitoring: A clear and intuitive interface should be available for caregivers to follow their child’s progress without disrupting or getting involved in the therapy session. This transparency aims not only to keep the parents informed, but also to motivate them to support their child’s ongoing development [citation]. 
Scenarios
Interactive Diagnostic Session

In a typical session, the therapist initiates the diagnostic process via a secure digital dashboard or the parent engages the robot via a button. The therapeutic companion engages the child through a series of interactive, game-like exercises that are designed to incorporate standard diagnostic questions in an entertaining manner. During the session, the system records high-fidelity audio, video, and sensor data, providing a dataset for later review and analysis. This approach not only aims to make the sessions more inviting for the child, but also to enable the therapist with comprehensive data on longer assessments, typically that are not possible to do otherwise.

Remote and Asynchronous Review

After the interactive session, the therapist logs into a secure platform where all recorded data is available for review. They can pause, rewind, and annotate key segments, which allows for a detailed and thoughtful analysis of the child’s responses and non-verbal cues.

Collaborative Consultation

For some cases, additional input or consultation from colleagues is required, therefore the recorded sessions can be shared with multiple experts after obtaining the necessary consents. This collaborative consultation allows for reviewing and discussing the diagnostic findings, leading to more comprehensive and consensus-driven treatment plans. Such an approach fosters an environment of continuous professional development and shared expertise [citation].

Requirements
For the Therapist
  • Robust Hardware Integration: The system must incorporate reliable and durable hardware to ensure diagnostic sessions are completed without data loss or interruption. The design should aim to minimise technical failures during assessments, ensuring that every session's data remains intact and can be reviewed later.
  • User-Friendly Dashboard: An intuitive and efficient digital dashboard is required to present clinicians with information on each recorded session. The dashboard should facilitate rapid review and analysis, with the goal of enabling therapists to quickly identify patterns or issues in speech. By streamlining navigation and data review, the tool should help therapists manage multiple patients efficiently while maintaining high diagnostic accuracy.
  • Secure Remote Accessibility: In today’s increasingly digital healthcare environment, therapists must be able to access patient data remotely. The system must employ state-of-the-art encryption and robust user authentication protocols to protect sensitive patient data from unauthorised access. Having robust security is crucial for both clinicians and patients, as it reassures all parties that the integrity and confidentiality of the clinical data are maintained as per today’s standards.
For the Child and Parent
  • Engaging and Comfortable Design: For young children, the physical design of the robot plays a crucial role in therapy success. A soft, plush exterior coupled with interactive buttons and LED feedback systems can create a friendly, non-intimidating interface that reduces fosters a positive human robot interaction. Ultimately, the sessions should be a fun experience for the child, as otherwise no progress would be made and no speech data would be collected.
  • Responsive Feedback Systems: Dynamic auditory and visual cues are essential components that should guide children through each step of a session. Real-time feedback, such as flashing LEDs synchronised with encouraging sound effects, aims to help the child understand when they are performing correctly, and gently corrects mistakes when necessary. This immediate reinforcement not only keeps the child engaged and motivated but also provides parents with clear, observable evidence of their child’s progress. In essence, cues ensure that the therapy sessions are both interactive and instructive.
  • Robust Data Security: The system must implement comprehensive security measures such as end-to-end encryption and secure storage protocols to prevent unauthorised access or data breaches. The level of protection must reassure both parents and therapists that the child’s data is handled with the highest level of care and confidentiality. Adhering strictly to healthcare regulations is essential to maintain trust and protect privacy throughout the therapy process.

Society

Early, engaging assessments have been proven to significantly enhance long-term outcomes for children with speech challenges. Some studies have demonstrated that gamified diagnostic tools reduce anxiety and enable quicker detection of speech impediments, ensuring that children can receive timely interventions during formative year [citation]. This acceleration in diagnosis can shorten the long waiting lists currently experienced in many places, ultimately leading to better educational and social outcomes, for those on the waiting lists. Furthermore, digital platforms that enable “over-the-air” reviews and multi-organisation consultations have been shown to improve diagnostic accuracy and continuity of care [citation]. It has been shown that digital care networks enhance collaborative efforts among healthcare professionals, ensuring that each patient is given an opportunity to receive additional evaluation from other experts [citation]. Early interventions in speech impediments not only address immediate speech challenges but also contribute to improved educational achievements and social integration in the long term. Some research shows that children who receive timely, engaging assessments exhibit better communication skills later in life, promoting later inclusion in communities, activities and alike [citation]. These improvements in communication and social skills ultimately contribute to a healthier, more productive society overall.

Enterprise

By transforming lengthy, in-person diagnostic sessions into concise, interactive assessments, the solution enables therapists to focus on cases that require direct intervention while still collecting high-quality diagnostic data. Reports have shown [citation] that digital diagnostic tools can increase clinician productivity significantly, thereby enhancing overall workflow efficiency and allowing for more timely patient care. The system’s capacity to collect anonymised diagnostic data offers opportunities in refining assessment algorithms and personalising treatment plans. Likewise, such refinement can be of service to a broader research initiative and other healthcare experts. Although reducing the need for extended in-person sessions inherently lowers operational costs, the true value of the technology lies in its ability to improve patient outcomes and support innovative care models. Some studies have highlighted that integrating such digital tools leads to a more adaptive healthcare ecosystem that better meets the needs of both patients and providers [citation].  

State of the Art

Existing robots

To understand how we can create the best robot for our users, we have to look at what robots already exists relating to our project. We analyzed the following robots and related them to how we can use them for our robot.
The RASA robot

RASA robot

The RASA (Robotic Assistant for Speech Assessment) robot is a socially assistive robot developed to enhance speech therapy sessions for children with language disorders. The robot is used during speech therapy sessions for children with language disorders. The robot uses facial expressions that make therapy sessions more engaging for the children. The robot also uses a camera that uses facial expression recognition with convolutional neural networks to detect the way the children are speaking. This helps the therapist in improving the child's speech. Studies have shown that incorporating the RASA robot into therapy sessions increases children's engagement and improves language development outcomes.

Automatic Speech Recognition

The Nao robot
Recent advancements in Automatic speech recognition (ASR) technology have led to systems capable of analyzing children's speech to detect pronunciation issues. For instance, a study fine-tuned the wav2vec 2.0 XLS-R model to recognize speech as pronounced by children with speech sound disorders, achieving approximately 90% accuracy in matching human annotations. ASR technology streamlines the diagnostic process for clinicians, saving time in the diagnosing process.

Nao robot

Developed by Aldebaran Robotics, the Nao robot is a programmable humanoid robot widely used in educational and therapeutic settings. Its advanced speech recognition and production capabilities make it a valuable tool in assisting speech therapy for children, helping to identify and correct speech impediments through interactive sessions.

Kaspar robot

Kaspar the social robot, University of Hertfordshire
Kaspar is a socially assistive robot whose purpose is to help children with autism learn social and communication skills. A child-centric appearance and expressive behavior are given prominence in the design in order to invite users to engage in interactive activity. Studies [1] have indicated that children working with Kaspar show improved social responsiveness, and the same design principles could be applied to enhancing speech therapy outcomes.

Requirements

The aim of this project is to develop a speech therapy plush robot specifically designed to address issues in misdiagnoses of speech impediments in children. Our plush robot solution addresses these issues by transforming lengthy, traditional speech assessments into engaging, short, interactive sessions that are more enjoyable and less exhausting for both patients (children aged 5-10) and therapists. By providing an interactive plush toy equipped with audio recording, playback capabilities, local data storage, intuitive user interaction via buttons and LEDs, and secure data transfer, we aim to significantly reduce the strain on SLT resources and enhance diagnostic accuracy and patient engagement.

To be able to achieve all of this we must satisfy a certain set of requirements that will outline the requirements, design considerations, user interactions, data handling, performance goals, and a test plan, all of which will be done following a MOSCOW prioritization method:

MOSCOW Prioritization Model
Category Definition
Must Have Essential for core functionality; mandatory for the product's success.
Should Have Important but not essential; enhances usability significantly.
Could Have Beneficial enhancements that can be postponed if necessary.
Won’t Have Explicitly excluded from current scope and version.

Design Specifications

Physical and Interaction Design Requirements
Ref Requirement Priority
DS1 Plush toy casing made from child-safe, non-toxic materials. Must
DS2 Secure internal mounting of microphone, speaker, battery, and processing units. Must
DS3 Accessible button controls (Power, Next Question, Stop Recording). Must
DS4 Visual feedback via LEDs (Recording, Test Complete, Error, Low Storage, Low Battery). Must
DS5 The robot casing should withstand minor physical impacts or drops. Should
DS6 Device will not be waterproof. Won’t

The plush robot’s physical design prioritizes child safety, comfort, and ease of interaction. Using child-safe materials (DS1) and securely mounted electronics (DS2) ensures safety, while intuitive button controls (DS3) and clear LED indicators (DS4) simplify usage for young children. Durability considerations (DS5) shall ensure the product withstands typical child handling (or any handling...), though waterproofing (DS6) is excluded due to practical constraints and the lack of its necessity given the setting it will be used in.


The test plan is as follows:

Test Cases Design Specifications
Ref Precondition Action Expected Output
DS1 N/A Inspect the plush toy casing and its material certificates/specifications. Plush toy casing is certified as child-safe and non-toxic.
DS2 N/A Inspect internal components of the plush toy. Microphone, speaker, battery, and processing units are securely mounted internally.
DS3 Plush robot is powered on. Press each button (Power, Next Question, Stop Recording). Each button activates its designated functionality immediately.
DS4 Plush robot is powered on. Observe LED indicators when performing different operations (recording, completing test, causing an error, storage nearly full, battery low). LEDs correctly indicate each state as intended.
DS5 N/A Simulate minor drops or impacts on a safe surface. Plush robot casing withstands minor impacts without significant damage or loss of functionality.
DS6 N/A N/A N/A

Functionalities

Functional Requirements
Ref Requirement Priority
FR1 Provide pre-recorded prompts for speech exercises. Must
FR2 Capture high-quality audio recordings locally in WAV format. Must
FR3 Securely store audio locally to maintain data privacy. Must
FR4 Enable intuitive button-based session controls. Must
FR5 Support secure data transfer via USB and/or Bluetooth. Must
FR6 Implement basic noise-filtering algorithms. Should
FR7 Automatically shut down after prolonged inactivity to conserve battery. Should
FR8 Optional admin panel for therapists to configure exercises and session lengths. Could
FR9 Optional voice activation for hands-free interaction. Could
FR10 Explicitly exclude cloud storage for privacy compliance. Won’t

These functional requirements reflect: the necessity to simplify lengthy therapy sessions into manageable segments (FR1, FR4), securely capture and store speech data for later analysis (FR2, FR3, FR5), and maintain user-friendly interaction. Privacy and data security are prioritized by excluding cloud storage (FR10), while noise filtering (FR6), auto-shutdown (FR7), admin panel (FR8), and voice activation (FR9) further enhance practical usability and efficiency but are not essential for the functionality of the robot per se.


The test plan is as follows:

Functionalities Test Plan
Ref Precondition Action Expected Output
FR1 Plush robot is powered on. Initiate a therapy session. Robot plays clear pre-recorded speech exercise prompts correctly.
FR2 Plush robot is powered on and ready to record. Record speech audio using provided prompts. High-quality WAV audio recordings are stored locally.
FR3 Recording completed. Inspect local storage on plush robot (either via ssh or usb etc.) Audio recordings are securely stored locally.
FR4 Plush robot is powered on. Use button controls to navigate through a session. Session navigation (Start/Stop, Next Question) operates smoothly via buttons.
FR5 Plush robot has recorded data. Connect robot via USB/Bluetooth to transfer data securely. Data transfer via USB/Bluetooth completes successfully with no data corruption or leaks.
FR6 Robot is powered on and ready to record. Record speech in a moderately noisy environment. Recorded audio demonstrates effective noise filtering with reduced background noise.
FR7 Plush robot powered on, idle for prolonged period. Leave robot idle for X+ minutes. Robot automatically powers off to conserve battery.
FR8 Admin panel feature implemented. Therapist configures new exercises/session lengths via admin panel. Admin panel accurately saves and applies changes to exercises/session durations.
FR9 Voice activation implemented. Use voice commands to navigate exercises. Robot successfully responds to voice commands.
FR10 Check product documentation/design specification. Inspect data storage and upload protocols. Confirm explicitly stated absence of cloud storage capability.

UI/UX

User Interaction Requirements
Ref Requirement Priority
UI1 Clear visual indication of active recording status through LEDs. Must
UI2 Easy navigation between prompts using physical buttons. Must
UI3 Dedicated button to stop recording and securely store audio. Must
UI4 Audio or Visual notifications/indications when storage or battery capacity is low. Should
UI5 Optional voice commands to navigate exercises. Could
UI6 Exclude advanced manual audio processing controls for simplicity. Won’t

User interactions are designed to be intuitive, enabling children to comfortably navigate therapy sessions (UI1, UI2, UI3) without assistance. Additional audio notifications (UI4) provide helpful prompts, and potential voice command options (UI5) may further simplify operation. Advanced settings are deliberately excluded (UI6) to maintain simplicity for the primary child users (and also due to the lack of time available to work on the project).


The test plan is as follows:

Ref Precondition Action Expected Output
UI1 Plush robot is powered on. Initiate audio recording session. LEDs clearly indicate active recording status immediately.
UI2 Session ongoing. Press "Next" button. Plush robot navigates to next prompt immediately and clearly.
UI3 Session ongoing, recording active. Press dedicated "Stop" button. Recording stops immediately and audio securely stored.
UI4 Low storage/battery conditions simulated. Fill storage nearly full and/or drain battery low. Robot issues clear audio or visual notifications indicating low storage or battery.
UI5 Voice commands implemented. Navigate prompts using voice commands. Robot accurately navigates prompts using voice interaction.
UI6 Check product documentation/design specification. Verify available UI options. Confirm explicitly stated absence of advanced audio processing UI controls.

Data Handling and Privacy

Data Handling and Privacy Requirements
Ref Requirement Priority
DH1 Local, offline storage of all collected data. Must
DH2 No cloud uploads or third-party data sharing. Must
DH3 Optional encryption of stored data. Should
DH4 Facility for deletion of data post-transfer. Should
DH5 Optional automatic deletion feature to manage storage space. Could
DH6 No external analytics or third-party integrations. Won’t

Data privacy compliance is paramount, thus mandating the need for local storage (DH1) without cloud exposure (DH2). Encryption (DH3) and data deletion capabilities (DH4, DH5) enhance security measures, while explicitly excluding third-party integrations (DH6) aligns with data protection and privacy goals.


The test plan is as follows:

Data Handling and Privacy Test Plan
Ref Precondition Action Expected Output
DH1 Robot has collected data. Inspect robot’s storage location and methods. Confirm data is stored locally/offline with no online/cloud-based storage.
DH2 Data transfer process. Attempt transferring data and monitor network activity. No cloud uploads or third-party data sharing occurs during or after data transfer.
DH3 Data encryption implemented. Attempt accessing stored data directly without proper keys. Data is inaccessible or unreadable without proper decryption.
DH4 Data has been transferred. Delete data via provided facility after transfer. Data is successfully deleted from local storage immediately after confirmation.
DH5 Data exists (old) Fill storage and observe automatic deletion feature. Old data automatically deletes to maintain adequate storage space.
DH6 N/A Inspect software. Confirm absence of external analytics and third-party integrations...

Performance

Performance Requirements
Ref Requirement Priority
PR1 Prompt playback latency under one second after interaction. Must
PR2 Recording initiation latency under one second post-activation. Must
PR3 Real-time or near-real-time audio noise filtering. Could
PR4 Optional speech detection for audio segment trimming. Could
PR5 Exclusion of cloud-based or GPU-intensive AI processing. Won’t

Performance standards demand rapid, seamless interaction (PR1, PR2) to maintain user engagement and provide a good user experience, supplemented by noise filtering capabilities (PR3) to provide the therapists a good experience though it is not exactly necessary for the functioning of the robot. Advanced optional speech-processing features (PR4) are nice to haves but again not necessary, and high-resource cloud-based AI operations (PR5) are explicitly omitted to maintain simplicity and most importantly data security.


The test plan is as follows:

Performance Test Plan
Ref Precondition Action Expected Output
PR1 Plush robot powered on. Initiate speech prompt via interaction/button press. Prompt playback latency is less than or equal to X seconds.
PR2 Plush robot powered on. Start recording via interaction/button press. Recording initiation latency is less than or equal to X second.
PR3 Noise filtering feature implemented. Record audio in background-noise conditions and observe immediate playback. Real-time or near-real-time audio noise filtering noticeably reduces background noise.
PR4 Speech detection implemented. Record speech with silent pauses. Audio segments are correctly trimmed to include only relevant speech portions.
PR5 N/A Inspect software. Confirm absence...

Legal & Privacy Concerns

Note: We are only going to concern ourselves with EU legislation and regulations as this is our country of residence. Furthermore most of these regulations concern themselves with a full scale implementation of this robot.

We will mainly be making reference to the following regulations/Legislation:

Data Collection & Storage

The robot we want to build for this project requires that some specific audio snippets and data to be collected and stored somewhere where, therapists and professionals that are responsible for the patient's care can access it. This data however is sensitive and must be secured and protected so it is only accessible to those who are permitted to access it. We should also focus on storing the minimum required amount of data on the patient using the robot to make sure only necessary data is stored. These specific data collection and storage concerns, in the EU, are outlined in articles 5 and 9 of the GDPR.

in this context this means the data collected by the robot should at most include:

  • Speech audio data of the patient needed by the therapist to help treat the patients impediment
  • minimal identification data to know which patient has what data.
  • Other data may be needed but must specifically be argued (subject to change)

Furthermore all the data collected by the robot must be:

  • encrypted, so if somehow stolen cannot be interpreted
  • securely stored, so it can be accessed by the relevant permitted parties

User Privacy & Consent

In order for the robot to be used and for data to collected and shared with the relevant parties, the patient user must consent to this and they must also hold specific rights over the data (creation, deletion, restriction etc). On top of this depending on the age of the patient certain restrictions must be placed on the way data is shared, and all patients must have a way to opt-out and withdraw consent from data collection if necessary. These are all covered in articles 6,7,8 of the GDPR.

In essence this means the user must have the most power and control over the data collected by the robot, and the data collected and its use must be made explicitly clear to the user to make sure that its function is legal and ethical.

Security Measures

Since we must exchange sensitive data between the patient and therapist, data must be secured and protected in its transmission, storage and access. These relevant regulations are specified in article 32 of the GDPR (Data Security Requirements).

This means that data communication must be end-to-end encrypted, and there must be secure and strong authentication protocols across the entire system. On the therapists end of things there must be relevant RBAC (role based access control) so only the relevant admins can access the data. In real time use over long periods of time there should be the possibility of software updates to improve security.

Legal Compliance & Regulations

Since this robot can be considered as a health related or medical device, we must check and make sure that the data collected is used and treated as medical data. All regulations relevant to this are specified in the Medical Device Regulation.

This Robot may also have certain AI specific features or functionalities so this must also fall within and adhere to regulations and laws present in the AI act so that the functionality and usage of the robot is ethical.

Ethical Considerations

Since the patients using this device and interacting with it are children, we must make sure that the interactions with the child are ethical and the way in which data is used and analysed in order to form a diagnosis is not biased in any sort of way.

The robot must minimize psychological risks of AI-driven diagnosis, prevent any possible distress, anxiety and deception that interaction could cause. Training assessments should be analysed in a fair and unbiased manner and decisions on treatment and required data for a particular stage of treatment should be almost entirely decided by the therapist with little to minimal AI involvement.

These are all outlined in the AI Ethics Guidelines and article 16 of the UN Convention on the Rights of the Child.

Third-Party Integrations & Data Sharing

Since we are sharing the data collected from the robot to the therapist, we must ensure that strict data-sharing policies are in place that require parental/therapist consent. Furthermore if we use any 3rd party services, like cloud storage providers, AI tools, or healthcare platforms we must make sure data is fully anonymised so no there is no risk of re-identification.

This is so we adhere to article 28 of the GPDR

Liability & Accountability

In case of issues with function or potential data leak we must make sure to hold the responsible parties accountable. This is especially important in the case of AI functionalities as no responsibility can be placed on such actors, and must be placed on manufacturers and programmers.

If there are technical issues with the function of the robot or issues with data transmission encryption we (the creators of the product) must be held accountable for the issues with robot. If there are issues with storage of data due to failures of 3rd party systems those system creators must also be held accountable for the issues in their system. If the therapist or medical professional that is treating a patient leaks data or provides bad treatment intentionally or otherwise they must also be held accountable for their conduct and actions.

These are all specified in the Product Liability Directive under manufacturer accountability.

User Safety & Compliance

Since the robot interacts directly with children, we ensure its physical safety through non-toxic materials and a child-safe design, comply with toy safety regulations if applicable, prevent harmful AI behaviour by avoiding misleading feedback and ensuring therapist oversight, and adhere to assistive technology standards for accessibility and alternative input methods.

This is so we adhere with ISO 13482 and EN 71 (EU toy safety standard).

Interviews and Data collection

We will also be conducting interviews with several relevant volunteers upon this matter as well as experts in the field. To ensure ethical standards, we will obtain informed consent from all participants, clearly explain the purpose of the interviews, and allow them to withdraw at any time. Confidentiality will be maintained by anonymizing responses and securely storing data. Additionally, we will follow ethical guidelines for research involving human subjects, ensuring that no harm or undue pressure is placed on participants.

Design

Prototype

The prototype proposed, is focused in addressing the challenges and requirement specified earlier in the report. The traditional diagnostic tests are often extremely long—two to three hours—which leads to fatigue on both the patient's and therapist's side. As a result, the patient will experience the test as extremely uncomfortable, and the therapist's exhaustion will lead to a reduced accuracy of the diagnosis. The prototype will therefore break down these diagnoses into short, disguised games, without the need for speech therapist supervision. It will be an interactive device that will ask closed/open-ended questions to the patient that are specifically chosen by a speech therapist or from an already existing test. Once the question is posed, the robot will then record both in audio and (perhaps) video the response of the patient to be then reviewed at a later stage. Since all the responses are stored digitally, this will allow diagnostics to be performed abroad in areas, especially in rural areas lacking speech therapists. By allowing the therapist to rewind, replay, or pause the digital diagnosis, it would guarantee a more thorough analysis and lower the risk of missing details.

The prototype will hopefully reduce patients' stress and fatigue due to the test being broken down. It will lessen the workload of the speech therapists while also improving reliability. allowing multiple therapists to review the recording whenever it is convenient for them, reducing individual biases.


Device Description

Prototype Image.png



Plush Appearance:

The prototype was chosen to take the form of a friendly plush toy in order for young patients (ages 5-10) to engage more willingly with the speech and articulation exercises proposed by the plush. The friendly plush toy will disguise the assessment process as interactive play, trying to deceive the child into believing it is playing.

The soft appearance will enable the device to be more durable by acting as cushioning for the electronics inside it. This will increase the life expectancy of such a device.


Buttons

Multiple buttons are built into the plush's surfaces, which will allow you to control the plush's behavior. Here are the following buttons to be installed:

- Turn on/off button

- Initiate the plush next question button.

- End recording response


LEDs

A number of LEDs are on the device to enable visual feedback to both the patient and the supervisor (parent or speech therapist). Here are the following indicator LEDs on the device:

- "Recording" LED, ON if recording

- "Test complete" LED, ON is complete.

- "Error" LED, ON if error present

- "ON" LED, ON if plush is turned on




Microphone, Speaker, Camera:

A discrete, high-quality, sensitive microphone, hidden inside the plush, to ensure clear recording of the patient's speech, with minimal obstructive sound.

An internal speaker is placed, for example, on the chest of the plush, allowing the device to deliver audibly the questions, but also feedback and fun sounds.

A small camera is hidden inside plush eyes, enabling the recording of patients facial expressions and reactions to prompts. Enable more in-depth diagnostics.



Internal Hardware:

The device will contain a processing unit such as Arduino or Raspberry Pi for processing and managing all electrical components. A large storage unit, such as an SSD card, is needed to store the video and audio recording until retrieval. Finally, a rechargeable battery stored accessibly inside the plush for safety and convenience.

System Specification

Software

Testing

Interviews

Introduction

To develop a robot that effectively assists in speech therapy, it's important to understand current therapy practices and identify potential areas for improvement. We conduct interviews with speech therapists and parents of children with speech impairments to gain insight into existing methods, challenges, and how a robot might assist in this process.

Current Practices and Challenges

Participants (speech therapists) are asked about challenges they commonly face in data analysis, therapy workloads, and the overall diagnostic process. Feedback is collected regarding their experiences, highlighting the most demanding or time-consuming aspects of existing methods. Therapists are also asked about which speech impairments could detected by voice recognition (i.e. the robot), including specific signs or indicators commonly used for diagnosis.

User Experience and Engagement

The interviews aim to provide a better understanding of children's experiences during diagnostic assessments, particularly regarding factors leading to fatigue, anxiety, or disengagement. Identifying these factors is important in order to design the proposed speech therapy robot so that it may make therapy sessions more engaging and less stressful. Participants discuss factors they believe are most important for keeping children actively involved, including interactive elements, rewards, vocal feedback, and gamification strategies.

Potential for Robotic Assistance

Participants are asked to share their feelings on integrating a plush robot into speech therapy sessions. Discussions cover how child patients might respond to such a robot, whether positively or negatively, and therapists' feelings regarding interactions with social robots. Therapists should address how comfortable they are working alongside a robot, their views on robots potentially replacing certain tasks, and the inherent limitations a robot has compared to human therapists. These insights help clarify how the user groups view social robots, whether as complementary tools or replacements for existing therapist tasks.

Practical Considerations

To better refine the practical design of the robot, the interviews cover preferred methods for interacting with the device, including physical buttons, voice commands, and ways to review collected speech data (e.g., via a mobile app, web application, or directly from the robot). Participants are also asked to share how frequently they could imagine using or recommending a speech therapy robot. Furthermore, the therapists are asked to consider whether they believe the robot might increase children's engagement or participation in therapy. Finally, participants provide their preferences and wishes regarding the physical appearance of the robot.

Privacy & Ethics

Due to the sensitive nature of speech data collection and pediatrics in general, the interviews explicitly address privacy and ethical considerations. Participants are asked if they would be comfortable with the robot recording children's speech for later review by therapists, and their opinions on third-party access to such data, especially involving AI technologies such as large language models (LLMs). The interviews further discuss whether data should be stored locally on the robot or if participants would accept secure storage on remote servers. Participants are also asked to share their thoughts on whether the robot should serve as a complement or potentially replace certain aspects of traditional, in-person therapy sessions. The participants highlight some of their biggest hesitations or concerns related to integrating technology into speech therapy practices.

Method

Interviews are conducted either in-person or through video calls, each lasting around 15 to 30 minutes. With participants' consent, these interviews are recorded and transcribed for detailed analysis. Participants answer open-ended questions about current diagnostic methods, challenges in analyzing speech data, potential biases in diagnosis, and general experiences with speech therapy sessions. Additionally, we ask the participants' to share their thoughts on using a friendly, plush robot during therapy. Topics include possible benefits of the robot, preferred design features, usability, and data privacy.

Interview questions are adjusted based on the participant’s role. Speech therapists are mainly asked about technical and diagnostic challenges and practical robot-related considerations. Parents are encouraged to discuss their children's experiences with therapy and assessments, as well as share their experiences as the parent in this regard. The insights from these interviews aim to guide us in improving the design of the proposed speech therapy robot.

Analysis

Results

Conclusion

Bibliography

https://my.clevelandclinic.org/health/articles/24602-speech-language-pathologist

https://idcchealth.org/blogs/how-much-does-online-speech-therapy-cost/

https://www.hollandzorg.com/insured/reimbursements2025/speech-therapy

https://pmc.ncbi.nlm.nih.gov/articles/PMC7383695/

https://www.belganewsagency.eu/nearly-one-fifth-fewer-speech-therapy-students-in-ten-years-time

chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://files.eric.ed.gov/fulltext/EJ1135588.pdf
  1. ↑ https://www.jbe-platform.com/content/journals/10.1075/pc.12.1.03dau
https://www.researchgate.net/publication/384968031_Exploring_solutions_to_reduce_waiting_lists_for_speech_and_language_services_in_the_Netherlands

https://pubmed.ncbi.nlm.nih.gov/36467283/

https://arxiv.org/abs/2403.08187

Appendix

Time reporting

Week 1

Name Task Time spent
Andreas Sinharoy Robot and Problem Ideation and Research into the Idea 2 hours
Luis Fernandez Gu Problem Ideation and Research 2 hours
Alex Gavriliu Research into data privacy requirements in EU 1 hour
Theophile Guillet
Petar Rustić Creating the wiki structure, literature research 3 hours
Floris Bruin
All

Week 2

Name Task Time spent
Andreas Sinharoy Writing the Planning and Introduction sections of the wiki page 3 hours
Luis Fernandez Gu Wrote and planned Requirements Section 3 hours
Alex Gavriliu creating appropriate structure for legal and privacy section 1 hours
Theophile Guillet
Petar Rustić literature research, writing the USE analysis 6 hours
Floris Bruin
All

Week 3

Name Task Time spent
Andreas Sinharoy Helped obtain materials for the prototype, contacted and scheduled interviews 3 hours
Luis Fernandez Gu
Alex Gavriliu
Theophile Guillet
Petar Rustić
Floris Bruin
All

Week 4

Name Task Time spent
Andreas Sinharoy Started construction of the robot 6 hours
Luis Fernandez Gu
Alex Gavriliu
Theophile Guillet
Petar Rustić
Floris Bruin
All

Week 5

Name Task Time spent
Andreas Sinharoy Was not in the Netherlands 1 hour
Luis Fernandez Gu
Alex Gavriliu
Theophile Guillet
Petar Rustić
Floris Bruin
All

Week 6

Name Task Time spent
Andreas Sinharoy Helped construct the circuitry of the robot, helped created the code/software responsible for communicating beteen the robot and the child, updated the wiki page, filled out the ethical consent form, and messaged and contacted the ethics committee to streamline obtaining ethical consent in time. 20 hours
Luis Fernandez Gu
Alex Gavriliu
Theophile Guillet
Petar Rustić
Floris Bruin
All

Week 7

Name Task Time spent
Andreas Sinharoy
Luis Fernandez Gu
Alex Gavriliu
Theophile Guillet
Petar Rustić
Floris Bruin
All
Retrieved from "https://dsdwiki.wtb.tue.nl/index.php?title=PRE2024_3_Group3&oldid=131120"