PRE2023 3 Group2

Group Members

Name	Student ID	Department
Satya Dharmawan	1545892	Electrical Engineering
Alonso Geeksink Antón	1710508	Computer Science and Engineering
Sander Erbrink	1862146	Industrial Engineering
Gabriël van Dijk	1862081	Industrial Engineering
Antoni Liberak	1719696	Automotive Technology
Valentin Nikushor	1847120	Mechanical Engineering

Introduction

With an aging population, also comes a higher percentage of the population having dementia, as approximately 70% of individuals diagnosed with dementia are aged 75 years or older where over 30% of the patients resides in residential long-term care (RLTC) facilitieswhich rely heavily on caregivers for operation. However, fewer people are taking caregiver positions due to varying factors such as comparatively lower wages and benefits, especially when compared to positions in hospitals leading to a high turnover of just 12 months. The combination of a relatively old population combined with a lack of available caregivers results in a growing imbalance between the supply and the demand.

Elderly individuals, particularly those with dementia, often exhibit low medical adherence rates, ranging "from 17% to 42%, and medication discontinuation before the end of treatment ranged from 37% to 80%" . The shortage of caregivers exacerbates this issue, as there are fewer individuals available to monitor and ensure proper medication intake and overall health management.  An earlier study has shown that one of the methods to increase the medication of adherence of patients is through telehealth which is another method of checking up and monitoring the patient through the use of of telecommunications.

To address the shortage of caregivers and the growing demand for their services, we propose the implementation of an AI-powered caregiving assistant. This solution aims to fill the gap in caregiver availability and help balance the supply and demand in caregiving services.

Problem Statement

The goal is to address the need for caregivers in a way that reduces the frequency of in-person visits, ideally conducting a weekly check-up. This is where we propose to design an AI caregiving assistant to help with medication adherence and the day to day life of the user. The first device that will be integrated in the system is the pill box which would monitor the intake of the prescriptions to ensure that the user takes in the correct amount at the correct time.

Additionally, elderly individuals, often experiencing loneliness due to limited visitors, may feel further isolated if their caregiver only checks in weekly. To mitigate this, we propose integrating an AI companion feature that enables users to interact and engage in conversation, providing companionship and support.

Objectives

The aim is to enhance medication adherence among the elderly and individuals facing challenges in this regard by devising a system that integrates an AI caregiving assistant to assist them in their daily activities. The primary focus of this project is to initiate this endeavour by integrating the AI assistant with a pill dispenser, serving as the foundational step towards achieving improved support and management for this demographic group.

• The ability to converse with the AI companion
• Daily reminder to use medicine
• Checking whether the user has taken their medicine
• Checking whether the pill box is empty
• Reminder to stock up pill box
• To design the framework for connecting to future devices

Planning

Week	Task
1	Planning and setting up
2	Literature research
3	Functional requirements
	Potential problems
	First conceptual design
	Start making the app
4	Second conceptual design
5	Building prototype+electronics
6	Continuing on the prototype
	Debugging the app + electronics
7	Testing prototype
	Evaluate results
	Make presentation
8	Finish the wiki

Initial Functional Requirements & Deliverables

• A prototype with one or two compartments of the pill box working (with sensors)
  • English voice recognition and a companion AI app to assist the user with daily reminders and initiate conversations (as oppose to only responding to commands)
    • Speech recognition
      • Passed to an AI chatbot with text to speech functionality
• A completed Wiki page functioning as a report on the development of the aforementioned product
• A final presentation showcasing the prototype and any relevant parts of it's construction and the report

USE

User:

• The main user of this technology will be the elderly and patients with early stages of dementia that does not have a caretaker assigned to them nor other people to give them assistance. However, this does not limit to just the elderly, but whoever has a medical condition that does not allow them to adhere to the prescribed medication doses along with no one being able to take care of them doing their day to day tasks.

Society:

• Caretakers:
  • This would potentially reduce the burnout rate from caretakers as they have been reported to be burnt out due to low pay in addition to long hours due to the caretaker shortage.
• Government:
  • The main benefit for them would be that they could allocate less resources or subsidies to train/hire more caretakers and potentially increase more spending on healthcare providers as the elderly, especially those with dementia has a lower medication adherence. Thus the likelihood of coming back and using healthcare services would be higher.
• Healthcare services:
  • As mentioned previously, due to potentially having lower intake from the elderly, the burden on the healthcare services would be lower.

Enterprise:

• Given the current socioeconomic state globally, the trend of aging population will continue to increase unless large changes to the economic and social policies are made. Thus, this can be seen as a growing market.
• There will be a lot of business opportunities with AI chatbot companies as they will be the ones advancing the algorithm for the interaction between the robot and the user.

Methodology

To properly design and create a feasible and good product, it was important to first design a proper study of our target group and product specification. To achieve this, a mix of researching existing literature and qualitative data gathering in the form of interviews was used. For this, the objective ontological approach together with a epistemological perspective was employed. This framework facilitates collecting and sampling data, which is then empirically analyzed . This approach goes very well with the epistemological perspective, which tries to find the source and reason for knowledge .

Research Design:

The research consists of qualitative interviews and literature research. For the interviews, basic, follow-up and in depth questions were asked to participants chosen on either their experience within the field of caregiving, preferably with specific elderly care experience, and the elderly themselves who used a similar product as our product design. By interviewing this demographic, it enables the research to more accurately portray the current situation within the elderly care sector.

This would then be backed up by published research on a variety of field. State of the art research could paint a picture of different products on the market and where the design goals could be changed, resulting in a more unique and preferably better product. Research on the interaction between robots and the elderly was also conducted, this to gather insight on potential problems or focus points within human-robot interactions.

This is then followed by prototyping and finally an evaluation of the protoype.

State Of The Art

Smart drugs:Improving healthcare using Smart Pill Box for Medicine Reminder and Monitoring System

This smart pillbox epitomizes an innovative blend of advanced technologies meticulously crafted to streamline medication management, particularly for seniors. By harmonizing Arduino Nano and ESP8266 NodeMCU modules, the pillbox orchestrates precise dispensing sequences through its intricately designed 3D-printed components. This includes the pill storage cylinder, pill hatch, pill chamber, and pill pipe, each playing a pivotal role in ensuring accurate and controlled medication distribution. Additionally, the integration of a bespoke mobile application extends convenient remote control functionalities, empowering users to effortlessly input medication details, set dose reminders, and monitor the pillbox's operations in real-time.

Development of a Low-Power IoMT Portable Pillbox for Medication Adherence Improvement and Remote Treatment Adjustment

The article presents an Internet of Medical Things (IoMT) platform anchored by a 3D-printed pillbox, designed to bolster medication adherence. Beyond its compact form and energy-efficient design, the pillbox incorporates a camera, LCD display, and LED indicators. Connectivity to a remote server, web application, and third-party APIs enrich its functionality. User feedback underscores high acceptance and ease of use, particularly praising medication management features. Comparative trials against a dummy pillbox demonstrate significantly reduced intake delays, indicative of enhanced adherence. Future iterations aim to refine stability, reduce size, and introduce features like remote patient monitoring and accessibility options.

Tricella smart pillbox

Tricella's smart pillbox makes taking medicine easy. The app reminds users to take their medication, and caregivers get alerts, making it simple for families to stay connected. Phantom Sync technology makes managing complicated medication schedules simple. Users say it helps with busy lifestyles and ensures important treatments are never missed. Tricella's Remote Patient Monitoring Platform provides covered services for Medicare, helping medical practices earn money and improve patient outcomes. There are no upfront costs for practices, making it easy to enroll patients, review data, and bill for services. During the pandemic, Tricella provides free monitoring devices, prioritizing health and safety.

CYCO smart pillbox

CYCO is an advanced medication management solution with cutting-edge features that make handling medication schedules easy. It has dual sensors for precise dispensing and multiple layers for storing different medications efficiently. CYCO includes an alert system for wrong doses, reducing the risk of errors. The scheduling by doses feature allows users to create personalized medication plans, improving flexibility and adherence. With a user-friendly interface and simple setup, CYCO is dedicated to revolutionizing medication management and empowering individuals to take control of their health journey confidently.

Voice Based Medicine Remainder Alert Application for Elder People

The paper introduces a Voice Based Medicine Reminder Alert Application tailored for elderly individuals, aiding in medication adherence and emergency assistance. Utilizing Android smartphones, the app allows users to schedule medication dosages, receive voice-based reminders, and trigger emergency alerts via color-coded buttons. Cloud-based communication enables feedback between users and healthcare providers, enhancing medication monitoring. Literature review underscores the importance of personalized reminders for improved adherence. The system's methodology encompasses user registration, medication input, and text-to-voice conversion. Experiment results demonstrate the app's efficacy and user-friendliness. Software specifications include Android Studio, XML, and SQLite. Screenshots depict the app's interface and functionalities. Overall, the system offers a practical solution to medication management challenges among the elderly, fostering independence and better health outcomes. Authors acknowledge funding and provide profiles highlighting their expertise.

The product will be different from other State of the Art as a companion assistant will be integrated within the system. This can be used not only from medicine reminders but also to allow the user to interact with the device for other reminders and using it as an assistant. This assistant serves multiple purposes, extending beyond medication reminders to facilitate user-device interaction for various reminders and assistant functions. One secondary objective of incorporating the assistant is to address the potential loneliness experienced by the main target demographic, namely the elderly who lack consistent human interaction for daily support. By enabling users to engage in conversation with the assistant, they can seek engage in casual dialogue for things such as household queries or talking about random topics. This would foster a sense of companionship in the form of anthropomorphism, where which users may develop deeper emotional connections with the system, further enhancing its effectiveness in alleviating loneliness and providing support..

With these implementations as a starting point for future development, a more complete system of a an assistant can be made starting with the integration of the chatbot assistant and the pill dispenser.

Literature Research

An extensive literature research had been conducted on the topic of the medication prescribed to the patients with dementia. First of all, it is important to differentiate different causes for dementia, as this will have a direct effect on the treatment. Most common cause is considered to be Alzheimer's disease - it is involved in as many as 80% of cases (Birks & Harvey, 2018) ; next are dementia caused by blood vessels' disease within the brain, called vascular dementia, as well as a mixed dementia, where both Alzheimer's and vascular conditions are having an effect (VaD) There are other causes (Parkinson's, Lewy Body) but they are less common. Since the amount of time and resources is limited, it is chosen to focus mainly on the treatment prescribed to the patients with Alzheimer's. Most common treatment are:

• Acetylcholinesterase inhibitors, such as rivastigmine, galantamine, donepezil, mostly ingested together with food.
• NMDA antagonist medication - memantine (in combination with aforementioned inhibitors, for moderate to severe cases).

Some medications need to be taken at set intervals or a specific time of day, with or after food, or on an empty stomach. The exact number of pills, timing, and other specificities of medical regimen vary heavily from person to person, thus it is rather complicated to make a good generalization that would fit every person perfectly. In addition, some patients may also experience depression and anxiety in the early stages and delusions and hallucinations in the later stages. which implies a prescription of antidepressants or antipsychotics. Most common antidepressants are sertraline and mirtazapine, but there are others. Typical intake for those is about 150 mg/day and 45 mg/day respectively, typically ingested as one pill, however a study "..suggests that antidepressants should not be prescribed as a first-line treatment for people with depression in Alzheimer's disease who are referred to old-age psychiatry services, as many cases will resolve with usual care without sertraline or mirtazapine" ; Thus, it might not be reasonable to assume antidepressants. Antipsychotics and their dosage had not been studied yet as they are prescribed only in very severe cases of dementia and thus make the minority of the cases; they should however be analyzed in the future.

When it comes to the inhibitors, they are typically prescribed as "first-line treatment in patients with mild to moderate Alzheimer's disease", especially donepezil. Common dosage is a 5 or 10 mg pill per day. It improves cognition and daily function and its effects are extensively studied. In addition to or as a substitute, rivastigimine is used. It is either in a capsule (1-2 pills per day, 6mg pill), or as a on-skin patch. "Use of rivastigmine in doses of 6 to 12 mg daily is associated with statistically significant benefits in terms of cognitive function" and thus is also quite common. Galantamine had not been studied yet, while memantine had been researched and it is usually prescribed in the severe cases, usually in form of 2x10 mg pills per day, but can be one 20 mg pill. . A study that tried to establish efficacy and safety of memantine for people with dementia found that it indeed slightly cognitive functions, performance on activities of daily living, behavior and mood, etc. compared to those who did not take such medicine .

From these, we can assume that on average the user will take about medicines about 2 times a day.

Robots designed for the elderly offer more than practical help; they can provide emotional support and companionship, crucial for combating loneliness . Engaging in conversations, providing reminders, and offering entertainment, they help alleviate isolation and nurture connections. Their presence brings reassurance and comfort, especially for those living alone or in care facilities. By blending human interaction with technological innovation, these robots not only improve quality of life but also serve as steadfast companions, easing the burden of loneliness among the aging population. This would alleviate their loneliness especially since the aim of the product is to reduce the number of visits by the caregiver per week.

As for ergonomics, a metastudy has been conducted by Richard Sather III et al. Here the researchers looked at various pillbox designs and how they changed as time went on. At first the pillbox compartments were simple square structures. They later evolved to use a more rounded form, which proved easier to grab pills from, as the pills mostly stayed in the middle of the compartment with this design. However, this wasn't the standard or agreed on design. A multitude of different shapes and forms were and are still used. From simple squared, to tubes. Some also required to tilt the box in order to get the pills out of the box.

Using the information gathered from research and pin-pointing it together with the results of the interviews, the group should be able to come up with a useful pill box design.

Interview 1

Questions interview 1

Guideline: introduction Approximately 2 min

• short introduction interview

• Stating the research purpose: How would an automated pill dispensing system with conversational abilities help in elderly care, more so with early stages of dementia.

• Stating basic informal consent
  • Do i have permission to use this interview
  • Am i allowed to use this interview in our research
  • Do you want to stay anonymous

• How much assistance did your patients receive in their daily life?

• To what degree are elderly patients affected by loneliness?
• How much are robots used in caregiving, should it be more/less? (give an example of the animal robot companion)

• What was the memory like of the average patients, and those with early stage dementia?
  • What kind of problems do they encounter in their daily life?

• How often do most patients receive medication?
• Do they often adhere to the prescription of their medication?
  • If the adherence is low, what were the reasons? (do they often forget and need constant reminders or have a distrust in the prescribed medicine?)
• Do they need a lot of encouragement to exercise, such as taking a walk just to keep them physically active?
  • If so, what were your methods in encouraging them?

Short explanation of our product and its functionality

• Would such a product have a positive effect in patients?

• Do you recommend any extra specifications, or think some functionalities should be dropped?

• Would this product make your work, or the work of colleagues easier?
• Do you foresee any problems that might arise from using this product?
  

Short summary of interview approximately 1 minute

• Short summary of interview

Interesting Findings interviews:

Luuc:

"Ja... het is wel echt...(eenzaam) ook op gesloten afdelingen vooral. Dan merkt je toch wel dat eenzaamheid wel een dingetje was. En op een niet gesloten afdeling, ja, komt wat meer familie,"

Translation: On the "Closed care" (Care where patients cannot leave because of locked doors, often to make sure they don't wander away) there is a lot of loneliness. On the normal care the family visits a bit more.

Note: He insinuated both normal and closed care is lonely, but the closed care moreso.

"... Ja, ik denk dat het vaak bij patiënten die nog redelijk op zichzelf waren, als uit de patiënten of hoe je denkt die het juist het beste nog zelf kunnen nemen. Die hadden vaak zijn apparaat"

Translate: Patients who where still relatively capable often use a pillbox, often those who you wouldn't consider for such a function.

"Volgens mij worden die nou vaak in die automatische zakjes, zoals die Backstrolls, ziten ze vaak. En die, volgens mij zijn die wekelijks. Dat is die wekelijks waar de bijgevuld."

Translation: They come in Backstrolls (a roll of little bags containing the pills), and they are delivered on a weekly basis.

"Ja, die kamers zijn niet groot van patiënten vaak. ... hij moet niet heel groot zijn, maar ik denk niet dat het heel erg uitmaakt, het formaat is niet echt heel belangrijk, het moet om een kastje kunnen staan."

Translation: The rooms of the elderly that are cared for aren't very big so the box shouldn't be of excessive size, but it doesn't really matter as long as it fits on a cupboard.

"Ja, in principe, meestal staat die op een kastje of in de keuken en blijft hij eigenlijk altijd wel daar staan. Dus ik denk niet dat je hem heel vaak kwijt raakt. Alleen op gesloten afdelingen gebeurt het nog wel eens dat er iemand bij iemand anders spullen mee neemt, en dan raak je wel dingen kwijt."

Translation: Usually the boxes stay in the same location, so they aren't lost often. Only sometimes in closed care there are patients who take other peoples stuff. (usually out of confusion not malice)

"Ik denk voor de ouderen zelf dat zo'n spraakfunctie wel fijner is, kijk zo'n alarm werkt, want je wordt er ook aan het herinnerd. Maar ik denk dat een spraakfunctie een fijnere manier is om er aan het herinnerd te worden. Dus ik denk dat dat iets prettiger is"

Translation: A speech function is nicer than a regular alarm, a regular alarm works but Luuc thinks utilising speech is a better experience for the elderly.

"Want zou bijvoorbeeld de familie ook diezelfde app kunnen, dat ze kunnen zien of de medicatie genoeg is?"

"Nou, ik denk als ze samen kan werken met het systeem dat de zorg gebruikt, dat ze een melding krijgen of de medicatie genomen is ja of nee, Dat je dan de belangrijkste functie al hebt."

Translation: If the app can use a system that works together with the care itself and can notify caregivers if the medication is used or not is the most important app function, and it could also notify family members.

Tine:

"Ja ik denk het wel (dat ze eenzaam zijn). Ik heb het daar niet zo heel vaak over met cliënten, maar ze zijn best wel vaak alleen, bij ons is het zo dat je met verschillende kamers werkt en dan een gezamelijke woon ruimte zegmaar"

Translation: She thinks the elderly are lonely, although she doesn't really talk to them about this. But they are often alone, they have their own rooms and a shared common room.

"Ja mensen weten wel vaak waar ze nog zijn en zo, dat stadium zitten ze nog wel. Maar ja, ze weten vaak niet bijvoorbeeld mijn naam en sommige mensen vragen wel vaker op een dag hetzelfde. Maar er zit een groot verschil tussen patiënten onderling."

Translation: Patients are often still aware of their surroundings, but details like names are often forgotten, and they often repeat questions during the day. But there is a big difference between the mental states of patients.

Main takeaways for the initial prototype:

• An oral notification is preferred
• Notifications to the caregiver and family members are crucial

Potential problems and Functional requirements

When conducting this research and developing a working product, there are a multitude of problems to consider. Some of these problems will need to be solved in order to create a acceptable prototype. These problems are as the following:

• Forgetting Medication Schedule: Memory decline is common among elderly individuals, especially those with dementia. They may forget to take their medications at the prescribed times, leading to missed doses or incorrect dosing schedules.
• Confusion with Medication Names: Elderly individuals may have difficulty remembering the names and purposes of their medications, increasing the risk of medication errors or confusion, especially if they are taking multiple medications.
• Physical Limitations: Arthritis, reduced dexterity, or visual impairment can make it challenging for elderly individuals to handle medication containers, open pill bottles, or accurately measure out doses, leading to difficulties in self-administration.
• Safety Features: Ensuring the safety of the users is paramount. The design should prevent accidental ingestion of multiple doses, incorporate child-proof mechanisms if necessary, and be resistant to tampering.
• Durability: The pillbox should be designed to withstand accidental drops or spills, as well as environmental factors such as humidity or temperature variations. Using durable materials and robust construction techniques can enhance longevity.
• Simplified Setup: Simplifying the initial setup process for caregivers or family members who assist the elderly individuals in using the device is essential. Clear instructions and minimal configuration steps can help alleviate frustration during setup.
• Time Management: Elderly individuals may struggle with time management, making it difficult for them to adhere to specific medication schedules or remember when they last took their medication.
• Travel and Mobility: Traveling or moving between different locations can disrupt medication routines and make it challenging for elderly individuals to remember to take their medications or pack them securely for travel. This means that the pillbox needs to have a way it can be battery-powered. Or a similar solution.
• Medication Adherence Stigma: Some elderly individuals may feel stigmatised or embarrassed about needing assistance with medication management, leading them to resist using supportive tools or seeking help from others.
• Unwillingness to use robots: Some elderly may be unwilling to use or listen to a robot.
• Decreased human interaction: A robot takes away a task that would usually be performed by a human caretaker. A daily interaction like someone coming in to get the patients to take their pills could be an important piece of interaction for lonely elderly patients.

Then there are some functions that the product must be able to do:

• Software
  • Recognize speech
  • Understand speech
  • Output in spoken language
  • Able to form sentences and have conversations
  • Ability to understand the medicine intake
  • Needs to log data
  • Secure communication
  • Reminders snooze, if the can take it immediately
  • Error Handling: The software should be able to handle errors gracefully and provide clear feedback to the user in case of misunderstandings or communication failures.
• Hardware
  • Give notifications with led and speech
  • Detect when lid is opened
  • Power management, battery for periods of time when on the move

Furthermore, not every problem is solvable within the scope of the course. However, this does not mean they should be disregarded. It is important to acknowledge these issues to show that every aspect has been considered, and as a list of potential improvements for hypothetical future iterations:

• Child safety features
• Giving medical advice
• Backup and restoration
• User Authentication: Implement user authentication to ensure that only authorized individuals can access sensitive data or features of the app, such as medication logs or personal information
• Integration with Health Records: Provide the option for users to integrate their medication intake data with their electronic health records or share it with healthcare providers for better monitoring and management of their health.
• Customizable Preferences: Users should be able to customize their preferences regarding speech recognition, language, reminders, and other features according to their individual need
• GPS Tracking: Implement GPS tracking functionality to help users locate the pillbox if it is misplaced or lost.
• Integration with Wearable Devices: Allow integration with wearable devices such as smartwatches or fitness trackers to provide additional insights into the user's health and activity levels.
• Offline capability

https://miro.com/app/board/uXjVNkFXOdg=/

Storyboard of the system

The ideal target demographic would be an elderly patient that are in the early stages of dementia.

The symptoms can be:

• Problems with their memories, especially when trying to recall recent events
• Increasing confusion and an overall reduced concentration
• Slight changes in their personality or behaviour

A simplified scenario of the system:

When it is time for the medication

• The user will receive a verbal notification through the app to take their medicine
• If the sensors does not detect the lid of the pill box opening within a time duration, for example 1 minute, then another verbal notification will be sent
• After some reminders, the caregiver would be notified.

The user receives new prescriptions or medications

• The caregiver would update the app on the list of new medications the user would have to take and when they should take them
  • This would also contain the instructions and the side effects and some basic description of the medication
• The caregiver would have to also refill the pill box for the user accordingly, as the notifications will state which box number the user should take the medication from.

The user is confused and wants to ask questions related to their health

• The basic instruction and description of the medications should already be in the app, and the user can either open the app or ask the AI assistant for those details.
  • The AI assistant will only access the database in the app for inquiries about prescriptions to limit the number of medications the AI can "choose" from in case the speech to text functionality is not fully accurate.
• The AI assistant will also prompt the user whether the amount of details that were given is enough
  • Otherwise the caregiver will be contacted to give a more comprehensive answer to the user's inquiries.
  • Due to privacy concerns, the AI assistant will not have access to the specific condition of the user and thus cannot answer specific inquiries related to the user's health.

There are no visitors for the user for a while and they feel lonely

• If the patient does not have a lot of visitors from friends or family or the caregiver and the user feels lonely, the companion AI assistant is there to keep them company through conversations that mimic human interactions.

Abnormalities from other telehealth devices (Out of scope for this project)

• If the connected devices for telehealth detects anything abnormal such as a higher blood rate or the wearable that the user might be wearing has detected a fall, then the caregiver will be immediately notified. The AI assistant will prompt the user basic questions such as whether they are bleeding and whether they need urgent assistance such as an ambulance.
  

Flowchart of some scenarios

This is used to align the main ideas of what we plan to design across the group.

Ethical Considerations and Limitations of the Technology

When developing this product, there were a couple of limitations that has been encountered.

• Speech recognition is limited in terms of accuracy. There are a plethora of factors that contribute to the accuracy of speech recognition, such as the language that the user is speaking in, the accents that they have which can vary greatly even in the same region. Additionally, background noise levels, microphone quality, and the complexity of the vocabulary being spoken all play crucial roles in determining the precision of speech recognition systems.
  • Due to these limitations, there is only so much accuracy that the speech recognition can provide in terms of transcribing speeches. In an attempt to improve the accuracy of the speech recognition system, only English will be used as it is often considered one of the best supported languages for speech recognition due to the extensive research, development, and resources dedicated to it. It benefits from a large amount of available training data, robust linguistic models, and a wide range of applications that have been optimized for English speakers. Additionally, the relatively standardized pronunciation and grammar of English compared to some other languages contribute to its suitability for speech recognition systems.
  • These limitations will be especially problematic in queries about prescriptions and medicines because if the speech recognition is incorrect or if the AI processing for the conversation is not adequate, then the device would provide incorrect information. To mitigate this issue, there is only a limited number of potential medicines it can refer to and they are all stored in the local database that the caretaker sets up at the beginning of the week or whenever there are changes in their prescription. These medicines are only the prescribed medicines to the user.
    • By limiting the number of medicines the system can recognise, there's less chance it will misunderstand what was being said. This helps ensure that the information it gives is accurate and specific to the user's prescribed medications. This approach not only makes the system more reliable but also makes it easier for users and caretakers to trust the information it provides. In addition to that, having fewer medicines to choose from makes it quicker to find the right info when needed, making the speech recognition system more helpful in medical situations.
  • As an initial countermeasure to this, the user can also check what the device has interpreted their speech as.
• Given the current capabilities of ChatGPT and other AI chat systems, it may not be reliable in offering precise information regarding specific medical inquiries. In instances where such questions arise, it's advisable to reach out to the caretaker instead, as they can offer personalized responses tailored to the individual's medical needs and circumstances. While AI technologies like ChatGPT can be helpful in various contexts, they may lack the expertise and contextual understanding required for accurate medical advice. The responses from the AI assistant will mainly have pre defined messages that has been set up by the caregiver (which they would get from the doctor or other reliable sources) and these can be things such as the instructions, basic descriptions of what it does and common side effects.
• Relying solely on AI for medical information could potentially lead to misunderstandings or incorrect guidance, highlighting the importance of seeking assistance from qualified caregivers who can provide tailored support and guidance based on their knowledge and experience.

The app

User Interface

Notification sounds

Ensuring seniors take their medication on time is crucial, especially in settings where loneliness is prevalent, such as closed care facilities. According to Luuc, who works in elderly care, traditional pillboxes are common among relatively independent seniors. These pillboxes are often filled weekly and placed in accessible locations, like the kitchen or a bedside table. Luuc suggests enhancing medication reminders with speech functions, making them more user-friendly for seniors. He emphasizes the importance of ensuring the reminders are easy to locate and difficult to misplace, particularly in closed care settings where confusion may lead to items being taken by other residents. Tine, another caregiver, agrees that loneliness is a significant issue among the elderly. Although she doesn't frequently discuss this with her clients, she acknowledges their frequent solitude in care settings with individual rooms and shared common areas. Both Luuc and Tine highlight the importance of family involvement in medication management. They suggest notification systems that update caregivers and family members on medication intake, providing support and reassurance. In summary, integrating speech reminders and family notifications into medication management systems can enhance seniors' adherence to their medication schedules while addressing feelings of loneliness and providing peace of mind to caregivers and family members.

Implementation with Open-AI

Open-AI was used both to implement the companion features of the app and the voice commands. This was achieved in the following manner:

Anything said to the app, or typed into the chat page, is sent to the Open-AI API to determine if it is one of the possible commands, e.g. "Reply only yes or no, is the following asking to add a medication?" with the message sent. If Open-AI determines that the input does not match any of the possible commands it is added to the companion conversation and a response is generated. If it is determined that it is a command, Open-AI is sent a set of follow up questions to extract relevant information needed to service the command, e.g. "Reply only with the name, What is the name of the medication mentioned in the following?". Since there is some inherent unreliability in the answers given by a large language model, the app checks if any of the information given to perform the command does not conform to the requirements needed to execute the command, and if so it asks the user to give that detail again, e.g. "I'm sorry, I didn't quite catch the name of the medication you wanted to add. Could you please repeat it?" at which point the same procedure is repeated again to extract the information required.

Disclaimer

There are various limitations in the current AI model that is used both for transcription and for conversing, thus we have added a disclaimer whenever the app is first opened to warn the users of the various inaccuracies of the technology

Speech To Text

STT was implemented using the Android SpeechRecognizer library. When the "VOICE ASSISTANCE" button is clicked on the home page the library is called to listen in for what is said. When the user stops talking the library returns what was said, at which point the result is analysed as described in the above section.

Text To Speech

TTS was implemented using the Android TextToSpeech library. Whenever the app speaks this library is called with the text to be spoken passed to it. This includes when a medication needs to be taken, when having a conversation with the companion AI via the STT and when clarification is asked for the details of a command that was issued by STT.

Electronic components

The reed relay switch used

For this project it was decided an Arduino Uno microcontroller board will be used due to its availability as well as perfect compatibility with our requirements. The microcontroller is responsible for keeping track of two factors. Firstly, it checks whether the lid of a compartment is open or closed, allowing for the monitoring of pill intake. Additionally, whenever a pill compartment is empty, the setup also detects the state change. After a change is observed, the microcontroller will communicate the information over bluetooth to the Android phone app. In order for all of those functions to work properly a choice of sensors as well as a bluetooth module had to be made.

A. Lid sensor

For this task a sensor, which detects whether the lid is closed or open had to be employed. At first a gyroscope was considered due to its availability in our case, however there are downsides to this choice. Gyroscopes can be susceptible to external factors such as vibration, shock, and electromagnetic interference, which might affect their accuracy and reliability in certain situations. Additionally a gyroscope is less energy efficient than some of the alternatives. This matters considering further iterations of our design could be portable and battery powered. After further research two types of sensor options were elected: a Hall sensor and a reed relay. Both of these sensors detect magnetic fields, which is perfect for our application as with an addition of a magnet to the lid, the change of state can be easily detected. As both of these components are similar in terms of availability, price and reliability, the choice was made based on their energy efficiency. A hall sensor requires a constant electrical circuit to operate and produce an output signal in the passive state. On the other hand a reed relay switch consumes no power and only operates in the presence of a magnetic field. For those reasons, our final choice is the reed relay.

B. Pill compartment sensor

The tcrt5000 infra-red sensor used

In order to keep track of whether the compartment is empty or not we had to consider a variety of possible solutions. The first type of sensor considered was the capacitive sensor. It could be placed at the bottom of the compartment and detect whether it is in contact with a pill. A major problem was encountered, however. Since the majority of pills on the market are non-conductive, only a very high accuracy sensor could be viable in this application. The availability and price of precise capacitive sensors is outside of our scope, so it was decided to abandon the idea. Another kind of sensor that could be used is a weight sensor. Similarly to our first idea, it turned out that weight sensors with enough precision for detecting the small weight of a pill or even multiple of them, were way too expensive for our project. After further research we decided on slightly changing the pillbox design in a way that will allow for the application of a regular infrared sensor. Making the pill compartment walls sloped makes it so that even with only one pill left it will be resting on top of the IR sensor. This way a state change is reported only once the compartment has been emptied.

C. Bluetooth module

Out of the Bluetooth modules the fit our requirements and budget, the HC-05 and HM-10 modules were the two best choices. The HC-05 was chosen over the other module even though both provided Bluetooth connectivity since it worked properly with the integrated phone application that was necessary for the project. In the end HC-05 module’s compatibility exceeded the HM-10’s benefits, even though the HM-10 has its advantages such as lower power consumption.

HC-05 bluetooth module

Prototyping

First iteration:

The first design was meant to serve as a test speciment. The idea was to have combined compartments in one detachable part, that would be inserted into the pill box. One big lid would be used to open and close the lid. At this stage electronics weren't yet considered, so the space for them was not designed with any dimensions in mind, simply as a reminder that they have to be there. This design was meant to be used as something to get feedback from interviews and professors on its strengths and weaknesses. With the help of the latter, the group had realized that a single detachable compartment was not a good idea, since it limits the user quite a lot in terms of how often they can refill the compartment. The lack of clear distinction between individual compartments could also lead to confusion about which pill should be taken.

First prototype

Cross-section of the first prototype

Isometric view

Second iteration:

Second prototype

Exploded view, second prototype

These prototypes are still a work-in-progress. As of end of week 4, the interview results as well as literature research should be analyzed and compiled together to define design specifications of the pill box. Currently, it is required to answer the following questions: how many pill compartments will be there? Should each compartment have a lid or not? Should LED sensors be used to highlight a specific compartment? and etc. Moreover, a safety mechanism should be introduced in order to make the product child-safe. Lastly, a compartment for the microcontroller and other necessary electronics has to be added.

The second prototype was a dive in a different direction, since the group tried to explore the idea of using separate compartments with separate lids. This would allow to track better when each of the compartment is open and which precisely. This would allow the software to have an accurate representation of what is happening in reality, for example whether or not user forgot to take the medicine or whether or not they took the right one.

Third iteration:

Isometric section view of the third prototype

Top view

Side section view of the prototype

In the final version, the idea of separate compartments had been refined. According to the feedback received, this was the right direction for the design of compartments. At this point, space for the electronics had also been implemented.

The dimensions became quite different from the previous prototypes - it is 278x131x43 mm (length, width, height). Previous prototypes were much smaller, about half or even one-third the size, which is explained by the lack of electronics space, lower number of compartents and less refined compartment design. The width is measured, if considering the side section view, from the leftmost part of the pillbox until the rightmost, where Arduino r3 controller is located. If considering just the pill box, it is 73 mm in width. This prototype is the closest to final design - it has 7 compartments (one per day) and each of them has a lid. Moreover, two of the compartments that serve as proof of concept, have space for a magnetic relay sensor and an IR sensor, that will be used to get information about lid opening/closing and pills being taken. There is some changes to be done regarding the electronics however, since the arduino "box" is designed just for the controller, not taking into account the future soldered wires. The shape of the pill compartment was chosen to be an obtuse triangle with a 100° angle and 12.5 mm height. The compartment itself is 35x36 mm. The low slope and the size of the compartment allows for an easy grab of any pill - a finger can be modelled as a sphere of up to 25 mm diameter. Thus, there is enough space to "insert" a finger inside and drag a pill out of the compartment thanks to the low slope, were the user can grab it with the second finger (if they couldnt do it already, because modelling a finger as a 25mm diameter sphere in most of the cases will be an overestimation of the size). Final changes will be added during week 6 and then the design will be ready for printing.

Each box will only be used for a single type of medicine instead of designating it for a specific time and day. This is for modularity, because it the user changes their prescription that requires 3 intakes per day, then there would be 3*7=21 compartments needed. The literature review had shown that the number of different types of pills is lower than the number of intakes per week. This way of sorting the pills makes the task of controlling the pills easier and allows the pill box to be more generally used, since now it is no longer dependend on the highly variable number of intakes per week. On the other hand, some pills are taken more often than others and a standardized size of the compartment may not be large enough to fit all of the pills of the same type that will be consumed during the week. In addition, the IR sensor is not really able to distinguish between multiple pills in one compartment. Due to this hardware limitation, there information about the number of pills taken from a compartment is not 100% accurate, which makes the software rely on he ability of the user to follow commands it gives precisely.

Last touches:

The group had an internal deadline to have the print ready at the week 6 to start the assembly. Some of the final touches implemented included making space for an IR sensor and a magnet (for the reed relay) in the box. The following images demonstrate these changes:

Changed design of the box, implementing space for both Reed Relay and IR sensor

Space for magnet in the lid, above space for reed relay

Next, the assembly was "sliced" (simulation of 3D printing process, splits the assembly into layers) in 3D printing software called "Cura". This allows to see an estimate of the printing time:

Sliced assembly, full version

Due to the size of the assembly, estimated print time, even with the fastest optimal settings, was at 14 hours, which proved to be an issue in the future. Due to the size of the assembly, estimated print time, even with the fastest optimal settings, was at 14 hours, which proved to be an issue in the future.

Assembly of Prototype

At the beginning of the week 6, the assembly was supposed to be printed out. However, at Proto/Zone at Tu/e a print time of up to 8 hours was allowed, due to other courses of Q3 requiring the 3D printers. Thus the assembly had to be changed in order to accommodate the restriction. Since the group was intending to show the functionality of the pill box only on 1-2 compartments, it was possible to get rid of other 5 "dummy" compartments as well as space for arduino, as these are secondary.

The images below demonstrates the submitted print job and the received printed parts:

Submitted assembly

Next, the printed parts were taken care of by getting rid of the brim (thin layer of plastic serving as a base for print) and supports (required to make parts of the assembly where a part of the layer is hovering, i.e is not built on top of another layer). Also, the axis was sanded down, as it came out too wide and would not fit through the holes.

Received printed parts

Interview 2

interview evaluation first prototype

1. How many pills do you take per time?
2. How many times per day do you take pills?
3. How often do you want / can you refill your medication?
4. Do you need to move the pillbox often? 1
5. Do the size of the box matter to you? If so, how big can de pillbox be at maximum?
6. Do you need a locate and find option in the pillbox?
7. How do you like to be reminded to take your medicine?
8. What features does the app really need to have?
9. Is there anything you would like to add to the pillbox or the app?

Response interview 2

Questions	1	2	3	4	5	6	7	8	9
Response 1	3 morning, 2 evening	2 times	once per week	No	no, doesn't matter	No	Yes	Only need to be able to check when i need to take my meds	No
Response 2	2 morning	1 times	more than 4 days	yes	yes, as easy to take with you as a water bottle	Yes	Yes	See the schedule, list of my medication with some information, see when I need to refill my box	No
Response 3	3 morning, 3 evening	2 times	once per week	Yes. once per week	is 14 bij 5 bij 2, have 2, not much bigger,	No	No, takes during brushing theeth	Already more information than he needs	No

Results

Demonstration:

Pictures of the finished product:

Picture of the closed pill box

Picture of the open pill box

Picture of the Electronics outside of the box

Picture of the back of the box

As it can be seen, the final pill box is of an orange color, which is due to re-printing of the box itself. This was necessary because the Reed Relay sensors, located at the front, would not fit into the designated space due to the wiring. The new box is 5 mm taller, and is printed of a orange PLA, which is an upside since it makes the pillbox much more noticeable for the user. The numbers at the front wall correspond to the number of a compartment, which is used in the software to indicate which pill should be taken.

The length and width of each compartment is about 3.5 cm and the depth is at about 1.5 cm. This allows for most of the pills, even the bigger ones, to easily be placed inside and the inclined surfaces allow to easily grab the pills, which had been tested afer assembly. At the bottom of each compartment there is an IR sensor which is, as mentioned in one of the previous chapters, used in order to determine whether or not a pill has been taken. The incline makes pills slide down to the bottom, right on top of the IR sensor, which decreases the chances of the sensor sending wrong outputs, such as "No pill present" when the pill is actually in the compartment. At the front of the compartment a Reed Relay is located, which when paired with a magnet, allows to distinguish between open and closed positions of the lid.

The lid itself is designed in a quite simple way. First the lids are put into their places on top of the compartments, and then a shaft is slid through the holes. The dimensions are fit in such a way that allows a semi-slide of the lid - if turned upside down, the lids arent going to open, their weight is not enough; instead a slight force should be applied to open them. The tip of the lid slightly overhangs the front wall of the box and has a shape that allows the user to easily get a grab with the tip of the finger and open it. However, due to inaccuracy in Reed Relay placement, in the assembled pill box the lids do not close completely, as shown in the pictures.

All of the sensors are connected from the back of the pill box. Initially, it was planned to have all of the wiring inside of the pill box, for safety, convenience and simply aesthetic reasons. However, due to the printing limitations at Proto/ZONE, this idea was not implemented into the real model - the Arduino together with breadboard take quite a lot of space and print time was already too long. Of course, in an actual product this would have to be addressed.

The following video demonstration firstly shows the process of adding medication to your schedule with use of speech-to-text capabilities of the app. Secondly the text-to-speech function for reading planned medication is shown. Lastly the mode of operation of the pill box itself is shown. When the app reminds the user to take medication, it stops after the compartment has been opened thanks to communicating the sensor states through bluetooth, which can be seen at the end in the application log.

The demonstration of the product is shown here: https://youtu.be/5F5EuKUZxCo

Limitations of the prototype:

• Voice recognition can only be so accurate
  • We noticed that using the phone's microphone to record the voices and using the SpeechRecognizer, it could only reliably transcribe a few of our group members. This is a limitation of the Speech to Text model and the quality of the phone's microphone.
• Giving medical advice
  • As mentioned previously, due to the fact that integrated ChatGPT is by no means certified to give medical advice and nor are the group members, there is no reliable information available in the software in order to be able to give such advices. This could be adressed by having an option for a doctor, who is treating the user, to add a sort of database for the medication that their patient is prescribed.
• Notifying third parties
  • Due to problems with debugging for other parts of the app, this was not possible in the time frame though this is a crucial aspect that needs to be implemented.
• User authentication
  • There’s no way to tell who’s taking the medicine, as other people might be able to take it.
  • Additionally, at the moment due to the limitation of the system, there is no way of the system knowing whose voice is who because of the voice recognition API that we used. This might be a possible improvement in the future as there might be other API that would have the additional functionality of profiling the recorded voice.
  • As an additional measure, there is the possibility of implementing passwords for adding/deleting prescriptions so that only the caregiver can have access to it in in case someone else other than the user and the caregiver wanting to change it randomly.
• Capacity of the pill compartments
  • Due to 3d printing time and cost for additional components, only 2 compartments were printed.
• Knowledge of the actual pill consumption is limited by the sensors
  • There is no way to actually tell how many pills has been taken due to sensor limitations. This could be fixed by adding a highly sensitive weight sensor, however it would be costly.

Discussion

Our journey in developing the pillbox prototype and companion app is made to address the medication management challenges confronting elderly individuals and those in the early stages of dementia. Our prototype is a pillbox sensor technology and AI-driven companion features, aimed at providing a solution that enhances medication adherence and promotes overall well-being.

Prototype Evolution: Our development process has been guided by the trough multiple interviews throughout the process to analyse the user needs, valuable insights from caregivers and healthcare professionals and extensive literature research. The current version of our prototype showcases notable enhancements, boasting a compact design with seven pill compartments, for which only 2 have been printed.

Sensor: The seamless integration of sensors, such as the reed relay and IR sensor, contributes to our success. These sensors facilitate precise lid opening and pill intake detection, thereby ensuring medication tracking. Additionally, in the future the incorporation of LED sensors looks promising in providing visual cues, thereby augmenting user experience and accessibility, particularly for individuals with visual impairments. Also, looking at ways to make the sensors smaller and lighter can improve user experience and portability.

Companion AI App: The companion AI app is a big part of the functionality of the pillbox, offering voice recognition capabilities and initiating engaging conversations with users. Leveraging state-of-the-art OpenAI technology, we set out to create interactive voice assistance that went a step further than conventional command-response assistance, like Siri and Google, by trying to implement meaningful engagement and emotional support for users.

Future Trajectory: Referencing the prototype based on user feedback and usability testing involves looking ahead. This includes improving sensors and adding an LED for better visibility. Also, improvements in the app are needed, the voice assistance needs to be trained better to give the user more meaningful interactions and support.

Conclusion

During the project, we have gained valuable insight into the caregiving industry. While we believe that the product we are designing can make a change in the industry and substitute the demand for caregivers, it is a complex task to integrate various devices to the point that it can replace most if not all the daily tasks of a caregiver. The speech to text limitations along with the limitations of the AI processing in conversations can already pose a large issue when asking for medication details and would also limit its functionality as a companion.

Our pillbox prototype and companion app aim to address medication management challenges for elderly individuals and those in the early stages of dementia. Through user interviews, insights from caregivers, and extensive literature research, our prototype has evolved to feature seven pill compartments, with sensors like reed relays and IR sensors facilitating precise medication tracking. Integration of LED sensors for visual cues is a great future step, along with efforts to make sensors smaller and lighter for improved portability. The companion AI app, powered by state-of-the-art OpenAI technology, offers voice recognition capabilities and engaging conversations to provide meaningful support. Plans involve refining the prototype based on user feedback and usability testing, including enhancements to sensors and voice assistance for better user interaction and support.

This product could alleviate the burden of this epidemic. Nevertheless, we strongly advocate for reforms to incentivize individuals to pursue caregiving positions. The primary beneficiaries of this product are patients in the early stages of dementia, which already presents a significant challenge. As the population continues to age, the proportion of individuals entering later stages of dementia will rise, increasing the demand for real caregivers.

Logbook

Week 1

Name	Hours	Tasks
Alonso Geesink Antón	6	Meeting (1 hr), Deliverables (30 min), Literature search (3 hr), Summarising (1.5 hr)
Satya Dharmawan	9	Meeting(1 hr), Deliverables (30 min), Literature search (3 hr), Summarising (1.5 hr), Brainstorming (3 hr)
Gabriël van Dijk	6	Meeting (1 hr) Deliverables (30 min), Literature search (3 hr) Summarising (1.5 hr)
Sander Erbrink	6	Meeting (1 hr), Deliverables (30 min), Literature search (3 hr), Summarising (1.5 hr)

Week 2

Name	Hours	Tasks
Alonso Geesink Antón	7	Feedback Session (30 min), Meetings (1.5 hr), Researching ChatGPT integration (2 hr), ChatGPT API tutorial and experimenting (4 hr)
Satya Dharmawan	7	Feedback Session (30 min), Meetings (1.5 hr), Researching ChatGPT integration and speech-text and text-speech (2 hr), Researching Android development (4hr)
Gabriël van Dijk	3.5	Feedback Session (30 min), Meetings (1.5 hr), Creating interview guidelines and contacting potential interviewees (1.5 hr)
Antoni Liberak	3.5	Meetings (1.5 hr), Literature search (1 hr), Researching electronic components (1 hr)
Sander Erbrink	3.5	Feedback Session (30 min), meeting (1.5 hr), literature search state of the art (30 min), looking into frontend development of a app (30 min), looking at 3d modeling (30 min)

Week 3

Name	Hours	Tasks
Alonso Geesink Antón	8	Feedback Session(30 min), Meeting (1 hr), Researching bluetooth connection to app (2 hr), Setting up app code project files and github (1 hr), Implementing bluetooth connection (3.5 hr)
Satya Dharmawan	13	Feedback Session (30 min), Researching bluetooth-android connection+ learning java (8 hr), researching parts for the sensors to be used and making a draft of the implementation for it (4hr), editing the blog (30 minutes)
Gabriël van Dijk	8	Feedback Session (30 min), Meeting (1 hr), Potential problems and (1.5 hr), Functional requirements (1.5 hr), Editing Interview (0.5 hr), Conducting Interview (0.5 hr), Transcribing Interview (2 hr), Updating Wiki (0.5 hr)
Antoni Liberak	8	Feedback Session (30 min), Meeting (1 hr), Researching sensors and other arduino components (4 hr), Testing Arduino (2 hr), Editing the wiki (30 min)
Sander Erbrink	10	Feedback session (30 min), Meeting (1 hr), State fo the art (30 min), Potantial problems (1.5 hr), Functional requirment (1.5 hr), UI in android studio (5 hr)
Valentin Nikushor	9	Getting acquainted with the project/reading the literature (3 hr), dealing with modelling software (1.5 hr), modelling the prototype (4.5 hr)

Week 4

Name	Hours	Tasks
Alonso Geesink Antón	9	Feedback Session (30 min), Meeting (1 hr), implementing activities and navigation between them (2.5 hr), bugfixing(1 hr), implementing TTS (1 hr), refactoring bluetooth as a service (1.5 hr), cleaning code(30 min), changing bluetooth and audio to constant instead of on button click & editing UI to reflect this (1 hr)
Satya Dharmawan	4	Feedback Session (30 min), Meeting (1), Creating the storyboard (3), editing the blog (30 minutes)
Gabriël van Dijk	14	Feedback + Meeting (1.5 hr), Storyboard (1 hr), Interviewing (1,5 hr ), transcribing and translating (2 hr), Gathering usefull information (1 hr), getting familliar with arduino (2 hr), learning c++ basics (4 hr), coding arduino (1 hr)
Antoni Liberak	9	Feedback + Meeting (1.5 hr), Choosing final sensors for ordering (4 hr), Arduino code (2 hr), editing wiki (30 min)
Sander Erbrink	9	Feedback + Meeting (1.5 hr), Storyboard (3 hr), making landscape mode for UI (3 hr), editing wiki (30 min), interviewing (1 hr)
Valentin Nikushor	11	Feedback session (0.5), Meeting (1), research and reading articles (5), modelling (3.5), wiki update (1)

Week 5

Name	Hours	Tasks
Alonso Geesink Antón	7	Feedback session (0.5), Meeting (1), Debugging (1), Helping others with coding (1), implementing database (3.5)
Satya Dharmawan	22.5	Meeting (1), Making the chatgpt app implementation (21), editing wiki (0.5)
Gabriël van Dijk	12	Meeting (1 hr), Literature (1 hr), Learning Java (2 hr), coding app file manager (8 hr)
Antoni Liberak	7	Feedback + Meeting (1.5 hr), Testing ordered sensors (4 hr), Arduino code (1 hr), editing wiki (30 min)
Sander Erbrink	12	Feedback session (0.5), Meeting (1), learning java (3 hr), working on frontend app (3 hr), working on backend app (4 hr), editing wiki (30 min)
Valentin Nikushor	8	Feedback + Meeting (2), prototyping (6)

Week 6

Name	Hours	Tasks
Alonso Geesink Antón	15	Feedback + meeting (2), ChatGPT implementation (2), implementing database (4), Implementing reminders (5), other backend tasks (2)
Satya Dharmawan	8	Feedback + meeting (2), ChatGPT implementation (2), reorganising and adding stuff on wiki (most still not uploaded yet) (4),
Gabriël van Dijk	8	Feedback + meeting (2 hr), building speech to text framework (6 hr)
Antoni Liberak	9	Feedback + meeting (2 hr), Writing working arduino code for the entire setup (3 hr), editing wiki (4 hr)
Sander Erbrink	7.5	Feedback + meeting (2 hr), Editing wiki (30 min), literature research (2 hr), Working on app (3 hr)
Valentin Nikushor	9.5	Feedback + Meeting (2), Prototyping (3), changes for print job sumbission (2), assembly reception and processing (2.5)

Week 7

Name	Hours	Tasks
Alonso Geesink Antón	20	Feedback + meeting(2), implementing schedule page (4), integrating chatGPT (4), bluetooth and related functions (6), speechToText integration (2), bug fixing (1), code cleaning (1)
Satya Dharmawan	14	Feedback + meeting(2), testing the app and the bluetooth (6), wiki (6)
Antoni Liberak	13	Feedback +meeting (2), Improving Arduino code (3), Working on the Bluetooth connection (5), editing wiki (3)
Sander Erbrink	10	Meeting (2 hr), working on the app (5 hr)
Valentin Nikushor	10	Meeting (2hr), modifications to model (6hr), model assembly (2hr)

Week 8

Name	Hours	Tasks
Alonso Geesink Antón	16	Testing the app and bluetooth (3), bug fixing (2), implementing showing of next med (2), improving robustness (2), code cleaning (1), implementing functions for processing speechToText result (5), wiki (1)
Satya Dharmawan	18	Meeting(2), testing the app and the bluetooth (4), wiki (12)
Antoni Liberak	13	Feedback + meeting (2), Fixing Bluetooth connection issues (6), Fine tuning Arduino code and electronic components for the pill-box (4), editing wiki (1)
Sander Erbrink		Meeting (2 hr), Editing wiki (1 hr),
Valentin Nikushor	10	Meeting (2hr), new print job submission and assembly (8hr)

References

Caregiver and Clinician Shortages in an Aging Nation

The nursing home industry experiences a higher turnover rate, attributed to comparatively lower wages and benefits, especially when compared to positions in hospitals. This is exacerbated by staffing shortages, leading to increased workloads and often necessitating staff members to take on multiple shifts in a day. Simultaneously, societal values are shifting towards individualism, contributing to the fragmentation of families due to factors like divorce or geographic distance. This trend results in a decline in the availability of "informal care providers," typically family members. The rise in nonmaternal caregivers, such as nannies or day-cares, not only weakens emotional bonds between parents and children but also contributes to a reduction in the pool of accessible informal care providers, further increasing the demand for nursing homes

Medication Adherence in Older Patients With Dementia: A Systematic Literature Review

This article conducts a thorough analysis of medication adherence among elderly individuals with dementia, encompassing adherence levels, influential factors, and strategies for intervention. The review consistently establishes a link between dementia/cognitive impairment and medication nonadherence in diverse studies. It emphasises factors like patient characteristics, disease conditions, and medication types as contributors to nonadherence. It highlights the necessity for customized interventions to enhance medication adherence in older individuals with dementia, emphasising the crucial role of caregiver support in this context.

The impact of resistance exercise on the cognitive function of the elderly

This research looks into how resistance exercise affects the health of older people, focusing on their physical and mental well-being. The results show that doing resistance exercise, whether at a moderate or high intensity, helps improve muscle strength, lean mass, mood, quality of life, and cognitive function in older individuals. Interestingly, moderate-intensity exercise seems better for boosting mood and certain cognitive aspects. Overall, this study emphasizes the positive impact of resistance exercise on the overall health of older adults, underscoring the importance of personalized exercise plans for this group

The Efficacy of Exercise as a Long-term Antidepressant in Elderly Subjects: A Randomized, Controlled Trial

This study discovered that exercise remains helpful for reducing depression even when people switch from supervised workouts to doing weight lifting on their own at home or in a gym. Even more than two years later, those who exercised were still less depressed than those who didn't. The research also found that people's confidence and mood got better with exercise, even when they did it without supervision for up to 20 weeks. After 26 months, about a third of those who initially exercised were still keeping up with their exercise routine. The study shows that exercise is quite effective, resolving depression in 73% of those who exercised, compared to 36% who didn't. It suggests that doing weight lifting on your own can be a good, long-term way to reduce depression in older adults.

Ability of older people with dementia or cognitive impairment to manage medicine regimens: a narrative review

This review highlights the substantial impact of cognitive impairment and dementia on the medication management skills of older individuals, emphasising the critical importance of evaluating their ability for self-administration through thorough assessments conducted by competent health professionals. Neglecting to address medication management skills in those with cognitive impairment could lead to heightened economic and healthcare burdens. Recognizing this potential consequence, the review urges immediate research into strategies for improving medicine-taking among individuals with cognitive impairment, emphasising the existing limitation in studies focused on this particular area.

The examination of the factors affecting the feeling of loneliness of the elderly

This study investigated the impact of a 12-week intervention program involving interactions with a seal robot on the well-being, mood, and loneliness of elderly residents in a nursing home. The findings indicated a notable reduction in loneliness scores among participants who engaged with the seal robot twice a week, in contrast to a control group. This observation is noteworthy, given that prior research on companion robots, such as Paro and AIBO, primarily emphasized mood enhancement without specifically addressing loneliness. The study suggests that companion robots, serving as substitutes for live animals, can effectively contribute to alleviating loneliness within nursing home environments.

Frontiers | Reducing Loneliness Among Aging Adults: The Roles of Personal Voice Assistants and Anthropomorphic Interactions

The research brings attention to a unique discovery indicating a direct link between anthropomorphism and a reduction in loneliness. Participants experienced a decline in perceived loneliness following the use of Personal Voice Assistants (PVAs), consistent with earlier studies that propose the effectiveness of technology interventions, such as communication tools and virtual pet companions, in addressing loneliness among older adults. The results underscore the significance of anthropomorphism in alleviating loneliness, particularly evident in relational greetings. Notably, baseline loneliness served as a predictor for such interactions, revealing that individuals experiencing higher levels of loneliness were more inclined to engage with the PVA, initiating personal greetings with polite language and adherence to interaction rituals. This behavior suggests a strong desire for connection, highlighting the role of anthropomorphism in fostering meaningful interactions with the device.

Designing Robots for Elderly from the Perspective of Potential End-Users: A Sociological Approach

This research looks to adress some issues for end-users regarding age-friendly robots. The main issues here were: How robots are envisioned by the elderly, how knowledgeable the elderly are on the robotics development, and how the elderly are involved in robot design. The research uses qualitative methods, notably interviews. Some interesting results were that most elderly rated themselves as capable enough regarding current technologies, or digital competence as the research calls it. However, during interviews most released that they in fact were no able to keep up with the new developments, often also unwilling to try. This study gives a good insight in how the elderly treat and see technology. Therefor it is quite important and interesting for our own research and creation of a product that is aimed at the elderly.

Robots for Elderly Care: Review, Multi-Criteria Optimization Model and Qualitative Case Study

For the first part this paper reviews the current knowledge about robots for the elderly, focusing on service and social robots. The review aims to highlight how robots interact with the elderly in healthcare and social situations. Also events like covid are considered. For the second part it focusses on a conceptual model. As goal it has to maximize efficiency and robot utilization and minimizing the stress for human caregivers. This is a optimisation conceptual model. It first builds on the ever expanding fields and level of robotic presence in society. It then tries to formulate a formula based on gathered data. Finally the study takes a look at the findings of other research papers regarding which needs, requirements and technological adaptations there are in the elderly care. They also used focus groups as part of their data set. Some of the more important findings were that using robots could help loneliness for example. Also that different people had different sets of expectations for robots, even for the robots appearance.

Socially Assistive Robots in Elderly Care: A Systematic Review into Effects and Effectiveness

This research focusses on robot intervention in social assistance for the elderly and its effectiveness. It takes data from previous researches. They took a lot of information from a previous meta study. To improve upon this they conducted thorough quality control on the used sources. They found that from all 2891 found publications, only 17 studies were really relevant to socially assistive robot care for the elderly. These used 5 different robots from which one was undefined. For the results from these different robots, they generally all seem to have various positive influences on the elderly they cared for. The researchers however did not that more intensive studies must be done on a larger scale to give properly confirm these findings.

Robots in Elderly Care

This study looks at robots in the elderly care. They state that longer life expectancy and lower birthrates result in a relatively larger elderly group. This larger group needs more care from less people, which is why robots in elderly care become more important, even more-so considering rising health care costs. First they divide the needs into some categories, such as daily activities, physical, cognitive, psychological and social needs. They then outline different robots for each job, and the rules they must adhere to (Isaac Asimov's Three Laws of Robotics). The researches look at previous studies on these robots and their impact. In their findings the negative effects are prominently discussed, as opposed to many other researches. Noteworthy worries were the huge data collection that could happen trough robots, and how reducing real human contact influences the elderly, for example by thinning the line between the reality and the appearance of robots. They do conclude that further development on care robots for the elderly must be pursued and that it could prove a very useful tool.

Assistive social robots in elderly care: a review.

This paper aims to check how effective assistive social robots are in the care for the elderly. To gather this information, a systematic review of previous literature was done. The robots are divided into two types: assistive robots that do supporting tasks for their patients, and social service robots which try to act more as a companion. They studied the same robots as other provided literatures, partly because there is a small selection to choose from. Studies were once again filtered and resulted in 43 relevant studies. The results are generally positive in all aspects, from mental to physical wellbeing. The researches did note some patterns that limited the significance of the results. For example: most research was done on two robots, Paro and Aibo, thus limiting how varied the use and capabilities of robots was. Also, most research was done in Japan. Furthermore the elderly lived in nursing homes and not on their own, and finally the methodology was lacking, as they had no control over the other studies and how they were performed/measured.

The use of care robots in aged care: A systematic review of argument-based ethics literature

This study explores the effects of resistance exercise on the physical and mental well-being of older individuals. The research findings suggest that engaging in resistance exercise, whether at moderate or high intensity, leads to improvements in muscle strength, lean mass, mood, quality of life, and cognitive function among older adults. Notably, moderate-intensity exercise appears particularly beneficial for enhancing mood and certain cognitive aspects. The study underscores the importance of personalized exercise plans tailored to the needs of older adults to promote their overall health and cognitive function.

Ethical challenges in the use of social service robots for elderly people

The use of socially assistive robots (SARs) for supporting elderly individuals in care and domestic settings raises significant ethical concerns. Key issues include privacy and data protection, safety and responsibility, involvement of vulnerable individuals, and deception. To address these challenges, ethical principles such as nonmaleficience, beneficence, autonomy, and fairness need to be applied to robotics. While clear answers may not yet be available for every ethical dilemma, the development of ethical guidelines for SAR deployment and research is crucial. This article suggests practical measures from a robotics project to navigate these ethical considerations effectively

Granny and the robots: ethical issues in robot care for the elderly

As the elderly population grows and robotics advances, the use of robots in elder care becomes increasingly feasible. They explore various applications of robots in assisting the elderly and their caregivers, monitoring health and safety, and providing companionship. While recognizing potential benefits, they address six primary ethical concerns, including reduced human contact, feelings of objectification, loss of privacy and personal liberty, deception, infantilization, and control over robots. By weighing care benefits against ethical considerations, they conclude that introducing robots with careful planning and guidelines could enhance the lives of the elderly, fostering independence and facilitating social interaction.

Service robots, care ethics, and design

The rapid growth of service robots in various sectors demands ethical attention in their design and implementation. This paper introduces the Care Centered Value Sensitive Design (CCVSD) approach, originally designed for healthcare robots, as a framework for evaluating personal and professional service robots. By integrating care ethics principles, CCVSD aims to systematically incorporate ethical considerations into robot design. The paper discusses the necessity for ethical evaluation of service robots, presents the CCVSD approach, and explores its applicability beyond healthcare. Through examples of current robot prototypes, the paper illustrates how CCVSD can be used to assess service robots based on their integration into care practices. Ultimately, the paper advocates for proactive ethical evaluation in robot design to address the ethical challenges arising from their increasing presence in daily life.

Emotional Attachment to AI Companions and European Law

This paper discusses ethical issues surrounding the use of AI companions for the elderly, and how EU law currently deals with these. It primarily discusses the potential risks of emotional dependence, giving harmful advice/saying harmful things, harming the users relationships, and amplifying problematic social dynamics. The paper contends that EU law currently deals with these risks in 2 major ways: first in a preventative way by establishing a framework of risk from which AI systems are regulated accordingly and secondly through broad liability laws that mean that the maker of one of these companions does not have to be at fault to be considered liable, and that contracts do not invalidate liability. The potential issue of privacy is then discussed and how the EU currently aims to tackle this through the GDPR, although it is considered by the author to have significant limitations due to a lack of informed consent.

AI Companion Robot Data Sharing: Preferences of an Online Cohort and Policy Implications

This article highlights how current policy surrounding AI companions and privacy seldom takes into account the elderly, which is one of the most vulnerable groups, and discusses how future policy could address this. The researchers use surveys to show that old people are less comfortable with the use and storage of their data by these AI companions and that most want more transparency by companies on how their data is being used, as well as heavier enforcement of privacy laws by the government. It is further contended that the elderly are particularly vulnerable due to their lower familiarity with modern technology and higher prevalence of cognitive issues making it easier to obtain consent that was not properly informed.

‘He knows when you are sleeping’ – Privacy and the personal robot companion

This paper examines privacy concerns surrounding personal robot companions by conducting an exploratory study with 12 participants. It aims to understand user attitudes towards data collection and privacy safeguards in personal robot companions. Participants were exposed to scenarios involving interactions with a robot companion, simulating conversations about personal habits. Findings indicate a trade-off between the perceived utility of data collection and concerns about information disclosure. Participants expressed discomfort with the idea of robots storing personal information, citing risks of unintentional disclosure and emphasizing the need for security measures. Cultural backgrounds influenced attitudes towards privacy, with differences noted in perceptions of data ownership and sharing. Overall, the study highlights the importance of addressing privacy concerns in the development of personal robot companions to ensure user acceptance and adoption.

User Centered Design of a Pill Dispenser for the Elderly

The paper discusses the increasing challenge of patient multimorbidity, particularly among the elderly population, driven by demographic changes in Germany and Europe. With the aging population, there's a rising risk of medication intake among older individuals, with a significant portion already taking multiple medications. However, studies indicate a high rate of incorrect or missed medication doses among patients with chronic diseases. Managing medications correctly is crucial for maintaining independence and quality of life in old age. To address this, research has focused on developing age-appropriate pill dispensers through a user-centered design approach including user surveys, expert evaluations, and design concept variations. Findings highlighted key requirements for pill dispensers, including reliability, long-life functionality, ease of use, and non-stigmatizing design. Based on user feedback and expert evaluation, a modular pill dispenser system with mobile daily units was selected for further development.

Is a big button interface enough for elderly users? Towards user interface guidelines for elderly users

This report provides guidelines for improving software performance for elderly users, considering their physical and mental declines. Designing for elderly users requires simplifying interactions and reducing complexity. Key suggestions include removing rarely used functions, reducing items per page, and avoiding complex interactions. Clear task structures are essential, with one key performing one function and one page dedicated to one task. Consistency in information presentation, rapid feedback, and user support are crucial. Interface optimization involves proper sizing of components, avoiding scroll bars, presenting text simply, and using real object-like interfaces. Customizability, both for disabilities and remotely, is vital to accommodate the diverse needs of elderly users. These guidelines aim to enhance the usability and accessibility of software for the elderly population.

Users' Perceptions of an in-Home Electronic Medication Dispensing System: A Qualitative Study

The article examines challenges in medication adherence among older adults, particularly due to complex regimens, and the role of medication dispensing technology. It highlights research showing the potential of such technology to improve adherence and health outcomes. However, challenges like cost and lifestyle compatibility persist. The study aims to understand users' perspectives on a specific home-based medication dispensing system for individuals with chronic conditions living at home.

Older Adults’ Opinion on Social Robot as Companion

The article examines older adults' attitudes toward social robots, particularly among those aged 85 and older, who are underrepresented in research. Through questionnaires, demonstrations, and focus group discussions, the study found that acceptance of social robots as companions was influenced by participants' exposure and interaction with the technology, especially among early and middle-of-the-road adopters. The findings highlight the importance of considering older adults' experiences and attitudes when designing and implementing social robots for improving quality of life in aging populations.