PRE2015 3 Groep5

The FoodFetcher

---

Introduction

The FoodFetcher is a system that collects information on what kind of food you have in stock and based upon this information it orders new groceries. The FoodFetcher operates through an app in which the user can put a standard list of groceries that need to be bought every x amount of time. The app also keeps track of what groceries are in stock and suggest a meal of the day based upon it. This supports a healthy lifestyle and tries to let the user eat various different cuisines.

Group members

Buster Franken 0912005 b.franken@student.tue.nl

Dayo Oladapo 0908184 d.oladapo@student.tue.nl

Jari van Steen 0914013 j.j.v.steen@student.nl

Jesper van de Grift 0961129 j.v.d.grift@student.tue.nl

Joost Pieterse 0848231 N/A

Michael Verhage 0848768 m.verhage@student.tue.nl

State of the Art

Attempts have been made to produce a smart fridge for various purposes:

• sdf

Users

Primary Users

Homeowners

These are the user(s) that possess a room where an intelligent system, that can track the user(s) groceries, is installed. They determine, primary, which products are used, throw away used products and pay the bills of said products. They interact with secondary and tertiary users

Implications

The most important implications for the primary user are that it is both easier to get your food since you don't have to go to the supermarket anymore to look for what you want to have, and you also run out of a product less easy since the FoodFetcher keeps track of your needs and adjusts its ordering mechanism to this. This will make it a lot easier for handicapped people or older people who don't find it that easy to go all the way to the supermarket, to get everything they want. It is also good for people who just don't like shopping or forget about it, or people who forget to buy some things when they are at the supermarket. The FoodFetcher will make the life of these people easier.

Needs

The primary user has a lot of needs. First of all, it needs a way of accessing all of the products so it can determine what he wants without fysically going to a supermarket or something like that. The user also needs a way of accepting products that it has ordered. The interaction with the FoodFetcher system should be easy as well for the main user, so he will not make any misorders for example. The user should also not get unwanted products because the system incorrectly thought the user would like them. The instructions on how the FoodFetcher works should be very clear to make communication with the FoodFetcher system even easier. The system also needs to be built in in existing homes, so that you don't have to redesign your whole kitchen for a FoodFetcher in your own home. At last, it is also necessary that the user can pay for their groceries.

Secondary Users

Family members & Friends

These are the people who might come over to the homeowner's house where they want to interact with the FoodFetcher. They should know how the system works and can interact with it via the homeowner's mobile app.

Implications

Right now, if an elderly of handicaped person is not anymore capable of going to the supermarket by him- or herself, his or her family members usually have to take over the job by shopping for them, even without perfectly knowing all of their preferences sometimes. One of the implications is that family members of elderly people do not have to shop for those people anymore if they can't shop for themselves, they can look it up for themselves what they want on the FoodFetcher app and then have it delivered. The possibility to pay together for foods and drinks may also come with the foodfetcher system, since people can order their own beers when they hang out for example. Besides that, food and drinks will be available for family and friends as the foodfetcher system can keep these kinds of things in mind and order items that your friends like, but are used up.

Needs

One of the needs for the family and friends of the homeowner, is that they can be able to access the foodfetcher on their own mobile phone outside the home of the homeowner, since this would enable them to help ordering food for people who can't do this themselves that easily, for example dementing people. However, this is not the most important need. Another need is the need for sharing in pay on the foodfetcher. For example, when you order the ingredients for a meal, it would be nice if everyone could pay for their own share.

Food distribution companies

The food distribution companies are the link between food producers and the food ordered by the FoodFetcher. They sell the products too the FoodFetcher (homeowner) and give order for delivery (either done by themselves or by an external delivery company).

Implications

For the food distribution companies, a lot changes when the foodfetcher will be used by many people. There will be a change in generating revenue and the type of jobs will also change. Instead of restockers, most employees will be delivery boys. There is also the opportunity to implement advertisements for certain food types. If there are two brands that sell the same product, the foodfetcher can suggest one of the two that has paid for advertising.

Needs

The needs for the food distribution companies are first of all enough people and materials to supply people with enough groceries, since you don't want people to wait an entire day for their groceries. The other important need is feedback that groceries are indeed delivered.

FoodFetcher producers

This is the company that designs, produces, sells and take feedback on the FoodFetcher. It is responsible for the FoodFetcher’s quality and workings.

Tertiary Users

Employees of grocery shops

The employees might face a change in job responsibilities in the grocery shop market. People’s physical shop buying habit might change into digital buying through the FoodFetcher. This could result in either job loss or occupation change of grocery shop workers.

Advertisement companies

The FoodFetcher may be a perfect platform for directed personal advertisement. A product can be advertised through the mobile app, by any advertiser. This can mean that a personalized shopping list may be offered to the primary user (homeowner). Advertisement companies can use the FoodFetcher for revenue.

Delivery of products

When an order is placed by the FoodFetcher, the products need to be shipped from the distribution companies. This delivery can be done either by standard delivery companies or a new service can be designed.

Maintainers of the FoodFetcher

When the FoodFetcher experiences failure or bugs it is important that the problem gets fixed in a short amount of time. Any food product contained by the FoodFetcher might go bad if the problem is not fixed. An external maintenance company might take this role.

Government

The FoodFetcher can provide a platform for directed personal health and lifestyle campaigns. Also any new system must adhere to regulations in order to be safe for use and can be produced without causing problems.

Implicatons

The first implication for the government is that there is a possibility for a healthy food campaign by making the foodfetcher propose healthy meals instead of junkfood. With the growing number of overweight people, this may be a really smart thing to do. Besides that, the government will get taxation income from the selling of the foodfetcher and products through the foodfetcher.

Needs

The job of the government is mainly to keep everything under the control. To achieve this, regulations will need to be put up, for example to clear up what happens if a person buys food and does't get it delivered at all. Advertisements will also need to be regulated in order to not make it go out of control. Also, trying to make the system hack-proof is something that the government is part of, so making the system as hack-proof as possible is also a need.

Scenario

What would it be like in a not to distant future with the FoodFetcher implemented?

A phone rings in the cubicle of a busy office. The man sitting in the cubicle answers “Hello?” his mind still with his work. On the other side of the line a woman's soft voice says; ‘Honey, have you thought about Valentines for tonight?’ The man’s heart rate rises, his mind runs down all the options. Restaurants are full and he didn’t make reservations. He mutters; ‘Err, sure honey...’ She says; ‘Oh you are so thoughtful, my mother was wrong in saying I shouldn’t marry you.’ Sweat runs down the man’s face.. But suddenly, he remembers. FoodFetchers. The man answers to his love ‘I am going to cook diner, just for the two of us’ He grabs his smartphone and fires up the FoodFetchers app. In the menu under the heading suggestions an extra option appeared for Valentines, as happens with every holiday. The man’s slowly steadying finger taps the option and waits while the data from his fridge and pantry at home is retrieved. Normally the man would use the daily FoodFetchers-diner-suggestion that sends a push-message with a diner suggestion at 12 o'clock. But today he was so busy he forgot to check his phone.

A notification appears, -FoodFetchers has made a selection for Valentines-diners based on what you’ve already got at home- The man rejoices. He can always trust his personal FoodFetcher-system. ‘Are you still there?’ His love asks. ‘Yes, I was.. just distracted, I’ll be home in a bit.. Love you’ ‘Love you to’ *click*. On the drive home the man sees a supermarket and recalls.. that just two years ago he and so many others, used to frequent this now almost abandoned building, daily. Most supermarkets now have changed layout and turned their shop employees into delivery boys and girls. It is understandable. FoodFetchers is so easy. It coming with standard food-stocking settings and downloadable upgrades. Even the man’s mother didn’t have to go to an elderly home because reheatable food-packages are now delivered daily. FoodFetchers even monitores for a variated and healthy eating habit.

The man pulls his car onto his driveway and sighs in relief, thinking; ‘with the little one on the way, I don’t know how I’d ever do without FoodFetchers. How did mý parents manage this..’

Prototype

The prototype of the FoodFetcher is what we want to fysically build within the few following weeks. Here, some specifications are given.

Requirements

These are the main requirements for the prototype of the FoodFetcher, derrived from the user needs. Without these requirement, the system would not function. Each requirement contains a fixed action it has to be able to do to make the system work, which can be tested on whether it works or not.

• The foodfetcher needs to be able to scan which products are in stock in order to know how much is in the inventorty.
• The foodfetcher needs to be able to communicate with the user through the FoodFetcher app.
• The foodfetcher needs to be able to take into account user preferences about food, so that the user gets what hee wants.
• The foodfetcher needs to be able to order products and have a way of accepting products so that it is possible to retrieve a product after ordering it.
• The foodfetcher needs to be built in in current homes; it won't cost too much in that case.
• The foodfetcher needs to to know how full a product in a package is, for example, there is a difference between a half-full pack of milk and a full pack of milk.

Preferences

These preferences also come directly from the user needs, however, they aren't required to make a working product. Besides, these preferences do not have a specific quantity they need to have, they have to be fulfilled as good as possible. They are not less important however, since a user-unfriendly product will not be tolerated.

• The interaction with the FoodFetcher system should be as easy as possible.
• All the functionalities need to be as clear as possible through instructions.
• There should be the least amount of work that needs to be done by the user to make the system work, the system should work autonomously.
• The system has to be compatible with as many kitchens as possible
• The system should be as difficult as possible to hack.
• The system should be as cheap as possible.

Hardware

• Pressure plates: to detect a product and how much it weighs
• Label on the product: to be able to scan what product it is and what expiration date it has
• Moving light sensor: to detect the label on the products
• Arduino: to control and use the hardware
• Raspberry pi: to control the software

Software

We eventually want to have an app in which we can keep track of every product in our kitchen. We want our app to do the following:

• Giving an overview of products you have
• A list which tells us to remove an expired product
• The expiration dates of the products in your kitchen

To manage our app, we are planning to use the systems as stated below:

• ZXing library android in order to scan the barcodes on the labels
• Outpan API to retrieve the information of a product (unfortunately incomplete)
• zinc.io API to order food from amazon UK (future)

Hardware update

First we will design a prototype. This will be to test the proof of concept. The prototype will consist of a basis, supporting the pressure plates and food products, and a guiding system; holding the camera and servos. The prototype will enclose all the necessary sensors and hardware in order to test the scale and workings of the system.

The casing

The casing is designed to hold 3 Interlink Electronics Force Sensitive Resistors (FSR); model 406. These are spaced so that each FSR can detect one milk carton at a time. Milk cartons being our chosen food product for the prototype. Furthermore the casing consist of a lid that can be opened and closed in order to place and remove the milk cartons. The FSR have a longer connection strip than the width of the basis. This is due to the fact that the FSR’s connections steps cannot be shortened for this prototype because this will reduce their sensing capabilities. The casing cannot be any wider for than a milk carton can be placed, by the user, besides a pressure plate resulting in a loss of data measurement (force). Therefore some of the strip will extend beneath the door lid, the FSR’S will also be connected to jumper cables on a breadboard. The casing will have closed side walls in order to compensate for the forces produced on top by the moving camera. The back wall can be partially open without structural problems. Furthermore a closed container is placed at the side of the casing wall. It contains the Arduino board and connecting cables. It is also the place where the power cable can be attached too.

In order to produce (first round estimate) dimensions for the casing, several milk cartons have been measured. The following (mean) data is acquired (100% error analysis):

Caption: Milk Carton Dimension Data

Milk Carton Type	Width ±0,4 (cm)	Length ±0,4 (cm)	Height± 0,4 (cm)	Perturbing Parts ±0,4 (cm)
A	7,0	9,5	24,5	Lid height: 1,0	Diameter: 2,5
B	7,2	7,2	23.5	Lid height: 1,0	Diameter: 2,5
C	9,0	6,1	19,5	Lid Height: 0,9	Diameter: 2,5

Casing Dimensions

The milk carton dimension are used to define the dimensions of the basis. The largest carton is set as mean; type A. Furthermore the basis is taken ±10, 0 cm higher than type A in order to account for placement movements. In the final prototype this may need to be changed. The basis has the following dimensions:

Caption: Milk Carton Dimension Data

Width (cm) without Arduino holder	Length (cm)	Height (cm)	Width (cm) with Arduino holder
30,0	10,0	34,05	38,0
Lid Height (cm)	Lid Width (cm)	Lid Depth (cm)
5,0	28,0	2,1

Pressure plates system

The pressure plate system will consist of three FSR Interlink Electronics 406. Only two will be installed in the prototype because of budget control. These FSR's will be placed on the basis of the Tray. Each will be placed [..] cm in-between (measured edge to edge of FSR). And [..] cm measured from the back wall to the edge. The outer FSR’s are placed [..] to each wall (measured edge-wall) See following figure:

The FSR 406

As said the pressure plates will consist of so called Force Sensitive Resistors (FSR). These are resistors that can change their resistance by influence of an external force, unlike solid resistors which have a fixed resistance (e.g. 10 kOhm). The FSR consist of a conductive polymer which can change it’s inherit resistance by application of a force to its surface area. The FSR come in various shapes and sizes, but most common are flat FSR’s. In our prototype we use square and flat FSR’s of model type 406 made by Interlink Electronics. See figure [..] for an image. Each polymer sheet is covered with a very thin film of both electrically and non-electrically components. This components are produced in a square grid, also known as a matrix. The primary task of these components is protecting the polymer sheet. Secondary tasks include improvement of mechanical properties like bendability and stability and increasing the surface friction. It also protects the polymer sheet against temperature changes. When the sheet undergoes a force, its outer film conductive components get squeezed against the polymer sheet, changing the resistance of the film. This change can be detected when a voltage is applied across the sheet, usually in the range of 3.3-5V. This voltage is delivered by external electronical components like the Arduino board. A simple schematic can be used to build a force sensitive system. In our prototype we use an Arduino Uno and a so called voltage divider (see section [..]). A complete overview of the FSR 406, including dimensions can be seen in figure [7.3-1].

• 

  FSR 406 specifications

• 

  FSR 406 picture, comparisson to coin

• 

  voltage divider setup

Each pressure plate will be connected to the Arduino via a voltage divider setup (see image above.) This allows for measurement of the change in voltage when a load is applied to the FSR. The resistance of the FSR will decrease as more pressure is applied to the FSR, therefore increasing the voltage (and current). The fixed resistor (R) will take a role as a pull-down resistor. This pull-down resistor R is connected to the ground and ensures that logic signal is low when no load is applied on the FSR. A breadboard is used to allow for simple construction of the setup. It can also be easily modified and removed. A final prototype should hold a more reliable system like a PCB board. This is not included in this prototype version.

The divider needs a ground connection delivered by the Arduino board: Digital Ground (DGND). The +5V voltage will be delivered by the Arduino also and is connected to one FSR pin. The other FSR pin is connected to an analog pin (A0-A2) on the Arduino. This is where the data is read from. Furthermore a 10 kΩm resistor (R) as a pull-down resistor. So for the earlier mentioned two fsr setup; two voltage dividers are needed. The following figure shows this setup. The FSR in this figure are shown as round, the 406 model used in the prototype are square.(This is due to a software liberary)

Software update

For now, the software has been written partly in the program Android Studio. The interface is clear and items can be added to a shopping list. We also created a function to recognize a qr-code or barcode with a camera using the ZXing library. These functions have now been combined into one working app. The QR-codes that are scanned refer to products that automatically appear in your inventory. This barcode unmber is sent to the outpan API, which will give some information about the product. Unfortunately, the outpan API is not very complete: it has information about 3 of the 20 products we tested. The arduino software has also been written. It can measure data from a pressure sensor, calculate the force belonging to this input and send this to a text-file. It is work in progress to use this data in the app to know what is in your inventory.

Prototype Walkthrough

A user perspective prototype walkthrough

In this text a complete user perspective walkthrough of the prototype is given

Hardware: placing product → pressure sensor → position product determined → camera approaches on rails [until position given by pressure sensor (standard positions)] → sticker (colour) recognition → “spoiled milk dataset” AND “milk left dataset” (“product database”) → write to rasp. pi [structures data] → ethernet cable to modem → computer on wifi reads rasp. pi structured data (via shared devices on home network)

Our story starts with placing, for example, a carton of milk in your fridge. This carton of milk is placed on a random spot on a FoodFetcher-tray with pressure sensors. These pressure sensors send a signal to the color-detector which

Future improvements

Of course we want our system to work as fine as possible and we want to be as user friendly as possible. However there are some improvements we would like to see in our final product which we can't imply now. The following lists states those functions.

• Suggestions for a recipe if, for example, some products expire soonly and you (onviously) don't want to throw them away
• Camera moving on both the X plane and the Y plane to be able to scan more products
• Removing products which are expired
• Order products you want or need and pay for them by using the app