PRE2015 3 Groep5

The FoodFetcher

---

Introduction

The FoodFetcher is a system that collects information on what kind of food you have in stock and based upon this information it orders new groceries. The FoodFetcher operates through an app in which the user can put a standard list of groceries that need to be bought every x amount of time. The app also keeps track of what groceries are in stock and suggest a meal of the day based upon it. This supports a healthy lifestyle and tries to let the user eat various different cuisines.

Group members

Buster Franken 0912005 b.franken@student.tue.nl

Dayo Oladapo 0908184 d.oladapo@student.tue.nl

Jari van Steen 0914013 j.j.v.steen@student.nl

Jesper van de Grift 0961129 j.v.d.grift@student.tue.nl

Joost Pieterse 0848231 N/A

Michael Verhage 0848768 m.verhage@student.tue.nl

State of the Art

Attempts have been made to produce a smart fridge for various purposes:

• sdf

Users

Primary Users

Homeowners

These are the user(s) that possess a room where an intelligent system, that can track the user(s) groceries, is installed. They determine, primary, which products are used, throw away used products and pay the bills of said products. They interact with secondary and tertiary users

Implications

The most important implications for the primary user are that it is both easier to get your food since you don't have to go to the supermarket anymore to look for what you want to have, and you also run out of a product less easy since the FoodFetcher keeps track of your needs and adjusts its ordering mechanism to this. This will make it a lot easier for handicapped people or older people who don't find it that easy to go all the way to the supermarket, to get everything they want. It is also good for people who just don't like shopping or forget about it, or people who forget to buy some things when they are at the supermarket. The FoodFetcher will make the life of these people easier.

Needs

The primary user has a lot of needs. First of all, it needs a way of accessing all of the products so it can determine what he wants without fysically going to a supermarket or something like that. The user also needs a way of accepting products that it has ordered. The interaction with the FoodFetcher system should be easy as well for the main user, so he will not make any misorders for example. The user should also not get unwanted products because the system incorrectly thought the user would like them. The instructions on how the FoodFetcher works should be very clear to make communication with the FoodFetcher system even easier. The system also needs to be built in in existing homes, so that you don't have to redesign your whole kitchen for a FoodFetcher in your own home. At last, it is also necessary that the user can pay for their groceries.

Secondary Users

Family members & Friends

These are the people who might come over to the homeowner's house where they want to interact with the FoodFetcher. They should know how the system works and can interact with it via the homeowner's mobile app.

Implications

Right now, if an elderly of handicaped person is not anymore capable of going to the supermarket by him- or herself, his or her family members usually have to take over the job by shopping for them, even without perfectly knowing all of their preferences sometimes. One of the implications is that family members of elderly people do not have to shop for those people anymore if they can't shop for themselves, they can look it up for themselves what they want on the FoodFetcher app and then have it delivered. The possibility to pay together for foods and drinks may also come with the foodfetcher system, since people can order their own beers when they hang out for example. Besides that, food and drinks will be available for family and friends as the foodfetcher system can keep these kinds of things in mind and order items that your friends like, but are used up.

Needs

One of the needs for the family and friends of the homeowner, is that they can be able to access the foodfetcher on their own mobile phone outside the home of the homeowner, since this would enable them to help ordering food for people who can't do this themselves that easily, for example dementing people. However, this is not the most important need. Another need is the need for sharing in pay on the foodfetcher. For example, when you order the ingredients for a meal, it would be nice if everyone could pay for their own share.

Food distribution companies

The food distribution companies are the link between food producers and the food ordered by the FoodFetcher. They sell the products too the FoodFetcher (homeowner) and give order for delivery (either done by themselves or by an external delivery company).

Implications

For the food distribution companies, a lot changes when the foodfetcher will be used by many people. There will be a change in generating revenue and the type of jobs will also change. Instead of restockers, most employees will be delivery boys. There is also the opportunity to implement advertisements for certain food types. If there are two brands that sell the same product, the foodfetcher can suggest one of the two that has paid for advertising.

Needs

The needs for the food distribution companies are first of all enough people and materials to supply people with enough groceries, since you don't want people to wait an entire day for their groceries. The other important need is feedback that groceries are indeed delivered.

FoodFetcher producers

This is the company that designs, produces, sells and take feedback on the FoodFetcher. It is responsible for the FoodFetcher’s quality and workings.

Tertiary Users

Employees of grocery shops

The employees might face a change in job responsibilities in the grocery shop market. People’s physical shop buying habit might change into digital buying through the FoodFetcher. This could result in either job loss or occupation change of grocery shop workers.

Advertisement companies

The FoodFetcher may be a perfect platform for directed personal advertisement. A product can be advertised through the mobile app, by any advertiser. This can mean that a personalized shopping list may be offered to the primary user (homeowner). Advertisement companies can use the FoodFetcher for revenue.

Delivery of products

When an order is placed by the FoodFetcher, the products need to be shipped from the distribution companies. This delivery can be done either by standard delivery companies or a new service can be designed.

Maintainers of the FoodFetcher

When the FoodFetcher experiences failure or bugs it is important that the problem gets fixed in a short amount of time. Any food product contained by the FoodFetcher might go bad if the problem is not fixed. An external maintenance company might take this role.

Government

The FoodFetcher can provide a platform for directed personal health and lifestyle campaigns. Also any new system must adhere to regulations in order to be safe for use and can be produced without causing problems.

Implicatons

The first implication for the government is that there is a possibility for a healthy food campaign by making the foodfetcher propose healthy meals instead of junkfood. With the growing number of overweight people, this may be a really smart thing to do. Besides that, the government will get taxation income from the selling of the foodfetcher and products through the foodfetcher.

Needs

The job of the government is mainly to keep everything under the control. To achieve this, regulations will need to be put up, for example to clear up what happens if a person buys food and does't get it delivered at all. Advertisements will also need to be regulated in order to not make it go out of control. Also, trying to make the system hack-proof is something that the government is part of, so making the system as hack-proof as possible is also a need.

Scenario

What would it be like in a not to distant future with the FoodFetcher implemented?

A phone rings in the cubicle of a busy office. The man sitting in the cubicle answers “Hello?” his mind still with his work. On the other side of the line a woman's soft voice says; ‘Honey, have you thought about Valentines for tonight?’ The man’s heart rate rises, his mind runs down all the options. Restaurants are full and he didn’t make reservations. He mutters; ‘Err, sure honey...’ She says; ‘Oh you are so thoughtful, my mother was wrong in saying I shouldn’t marry you.’ Sweat runs down the man’s face.. But suddenly, he remembers. FoodFetchers. The man answers to his love ‘I am going to cook diner, just for the two of us’ He grabs his smartphone and fires up the FoodFetchers app. In the menu under the heading suggestions an extra option appeared for Valentines, as happens with every holiday. The man’s slowly steadying finger taps the option and waits while the data from his fridge and pantry at home is retrieved. Normally the man would use the daily FoodFetchers-diner-suggestion that sends a push-message with a diner suggestion at 12 o'clock. But today he was so busy he forgot to check his phone.

A notification appears, -FoodFetchers has made a selection for Valentines-diners based on what you’ve already got at home- The man rejoices. He can always trust his personal FoodFetcher-system. ‘Are you still there?’ His love asks. ‘Yes, I was.. just distracted, I’ll be home in a bit.. Love you’ ‘Love you to’ *click*. On the drive home the man sees a supermarket and recalls.. that just two years ago he and so many others, used to frequent this now almost abandoned building, daily. Most supermarkets now have changed layout and turned their shop employees into delivery boys and girls. It is understandable. FoodFetchers is so easy. It coming with standard food-stocking settings and downloadable upgrades. Even the man’s mother didn’t have to go to an elderly home because reheatable food-packages are now delivered daily. FoodFetchers even monitores for a variated and healthy eating habit.

The man pulls his car onto his driveway and sighs in relief, thinking; ‘with the little one on the way, I don’t know how I’d ever do without FoodFetchers. How did mý parents manage this..’

Prototype

The prototype of the FoodFetcher is what we want to fysically build within the few following weeks. Here, some specifications are given.

Requirements

These are the main requirements for the prototype of the FoodFetcher, derrived from the user needs. Without these requirement, the system would not function. Each requirement contains a fixed action it has to be able to do to make the system work, which can be tested on whether it works or not.

• The foodfetcher needs to be able to scan which products are in stock in order to know how much is in the inventorty.
• The foodfetcher needs to be able to communicate with the user through the FoodFetcher app.
• The foodfetcher needs to be able to take into account user preferences about food, so that the user gets what hee wants.
• The foodfetcher needs to be able to order products and have a way of accepting products so that it is possible to retrieve a product after ordering it.
• The foodfetcher needs to be built in in current homes; it won't cost too much in that case.
• The foodfetcher needs to to know how full a product in a package is, for example, there is a difference between a half-full pack of milk and a full pack of milk.

Preferences

These preferences also come directly from the user needs, however, they aren't required to make a working product. Besides, these preferences do not have a specific quantity they need to have, they have to be fulfilled as good as possible. They are not less important however, since a user-unfriendly product will not be tolerated.

• The interaction with the FoodFetcher system should be as easy as possible.
• All the functionalities need to be as clear as possible through instructions.
• There should be the least amount of work that needs to be done by the user to make the system work, the system should work autonomously.
• The system has to be compatible with as many kitchens as possible
• The system should be as difficult as possible to hack.
• The system should be as cheap as possible.

Hardware

• Pressure plates: to detect a product and how much it weighs
• Label on the product: to be able to scan what product it is and what expiration date it has
• Moving light sensor: to detect the label on the products
• Arduino: to control and use the hardware
• Raspberry pi: to control the software (in the prototype this will be a laptop)

Software

We eventually want to have an app in which we can keep track of every product in our kitchen. We want our app to do the following:

• Giving an overview of products you have
• A list which tells us to remove an expired product
• The expiration dates of the products in your kitchen

To manage our app, we are planning to use the systems as stated below:

• ZXing library android in order to scan the barcodes on the labels
• Outpan API to retrieve the information of a product (unfortunately incomplete)
• zinc.io API to order food from amazon UK (future)

Hardware update

First we will design a prototype. This will be to test the proof of concept. The prototype will consist of a basis, supporting the pressure plates and food products, and a android phone camera. The prototype will enclose all the necessary sensors and hardware in order to test the scale and workings of the system.

The casing

The casing is designed to hold 3 Interlink Electronics Force Sensitive Resistors (FSR); model 406. These are spaced so that each FSR can detect one milk carton at a time. Milk cartons being our chosen food product for the prototype. Furthermore the casing consist of a lid that can be opened and closed in order to place and remove the milk cartons. The FSR have a longer connection strip than the width of the basis. This is due to the fact that the FSR’s connections steps cannot be shortened for this prototype because this will reduce their sensing capabilities. The casing cannot be any wider for than a milk carton can be placed, by the user, besides a pressure plate resulting in a loss of data measurement (force). Therefore some of the strip will extend beneath the door lid, the FSR’S will also be connected to jumper cables on a breadboard. The casing will have closed side walls in order to compensate for the forces produced on top by the moving camera. The back wall can be partially open without structural problems. Furthermore a closed container is placed at the side of the casing wall. It contains the Arduino board and connecting cables. It is also the place where the power cable can be attached too.

In order to produce (first round estimate) dimensions for the casing, several milk cartons have been measured. The following (mean) data is acquired (100% error analysis):

Caption: Milk Carton Dimension Data

Milk Carton Type	Width ±0,4 (cm)	Length ±0,4 (cm)	Height± 0,4 (cm)	Perturbing Parts ±0,4 (cm)
A	7,0	9,5	24,5	Lid height: 1,0	Diameter: 2,5
B	7,2	7,2	23.5	Lid height: 1,0	Diameter: 2,5
C	9,0	6,1	19,5	Lid Height: 0,9	Diameter: 2,5

Pressure plates system

The pressure plate system will consist of three FSR Interlink Electronics 406. Only two will be installed in the prototype because of budget control. These FSR's will be placed on the basis of the tray in the prototype. For early testing a breadboard setup will be used.

Hardware and software list

Before going into detail is perhaps good to start with the parts used in the subsequent paragraphs:

• FSR model 406 (2x)
• Breadboard (1x)
• Jumper cables (multiple)
• Arduino UNO (1x)
• USB cable (1x)
• Resistors (2x for each R (Ohm) value)
• Laptop (1x)

Also software is important to achieve a working prototype. Therefore the following software is used:

Controlling the arduino and acquiring sensor (FSR) data:

• Arduino IDE

Sending the sensor (FSR) data to the laptop for processing:

• CoolTerm

Analyzing the acquired sensor (FSR) data:

• Origin

Electronic Schematic software:

• Fritzing

The FSR 406

As said the pressure plates will consist of so called Force Sensitive Resistors (FSR). These are resistors that can change their resistance by influence of an external force, unlike solid resistors which have a fixed resistance (e.g. 10 kOhm). The FSR consist of a conductive polymer which can change it’s inherit resistance by application of a force to its surface area. The FSR come in various shapes and sizes, but most common are flat FSR’s. In our prototype we use square and flat FSR’s of model type 406 made by Interlink Electronics. See figure [..] for an image. Each polymer sheet is covered with a very thin film of both electrically and non-electrically components. This components are produced in a square grid, also known as a matrix. The primary task of these components is protecting the polymer sheet. Secondary tasks include improvement of mechanical properties like bendability and stability and increasing the surface friction. It also protects the polymer sheet against temperature changes. When the sheet undergoes a force, its outer film conductive components get squeezed against the polymer sheet, changing the resistance of the film. This change can be detected when a voltage is applied across the sheet, usually in the range of 3.3-5V. This voltage is delivered by external electronical components like the Arduino board. A simple schematic can be used to build a force sensitive system. In our prototype we use an Arduino Uno and a so called voltage divider (see section [..]). A complete overview of the FSR 406, including dimensions can be seen in figure [7.3-1].

Immediately after turn-on, the resistance decreases very rapidly. At slightly higher and then intermediate forces, the resistance follows an inverse power law. At the high forces the response eventually saturates to a point where increases in force yield little or no decrease in resistance. Saturation can be pushed higher by spreading the applied force over a larger actuator. www.interlinkelectronics.com

• 

  FSR 406 specifications

• 

  FSR 406 picture, comparisson to coin

• 

  Voltage divider setup

Calibration of the FSR 406

Theoretical weight calibration

The calibration will be done in the following way. We will again be using a voltage divider setup. Instead of having to calculate the conductance from the variable resistance R_FSR a simple voltage-force conversion will be examined. The measuring resistor, R₁, is chosen to maximize the desired force sensitivity range and to limit current. Multiple type of resistors are availible: 10 kOhm, 8,1 kOhm and [...]. Several weights of variaying mass (g) will be used in this calibration. The problem is that it is very important that the place of measurment remains the same throughout the calibration process. In order to maximize the accuracy the following performance optimization will be used:

1. We will provide a consistent force distribution on each FSR. The FSR is very sensitive to force distribution across its surface. Therefore we will use "dead weights" like metal blocks in each measurement to ensure cycle-to-cycle accuracy and repetability. Also a small polymer surface like an elastomer can absorb any errors in force distribution.

2. We will maintain the same sort of measuring material throughout the calibration process. This will reduce the response error of the FSR. Therefore we will for example not be using a force provided by a human finger.

3. We will maintain the same actuation cycle time in each measurement consistent. The FSR shows time dependency in the resistance of the applied force (drift). Therefore in each cycle of increasing measurent weight the same actuation time will be used.

After this performance optimization has been carefully considered the data needs to be examinded. A repeatable and reproducible system has been estabilished and it is time for data acquisition. We will record the output voltage for several weights, whereby the software Origin will be used for analysing the data. It is possible to plot a curve through the acquired data points in order to produce a mathematical relationship making the the output voltage (V) a function of force (g). If necessary we can calculate the force (g) to Newtons.

A possible calibration result is provided by Interlink Electronics and can function as a benchmark/comparison. The following graph is taken from Interlink Electronics FSR intergration guide/Measurement techniques.

It can be clearly seen, from the figure above, that a different behaviour in the resistance-force curve family can be acquired for different fixed resistor R₁ values. This results in different senstivity ranges, to be determined in our case for our prototype. Again the steep "cut-off" voltage can be seen when a small load (force) is applied. This is as expected.

Experimental calibration: setup and results

Weight calibration

The calibration was done according to the specified procedure above. Several measurements where done with different metal weights of varying mass. We had access to the following weights: • 33,8 gram • 50 gram • 200 gram ‘small’ • 200 gram ‘large’ • 500 gram ‘small’ • 500 gram ‘large’

Because the size of the weight matters when calibrating the FSR it was necessary to use larger weights: from 200 gram, the weights were classified in a small metal cylinder and large metal cylinders. The smaller size weights were sufficiently small to not influence the measurement. We measured with increments of 33,8 and 50 grams, starting with a zero measurement increasing to 600 grams. Each measurement was captured using the Arduino and FSR setup as mentioned above. It became clear both from constructing the calibration setup and the integration guide, specified by Interlink Electronics, that a soft supporting material was needed between a mass and the FSR surface. If this precaution where to be for sacked it was very likely that damage to the FSR would occur. Therefore we choose to use foam like filler that came with shipping our electronic components. It was found that this material offered sufficient damping and protection capabilities while keeping direct interaction between mass and FSR surface minimal.

Each increment of increasing mass placement followed and increase in voltage, as we expected from concideration in the paragaph above (theoretical calibration). The manual also introduced to concept of time actuation of the measurement (also known as ‘drift’) as explained above. During the experiments it became clear that this drift was sufficient to interfere with accurate measurements. The time before a steady voltage measurement occurred with some masses was long, in the order of a up to minute. It was most strong when very light weights where used: the 38,8 and 50,0 gram. When heavier weights where used the drift time became less notable: in the range of ~30-40 seconds. In order to investigate the drift time the following experiment was done.

The following data graph was acquired showing the relationship between mass [g] and voltage [mV] of the FSR.

For the 10 KOhm resitor the mass-voltage data was aquired to show that there excists a discrepancy between each FSR unit measurement results. This is as expected because no two FSR are the same. The polymers inside the FSR differ in distribution and quality from unit to unit. Therefore it is important that both FSRs are measured and analysed in the cailbration process. In the graph above this is shown only for the 10KOhm resistor R. Clearly for higher mass the discrepancy becomes notable. Also the smaller resistor value of 5 KOhm, shows a lower voltage curve, as is expected. Also a different value for the 5,1 KOHm resistor can be seen for the masses 200g and 33,8g. This is due to the fact that because of "drift" problems the measurement gave a wrong result. See next section for an overview of drift problems. The 3 KOhm resistor was not calibrated for reasons described in the next section.

Drift experiment

Several "pull-down" resistors R_n where available with the following resitance:

• n=1: 3 KOhm
• n=2: 5,1 KOhm
• n=3: 10KOhm
• n=4: 12KOhm

With each of these resistors a voltage divider setup was made (see [..]). The FSR was placed in this setup and connected to the Arduino Uno. Again several weights where available:

• 33,8 g #1
• 200 g #2
• 400 g #3

The drift experiment was conducted in the following way. At first no load (pressure) was applied to the FSR giving a zero-measurement of the voltage [mV] value; as expected. Now by placing the smallest weight, number 1, on the FSR surface area an increase in voltage could be detected. The drift in the FSR caused the voltage to rise for a certain amount of time [s] before becoming a steady value. The time between mass placement and steady-voltage value was measured for each weight (#m) and each resistor R_n. This experiment gave the following results:

Resistor Rn [KOhm]	Weight [g]	Drift time [s]	Voltage v [mV]
3	33,8	59	288
3	200	100	2956
3	400	110	3768
5,1	33,8	74	1622
5,1	200	69	3548
5,1	400	51	4213
10	33,8	50	2140
10	200	32	3792
10	400	21	4511
12	33,8	38	2605
12	200	31	4535
12	400	23	4535

From the table above it can be clearly seen that the drift time decreases when the load increases. This is easy to understand as the force acting on the FSR surface is greater for heavier weights, and thus there is little room fo deformation over time in the polymer areas of the FSR surface. Only when the resistor R_n=3 becomes too small, the drift time increases when the weight increases. This is due to the fact that the voltage V is very low, and the FSR resistance very high. It is more difficult to establish a good reading by the Arduino. Also the pull-down capablities of R_n=3 become almost none. Concluding from the data above, it becomes clear that the setup requires a fixed R_n resistor as high as possible, and therefore we will use the 12KOhm (n=4) resistor in our protopype setup for both voltage dividers.

The electronic setup

Each pressure plate will be connected to the Arduino UNO via a voltage divider setup (see image above.) This allows for measurement of the change in voltage when a load is applied to the FSR. The resistance of the FSR will decrease as more pressure is applied to the FSR, therefore increasing the voltage (and current). The fixed resistor R₁ will take a role as a pull-down resistor. This pull-down resistor R₁ is connected to the ground and ensures that logic signal is low when no load is applied on the FSR. A breadboard is used to allow for simple construction of the setup. It can also be easily modified and removed. A final prototype should hold a more reliable system like a PCB board. This is not included in this prototype version however.

The divider needs a ground connection delivered by the Arduino board: Digital Ground (DGND). The +5V voltage will be delivered by the Arduino also and is connected to one FSR pin. The other FSR pin is connected to an analog pin (A0-A2) on the Arduino. This is where the data is read from. Furthermore a 10 kΩm resistor R₁ as a pull-down resistor. So for the earlier mentioned two fsr setup; two voltage dividers are needed. The following figure shows this setup. The FSR in this figure are shown as round, the 406 model used in the prototype are square.(This is due to a limited Frizting software library)

• 

  Schematic electronical setup of the pressure plate system using Frizting software. Two FSR's are shown each connected to a voltage divider and the Arduino Uno Board

• 

  Picture of breadboard electronic setup

• 

  Picture of breadboard electronic setup

Mathematical consideration

Theoretical model

The mathematics describing the model for a voltage divider is quit straightforward. Each voltage divider "divides" the voltage between the two resistors: one fixed R₁ (Ohm) and one variable R_FSR (Ohm). This can be mathematically expressed as:

[math]\displaystyle{ V_{out}= V_{in}*{R_1\over R_1+R_{FSR}} }[/math] [V]

whereby V_out (V) is the voltage supplied over the resistance R₁. V_in (V) being the +5V supplied by the Arduino Uno Board. R₁ is the fixed resistance which is chosen to be 10kOhm. The resistor R_FSR is the variable resistor FSR. When no load (pressure) is applied to the FSR the resitance will be extremely high (~infinite). This is the same situation of having an open circuit, thus no voltage can be detected. In this situation the pressure plate software (see next chapter) knows that no load is applied. When a load of increasing weight is applied the resistance will drop significantly. At 100 N load the resistance will be a mere ~250 ohm. Looking at the equation above it is clear that in this case the voltage detected is very close to 5V (but still less!).

The calculation of the force (in Newton) cannot (or atleast for this preliminary results) be direclty taken from the equation above. Instead this has to be releated to the conductivity and calibration results delivered by the manufacterer of the FSR 406 (Interlink Electronics). We plan on doing the calibration later on ourselves too ensure even better accuracy as each FSR differs slighty from the specified calibration. But for now this specified data suffices. In our first force approximation we will use the conductance C:

[math]\displaystyle{ C= {1\over R_{FSR}} }[/math] [Ohm^-1]

in which the resistance R_FSR is calculated with equation [..] resulting in:

[math]\displaystyle{ R_{FSR} = R_{1}({{V_{in}\over V_{out}}-1}) }[/math] [Ohm]

As said we will use data delivered by the manufacter. In this case it is not directly the capacitance but the resistance R (ohm). See the graph below which is taken from the FSR integration guide from Interlink Electronics.

Looking at the graph above it becomes clear that indeed the resistance decreases very rapidly when a force is applied. After the initial steep decrease, the curve seems to be, in good approximation, lineary decreasing in this double log plot. We will approximate the force (F)-capacitance (C) curve in two domains (which is the inverse from the graph above):

If C<1000 => [math]\displaystyle{ F= {C \over 80} }[/math] [N]

else => [math]\displaystyle{ F= {{C-1000}\over 30} }[/math] [N]

This will be replaced with the Voltage V_out- Force(g) curve calibration as described in paragraph [..] but for preliminary results it is acceptable.

Calibration model

The calibration gave a mass-voltage curve from which a mathematical relation can be obtained. The curve used can be seen in the previous section [...] and is plotted below with a logarithmic fitting curve.

File

The fitting was done with the analysing software package OriginLab 20115. The fittning of the data gave the following mathematical relation:

[math]\displaystyle{ voltage = a*log (-b*ln(m)) }[/math]

Software update

Mobile application software

For now, the software has been written partly in the program Android Studio. This program mixes Java-code for programming and xml for the layout of the app and some other things. Different screens in the app represent different activities. These activities are kept seperately to assure that in case of errors, the entire code does not need to be checked for bugs. It is a so-called failsafe and has many more advantages over using one single activity. In the app, the following activities are currently used:

• A main activity with buttons linking to different activities
• A way of scanning bracodes
• A list where you can see your current inventory, together with the expiration date.
• The list where you can remove items for your inventory.
• An activity where you can see what items can be ordered and where you can actually order those items.
• A shopping list where you can see what items you have ordered.

The layout of the main activity is right now simple, but effective. Creating a beautiful layout is not something that has a priority right now, but will be something for in the future. It currently looks like this:

One note that has to be made is that there is a button for the suggestions, but there is no activity for this yet. Of course, this will be fixed later, but for now, we have not yet done this.

The barcode scanning activity is used when a product is placed into a fridge or other cabinet. Every product has a QR-code placed on top of it, which can be scanned by simply pressing the button on the mainscreen and facing the QR-code with the camera. In this QR-code, information is given about the type of product and its expiration date, which is done by using the ZXing library. Once the product is scanned, it is automatically added to the inventory.

The inventory list is opened when this button is clicked. It looks like this:

As said earlier, this inventory activity displays not only which products are in your fridge, but also what its expiration date is. Besides that, to make the user's life easier, a picture of each item is added in your inventory.

There was some discussion in the group about what to do with the removal of items, the 2 options are to either scan the product upon removal of the pressure sensor, or when the product is scanned for the second time. After some discussion, it was decided that it is the best from the user's perspective to scan for a second time. Even though this seems weird, the alternative is that the user is not allowed to change the position of any other item in the cabinet or fridge. This is not user-friendly at all, especially in someone's fridge. When someone takes away a product in the fridge, he usually lifts a lot of products with this and places them back. This would cause the system to not know which product is removed. We think that the user can appreciate a robust product over a product that works half of the time, but with more effort that has to be done.

Once a product is removed from the fridge, it is put on the remove-list. These products will be removed automatically after a certain amount of time, and besides that, the products can be removed manually when the remove-activity is opened from the mainscreen.

Items can also be added to a shopping list through an activity. Different types of products can be added in different amounts and your shopping list, which can be viewed through a seperate activity.

We are currently working on a way of communication between the arduino and a phone. This will make sure that the app also knows how much of a certain item is still left, since the arduino is connected to pressure sensors. Once this is implemented, the amount left of a product will be shown in the inventory activity.

Once this thing is implemented, we want to continue on the suggestions of the app and the automatic creation of a shopping list based on your preferences and what you have in your fridge. These are the next few steps in the software.

Prototype software

The arduino software has also been written. It can measure data from a pressure sensor, calculate the force belonging to this input and send this to a text-file.

The Arduino software has multiple purposes; it controls the analog pins on the Arduino UNO boad, and provides the +5V voltage to the FSR's. By reading the voltage value of the analog pin it is possible to calculate the resistance (and therefore force) as explained in the previous chapter. An if-else statement is used to display this calculated force; if there is no detection of a voltage the software knows no load (pressure) is applied of the FSR's. Else, if a load is applied on either of the two FSR's, this is translated into a force (N) value to the user.

[code]

• int fsrPin1 = 0; // the FSR_1 and 10K pulldown_1 are connected to a0.
• int fsrPin2 = 1; // the FSR_2 and 10K pulldown_2 are connected to a1.
• int fsrReading1; // the analog reading from the FSR_1 resistor divider.
• int fsrReading2; // the analog reading from the FSR_2 resistor divider.
• int fsrVoltage1; // the analog reading_1 converted to voltage.
• int fsrVoltage2; // the analog reading_2 converted to voltage.

• unsigned long fsrResistance1; // The voltage_1 converted to resistance_1, can be very big so make "long".
• unsigned long fsrResistance2; // The voltage_2 converted to resistance_2, can be very big so make "long".

• unsigned long fsrConductance1;
• unsigned long fsrConductance2;

• long fsrForce1; // Finally, the resistance_1 converted to force
• long fsrForce2; // Finally, the resistance_2 converted to force

void setup(void) {

 Serial.begin(9600);   // We'll send debugging information via the Serial monitor

}

void loop(void) {

 fsrReading1 = analogRead(fsrPin1);  
 Serial.print("Analog reading_1 = ");
 Serial.println(fsrReading1);

 // analog voltage_1 reading ranges from about 0 to 1023 which maps to 0V to 5V (= 5000mV)
 fsrVoltage1 = map(fsrReading1, 0, 1023, 0, 5000);
 Serial.print("Voltage reading FSR 1 in mV = ");
 Serial.println(fsrVoltage1);  

 if (fsrVoltage1 == 0) {
   Serial.println("No pressure on FSR 1 detected");  
 } else {
   // The voltage1 = Vcc * R / (R + FSR) where R = 10K and Vcc = 5V
   // so FSR = ((Vcc - V) * R) / V        yay math!
   fsrResistance1 = 5000 - fsrVoltage1;     // fsrVoltage_1 is in millivolts so 5V = 5000mV
   fsrResistance1 *= 10000;                // 10K resistor
   fsrResistance1 /= fsrVoltage1;
   Serial.print("FSR resistance FSR 1 in ohms = ");
   Serial.println(fsrResistance1);

   fsrConductance1 = 1000000;           // we measure in micromhos so 
   fsrConductance1 /= fsrResistance1;
   Serial.print("Conductance FSR 1 in microMhos: ");
   Serial.println(fsrConductance1);

   // Use the two FSR guide graphs to approximate the force
   if (fsrConductance1 <= 1000) {
     fsrForce1 = fsrConductance1 / 80;
     Serial.print("Force FSR 1 in Newtons: ");
     Serial.println(fsrForce1);      
   } else {
     fsrForce1 = fsrConductance1 - 1000;
     fsrForce1 /= 30;
     Serial.print("Force FSR 1 in Newtons: ");
     Serial.println(fsrForce1);            
   }
 }
 Serial.println("---------------");
 delay(1000);

 fsrReading2 = analogRead(fsrPin2);  
 Serial.print("Analog reading_2 = ");
 Serial.println(fsrReading2);

// analog voltage_2 reading ranges from about 0 to 1023 which maps to 0V to 5V (= 5000mV)

 fsrVoltage2 = map(fsrReading2, 0, 1023, 0, 5000);
 Serial.print("Voltage reading FSR 2 in mV = ");
 Serial.println(fsrVoltage2);  

 if (fsrVoltage2 == 0) {
   Serial.println("No pressure on FSR 2 detected");  
 } else {
   // The voltage2 = Vcc * R / (R + FSR) where R = 10K and Vcc = 5V
   // so FSR = ((Vcc - V) * R) / V        yay math!
   fsrResistance2 = 5000 - fsrVoltage2;     // fsrVoltage_2 is in millivolts so 5V = 5000mV
   fsrResistance2 *= 10000;                // 10K resistor
   fsrResistance2 /= fsrVoltage2;
   Serial.print("FSR resistance FSR 2 in ohms = ");
   Serial.println(fsrResistance2);

   fsrConductance2 = 1000000;           // we measure in micromhos so 
   fsrConductance2/= fsrResistance2;
   Serial.print("Conductance FSR 2 in microMhos: ");
   Serial.println(fsrConductance2);

   // Use the two FSR guide graphs to approximate the force
   if (fsrConductance2 <= 1000) {
     fsrForce2 = fsrConductance2 / 80;
     Serial.print("Force FSR 2 in Newtons: ");
     Serial.println(fsrForce2);      
   } else {
     fsrForce2 = fsrConductance2 - 1000;
     fsrForce2 /= 30;
     Serial.print("Force FSR 2 in Newtons: ");
     Serial.println(fsrForce2);            
   }
 }
 Serial.println("---------------");
 delay(1000);

}

[/code]

The above code can maintain two FSR's. The code will subsequently measure the output voltage of FSR 1 and FSR 2 each 1000ms. This value is ofcourse arbitrary at this stage. The data is tranported to the laptop from the Adruino UNO board in a .txt file, by the software called TermCool (see partlist). Each data acquisition is time stamped which may come in handy for further data analysis in the mobile app. [The code above is based on the guide: "Using an FSR" by Adana Fruit company]

Prototype Walkthrough

A user perspective prototype walkthrough

In this text a complete user perspective walkthrough of the prototype is given

Hardware: scanning product → placing product → pressure sensor → position product determined → sticker/code recognition → “spoiled milk dataset” AND “milk left dataset” (“product database”) → write to rasp. pi [structures data] → ethernet cable to modem → computer on wifi reads rasp. pi structured data (via shared devices on home network)

Our story starts with placing, for example, a carton of milk in your fridge, after scanning your product. This carton of milk is placed on a random spot on a FoodFetcher-tray with pressure sensors. These pressure sensors send a signal

Future improvements

Of course we want our system to work as fine as possible and we want to be as user friendly as possible. However there are some improvements we would like to see in our final product which we can't imply now. The following lists states those functions.

• Suggestions for a recipe if, for example, some products expire soonly and you (onviously) don't want to throw them away
• Camera moving on both the X plane and the Y plane to be able to scan more products
• Removing products which are expired
• Order products you want or need and pay for them by using the app