PRE2015 3 Groep5

The FoodFetcher

---

Introduction

The FoodFetcher is a home system that monitors what and how much food is in stock and their expiration dates with use of pressure mats, a scanner and receipt information. The FoodFetcher app communicates this information to the user and checks with a personal food profile of preferences, history and health information to give dinner and store-order suggestions. FoodFetcher has the potential to not only save you a lot of time, but also make you eat healthy in a fun and befitting way.

Group members

Buster Franken 0912005 b.franken@student.tue.nl

Dayo Oladapo 0908184 d.oladapo@student.tue.nl

Jari van Steen 0914013 j.j.v.steen@student.nl

Jesper van de Grift 0961129 j.v.d.grift@student.tue.nl

Joost Pieterse 0848231 N/A

Michael Verhage 0848768 m.verhage@student.tue.nl

State of the Art

Every seperate component of the FoodFetcher has been made before, but never has it been combined in this manner. Components for example are food ordering apps or the attempts that have been made to produce a smart fridge for various purposes, but all are total systems and very expensive. Examples of existing systems are:

• Albert Hein food order app
• Samsung smart fridge
• Recipe suggestion websites and food blogs

Users

Primary Users

Homeowners

These are the user(s) that possess a room where an intelligent system, that can track the user(s) groceries, is installed. They determine, primary, which products are used, throw away used products and pay the bills of said products. They interact with secondary and tertiary users

Implications The most important implications for the primary user are that it is both easier to get your food since you don't have to go to the supermarket anymore to look for what you want to have, and you also run out of a product less easy since the FoodFetcher keeps track of your needs and adjusts its ordering mechanism to this. This will make it a lot easier for handicapped people or older people who don't find it that easy to go all the way to the supermarket, to get everything they want. It is also good for people who just don't like shopping or forget about it, or people who forget to buy some things when they are at the supermarket. The FoodFetcher will make the life of these people easier.

Needs The primary user has a lot of needs. First of all, it needs a way of accessing all of the products so it can determine what he wants without fysically going to a supermarket or something like that. The user also needs a way of accepting products that it has ordered. The interaction with the FoodFetcher system should be easy as well for the main user, so he will not make any misorders for example. The user should also not get unwanted products because the system incorrectly thought the user would like them. The instructions on how the FoodFetcher works should be very clear to make communication with the FoodFetcher system even easier. The system also needs to be built in in existing homes, so that you don't have to redesign your whole kitchen for a FoodFetcher in your own home. At last, it is also necessary that the user can pay for their groceries.

Secondary Users

Family members & Friends

These are the people who might come over to the homeowner's house where they want to interact with the FoodFetcher. They should know how the system works and can interact with it via the homeowner's mobile app.

Implications Right now, if an elderly of handicaped person is not anymore capable of going to the supermarket by him- or herself, his or her family members usually have to take over the job by shopping for them, even without perfectly knowing all of their preferences sometimes. One of the implications is that family members of elderly people do not have to shop for those people anymore if they can't shop for themselves, they can look it up for themselves what they want on the FoodFetcher app and then have it delivered. The possibility to pay together for foods and drinks may also come with the foodfetcher system, since people can order their own beers when they hang out for example. Besides that, food and drinks will be available for family and friends as the foodfetcher system can keep these kinds of things in mind and order items that your friends like, but are used up.

Needs One of the needs for the family and friends of the homeowner, is that they can be able to access the foodfetcher on their own mobile phone outside the home of the homeowner, since this would enable them to help ordering food for people who can't do this themselves that easily, for example dementing people. However, this is not the most important need. Another need is the need for sharing in pay on the foodfetcher. For example, when you order the ingredients for a meal, it would be nice if everyone could pay for their own share.

Food distribution companies

The food distribution companies are the link between food producers and the food ordered by the FoodFetcher. They sell the products too the FoodFetcher (homeowner) and give order for delivery (either done by themselves or by an external delivery company).

Implications For the food distribution companies, a lot changes when the foodfetcher will be used by many people. There will be a change in generating revenue and the type of jobs will also change. Instead of restockers, most employees will be delivery boys. There is also the opportunity to implement advertisements for certain food types. If there are two brands that sell the same product, the foodfetcher can suggest one of the two that has paid for advertising.

Needs The needs for the food distribution companies are first of all enough people and materials to supply people with enough groceries, since you don't want people to wait an entire day for their groceries. The other important need is feedback that groceries are indeed delivered.

FoodFetcher producers

This is the company that designs, produces, sells and take feedback on the FoodFetcher. It is responsible for the FoodFetcher’s quality and workings.

Tertiary Users

Employees of grocery shops

The employees might face a change in job responsibilities in the grocery shop market. People’s physical shop buying habit might change into digital buying through the FoodFetcher. This could result in either job loss or occupation change of grocery shop workers.

Advertisement companies

The FoodFetcher may be a perfect platform for directed personal advertisement. A product can be advertised through the mobile app, by any advertiser. This can mean that a personalized shopping list may be offered to the primary user (homeowner). Advertisement companies can use the FoodFetcher for revenue.

Delivery of products

When an order is placed by the FoodFetcher, the products need to be shipped from the distribution companies. This delivery can be done either by standard delivery companies or a new service can be designed.

Maintainers of the FoodFetcher

When the FoodFetcher experiences failure or bugs it is important that the problem gets fixed in a short amount of time. Any food product contained by the FoodFetcher might go bad if the problem is not fixed. An external maintenance company might take this role.

Government

The FoodFetcher can provide a platform for directed personal health and lifestyle campaigns. Also any new system must adhere to regulations in order to be safe for use and can be produced without causing problems.

Implicatons The first implication for the government is that there is a possibility for a healthy food campaign by making the foodfetcher propose healthy meals instead of junkfood. With the growing number of overweight people, this may be a really smart thing to do. Besides that, the government will get taxation income from the selling of the foodfetcher and products through the foodfetcher.

Needs The job of the government is mainly to keep everything under the control. To achieve this, regulations will need to be put up, for example to clear up what happens if a person buys food and does't get it delivered at all. Advertisements will also need to be regulated in order to not make it go out of control. Also, trying to make the system hack-proof is something that the government is part of, so making the system as hack-proof as possible is also a need.

Scenario

What would it be like in a not to distant future with the FoodFetcher implemented?

A phone rings in the cubicle of a busy office. The man sitting in the cubicle answers “Hello?” his mind still with his work. On the other side of the line a woman's soft voice says; ‘Honey, have you thought about Valentines for tonight?’ The man’s heart rate rises, his mind runs down all the options. Restaurants are full and he didn’t make reservations. He mutters; ‘Err, sure honey...’ She says; ‘Oh you are so thoughtful, my mother was wrong in saying I shouldn’t marry you.’ Sweat runs down the man’s face.. But suddenly, he remembers. FoodFetchers. The man answers to his love ‘I am going to cook diner, just for the two of us’ He grabs his smartphone and fires up the FoodFetchers app. In the menu under the heading suggestions an extra option appeared for Valentines, as happens with every holiday. The man’s slowly steadying finger taps the option and waits while the data from his fridge and pantry at home is retrieved. Normally the man would use the daily FoodFetchers-diner-suggestion that sends a push-message with a diner suggestion at 12 o'clock. But today he was so busy he forgot to check his phone.

A notification appears, -FoodFetchers has made a selection for Valentines-diners based on what you’ve already got at home- The man rejoices. He can always trust his personal FoodFetcher-system. ‘Are you still there?’ His love asks. ‘Yes, I was.. just distracted, I’ll be home in a bit.. Love you’ ‘Love you to’ *click*. On the drive home the man sees a supermarket and recalls.. that just two years ago he and so many others, used to frequent this now almost abandoned building, daily. Most supermarkets now have changed layout and turned their shop employees into delivery boys and girls. It is understandable. FoodFetchers is so easy. It coming with standard food-stocking settings and downloadable upgrades. Even the man’s mother didn’t have to go to an elderly home because reheatable food-packages are now delivered daily. FoodFetchers even monitores for a variated and healthy eating habit.

The man pulls his car onto his driveway and sighs in relief, thinking; ‘with the little one on the way, I don’t know how I’d ever do without FoodFetchers. How did mý parents manage this..’

Prototype

The prototype of the FoodFetcher is what we want to fysically build within the few following weeks. Here, some specifications are given.

Requirements

These are the main requirements for the prototype of the FoodFetcher, derrived from the user needs. Without these requirement, the system would not function. Each requirement contains a fixed action it has to be able to do to make the system work, which can be tested on whether it works or not.

• The FoodFetcher needs to be able to scan which products are in stock in order to know how much is in the inventory.
• The FoodFetcher needs to be able to communicate with the user through the FoodFetcher app.
• The FoodFetcher needs to be able to take into account user preferences about food, so that the user gets what he/she wants.
• The FoodFetcher needs to be able to order products and have a way of accepting products so that it is possible to retrieve a product after ordering it.
• The FoodFetcher needs to be built in in current homes; it won't cost too much in that case.
• The FoodFetcher needs to to know how full a product in a package is, for example, there is a difference between a half-full pack of milk and a full pack of milk.
• The FoodFetcher needs to know an estimate or the exact expiration date.

Preferences

These preferences also come directly from the user needs, however, they aren't required to make a working product. Besides, these preferences do not have a specific quantity they need to have, they have to be fulfilled as good as possible. They are not less important however, since a user-unfriendly product will not be tolerated.

• The interaction with the FoodFetcher system should be as easy as possible.
• All the functionalities need to be as clear as possible through instructions.
• There should be the least amount of work that needs to be done by the user to make the system work, the system should work autonomously.
• The system has to be compatible with as many kitchens as possible
• The system should be as difficult as possible to hack.
• The system should be as cheap as possible.

Hardware

• Pressure plates: to detect a product and how much it weighs
• Label on the product: to be able to scan what product it is and what expiration date it has
• Moving light sensor: to detect the label on the products
• Arduino: to control and use the hardware
• Raspberry pi: to control the software (in the prototype this will be a laptop)

Software

We eventually want to have an app in which we can keep track of every product in our kitchen. We want our app to do the following:

• Giving an overview of products you have
• A list which tells us to remove an expired product
• The expiration dates of the products in your kitchen

To manage our app, we are planning to use the systems as stated below:

• ZXing library android in order to scan the barcodes on the labels
• Outpan API to retrieve the information of a product (unfortunately incomplete)
• zinc.io API to order food from amazon UK (future)

Hardware update

First we will design a prototype. This will be to test the proof of concept. The prototype will consist of a basis, supporting the pressure plates and food products, and a android phone camera. The prototype will enclose all the necessary sensors and hardware in order to test the scale and workings of the system.

The casing

The casing is designed to hold 2 Interlink Electronics Force Sensitive Resistors (FSR); model 406. These are spaced so that each FSR can detect one milk carton at a time. Milk cartons being our chosen food product for the prototype. The FSR have a longer connection strip than the width of the basis. This is due to the fact that the FSR’s connections steps cannot be shortened for this prototype because this will reduce their sensing capabilities. The casing cannot be any wider for than a milk carton can be placed, by the user, besides a pressure plate resulting in a loss of data measurement (force). Therefore some of the strip will extend beneath the casing, the FSR’S will also be connected to jumper cables on a breadboard. The casing will have a seperate slot for every carton and a pressure dividing surface, since the pressure sensor is smaller than the width of the carton. The back wall will be higher than the front wall to support the control electronics. The arduino and breadboard will be placed on the back of the casing. It is also the place where the power cable can be attached to.

In order to produce (first round estimate) dimensions for the casing, several milk cartons have been measured. The following (mean) data is acquired (100% error analysis):

Caption: Milk Carton Dimension Data

Milk Carton Type	Width ±0,4 (cm)	Length ±0,4 (cm)	Height± 0,4 (cm)	Perturbing Parts ±0,4 (cm)
A	7,0	9,5	24,5	Lid height: 1,0	Diameter: 2,5
B	7,2	7,2	23.5	Lid height: 1,0	Diameter: 2,5
C	9,0	6,1	19,5	Lid Height: 0,9	Diameter: 2,5

Casing Dimensions

The casing is first graphically designed to get an overview of what needed to be sawn. These were mostly made to be clear for the executer of this task. The panels of the casing were designed in a manner that they would "click" together. This instead of glued, so that it was easier to transport and adjust the pressure plates inside. The individual panels can be seen in image ...

Pressure plates system

The pressure plate system will consist of three FSR Interlink Electronics 406. Only two will be installed in the prototype because of budget control. These FSR's will be placed on the basis of the tray in the prototype. For early testing a breadboard setup will be used.

Hardware and software list

Before going into detail is perhaps good to start with the parts used in the subsequent paragraphs:

• FSR model 406 (2x)
• Breadboard (1x)
• Jumper cables (multiple)
• Arduino UNO (1x)
• USB cable (1x)
• Resistors (2x for each R (Ohm) value)
• Laptop (1x)

Also software is important to achieve a working prototype. Therefore the following software is used:

Controlling the arduino and acquiring sensor (FSR) data:

• Arduino IDE

Sending the sensor (FSR) data to the laptop for processing:

• CoolTerm

Analyzing the acquired sensor (FSR) data:

• Origin

Electronic Schematic software:

• Fritzing

The FSR 406

As said the pressure plates will consist of so called Force Sensitive Resistors (FSR). These are resistors that can change their resistance by influence of an external force, unlike solid resistors which have a fixed resistance (e.g. 10 kOhm). The FSR consist of a conductive polymer which can change it’s inherit resistance by application of a force to its surface area. The FSR come in various shapes and sizes, but most common are flat FSR’s. In our prototype we use square and flat FSR’s of model type 406 made by Interlink Electronics. See figure [..] for an image. Each polymer sheet is covered with a very thin film of both electrically and non-electrically components. This components are produced in a square grid, also known as a matrix. The primary task of these components is protecting the polymer sheet. Secondary tasks include improvement of mechanical properties like bendability and stability and increasing the surface friction. It also protects the polymer sheet against temperature changes. When the sheet undergoes a force, its outer film conductive components get squeezed against the polymer sheet, changing the resistance of the film. This change can be detected when a voltage is applied across the sheet, usually in the range of 3.3-5V. This voltage is delivered by external electronical components like the Arduino board. A simple schematic can be used to build a force sensitive system. In our prototype we use an Arduino Uno and a so called voltage divider (see section [..]). A complete overview of the FSR 406, including dimensions can be seen in figure [7.3-1].

Immediately after turn-on, the resistance decreases very rapidly. At slightly higher and then intermediate forces, the resistance follows an inverse power law. At the high forces the response eventually saturates to a point where increases in force yield little or no decrease in resistance. Saturation can be pushed higher by spreading the applied force over a larger actuator. www.interlinkelectronics.com

• 

  FSR 406 specifications

• 

  FSR 406 picture, comparisson to coin

• 

  Voltage divider setup

The electronic setup

Each pressure plate will be connected to the Arduino UNO via a voltage divider setup (see image above.) This allows for measurement of the change in voltage when a load is applied to the FSR. The resistance of the FSR will decrease as more pressure is applied to the FSR, therefore increasing the voltage (and current). The fixed resistor R₁ will take a role as a pull-down resistor. This pull-down resistor R₁ is connected to the ground and ensures that logic signal is low when no load is applied on the FSR. A breadboard is used to allow for simple construction of the setup. It can also be easily modified and removed. A final prototype should hold a more reliable system like a PCB board. This is not included in this prototype version however.

The divider needs a ground connection delivered by the Arduino board: Digital Ground (DGND). The +5V voltage will be delivered by the Arduino also and is connected to one FSR pin. The other FSR pin is connected to an analog pin (A0-A2) on the Arduino. This is where the data is read from. Furthermore a 10 kΩm resistor R₁ as a pull-down resistor. So for the earlier mentioned two fsr setup; two voltage dividers are needed. The following figure shows this setup. The FSR in this figure are shown as round, the 406 model used in the prototype are square.(This is due to a limited Frizting software library)

• 

  Schematic electronical setup of the pressure plate system using Frizting software. Two FSR's are shown each connected to a voltage divider and the Arduino Uno Board

• 

  Picture of breadboard electronic setup

• 

  Picture of breadboard electronic setup

Calibration of the FSR 406

Theoretical weight calibration

The calibration will be done in the following way. We will again be using a voltage divider setup. Instead of having to calculate the conductance from the variable resistance R_FSR a simple voltage-force conversion will be examined. The measuring resistor, R₁, is chosen to maximize the desired force sensitivity range and to limit current. Multiple type of resistors are availible: 10 kOhm, 8,1 kOhm and [...]. Several weights of variaying mass (g) will be used in this calibration. The problem is that it is very important that the place of measurment remains the same throughout the calibration process. In order to maximize the accuracy the following performance optimization will be used:

1. We will provide a consistent force distribution on each FSR. The FSR is very sensitive to force distribution across its surface. Therefore we will use "dead weights" like metal blocks in each measurement to ensure cycle-to-cycle accuracy and repetability. Also a small polymer surface like an elastomer can absorb any errors in force distribution.

2. We will maintain the same sort of measuring material throughout the calibration process. This will reduce the response error of the FSR. Therefore we will for example not be using a force provided by a human finger.

3. We will maintain the same actuation cycle time in each measurement consistent. The FSR shows time dependency in the resistance of the applied force (drift). Therefore in each cycle of increasing measurent weight the same actuation time will be used.

After this performance optimization has been carefully considered the data needs to be examinded. A repeatable and reproducible system has been estabilished and it is time for data acquisition. We will record the output voltage for several weights, whereby the software Origin will be used for analysing the data. It is possible to plot a curve through the acquired data points in order to produce a mathematical relationship making the the output voltage (V) a function of force (g). If necessary we can calculate the force (g) to Newtons.

A possible calibration result is provided by Interlink Electronics and can function as a benchmark/comparison. The following graph is taken from Interlink Electronics FSR intergration guide/Measurement techniques.

It can be clearly seen, from the figure above, that a different behaviour in the resistance-force curve family can be acquired for different fixed resistor R₁ values. This results in different senstivity ranges, to be determined in our case for our prototype. Again the steep "cut-off" voltage can be seen when a small load (force) is applied. This is as expected.

Experimental calibration: setup and results

Weight calibration without casing

The calibration was done according to the specified procedure above. Several measurements where done with different metal weights of varying mass. We had access to the following weights: • 33,8 gram • 50 gram • 200 gram ‘small’ • 200 gram ‘large’ • 500 gram ‘small’ • 500 gram ‘large’

Because the size of the weight matters when calibrating the FSR it was necessary to use larger weights: from 200 gram, the weights were classified in a small metal cylinder and large metal cylinders. The smaller size weights were sufficiently small to not influence the measurement. We measured with increments of 33,8 and 50 grams, starting with a zero measurement increasing to 600 grams. Each measurement was captured using the Arduino and FSR setup as mentioned above. It became clear both from constructing the calibration setup and the integration guide, specified by Interlink Electronics, that a soft supporting material was needed between a mass and the FSR surface. If this precaution where to be for sacked it was very likely that damage to the FSR would occur. Therefore we choose to use foam like filler that came with shipping our electronic components. It was found that this material offered sufficient damping and protection capabilities while keeping direct interaction between mass and FSR surface minimal.

Each increment of increasing mass placement followed and increase in voltage, as we expected from concideration in the paragaph above (theoretical calibration). The manual also introduced to concept of time actuation of the measurement (also known as ‘drift’) as explained above. During the experiments it became clear that this drift was sufficient to interfere with accurate measurements. The time before a steady voltage measurement occurred with some masses was long, in the order of a up to minute. It was most strong when very light weights where used: the 38,8 and 50,0 gram. When heavier weights where used the drift time became less notable: in the range of ~30-40 seconds. In order to investigate the drift time the following experiment was done.

The following data graph was acquired showing the relationship between mass [g] and voltage [mV] of the FSR.

For the 10 KOhm resitor the mass-voltage data was aquired to show that there excists a discrepancy between each FSR unit measurement results. This is as expected because no two FSR are the same. The polymers inside the FSR differ in distribution and quality from unit to unit. Therefore it is important that both FSRs are measured and analysed in the cailbration process. In the graph above this is shown only for the 10KOhm resistor R. Clearly for higher mass the discrepancy becomes notable. Also the smaller resistor value of 5 KOhm, shows a lower voltage curve, as is expected. Also a different value for the 5,1 KOHm resistor can be seen for the masses 200g and 33,8g. This is due to the fact that because of "drift" problems the measurement gave a wrong result. See next section for an overview of drift problems. The 3 KOhm resistor was not calibrated for reasons described in the next section.

Weight calibration with casing

The previous experiments showed how to calibrate the FSR’s, but now the finished casing is taking into account. The casing itself poses different conditions on the force measurement, for example the spread of weight (load) on the FSR surface. The 12 KOhm resistor was taken as a fixed resistor, because of the least amount of drift time (see next section). Because the drift is still a problem the measurement time between each increment of weight (50g) is chosen to be 40 s. The weights where measured from 50 grams up to 1300 grams, with 50 gram increment. We chose to have 1300 grams as a maximum, because the prototype casing only supports (e.g. milk) cartons and thus will not exceed this weight. The calibration was done for each FSR, denoted FSR1 and FSR 2. The graph below shows the measured voltage-mass curve for both FSRs:

Between both FSRs there is not much difference. The difference is only accounted to some fit parameters (see mathematical concideration). Each FSR will thus have the same fit formula (exponention) but with different fit parameters. In the Arduino IDE software this will be taken into account. Also each measurement was done every 40 seconds, this also have to be taken into account in the software.

Drift experiment

Several "pull-down" resistors R_n where available with the following resitance:

• n=1: 3 KOhm
• n=2: 5,1 KOhm
• n=3: 10KOhm
• n=4: 12KOhm

With each of these resistors a voltage divider setup was made (see [..]). The FSR was placed in this setup and connected to the Arduino Uno. Again several weights where available:

• 33,8 g #1
• 200 g #2
• 400 g #3

The drift experiment was conducted in the following way. At first no load (pressure) was applied to the FSR giving a zero-measurement of the voltage [mV] value; as expected. Now by placing the smallest weight, number 1, on the FSR surface area an increase in voltage could be detected. The drift in the FSR caused the voltage to rise for a certain amount of time [s] before becoming a steady value. The time between mass placement and steady-voltage value was measured for each weight (#m) and each resistor R_n. This experiment gave the following results:

Resistor Rn [KOhm]	Weight [g]	Drift time [s]	Voltage v [mV]
3	33,8	59	288
3	200	100	2956
3	400	110	3768
5,1	33,8	74	1622
5,1	200	69	3548
5,1	400	51	4213
10	33,8	50	2140
10	200	32	3792
10	400	21	4511
12	33,8	38	2605
12	200	31	4535
12	400	23	4535

From the table above it can be clearly seen that the drift time decreases when the load increases. This is easy to understand as the force acting on the FSR surface is greater for heavier weights, and thus there is little room fo deformation over time in the polymer areas of the FSR surface. Only when the resistor R_n=3 becomes too small, the drift time increases when the weight increases. This is due to the fact that the voltage V is very low, and the FSR resistance very high. It is more difficult to establish a good reading by the Arduino. Also the pull-down capablities of R_n=3 become almost none. Concluding from the data above, it becomes clear that the setup requires a fixed R_n resistor as high as possible, and therefore we will use the 12KOhm (n=4) resistor in our protopype setup for both voltage dividers.

Mathematical consideration

Theoretical model

The mathematics describing the model for a voltage divider is quit straightforward. Each voltage divider "divides" the voltage between the two resistors: one fixed R₁ (Ohm) and one variable R_FSR (Ohm). This can be mathematically expressed as:

[math]\displaystyle{ V_{out}= V_{in}*{R_1\over R_1+R_{FSR}} }[/math] [V]

whereby V_out (V) is the voltage supplied over the resistance R₁. V_in (V) being the +5V supplied by the Arduino Uno Board. R₁ is the fixed resistance which is chosen to be 10kOhm. The resistor R_FSR is the variable resistor FSR. When no load (pressure) is applied to the FSR the resitance will be extremely high (~infinite). This is the same situation of having an open circuit, thus no voltage can be detected. In this situation the pressure plate software (see next chapter) knows that no load is applied. When a load of increasing weight is applied the resistance will drop significantly. At 100 N load the resistance will be a mere ~250 ohm. Looking at the equation above it is clear that in this case the voltage detected is very close to 5V (but still less!).

The calculation of the force (in Newton) cannot (or atleast for this preliminary results) be direclty taken from the equation above. Instead this has to be releated to the conductivity and calibration results delivered by the manufacterer of the FSR 406 (Interlink Electronics). We plan on doing the calibration later on ourselves too ensure even better accuracy as each FSR differs slighty from the specified calibration. But for now this specified data suffices. In our first force approximation we will use the conductance C:

[math]\displaystyle{ C= {1\over R_{FSR}} }[/math] [Ohm^-1]

in which the resistance R_FSR is calculated with equation [..] resulting in:

[math]\displaystyle{ R_{FSR} = R_{1}({{V_{in}\over V_{out}}-1}) }[/math] [Ohm]

As said we will use data delivered by the manufacter. In this case it is not directly the capacitance but the resistance R (ohm). See the graph below which is taken from the FSR integration guide from Interlink Electronics.

Looking at the graph above it becomes clear that indeed the resistance decreases very rapidly when a force is applied. After the initial steep decrease, the curve seems to be, in good approximation, lineary decreasing in this double log plot. We will approximate the force (F)-capacitance (C) curve in two domains (which is the inverse from the graph above):

If C<1000 => [math]\displaystyle{ F= {C \over 80} }[/math] [N]

else => [math]\displaystyle{ F= {{C-1000}\over 30} }[/math] [N]

This will be replaced with the Voltage V_out- Force(g) curve calibration as described in paragraph [..] but for preliminary results it is acceptable.

Calibration model

The calibration with the casing (see previous section) requires a different model. The theoretical model uses the conductivity, but will not be used in the calibration model. Instead the relationship between voltage and mass is enough to give the weight in grams for a product placed on the FSRs. The previous section provides the experimental data which have to be fitted in order to find the mathematical relationship. A problem that occured is that no fit can be obtained that satifies the data for both low (0-250 grams) and high weight (250-1300 grams) domains. Therefore two ranges will be used: 0-250 grams and 250-1300 grams. Each range will have its own fit with different fit parameters. The value 250 gram was the point at which the data became unsufficient for both ranges and therefore chosen to be the limit between both.

The following graphs shows both fith in both ranges for FSR 1 calibration data (see previous section):

The following fits where made (red lines above): For FSR 1 in range 0-250 gram: [math]\displaystyle{ M= y1 + A1*e^{ {V \over t1}} }[/math]

with M the mass [g], V the voltage [mV] and y, A and t fit parameters:

fit parameter	value
y1	22.33971
A1	0.8375
t1	759.58707

For FSR 1 in range 250-1300 gram: [math]\displaystyle{ M= y2 + A2*e^{ {V \over t2}} }[/math]

fit parameter	value
y2	226.00072
A2	2.12925E-9
t2	179.05452

For FSR 2 range 0-250 gram: [math]\displaystyle{ M= y3 + A3*e^{ {V \over t3}} }[/math]

fit parameter	value
y3	29.63199
A3	0.29855
t3	634.63345

For FSR 2 range 250-1300 gram: [math]\displaystyle{ M= y4 + A4*e^{ {V \over t4}} }[/math]

fit parameter	value
y4	273.24213
A3	4.57473E-11
t3	157.93817

The fitting was done with the analysing software package OriginLab 20115.

Software update

Mobile application software

A major part of the project was the creation of a proof of concept of the mobile application of the foodfetcher. This was done by us through the creation of an android application, in which it is possible to check one's inventory, order items and get suggestions based on your preferences and what's in your inventory. This proof of concept of the application will now be discussed, in which 3 product types are mainly used: Milk, buttermilk and custard.

The software has been written partly in the program Android Studio. This program mixes Java-code for programming and xml for the layout of the app and some other things. Different screens in the app represent different activities. These activities are kept seperately to assure that in case of errors, the entire code does not need to be checked for bugs. It is a so-called failsafe and has many more advantages over using one single activity. In the app, the following activities are currently used:

• A main activity with buttons linking to different activities
• A way of scanning barcodes
• A list where you can see your current inventory, together with the expiration date.
• The list where you can remove items from your inventory that are removed from your fridge.
• An activity where you can see what items can be ordered and where you can actually order those items.
• A shopping list where you can see what items you have ordered.
• An activity for creating suggestions and adding suggestions
• An activity for giving in your own personal preferences

These activities will be disussed right now in the same order as the one listed above. For each activity, the use of it will be explained and what things can be done through the activity, including a very brief explanation on how it is implemented in Android Studio.

Main activity

When the application is opened, the first thing the user sees is the main screen, which is represented by the main activity. In this activity, all other activities can be opened, which works with an intent-filter in android studio. It looks like this:

As you can see, the layout of the main screen consists of all the buttons linking to different activities, with the FoodFetcher-logo in the background. There aren't really any other important uses for the main activity except for opening the other activities.

Barcode scanner

The barcode scanning activity is used when a product is placed into a fridge or other cabinet. Every product has a QR-code placed on top of it, which can be scanned by simply pressing the button on the mainscreen and facing the QR-code with the camera. In this QR-code, information is given about the type of product and its expiration date, which is done by using the ZXing library. This library retrieves the information from the QR code, which contains information about the product like the expiration date and the product type. Once the product is scanned, it is automatically added to the inventory. When the item is scanned for the second time, it is removed from your inventory and added to the removal list.

Inventory

The inventory list is opened when this button is clicked. It looks like this:

As said earlier, this inventory activity displays not only which products are in your fridge, but also what its expiration date is. Besides that, to make the user's life easier, a picture of each item is added in your inventory. It also shows the current weight of the product in grams, as well as the initial weight of the product. The initial weight of the product is determined by the first non-zero value of an unoccupied sensor after scanning the product. This is also how the app knows on which sensor a product is placed. If a product is taken out of the fridge and put back (and scanned twice in the process) the product does not have to be placed at the same place since the sensor can be redetermined.

The red background color for the milk item indicates that this product is expired and should no longer be consumed. Items that are expired are ignored by the suggestions activity. So even if there is 800 grams of expired milk in the Foodfetcher, it will still suggest to buy milk.

Removal activity

There was some discussion in the group about what to do with the removal of items, the 2 options are to either scan the product upon removal of the pressure sensor, or when the product is scanned for the second time. After some discussion, it was decided that it is the best from the user's perspective to scan for a second time. Even though this seems weird, the alternative is that the user is not allowed to change the position of any other item in the cabinet or fridge. This is not user-friendly at all, especially in someone's fridge. When someone takes away a product in the fridge, he usually lifts a lot of products with this and places them back. This would cause the system to not know which product is removed. We think that the user can appreciate a robust product over a product that works half of the time, but with more effort that has to be done.

Once a product is removed from the fridge, it is put on the remove-list. These products will be removed automatically after a certain amount of time, and besides that, the products can be removed manually when the remove-activity is opened from the mainscreen.

Ordering items

Items can also be added to a shopping list through an activity called ItemsActivity. Different types of products can be added and removed in different amounts and your shopping list, which can be viewed through a seperate different activity called ShoppingListActivity, which will be discussed next. The products on the shopping list are written to a textfile on the phone, so they won't be lost when you exit the app. New products are simply written to this text-file when these are added, and the file is read when the shopping list is printed. To add a product, one opens the activity through the button on the main screen. There, for each type of product, it is possible to fill in a text field with the amount of units of the product you want to add or remove to your shopping list and then you can click the button to add the products. It looks like this:

A few things can be noticed. First of all, there is a textfield with a message under the buttons. In this textfield, a message with confirmation of your addition to the shopping list is given. It confirms that you have added a type of product and also how many units you have added. If you remove products, this is also given in a confirmation which says how many products were actually removed and how many are left. If the amount of removed items is bigger than the total amount on the shopping list, all of that item on the shopping list is simply removed. When there is no valid number typed in the textfield for the amount of products, a notification is also given in the message textfield that there is no valid number inserted. When you are finished adding products to the inventory, you click sumbmit and return. The additions are then flushed to the shopping list textfile and you return to the main menu of the app. If you want to clear your shopping list, you simply click clear and then submit. When this is done, the entire shopping list is cleared.

Shopping list

Once you are done adding items, you can look at your shopping list through a separate activity which can be started via the main screen. This shopping list then contains all items you added including its amount, which it reads from the text-file created on your phone. This means that the shopping list will not be gone once you terminate the app. It looks like this:

What can be noticed is that the shopping list prints the product types with a digit for the amount of products, while the created textfile uses a long list of products. This is done by creating a counting algorithm to count the number of items of each product, which is printed for each product. It is not possible to make changes to the shopping list in the shopping list-activity, this has to be done in the activity for ordering items that is described above.

Suggestions

The FoodFetcher application also has to be a smart application, which means it has to create suggestions based on your inventory, current shopping list and your preferences. For now, an easy implementation of the suggestions is used, which is mainly for proving the concept of these suggestions. The suggestion will be based on the inventory, removal list, shopping list and preferences, which will be discussed next. The app will suggest you to put a product on the shopping list when:

• There is none of the product on the shopping list.
• The product is ticked in your preferences.
• The total mass of non-expired product is less than 400 gram, which means expired product will not count.

These points are looked at for each product type, which results in a list of products that can be added. This looks like this:

What can be noticed is that there is a button for simply adding the suggested products to your shopping list so you don't have to add them manually, which makes it easier for the user to add these suggestions.

Preferences

Without a possibility to add preferences, the app would not know what products to order, since the user might not want certain products (like buttermilk). Therefore, preferences are added in which you can tick whether you want to make the app suggest to buy new products of that type. These preferences are written to a textfile on your phone, so you won't lose your preferences when you terminate the app. Everything is done through the following interface:

It is clearly visible that the little boxes need to be ticked to add a product type to the prefered list of items. Once you are done, you just submit it and you will be able to use the suggestions, where it will not suggest products that are not on the prefered list of products.

Communication between android app and arduino

The arduino saves the data from the pressure sensor to a file on one of our laptops. A java program that runs on this laptop writes the relevant information (the sensor number and the pressure) to a dropbox file. We considered a couple of alternatives before using Dropbox:

• Sockets: the laptop would be the server socket and the android app could then connect to the laptop and retrieve information as it pleases. This initially seemed to be the most logical and the best way of doing this, but unfortunately we could not get it to work in practice. This was due to the fact that sockets connect by ip address, which is in some cases dynamically assigned. So it was not possible to guarantee communication using sockets.
• Bluetooth: bluetooth was the other alternative, but this had two disadavantages: the data in the FoodFetcher app is only updated if the user is near the laptop, which can be inconvenient in case he/she wants to check on the contents from further away. Another disadvantage is that it would be a lot of work to implement.

So in the end Dropbox seemed the best way of doing the communication, also because information can easily be retrieved via the download link that every file has. The only disadvantage is that we rely on the syncing of dropbox itself, but speed is not a very important issue in our system.

Since it takes quite a while before the pressure sensor finds the right pressure, the sensor data on dropbox is only updated every second, which also ensures that dropbox has enought time to sync the files with the cloud. This file can then be read by the android app in a seperate thread.

This information is represented in the app by a class, that contains the relevant attributes and that can write it to a file, such that all items can be retrieved again when the app is restarted.

Arduino software

The arduino software has also been written. It can measure data from a pressure sensor, calculate the force belonging to this input and send this to a text-file.

The Arduino software has multiple purposes; it controls the analog pins on the Arduino UNO boad, and provides the +5V voltage to the FSR's. By reading the voltage value of the analog pin it is possible to calculate the resistance (and therefore force) as explained in the previous chapter. An if-else statement is used to display this calculated force; if there is no detection of a voltage the software knows no load (pressure) is applied of the FSR's. Else, if a load is applied on either of the two FSR's, this is translated into a mass (g) value to the user.

[code] //Configuration

 //Pins
   int fsrPin1 = 0; // the FSR_1 and 10K pulldown_1 are connected to a0.
   int fsrPin2 = 1; // the FSR_2 and 10K pulldown_2 are connected to a1.
 //Readings
   int fsrReading1; // the analog reading from the FSR_1 resistor divider.
   int fsrReading2; // the analog reading from the FSR_2 resistor divider.
   int fsrReadingMin1 = 0; // the analog reading ranges of FSR_1 from 0 to 1023 this is the minimum 0.
   int fsrReadingMin2 = 0; // the analog reading ranges of FSR_2 from 0 to 1023 this is the minimum 0.
   int fsrReadingMax1 = 1023; // the analog reading ranges of FSR_1 from 0 to 1023 this is the maximum 1023;
   int fsrReadingMax2 = 1023; // the analog reading ranges of FSR_2 from 0 to 1023 this is the maximu 1023;
 //Voltages
   int fsrVoltage1; // the analog reading_1 converted to voltage.
   int fsrVoltage2; // the analog reading_2 converted to voltage.
   int fsrVoltageMax1 = 5000; // the voltage provided by the arduino to FSR_1.
   int fsrVoltageMax2 = 5000; // the voltage provided by the arduino to FSR_2.
   int fsrVoltageMin1 = 0; //the minimum voltage value of FSR_1.
   int fsrVoltageMin2 = 0; //the minimum voltage value of FSR_2.
  //Forces
   long fsrForce1; // Finally, the resistance_1 converted to force
   long fsrForce2; // Finally, the resistance_2 converted to force
  //Masses
   long fsrMass1; //fsr1 readings converted to mass in grams;
   long fsrMass2; //fsr2 readings converted to mass in grams;
   float fsr1A1 = 0.8375; //one of the constants found while putting mass against voltage on fsr
   long fsr1A2 = 0.00000000212925;
   float fsr1Y1 = 22.33971; //the other constant found doing the calibration tests
   float fsr1Y2 = 226.00072;
   float fsr1T1 = 759.58707;
   float fsr1T2 = 179.05452;
   float fsr2Y1 = 29.63199;
   float fsr2Y2 = 273.24213;
   float fsr2A1 = 0.29855;
   long fsr2A2 = 0.0000000000457453;
   float fsr2T1 = 634.63345;
   float fsr2T2 = 157.93817;
  //DelayConstants
   unsigned long timer1; // later on used to create a timestamp to create a delay
   unsigned long timer2; // same as timer1
   unsigned long delay1 = millis();  //millis() creates timestamps since the arduino is running this one is for the starting point and later changed to keep the delay
   unsigned long delay2 = millis(); //same as delay1

void setup() {

 Serial.begin(9600); //We'll send debugging information via the Serial monitor

}

void loop() {

 timer1 = millis();
 timer2 = millis();
 if(timer1>=delay1+40000){
 FSR1();
 delay1 = millis();
 }
 if(timer2>=delay2+40000){
 FSR2();
 Serial.println("---------------");
 delay2 = millis();
 }

}

void FSR1() {

 fsrReading1 = analogRead(fsrPin1);  
Serial.print("Analog reading_1 = ");
Serial.println(fsrReading1);

 // analog voltage_1 reading ranges from about 0 to 1023 which maps to 0V to 5V (= 5000mV)
fsrVoltage1 = map(fsrReading1, fsrReadingMin1, fsrReadingMax1, fsrVoltageMin1, fsrVoltageMax1);
Serial.print("Voltage reading FSR 1 in mV = ");
Serial.println(fsrVoltage1);  
 if (fsrVoltage1 == 0) {
  Serial.println("No pressure on FSR 2 detected");  
} else {
  if (fsrVoltage1 >= 4375) {
  fsrMass1 = fsr1A2 * exp(fsrVoltage1/fsr1T2) + fsr1Y2;
  }
  else {
  fsrMass1 = fsr1A1 * exp(fsrVoltage1/fsr1T1) + fsr1Y1;
  }
  Serial.print("Mass on FSR 1 in Grams");
  Serial.println(fsrMass1);
}

}

void FSR2(){

  fsrReading2 = analogRead(fsrPin2);  
Serial.print("Analog reading_2 = ");
Serial.println(fsrReading2);

 // analog voltage_2 reading ranges from about 0 to 1023 which maps to 0V to 5V (= 5000mV)
fsrVoltage2 = map(fsrReading2, fsrReadingMin2, fsrReadingMax2, fsrVoltageMin2, fsrVoltageMax2);
Serial.print("Voltage reading FSR 2 in mV = ");
Serial.println(fsrVoltage2);  
 if (fsrVoltage2 == 0) {
  Serial.println("No pressure on FSR 2 detected");  
} else {
   if (fsrVoltage2 >= 4375) {
  fsrMass2 = fsr2A2 * exp(fsrVoltage2/fsr2T2) + fsr2Y2;
  }
  else {
  fsrMass2 = fsr2A1 * exp(fsrVoltage2/fsr2T1) + fsr2Y1;
  }
  Serial.print("Mass on FSR 2 in Grams");
  Serial.println(fsrMass2);
}

}

[/code]

The above code can maintain two FSR's. The code will subsequently measure the output voltage of FSR 1 and FSR 2 each 40000ms. This value is ofcourse arbitrary at this stage. The data is tranported to the laptop from the Adruino UNO board in a .txt file, by the software called TermCool (see partlist). Each data acquisition is time stamped which may come in handy for further data analysis in the mobile app. [The code above is based on the guide: "Using an FSR" by Adana Fruit company]

Prototype Walkthrough

A user perspective prototype walkthrough

In this text a complete user perspective walkthrough of the prototype is given

Hardware: scanning product → placing product → pressure sensor → position product determined → sticker/code recognition → “spoiled milk dataset” AND “milk left dataset” (“product database”) → write to rasp. pi [structures data] → ethernet cable to modem → computer on wifi reads rasp. pi structured data (via shared devices on home network)

Our story starts with placing, for example, a carton of milk in your fridge, after scanning your product. This carton of milk is placed on a random spot on a FoodFetcher-tray with pressure sensors. These pressure sensors send a signal

Future improvements

Of course we want our system to work as fine as possible and we want to be as user friendly as possible. However there are some improvements we would like to see in our final product which we can't imply now. The following lists states those functions.

• Suggestions for a recipe if, for example, some products expire soonly and you (onviously) don't want to throw them away
• Camera moving on both the X plane and the Y plane to be able to scan more products
• Removing products which are expired
• Order products you want or need and pay for them by using the app