View source for Talk:PRE2015 4 Groep2

=Minutes=
==Week 1==
===Meeting 20-04===
Goal: Create a demo in which a robotic system is able to detect ripe strawberries and harvest them effectively. 

Subsystems:
Robotic arm
Machine Vision/Learning

Requirements for presentation:
-USE needs (Yannick + Raomi Team Awesome)
-Scientific literature -> EU projects -> How to go beyond? (Cameron)
-Requirements
--Moving to A to B along a fixed axis
--Cutting fruit
--Sensing for ripeness
--Ambient sensing
--Locating fruit
--Collecting/Handling fruit effectively
--Feedback from farmer to system

-Idea/Solution to the problem

--Moving to A to B along a fixed axis
Fixed railing
Cable 
Treads for conveying
4-wheel/2-wheel drive
Yo-yo

--Cutting fruit
Scissors
Custom cutting mechanism
Laser cutting
Sharp knife (fruit ninja)


--Sensing for ripeness
Kinect
Color sensor
pH sensor
Force sensor
Machine learning ripeness (with Kinect using training set of ripe fruit)

--Ambient sensing
Temperature sensor
Humidity
Air pressure
Sunlight exposure


--Locating fruit
Kinect for depth map
Probe for fruit (use color sensor to identify where fruit is located)
Touch sense for fruit (and then use color sensor)
Fixed location for fruit

--Collecting/Handling fruit effectively
Basket collection
Soft packaging
Grip from stem 

--Feedback from farmer to system

-Plan of approach
Divide task into sub-groups:
-Building the robotic arm (mechanical) (TB)
-Machine Vision/Learning (software) (TC)
-Control of the robotic arm (software and electronics) (TBC)
-Feedback from farmer (software) (TA)
-USE aspects (TA)

Key: 
TA: Yannick & Raomi
TB: Mark & Maarten
TC: Cameron and Birgit
     
Deadline Week1: 
Defining project plan and timeline
Specify USE aspects and identify multiple solutions
Elucidate requirements
Compile Literature 
Create a presentation

Deadline Week2:
First presentation
Parts list (BOM)
Order parts
Begin drawings/concepts

Deadline Week3:
Beginning of meetings
Mark abandons us
Consolidate drawings

Deadline Week4:
Separately working components
Working base for movement from A to B

Deadline Week5:
Build the arm

Deadline Week6:
Control of arm 

Deadline Week7:
Integration of subsystems

Deadline Week8:
Testing of system
Deadline Week9:
Phase-out period

==Week 2==
===Meeting 25-04===

'''Idea in steps:'''


Problem description:

Less people needed to work on farms, they can work in other fields (like help elderly)<br />

Facts and figures!

Move away from 3 group structure, no<br />
The whole group working on the use aspect first? <br />
The groups should not be too large,<br />

We should first redesign the whole system with the focus of the way the USE aspects interact and change the design (as seen from a purely technical perspective) <br />

We should specify more requirements with respect to the USE aspects, not as much from the technical side.<br />

'''Users:<br />'''
-Primary:<br />
:-Farmers:<br />
::	-Higher efficiency<br />
::	-More harvest<br />
::	-Lower costs due to less employees<br />
::	-They will have more time<br />
::	-Less physical work / Health benefits<br />
::	-Don’t have to train seasonal workers<br />
	
-Secondary:<br />
:-Supermarkets, Distributors:<br />
::	-Higher quality food<br />
::	-Lower cost for food<br />
::	-There will be more fresh food available(maybe?)<br />
::	-More efficient supply chain<br />
::	-The supermarkets might also be able to give feedback.<br />
-Tertiary:<br />
:-Company that creates the robots and maintains them:<br />
	-

'''Society<br />'''
:-There will not be enough food in the near future for all the people.<br />
:-Not enough workers, due to aging. <br />
:-Decrease in wage gap due to overabundance of food <br />
:-Post scarcity<br />


'''Enterprise:<br />'''
:-Large farms will start to dominate the market which will result in an increase in the gap between rich and poor.<br />
:-Lower worker costs<br />
:-Lower food prices might result in more sales<br />

'''Results from USE<br />'''
:-The supermarket might want to have a say in which fruits are ripe.<br />
::	-Direct and faster communication<br />
:-We can better specify the amount of food that we need.<br />
:-There is a lot of food waste, even due to food not looking good.<br />
::-Robots can determine what the best use for fruits is  based on many factors (e.g. looks)<br />
:-Farmer feedback:<br />
::-Via his/her smartphone.<br />
::-Online database for machine learning<br />
:::-Many farmers can have access -> lots of pictures means high accuracy<br />
::-Farmers and secondary users should have a separate application<br />
:::-Farmer should have more control (e.g. stop button)<br />
:-Greenhouses<br />
:-Robots save space<br />
::-Verticality<br />	
::-More plants<br />
::-Can work 24/7<br />

'''Society idea:<br />'''
:-every buyer: can indicate which fruit it wants and how many at which time.<br />
:-every farmer: can indicate which fruits it can deliver at what time.<br />

Raomi: Society part (needs with h2020)<br />
Birgit: identify supply chain<br />
Cameron: technical plan<br />
Yannick: planning<br />
Maarten: User part<br />
Mark: Enterprise part<br />

===Meeting 28-04===


'''How it's done right now (researched by Birgit)''' <br />
Supply chain state of the art: what they do now is sell not-nice-looking food to farmers for animals (not thrown away). There exists a machine that sorts potatoes on size, ugliness, etc., but probably not findable online (no research done on it).

User part: not done, put on Drive '''AP'''.

Society: found on H2020 (expand..), no figures and facts there.

Enterprise: not done, put on Drive '''AP'''.

'''Idea:''' <br />
- 1 system with Kinect, Raspberry Pi, … to classify fruit. Connects to database for classifying fruit.<br />
- Encasing with custom lighting for equal light situations.<br />
- Cloud database for the pictures with machine learning.<br />

Problem: images taken by different types of camera's with different color. Possible solution: one type of tablet. Other possible solution: only use Kinect for sensing (something else for interface). Thus we have only 1 system. This solution is chosen.<br />

Teams: <br />
'''App team''': UI/UX, app, interface, Raomi, Birgit, Yannick. <br />
'''Database/backend/Kinect''': machine learning, neural networks. Cameron, Maarten, Mark.

'''Ethical part''' <br />
- How broad, enterprise scale/impact analyzation? '''AP''' ask
- During the project make sure that the user is taken into consideration (in design).

[https://www.mendix.com/application-platform-as-a-service/mobile-paas/]

==Week 3==
===Meeting 02-05===
'''Focus'''
Isolated fruit detection in controlled lighting

'''Tasks'''<br />
Learning about CNN and basics, implement a non-fruit NN (look into TensorFlow) - MV, CW, MdJ<br />
Read literature about previously implemented CNNs - MV, CW, MdJ<br />
Basic CNN, demonstrate fruit detection- CW<br />
Create a basic API to POST images and catch the response - MV<br />
Screenflow for App - Raomi<br />
USE aspects of App, why does app help the solution? - YA, RvR<br />
Basic visuals of app with walkthrough - BvdS, YA<br />
Look into Mendix - BvdS<br />

Find a U, S, and E to interview about automation in their process/harvesting as well as our solution and how it impacts their sector. -MV ,U(E), BvdS (U,S)

==Week 4==
===Meeting 09-05===
'''Updated Planning:'''<br />
'''-Before thursday 12-05'''<br />
:Maarten: Find local farmer
:How we will define color? 
:Which lighting conditions do we want?
:Mark: Why CNN? (compared with for example support vector machine)
:Yannick: Powerpoint of app
:Birgit: First version of the app
:LED light ring

'''-Before Thursday 19-05'''<br />
:Cameron, Maarten, Mark: Gathering database with rasberry pi and kinect 
:Farmer: [http://www.kwekerij-jansen.nl/assortiment] 
:Farmer: Needed to assure the use of our product

'''-Week 3''' <br />
:Preliminary design for app 
:First implementation of app 
:Database/Server setup 
:CNN, and basics of neural networks 

'''-Week 4''' <br />
:App v1.0 with design fully implemented 
:Kinect interfacing to Raspberry Pi completed 
:USEing intensifies 
:Finish back-end design and choose frameworks
 
'''-Week 5''' <br />
:App v2.0 with design fully implemented and tested 
:Further training of CNN 
:Working database classification (basic) 
:Casing (with studio lighting LED shining on fruit)

===Meeting 12-05===
During this meeting we worked and talked about various subjects:
'''App'''
:-Screenflow
::Raomi made some first versions of the screenflow showing the functionality of the app and which screen should follow which.
:-Powerpoint
::Yannick made and showed a first version of the app made in Powerpoint, this followed the screenflows made earlier but also gave a first look and feel of the lay-out of the app and which button should be put where. It gave the opportunity to test and see whether some things should change or not.
:-"Real" app
::Birgit downloaded the files needed to make an app and started getting the required knowledge by following tutorials.<br \>
'''Neural Networks'''	
:Mark made an overview of different neural networks that exist and of all gave some upsides and downsides. He made this all into a word document which was posted on the Google Drive

==Week 5==
===Meeting 19-05===
This week we did not have that much to talk and work on beacuse we did not receive any feedback due to Whit monday. Therefore we mostly continued doing what we were doing and what we had planned. <br  \>During this meeting we worked and talked about various subjects:<br \>
'''App'''
:-"Real" app
::Birgit made a first version of the app based on the screenflows and Powerpoint version which was made earlier. This gave an even more complete overview and look and feel of the app.<br \>
'''Computer Vision'''<br\ >
:During this week we mostly worked on getting the Xbox kinect to take pictures and training of the CNN

==Week 6==
===Meeting 23-05===

During this week we only had one meeting since half of our team (Maarten, Mark and Cameron) went to the picknpack conference on the 26th. <br \>

'''App'''<br \>
On monday we decided to change the functionality of our app to include the tinder-like swiping feature in order for the farmer to be able to quickly rate pictures of strawberries. This was then also made in the powerpoint version of the app, after which Birgit worked on incorporating it in the "real" version of the app.<br \> <br \> 
'''Computer vision'''<br \> 
More work was done on creating the convolutional neural network used for classifying the strawberries.

==Week 7==
===Meeting 30-05===
During this meeting we once again worked on many different point in parallel, which caused us to have much progress, both during this meeting and the meeting after.<br \>
'''App'''<br \>
:-Powerpoint<br \>
::During this meeting we made a third version of the app, where the user can see the specifications of his/her farm per region instead of a total view.<br \>
<br\ >

===Meeting 02-06===
This meeting was mostly used to create different buttons for the app and handling the feedback given by the farmer. We used this to enlarge our system towards a higher level design of a farm which uses our system. Furthermore we also worked and wrote about the societal problems that arise and which our system can solve

==Week 8==
===Meeting 06-06===

===Meeting 09-06===

==Week 9==
===Meeting 16-06===