PRE2015 4 Groep2

From Control Systems Technology Group
Jump to navigation Jump to search

(Wiki markup cheatsheet)

We are developing an autonomous harvesting robot. The robot will be initially developed for harvesting strawberries. As creating a complete prototype is probably not feasible to do in nine weeks, we start with focusing on the detecting and sensing part. For that we will develop a system which scans fruits and determines their ripeness. It can also consider other factors like for example if the fruit looks appealing.

Group 2 members

  • Cameron Weibel (0883114)
  • Maarten Visscher (0888263)
  • Raomi van Rozendaal (0842742)
  • Birgit van der Stigchel (0855323)
  • Mark de Jong (0896731)
  • Yannick Augustijn (0856560)

Project description

To have a robot that can classify fruits based on their ripeness and appeal factors. The fruits are detected while on a transport belt. --or-- In the field.

Add problem description

Requirements

Functional requirements

  • The robot should be able to detect fruit using a Kinect camera.
  • It should be able to classify the ripeness of the fruit based on a convolutional neural network
  • The robot will query an online database about the ripeness of a certain fruit and the database will return the percentile of ripeness the fruit is in based on different fruit image sets
  • The farmer should be able to take pictures of overripe/underripe fruit to add to the training set to give feedback to improve the robot
  • The farmer should be able to interface with the database as well as different harvesting metrics through a mobile device.

Non-functional requirements

  • It should be relatively simple to add the Kinect+Raspberry Pi to an existing harvesting system.
  • The farmer should be able to use the system with minimal prior knowledge
  • The robot should perform better than a human quality controller
  • This robot should have all the safety features necessary to ensure no critical failures.

USE aspects

User

Primary users are farmers and their workers, who directly use the robot. The following aspects hold:

  • Their work becomes far less intensive and heavy. Instead of directly harvesting, farmers can let the robot do the work. They would now only occasionally need to check the harvest and possibly adjust some parameters. This work is less heavy than harvesting and therefore less health problems due to heavy work can be expected.
  • More free time for other things. This is because the new work takes far less time. Also there is no need anymore for training seasonal workers.

Secondary users are distributors that pick up the fruits from the farms. They use the robot occasionally when they need to get the fruits that are picked by the robot. Their work is mostly unaffected, however some parts of their work can be left to the robot, depending on how advanced the robot is. One of these things is selecting fruits based on ripeness and appealing factor. This can be done by the robot. The robot could also directly package and seal the fruits.

A tertiary user is the company that is developing and maintaining this robot. It indirectly uses the robot during development.

(I assume that this is a societal aspect:) Another tertiary user is the harvesting worker. A worker that is harvesting fruit manually does not directly come in contact with the robot. The robot does however influence these workers, as it takes away their jobs. This aspect should be researched more. These harvesting workers are likely people with a low education and students wanting to earn a little more. The people with low education can be expected to have a hard time finding a new job.

Society

See https://drive.google.com/file/d/0BxKlXUVjSWzHV0Y0RWh3UUpGYzg/view?usp=sharing .

Enterprise

Enterprise aspect for autonomous harvesting robot

More sales
A socioeconomic survey of 87 nurseries and greenhouses located in Mississippi, Louisiana, and Alabama has shown greenhouses or nurseries with higher levels of sales tend to also have a higher level of automation (Posadas 2008). The level of automation was measured as the percentage of tasks that were automated of mechanized, it was found that, “significant increases in total revenues were associated with the level of mechanization, the number of full time equivalent workers, and the number of acres in production. These positive coefficients indicate that an increase in total revenues, on the average, by $4,900/year was associated with a one-unit (1%) increase in the level of mechanization. “ The same survey also showed that within these 87 nurseries and greenhouses the automation level of ‘harvesting and grading production’ was 0 percent. This could mean that there is incentive to introduce automated harvesting robots in similar greenhouses and nurseries. Another study, on the farm development in Dutch agriculture and horticulture (Bremmer 2002) shows that “the degree of mechanization increases the probability of firm growth and firm renewal.”

Farms are getting bigger
The structure of agricultural production is experiencing fundamental changes, this is happening worldwide (Bremmer 2002). Farms are moving away from the traditional family-owned, small scale and independent farming model and moving towards an industry that is 2 structured in line with the production and distribution value chain. The result of these changes is that the average number of farms is decreasing, whereas the average farm size is increasing. Autonomous harvesting robots could help farmers deal with this increased size in a cost efficient way.

Farmers show interest in autonomous machines A recent survey of German farmers (Kester 2013) has shown that these farmers are interested in adopting (semi-)autonomous machinery. They see several benefits, amongst them are saving labour, saving time, higher precision and increasing efficiency. When asked about fully-autonomous machines, some farmers also saw some additional benefits, lower health risks and an advantage in documentation and evaluation. However, they were skeptical about the reliability and safety of the machines.

Lower costs
The increase in automation will be paired with lower wage costs and less health benefits that have to be paid by the farmers, since the amount of full time, part time and seasonal workers will decrease. This is already a reason for current development of machinery, “High labour costs in recent years have encouraged larger, wider and faster machinery for cost efficient production.” (Kester 2013).


Sources
Posadas, B. , Knight, P. , Coker, R. , Coker, C. , Langlois, S. , Fain, G., 2008, Socioeconomic Impact of Automation on Horticulture Production Firms in the Northern Gulf of Mexico Region, HortTechnology, vol 18. No 4, 697-704
Bremmer, J., Oude Lansink, A., Olsen, K., Baltussen, W., Huirne, R., 2002, Analysis of Farm Development in Dutch Agriculture and Horticulture, 13th International Management Congress, July 7-12 2002
Kester, C., Griepentrog, H., Hörner, R., Tuncer, Z., 2013, A survey of future farm automation – a descriptive analysis of survey responses, Precision agriculture ’13, 785-792

http://horttech.ashspublications.org/content/18/4/697.full
http://horttech.ashspublications.org/content/22/3/388.full
https://core.ac.uk/download/files/153/7082880.pdf
http://docsdrive.com/pdfs/academicjournals/ijar/2011/1-9.pdf
http://orbi.ulg.ac.be//bitstream/2268/30490/2/BSE02O.pdf
http://download.springer.com.dianus.libr.tue.nl/static/pdf/271/bok%253A978-90-8686-778-3.pdf?originUrl=http%3A%2F%2Flink.springer.com%2Fbook%2F10.3920%2F978-90-8686-778-3&token2=exp=1462031410~acl=%2Fstatic%2Fpdf%2F271%2Fbok%25253A978-90-8686-778-3.pdf%3ForiginUrl%3Dhttp%253A%252F%252Flink.springer.com%252Fbook%252F10.3920%252F978-90-8686-778-3*~hmac=6217afc1e745a0cf41bb39a7984154dfd19c5e70f919d20826231af0ec2b96a6

Planning

Week 2
Clarifying our project goals
Working on USE aspects
Finalize planning and technical plan
Sketch a prototype

Week 3
Preliminary design for app
First implementation of app
Database/Server setup
CNN, and basics of neural networks

Week 4
App v1.0 with design fully implemented
Kinect interfacing to Raspberry Pi completed
USEing intensifies
Finish back-end design and choose frameworks

Week 5
App v2.0 with design fully implemented and tested
Further training of CNN
Working database classification (basic)
Casing (with studio lighting LED shining on fruit)

Week 6
Improve CNN
Expand training sets (outside of strawberries (if possible))
User testing on app
Reflect on USE aspects and determine if we still preserve our USE values


Week 7
Implement feedback from testing app
Improve aesthetic appeal detection (if time)

Week 8
Finish everything
Buffer period
Final reflection on USE value preservation


Week 9
Improve wiki for evaluation
Peer review

Fallback: App for user to report feedback in the form of images of high/low quality fruit.
Have Rpi take pictures using Kinect and send to database
Choose a more binary classification (below 50%/above 50% quality)
Flesh out the design more (if implementation fails)

Technical aspects

Research

Harvesting robots

Sensing technology

(Older)

  • Yamamoto, S., et al. "Development of a stationary robotic strawberry harvester with picking mechanism that approaches target fruit from below (Part 1)-Development of the end-effector." Journal of the Japanese Society of Agricultural Machinery 71.6 (2009): 71-78. Link
  • Sam Corbett-Davies , Tom Botterill , Richard Green , Valerie Saxton, An expert system for automatically pruning vines, Proceedings of the 27th Conference on Image and Vision Computing New Zealand, November 26-28, 2012, Dunedin, New Zealand Link
  • Hayashi, Shigehiko, Katsunobu Ganno, Yukitsugu Ishii, and Itsuo Tanaka. "Robotic Harvesting System for Eggplants." JARQ Japan Agricultural Research Quarterly: JARQ 36.3 (2002): 163-68. Web. Link
  • Blasco, J., N. Aleixos, and E. Moltó. "Machine Vision System for Automatic Quality Grading of Fruit." Biosystems Engineering 85.4 (2003): 415-23. Web. Link
  • Cubero, Sergio, Nuria Aleixos, Enrique Moltó, Juan Gómez-Sanchis, and Jose Blasco. "Advances in Machine Vision Applications for Automatic Inspection and Quality Evaluation of Fruits and Vegetables." Food Bioprocess Technol Food and Bioprocess Technology 4.4 (2010): 487-504. Web. Link
  • Tanigaki, Kanae, et al. "Cherry-harvesting robot." Computers and Electronics in Agriculture 63.1 (2008): 65-72. Direct Dianus
    • Evaluation of a cherry-harvesting robot. It picks by grabbing the peduncle and lifting it upwards.
  • Hayashi, Shigehiko, et al. "Evaluation of a strawberry-harvesting robot in a field test." Biosystems Engineering 105.2 (2010): 160-171. Direct Dianus
    • Evaluation of a strawberry-harvesting robot.

State of the art

A small number of tests have been done with machines for harvesting strawberries. These are large, bulky and expensive machines like Agrobot. Cost prices are in the order of 50,000 dollar. Todo: add citations.

A lot of research is done towards inspection by means of machine vision. Todo: add citations and continue.

Further reading

Manual strawberry harvesting process

Source (move to citation)

Meetings

Moved to Talk:PRE2015_4_Groep2.

Retrieved from "https://dsdwiki.wtb.tue.nl/index.php?title=PRE2015_4_Groep2&oldid=26960"