PRE2018 3 Group5

Meaning of used colors

Colors should be removed in the final version. They are to make changes and remarks better visible

• Yellow background: remark what has to be done in section (maybe not directly possible).

General info

Group members

Name	Student ID
Ruben Haakman	0993994
Stan Latten	1257196
Tom Mulders	1008890
Jasper Stam	1006240
Mathijs Vastenhouw	1269496

Problem

When farmers grow crops, the have to deal with weeds growing on their fields in between their crops. To remove these weeds, pesticides are used. These pesticides can be harmful to insects, animals and humans and might even contaminate (ground)water. Clearly an alternative is needed.

Problem statement

In the current situation, a lot of pesticides are used in farming. These pesticides are used for treating bugs and diseases, but also for weeds. With the trend to be more environmentally friendly, we are looking for alternatives for pesticides and big farm trucks. A possible solution for this problem is a cooperation of small autonomous farming machines, which can control a field together. However, this solution is not new, people have already been working on the navigation of these small machines and on the detection of weeds in fields of crops. [1] That’s why we will try to make a weed picking device to be able to pick weeds without damaging the crops. For these small devices, we see future in the vertical agriculture as well, because they allow for a higher field density.

[1] https://ieeexplore.ieee.org/document/6740018

RPCs

Requirements

• The system recharges autonomously
• The system must be able to differentiate crops from weeds
• The system removes weed from the farm field and collects it for disposal
• The system moves itself around the farmfield, following a predefined pattern unique for each farmfield
• The system must not be harmful for the crops.

Preferences

• The system can operate for a long time before having to recharge
• The system should make minimal errors in recognizing weeds
• The system can work with 3D patterns, allowing applications in 'farm flats', reducing land area use
• The system should damage the ground as less as possible

Constraints

• The system has to be more cost-efficient than human workers
• The system has to be intelligent, has to know what to do
• The system does never run out of power, so it has to return to the charging station in time
• The system does not use pesticides

Users and other stakeholders

Should be changed to only users. Other stakeholders should be in USE stakeholder analysis. Objectives of users should also be added here.
The last thing can be done after contact with user (or if we don't get contact with user after additional research)

• Farmers
• Consumers
• Governments
• Society

Project setup

Approach

After reviewing the literature, we will determine the requirements for the system. Based on these requirements we will investigate implementations for these requirements and analyse their suitability. We will analyse the costs associated with a solution and compare this to the current costs of using pesticides, the effects on the stakeholders and on the future of farming. Finally we will conclude with a recommendation for or against the automated removal of weeds on farm fields without the use of pesticides and recommend future research topics.

Milestones

• State-of-the-art analysis
• Requirements Document
• Use analysis
• Implementation propositions
• Implementation analysis
• Cost analysis
• Conclusion

Deliverables

• Requirements document
• Implementation document
• Use analysis
• Cost analysis
• Conclusion

Who's doing what

• Ruben: Design(electronics), cost analysis.
• Stan: Design(general), Requirements, Use analysis.
• Tom: Design(general), Requirements, Use analysis.
• Jasper: Design(software), STOA analysis, Requirements, Use analysis.
• Mathijs: Design(general), STOA analysis, cost analysis.

State of the art

The literature study can be found on the page State of the art

Navigation

There has been a lot of research and patents on autonomous moving and navigating. Robots are able to detect humans and other objects and can distinguish the places where it can drive. Using object recognition and laser detection it is even possible to navigate in crowded places with many pedestrians and cyclists.

Trash collecting

Collecting trash is difficult for a robot, because of the many different shapes, materials and colors that the trash can have. Cleaning dust is easy with a vacuum cleaner but bigger trash can't be done with a vacuum cleaner and must be picked up by the robot. To do this the robot needs good object recognition and must be able to grab objects of many different shapes.

Weed removal

Robots are able to detect and remove weed. It is possible to accurately recognize different kinds of weed by machine vison and precisely remove these without effecting the surrounding area.

Communication

When there are multiple robots which must work together on one task it is important that they can communicate with each other and divide the tasks. Communication can also be used to prevent robots from colliding and let them work more efficiently. There are multiple papers and patents about communication between robots, human-robot interaction and multiple robots driving on sidewalks and other smart city applications.

Jasper

[1][2] Article about a trash collecting robot (team). It is about office cleanup, but with some changes the technique can also be relevant for outside use. It is about a competition. One document describes the solution of the winning team, the other gives some more information about the competition.

[3] A patent for sucking and filtering for a dust collection vehicle.

[4] A patent for an autonomous lawn mower robot. Also about navigating over the lawn.

[5] Paper about weed control, describing navigating through specific areas, detecting weed with a camera, making weed maps and spraying weed.

[6] A patent for a snow sweeper for sidewalks.

Tom

[7] Paper about the design of an autonomous vacuum cleaner.

[8] Paper about pathing algorithms for autonomous vacuum cleaner robots.

[9] Analysis of snow melting approaches.

[10] Paper about machine vision application for weed removal.

[11] Analysis of pavement maintenance methods.

[12] Research into small (< 20kg) urban robots for disaster relief.

[13] Small summary of robots in farming

[14] Autonomous tractors

Stan

[15] Paper about navigation on pavements, avoiding litter, pedestrians and bicycles.

[16] Paper about asphalt analysis, to detect whether the road needs maintenance. This paper was actually meant for airborne sensing, but could be used by our robot as well.

[17] Paper about stair-climbing methods for robots, useful for our robot to easily get on or off the pavement.

[18] Article about weather forecasting in the road [network. Could be used by our robot to predict which task it has to do (e.g. de-icing the road)

[19] Paper about autonomous docking at a recharging station for autonomous vehicles in general

[20] Article about an autonomous cleaning robot for outdoor use, including path-finding and memory of cleaned areas

[21] Article about different kind of weeds

Ruben

[22] Patent for communication of an autonomous sidewalk robot

[23] Patent for an autonomous neighborhood vehicle controllable through a neighborhood social network

[24] Patent for a system and method for navigating an autonomous vehicle using laser detection and ranging

[25] Paper about autonomous vehicles navigating trough sidewalks buildings and hallways

[26]] Paper about an Autonomous Robot for Garbage Detection and Collection

[27] Paper about multiple robots in smart city applications

Mathijs

[28] Paper on small autonomous robots working together to do big tasks.

[29] Paper on autonomous navigation on crowded sidewalks.

[30] Paper on robot navigation in highly populated pedestrian zones.

[31] Paper on human-robot interaction in urban environments.

[32] Paper on the design of a litter collecting robot.

[33] Article on electric snow removal by placing heating mats.

Weed detection and removal

With a growing world population and increasing demand of biological products, farmers are looking for new ways to improve their ways to remove weeds. Therefore there has been a lot of research on improvement of automatic and non-chemical weed removal.

Complete robots which are able to navigate autonomous on a field are already available (5, 36, 41). Also weed detection with a camera and machine vision is possible (5, 33, 34). Most of the weeding robots use chemicals to remove the weeds. There aren't many robots which are able to mechanicaly remove weeds. 42 and 43 describe both a possible method to remove weeds mechanically.

[34] Paper about an algorithm that can detect weeds and also classify it.

[35] Paper about weed recognition trough image processing.

[36] Paper about mechanical weed removal.

[37] Paper about an intelligent mechanical weeding machine

[38] Paper where test for mechanical weed control in greenhouses work not sure if accessible

[39] Thesis on weed control. Very interesting also techniques for removing weed are discussed

[40] Paper about a robot for plant-species–specific weed management using mechanical or chemical module to remove the weed

[41] Paper about different methods for non-chemical weed control

[42] Paper about the complete design of an autonomous weeder robot platform

[43] [44] 2 Papers about mechanical removing weed between plants

Planning

For each week, there are points what we plan to do in that week. Planning can change over the weeks, dependent on the progress in the project. Final versions of the documents will be delivered at the end of the quartile, but concept versions will be delivered earlier.

Week 1

• Introduction to course
• Brainstorming about problem
• Make problem statement
• First idea on plan for project
• Literature study on problem

Week 2

• Updated problem description
• Concrete planning for project
• Make plan more clear with introduction
• Analysis of literature found in week 1
• First idea on requirements
• Start on USE stakeholder analysis

Week 3

• Concrete decisions on prototype
• USE stakeholder analysis
• Make requirements ready to start on design

Week 4-6

• Work on prototype
• Analysis of requirements based on prototype and update if needed
• Analysis of decisions made for prototype and update if needed
• Update other documents if needed

Week 7

• Finalize prototype
• Prepare presentation

Week 8

• Presentation

Function definition

• Method of moving around the farmfield
• Differentiate crops from weeds
• Grab the weeds efficiently, universal for all weed types and orientations
• Move gripper from weed container to the weed on the farmfield and vice versa
• Battery monitoring to know when to stop
• Recharging at recharging station
• Weedcontainer monitoring and emptying mechanism
• Navigation around the farmfield

Navigation around the farmfield

As described in the problem statement, the robot should be able to move autonomously. For this document, we will investigate 1. how the robot autonomously maneuvers across fields, covering it fully and 2. How the robot operates when it needs to charge, needs to empty its container or has finished working a field.

For the autonomous maneuvering across fields we consider the following approaches: Full autonomy: the robot will given a field, determined by GPS coordinates, autonomously decide a(n) (optimal) route to fully cover a field. Semi-autonomy: the robot will follow set route, obtained from GPS data generated by the user or provider of the robot. For both approaches the robot will automatically detect obstacles including humans and animals and halt operation temporarily.

Full autonomy is highly desirable, as this improves the portability of the robot, reduces overhead for client and provider and potentially increases efficiency, in cases where a provided route is not optimal. However, full autonomy is hard to achieve, bears greater upfront costs and potentially is not cost-effective at all. Semi-autonomy, on the other hand, is cheaper and easier to implement, as following a set GPS route is near trivial. However it has the overhead that a GPS route has to be determined before operation can commence. Determining a (good) GPS is not hard, but costs time and effort, and has to be done for every individual field. As discussed in paper 14, determining a route for a tractor is not hard, it only requires the user to drive the desired route and then the autonomous system can replicate it, however in this use case, where the robot potentially is small, this would require a user to either use specialized equipment to determine a route, such as a remote drone, or to walk the route, which is less desirable.

Additionally, for bigger fields, a group of robots working together can be more cost-effective. In the case of fully autonomous robots, this means the robots must communicate and delegate parts of the field to each other. Depending on the implementation this can be done efficiently. Each robot could for example mark the visited coordinates or sectors on a shared digital data structure. Other robots can then avoid this areas and avoid doing extra work. Alternatively, at the initialization of the job, the robots could negotiate a sector of the field to be assigned to them and basically divide and conquer the work, as if the field was actually multiple small fields, worked by a single robot. Either way, this cooperation seems only marginally, if at all, more difficult than creating a fully autonomous robot and as such does not play for or against the fully autonomous system for deployment on bigger fields. As far as the semi-autonomous system is concerned, a possible cooperation technique for multiple such robots operating on the same big field could be a equal division of the pre-set path among all the cooperating robots. In conclusion, the size of the field does not particularly favor either implementation.

Next to the size of the field, let us look at how the shape of the field impacts our design decision. First of all we have (near) rectangular fields. Arguably, such fields are easier to handle, both for (semi-) autonomous as well as non-autonomous systems, than irregular fields. One simple, but possibly suboptimal, manner of dealing with (near) rectangular fields is to drive the full length (or width) of the fields, turn around and repeat until the entire field has been covered. For a semi-autonomous system, such an approach can be created easily as discussed in paper 14. For fully autonomous system, such a field should also be easy to operate on, as in the worst case, it should be able to do exactly the same as a semi-autonomous system; a simple reflex agent, which turns around once detects the field border (via GPS or some other feature) and some termination conditions might even suffice. Let us then turn to irregular field shapes. Currently, farmers are already dealing with irregular fields manually, as such a semi-autonomous system can be given a route without any issues. For autonomous systems, however, irregular fields might prove to be more challenging. Of course, this depends on the quality of the AI, but performance might be lower in the worse case (an approach might for example be a semi-bruteforce of the field), than the performance of a semi-autonomous system, but in the best case this might be better. Either system is able to handle an irregular field, but the performance of the autonomous system is heavily reliant on the quality of the AI and should be investigated for a conclusive recommendation.

Overall, when only looking at the maneuvering of the robot on the field, TODO: safety , a robot is able to autonomously deal with an entire field. A semi-autonomous system is guaranteed to fully cover the field, and is cheaper than a fully autonomous solution, however it has the additional overhead that a route has to be predefined. A fully autonomous robot has the potential to be more efficient in its route, but is more technically challenging. Since the size of the field, and the cooperation between robots, does not favor any particular solution and the shape of the field inconclusively favors a semi-autonomous system, our recommendation is to utilize a semi-autonomous system for now, until fully autonomous systems become more mainstream and less expensive.

Next we discuss how a system should behave when it needs to charge, needs to empty its container or has finished working the field. We can distinguish a couple of behaviours in such cases: The robot signals the user and waits for it to be picked up The robot stops somewhere in the field The robot moves to the edge of the field or even a designated point at the edge of the field The robot autonomously moves to a point, where it can charge, empty or wait The point is at the field edge The point is further away The robot autonomously moves to work another field

Before we discuss the pros and cons of the above behaviours, we must point out that the desired behaviour likely depends on the user. Some farms have all its fields close to or adjacent to each other, whereas other farms have their fields scattered around the vicinity. This heavily impacts whether a user is willing to pick up the robots or wants the robots to come to some “waiting place” or continue working another field. Also the wealth of the user determines whether each field has a charging point, or not. Finally the implementation of the charging point impacts the above: is it a fixed-in-place charging point, or more like a powerbank that you place where and when needed? TODO: specify charging point On closed farms, meaning farms where you can move between fields without encountering random people or vehicles anywhere, a more automated approach is viable: The robot could move from a field to a charge, empty or wait point somewhere else on the farm easily by just following a set route. It would then be the user’s responsibility that no objects or people get in the way, which is very manageable. Alternatively, again, a fully autonomous system could be used for the routing and maneuvering to the point. We can present similar pros and cons for a fully autonomous versus a semi-autonomous system again as above, but for this particular use case (closed farm) a semi-autonomous system would do just fine.

Concept gripper

The gripper should grab all kinds of weeds and remove them efficiently. The first concept uses two long parallel rods, which extend under the weed so the stem of the weed will be in between the two rods. When the gripper is at its place, the rod, mounted to a carriage, will be moved towards the stationary rod and the stem will be clamped in between the two rods. Now the weed is hold tightly at a relatively strong point, it can be pulled out of the soil, together with its roots. The gripper can then move to a weed container and dropping the weed in there. Below, a drawing of this gripper is shown, together with a carriage that will move one of the rods. The movement of the carriage is done by a rack and pinion actuator. The drawing is just a concept: manufacturing margins, motors, bearing seatings etc. still have been left out of the picture.

Side view of the gripper

Isometric view of the gripper

Battery

The robot needs to have energy to operate. The easiest way to provide the robot with energy is using a battery. Putting solar cells on the robot will not be enough, because the robot will operate underneath and between trees. The trees are blocking the sun, so the solar panels will not generate that much energy. Another option is providing energy by cables. A very long cable is needed in this situation, because the scope in which the robot works is very big. The cable can also be stuck behind some tree or other obstacle. Stationary cables providing energy (like is used by electric trains) is also not convenient. It is dangerous for human that enter the area, because they can get an electric shock. Also lots of cables are needed and it can be damaged easy. A battery doesn't have those disadvantages, but there is another disadvantage. A battery can run out of energy.

A battery can be recharged, but that takes time. If the battery is in the robot, the robot has to wait till the battery is charged and cannot do the tasks for which it is intended. To solve this problem, there is an existing solution, namely a battery change system. In such a system, the battery can be easily removed from the robot and another battery can be placed. Currently the battery change is mostly done by humans, however the process is not difficult. We found a patent on a battery change system. The battery itself has contacts two opposite sides. At one side the contacts are used for charging and the contacts at the other side are used when the battery is in use. To remove the battery is something like a conveyor belt used. When the battery is removed, then a full battery needs to be placed in front of the battery space in the robot. This can be done with a linear movable battery rack. When the full battery is in front of the gap the conveyor belt is applied again to place the full battery in the robot. To see a picture of this, see [45].

After measuring the battery usage the robot knows how long it can operate before getting the next battery. In the navigation, this can be taken into account. If this is done correctly, the robot will in practice never run out of battery.

Weed container

If you put weed in a container, then it doesn't nicely fill up the container. To solve this problem, you can use a press, like in a garbage truck. This presses the weed, such that it takes less place and the robot can work longer without emptying the container. To detect how full the container is, you can measure to which position the press comes with a specified force. Emptying the container works the same as with garbage trucks: tilting the container above some bigger container.

To Do

ToDo group 5

USE Analysis

In this analysis we will first sketch the ideal operation of the robot: its functionality and deployment. Then we will analyse the effects and implications of the robot for each of the following stakeholders: Farmers, consumers, governments, society.

Functionality and deployment The robot operates exclusively on farm fields (potentially in a greenhouse). It removes weeds from in between the crop, without damaging the crop and without using pesticides. Potentially multiple robots work together on the same field.

Stakeholders Farmers: First of all, farmers will no longer have to purchase pesticides, but will have to buy and maintain the robot. While the upfront cost of a (set of) robot(s) might be bigger than the cost of pesticides, the maintenance cost will be lower and hence will be more cost effective in the long run. Without the use of pesticides, farmers will no longer have to worry about any of the negative effects of the pesticides and hence will never suffer the consequences of potentially harmful product. Farmers may see an increase in demand and hence revenue, as people are potentially more inclined to buy pesticide-free products. Removed weeds can potentially be used as fertilizer or fuel.

Consumers: No pesticides on food suggests healthier food and hence healthier and happier consumers. Since no pesticides have to be purchased, the product are cheaper.

Governments: Do not deal with the consequences of harmful product, contaminated (ground) water etc.

Society: Less pesticides implies healthier ecosystem, hence better world to live in.

User analysis

The document describes potential user groups, and discusses which solutions fit which customer and on which customer we will focus. In general, our product is aimed towards farmers who can deploy the robot on their fields. However, there are different kind of farms and different kind of farmers. By the nature and purpose of the robot, it should be evident that robot is aimed towards farmers who grow crops and experience negative effects from weed growth on their fields. This excludes farmers who only keep animals, or do not have a weeds growing on their fields (like in greenhouses). Hence the main prospective user group is farmers who grows crops outside (so not in a greenhouse or anything alike).

We can identify multiple different type of farms in this subgroup, however. Open farms: farms with their fields scattered around the area, where the areas between the farms can contain roads, buildings or other entities, not owned or controlled by the user. On such farms, the environment is highly variable and uncontrolled. People or animals can be found around or even on the fields. Closed farms: farms with their fields on a single, or set of, properties owned and managed by the user. These properties are closed for the public and the environment is controlled. Anything that happens on the farm can be controlled and adjusted in a way the user desires. Next-gen automated farms: farms which are in an experimental phase and are aimed towards full automation. As such, these farms are designed and managed for and by robots. Outside interaction is (very) limited. Conditions are controlled. (example: pixel farming)

Each of these user groups will require the robot and autonomous system we are discussing to behave differently. For open farms, a user must either pick up the robots from fields and place them at fields themselves, or the robot must be able to maneuver public grounds and roads autonomously. Moverover, each field or set fields that is disconnected from other fields, must either have a charging and emptying point, or the user must manually move the robots to such points. Or even have to empty and charge the robot manually. Clearly, a fully autonomous robot could do such things autonomously, but for the near future such autonomy does not exist yet. Designing, implementing, and testing it would cost a lot of time and money and hence we decide to not offer such autonomy. The user-unfriendly nature of manual pick-ups and drop-offs of the robots is unlikely to appeal to any real customer, and as a result, we will not focus on this user group our system. As far as closed farms are concerned, an automated system is easier to realise. The movement between fields, charging and emptying points is not hindered by any entities, assuming the user enables this: proper briefing of staff and keeping routes obstacle free will allow an automated system to function effectively. Given this, our robot system will be able to operate effectively and efficiently: since we do not offer full autonomy, the routes in between fields, charging and emptying points will have to be provided to the system, as such, these routes should be kept fully accessible at all time. In conclusion, closed farms are a consideration for the main target group for this project.

Finally, there is next-gen automated farms. By the nature of these farms, our robot would be a perfect fit in such environments. Hence such farms are a consideration for the main target group for this project.

In order to maximize the number of potential users, and the experimental nature of “next-gen” farming, we have chosen to focus our product towards closed farms.

Design

This document describes the high level design and design decisions regarding an automated weeding robot for closed farms. This document continues from the RPCs that have been decided and the user analysis, as well as the STOA.

Mobility and overall package Wheels? Motors Describe performance: speed, turning circle etc. Size Capabilities: waterproof? Can work at night?

Navigation As decided in the user analysis, the robot will navigate using a predefined GPS route. To that extent, the robot’s software must be programmed to follow such a route, halt and continue a route when obstacles present itself, and allow the weeder to clear out weeds. how to obtain gps route Behaviour when encountering obstacles Weeding during driving? Charging, emptying, finished

Charging and emptying Describe station Charging, emptying process Charging details Emptying details

Weeds detection Camera, paper Performance: accuracy, speed

Weeding Arm Gripper performance

Communication Communicate status with user

Contact with users and other research teams

availability

• Tom: maandag middag, dinsdag, donderdag middag
• Jasper: maandag middag, dinsdag ochtend niet 19-2, woensdag ochtend, donderdag middag
• Ruben: Maandag middag, woensdag vanaf 13 maart, donderdag 21 en 28 maart, vrijdag middag
• Mathijs: Maandag middag, dinsdag middag niet 26, woensdag, donderdag.
• stan: Maandag middag, dinsdag middag, woensdag, donderdag.

Questions

User

• Is weed a big problem?
• How do you currently fight weed?
• How how many time costs it take to fight weed?
• How many people are needed to fight weed?
• What is the planning in removing weed? i.e. after how many time should you start again with the fields, how many times a season do you need to go over each field.
• How big is the area in which weed should be removed?
• Are all the locations reachable without using public roads?
• Are there many animals such as rabbits, birds in the fields?
• Which tools do you use currently?
• What is the cost of the tools and how long do they last?
• What it the cost of the people that are removing the weed?

Other researchers

• What are the main advantages of fighting weed with robots?
• What is the main problem of pesticides?
• What is the current solution, you have?
• What is the weight of that solution?
• How many energy consumes the current solution?
• Is the solution dependent on the type of crop and on other circumstances?

Interview

Overview of situation

Twigs that should be removed

Obstruction by dead tree and dead weeds

Obstruction by tree trunks

Obstruction by broken guide wood

Interview with the Farmer

We went to a farmer and came to the conclusion that greenhouses are not the right location for our robot plans and ideas. However, for the outside fruit cultivation it could be very useful. We got the tip to look at fruit trees like apples, pear and cherries, instead of his strawberries. These are grown in the neighborhood as well. We found a fruit farmer who was willing to tell us something about his farming and he answered all our questions. After the interview we went to have a look in the fields. We made pictures of the situation and from a lot of the common obstructions. This gives us a good image for making a model of the environment we are facing.

 Is weed a big problem? Weed is quite an issue as it takes away moisture and nutrients from the crops growing and thus having a negative impact on harvest rates as the crops cannot grow as good as they would without weeds.  How do you currently fight weed? We are currently using herbicides to fight the weeds. As the biggest problem we have is actually the weed couch-grass, which is found everywhere around the trees, removing this manually without the use of herbicides is almost impossible as it is a very time consuming task.  How much time does it take to fight weed? We can treat the complete farm with herbicides in a couple of hours. However, before we can do the treatment, the land has to be dry and the weather forecast should not give any rain for the coming day, in order for the herbicides to do their work.  How many people are needed to fight weed? To apply the herbicides to the farmland, we drive in between the tree rows with a small tractor, including a trailer which contains a tank with the herbicide mixture and a spraying device which sprays the herbicides just next to the trees.  What is the planning in removing weed? i.e. after how many time should you start again with the fields, how many times a season do you need to go over each field.

 How big is the area in which weed should be removed?  Are all the locations reachable without using public roads?  Are there many animals such as rabbits, birds in the fields?  Which tools do you use currently?  What is the cost of the tools and how long do they last?  What it the cost of the people that are removing the weed?