PRE2018 3 Group5

Group members

Student ID
0993994
1257196
1008890
1006240
1269496

Project setup

Approach

After reviewing the literature, we will determine the requirements for the system. Based on these requirements we will investigate implementations for these requirements and analyze their suitability. We will analyze the costs associated with a solution and compare this to the current costs of using pesticides, the effects on the stakeholders and on the future of farming. Finally we will conclude with a recommendation for or against the automated removal of weeds on farm fields without the use of pesticides and recommend future research topics.

Milestones

• State-of-the-art analysis
• Requirements
• USE analysis
• Design description
• Model
• Cost analysis
• Conclusion

Deliverables

• Requirements
• Design
• USE analysis
• Cost analysis
• Conclusion

Planning

planning

Week 2

• Arguments for problem - Everyone
• Concrete planning - Jasper
• Goal of the project, some introduction for project. Little story - Mathijs
• State of the Art analysis - Ruben
• Start with requirements - Stan
• USE analysis - Tom
• Function definition - Everyone
• Concepts for functions - Everyone

Week 3

• Contact zoeken - Mathijs
• State of the Art aanpassen - Ruben
• Problem statement aanpassen - Tom
• Nieuwe state of the Art content - Tom, Jasper
• Idee voor grijper - Stan

Week 4

• Contact boeren, domeinkennis onkruid - Mathijs
• Contact onderzoekers - Mathijs, Stan
• Vragenlijst - Jasper
• Requirements - Jasper
• User analysis - Tom
• Movement system - Ruben

Week 5

• Update wiki general info - Tom
• Literature study/refer to articles
  • Weed - Ruben
  • Navigation - Tom
  • All other functions (charging, mobility, communication) - Jasper
• Visit greenhouse (inclusive preparation and reporting on wiki) - Mathijs, Stan
• Requirements (after visit greenhouse) - Jasper, Tom

Update after Friday meeting

• Update structure of Wiki - Jasper
• Structure of State of the art (not grouped on name, sources notation etc.) - Jasper
• Function definition update - Jasper
• Report on visit farmer - Stan, Mathijs
• Requirements - Tom
• Modeling of solution - Stan, Mathijs
• Function definition navigation - Tom
• Literature study weed - Ruben
• Planning - Tom

Week 6

• UI - Jasper
• Communicatie user - Jasper
• Weed container - Jasper
• STOA - Jasper
• Cost analysis, lease, etc - Tom
• Visualisatie - Mathijs, Stan
• Design arm/gripper - Mathijs, Stan
• Weed detection, types of weeds, interval - Ruben
• Planning - Tom
• Target User, schets operation - Tom
• Safety - Ruben

Update after Friday meeting

• Change UI mockup - Jasper
• Topics presentation - Jasper
• Cost analysis - Tom
• Target user - Tom
• Schets operation - Tom
• Weed detection etc. - Ruben
• Safety - Ruben
• Modeling design - Mathijs, Stan

Week 7

• Cost analysis - Tom
• Design decisions - Tom
• Presentation - Ruben, Jasper
• Missing answers to interview - Stan, Mathijs
• CAD model - Stan, Mathijs
• Video of design - Stan, Mathijs
• 3D printed gripper? - Stan, Mathijs
• Sources in weed - Ruben
• Safety - Ruben
• Structure wiki - Jasper

Week 8

• Validation - Stan, Mathijs
• Problem description - Ruben
• Cost analysis - Tom
• Structure wiki - Jasper
• Cleanup wiki - Jasper
• Conclusion/recommendation - Tom
• Design decisions - Stan, Mathijs
• Pictures CAD model - Stan, Mathijs

Problem statement

Weed control has always been a big challenge in farming. Weeds can decrease the amount of space, light, water and nutrients available for the crops. Good weed control has therefore become an important part in farming because it has a big impact on the amount which can be harvested from the plants. To remove these weeds, often pesticides are used. These pesticides can be harmful to insects, animals and humans and might even contaminate (ground)water. This could eventually lead to those pesticides end up in our food. There are also concerns about the health for the farmers and people living nearby the fields where pesticides are used.

Due to all these disadvantages and concerns about pesticides there is an increasing demand for biological products. With this trend of increasing demand of biological products and the industry to become more environmentally friendly, we are looking for alternatives for pesticides and big farm trucks. A possible solution for this problem is a cooperation of small autonomous farming machines, which can control a field together. However, this solution is not new, people have already been working on the navigation of these small machines and on the detection of weeds in fields of crops. [28] That’s why we will try to make a weed picking device to be able to pick weeds without damaging the crops.

State of the Art

Articles

Article about a trash collecting robot (team). It is about office cleanup, but with some changes the technique can also be relevant for outside use. It is about a competition. One document describes the solution of the winning team, the other gives some more information about the competition.

A patent for sucking and filtering for a dust collection vehicle.

A patent for an autonomous lawn mower robot. Also about navigating over the lawn.

Paper about weed control, describing navigating through specific areas, detecting weed with a camera, making weed maps and spraying weed.

A patent for a snow sweeper for sidewalks.

Paper about the design of an autonomous vacuum cleaner.

Paper about pathing algorithms for autonomous vacuum cleaner robots.

Analysis of snow melting approaches.

Paper about machine vision application for weed removal.

Analysis of pavement maintenance methods.

Research into small (< 20kg) urban robots for disaster relief.

Small summary of robots in farming

Autonomous tractors

Paper about navigation on pavements, avoiding litter, pedestrians and bicycles.

Paper about asphalt analysis, to detect whether the road needs maintenance. This paper was actually meant for airborne sensing, but could be used by our robot as well.

Paper about stair-climbing methods for robots, useful for our robot to easily get on or off the pavement.

Article about weather forecasting in the road [network. Could be used by our robot to predict which task it has to do (e.g. de-icing the road)

Paper about autonomous docking at a recharging station for autonomous vehicles in general

Article about an autonomous cleaning robot for outdoor use, including path-finding and memory of cleaned areas

Article about different kind of weeds

Patent for communication of an autonomous sidewalk robot

Patent for an autonomous neighborhood vehicle controllable through a neighborhood social network

Patent for a system and method for navigating an autonomous vehicle using laser detection and ranging

Paper about autonomous vehicles navigating trough sidewalks buildings and hallways

Paper about an Autonomous Robot for Garbage Detection and Collection

Paper about multiple robots in smart city applications

Paper on small autonomous robots working together to do big tasks.

Paper on autonomous navigation on crowded sidewalks.

Paper on robot navigation in highly populated pedestrian zones.

Paper on human-robot interaction in urban environments.

Paper on the design of a litter collecting robot.

Article on electric snow removal by placing heating mats.

Paper about an algorithm that can detect weeds and also classify it.

Paper about weed recognition trough image processing.

Paper about mechanical weed removal.

Paper about an intelligent mechanical weeding machine

Paper where test for mechanical weed control in greenhouses work

Thesis on weed control. Very interesting also techniques for removing weed are discussed

Paper about a robot for plant-species–specific weed management using mechanical or chemical module to remove the weed

Paper about different methods for non-chemical weed control

Paper about the complete design of an autonomous weeder robot platform

2 Papers about mechanical removing weed between plants

Analysis of articles

We found reports complete farming robots that are fighting weed. Some of these robots are spraying pesticide to fight weed. Our intention is to not use pesticides, but remove the weed mechanically. The reports are on robots that can be used in other farming disciplines, for example outside vegetable growing and in greenhouses. In those reports there is useful information about different important topics for us, like navigation and weed recognition. Examples of those reports are

Navigation

Navigation is an important topic. The robot should be able to find its way trough the fields in which it operates. Navigation is not only finding a route, such that the complete field is covered, but also the lifespan of the battery has to be taken into account and obstacles has to be avoided. We found several papers on navigation, both in farming and other robotic areas. The navigation in other robotic areas can be useful for the way of navigating in farming, because of similarities.

Recognition of weed

The robot has to detect weed autonomously. The robot has to remove only the weed and not the good vegetation. For this end, artificial intelligence is needed. When the robot detected the weed, it can move the arm with the tool to the weed and remove it. We found some papers about detection of weed. Those papers show that weed can be detected and that weed can be differentiated from crops. For this robot, this thing cannot be copied one to one from another application, but the same methodology can be followed to make weed recognition from a camera working on this robot. This is, because in this area, there appear other types of weed.

Collaboration of different robots

When the robot is applied on a larger scale, it can be that multiple robots have to work together. When robots work together, they have to communicate to each other, because if they don't do that, situations can appear that they are constantly waiting for each other and they are not efficient. It can even be that the robots recognize each other as an unknown obstacle and stop with their task for safety. To prevent this, we found also some articles about robot collaboration, such that we can think about a way of controlling the robot, such that it can work together.

Weed control

Research into weed is very important. We have to know which types of weed there are and how we can remove it. The robot should also not damage the crops. We also have to know with which frequency the robot has to maintain the fields, to keep it, until an acceptable level, free of weed. We also found papers about types of weed in our case and papers on how to remove wee mechanically.

General design of robot

To get an idea for a design for a robot, we found some papers on different kinds of robots. From those papers we maybe can use information for our design.

Battery and charging

Our robot has to be provided with energy. We also did some research into charging.

Current situation

To know the environment in which the robot has to work, we contacted a potential user. In his fields, we could see some examples of obstacles and weed and we know how big the robot can be and on what surface it has to drive. We also asked the potential user how he currently fights the weed and asked about his knowledge about weed and the crops. For more information, see this section.

Users

USE Analysis

In this analysis we will first sketch the ideal operation of the robot: its functionality and deployment. Then we will analyze the effects and implications of the robot for each of the following stakeholders: Farmers, consumers, governments, society.

Functionality and deployment The robot operates exclusively on fruit orchards. It removes weeds from in between the trees, without damaging the trees and without using pesticides.

Stakeholders Farmers: First of all, farmers will no longer have to purchase pesticides, but will have to buy and maintain the robot. While the upfront cost of a (set of) robot(s) might be bigger than the cost of pesticides, the maintenance cost will be lower and hence will be more cost effective in the long run. Without the use of pesticides, farmers will no longer have to worry about any of the negative effects of the pesticides and hence will never suffer the consequences of potentially harmful product. Farmers may see an increase in demand and hence revenue, as people are potentially more inclined to buy pesticide-free products. Removed weeds can potentially be used as fertilizer, fuel or fodder, further highlighting the financial benefits.

Consumers: No pesticides on food suggests healthier food and hence healthier and happier consumers. Since no pesticides have to be purchased, the product are cheaper. No pesticides in orchards reduces the chances of accidental consumption of contaminated produce by for example dogs. The means consumers have less to worry about and are generally more happy.

Governments: Do not deal with the consequences of harmful product, contaminated (ground) water etc. Reduced costs for farming will allow poor government to produce more food, reducing famine.

Society: Less pesticides implies healthier ecosystem, hence better world to live in. More cost-effective farming means more money for other sectors such as healthcare. More cost-effective farming means more food and less famine.

Target user

The document describes potential user groups, and discusses which solutions fit which customer and on which customer we will focus. In general, our product is aimed towards farmers who can deploy the robot on their orchards. However, there are different kind of farms and different kind of farmers. By the nature and purpose of the robot, it should be evident that robot is aimed towards farmers who grow fruit trees and experience negative effects from weed growth on their fields. This excludes farmers who only keep animals, or do not have a weeds growing on their fields (like in greenhouses). Hence the main prospective user group is farmers who grows fruit trees outside (so not in a greenhouse or anything alike).

We can identify multiple different type of farms in this subgroup, however. Open farms: farms with their fields scattered around the area, where the areas between the farms can contain roads, buildings or other entities, not owned or controlled by the user. On such farms, the environment is highly variable and uncontrolled. People or animals can be found around or even on the fields. Closed farms: farms with their fields on a single, or set of, properties owned and managed by the user. These properties are closed for the public and the environment is controlled. Anything that happens on the farm can be controlled and adjusted in a way the user desires. Next-gen automated farms: farms which are in an experimental phase and are aimed towards full automation. As such, these farms are designed and managed for and by robots. Outside interaction is (very) limited. Conditions are controlled. (example: pixel farming)

Each of these user groups will require the robot and autonomous system we are discussing to behave differently. For open farms, a user must either pick up the robots from fields and place them at fields themselves, or the robot must be able to maneuver public grounds and roads autonomously. Moreover, each field or set fields that is disconnected from other fields, must either have a charging and emptying point, or the user must manually move the robots to such points. Or even have to empty and charge the robot manually. Clearly, a fully autonomous robot could do such things autonomously, but for the near future such autonomy does not exist yet. Designing, implementing, and testing it would cost a lot of time and money and hence we decide to not offer such autonomy. The user-unfriendly nature of manual pick-ups and drop-offs of the robots is unlikely to appeal to any real customer, and as a result, we will not focus on this user group our system. As far as closed farms are concerned, an automated system is easier to realise. The movement between fields, charging and emptying points is not hindered by any entities, assuming the user enables this: proper briefing of staff and keeping routes obstacle free will allow an automated system to function effectively. Given this, our robot system will be able to operate effectively and efficiently: since we do not offer full autonomy, the routes in between fields, charging and emptying points will have to be provided to the system, as such, these routes should be kept fully accessible at all time. In conclusion, closed farms are a consideration for the main target group for this project.

Finally, there is next-gen automated farms. By the nature of these farms, our robot would be a perfect fit in such environments. Hence such farms are a consideration for the main target group for this project.

In order to maximize the number of potential users, and the experimental nature of “next-gen” farming, we have chosen to focus our product towards closed farms. Moreover, from our interview with a farmer with a fruit orchard, we have concluded that a fruit orchards is the ideal farm for a first-generation automated weeding system, which we aim to design. A fruit orchard is usually completely closed off from the public, has a clear layout and is designed in such a way that a tractor can easily move trough its entirety. The robot can thus move in a similar way as a tractor through the orchard and not get stuck on random objects or inconsistencies in the ground. Also, since the branches of the trees are not close to the ground, the robot will encounter few obstacles and not harm the trees or its branches.

Contact with users and other research teams

Availability

• Tom: Monday afternoon, Tuesday, Thursday afternoon
• Jasper: Monday afternoon, Tuesday morning not 19-2, Wednesday morning, Thursday afternoon
• Ruben: Monday afternoon, Wednesday after 13-3, Thursday 21-3 and 28-3, Friday afternoon
• Mathijs: Monday afternoon, Tuesday afternoon not 26-2, Wednesday, Thursday
• Stan: Monday afternoon, Tuesday afternoon, Wednesday, Thursday

Questions

We came up with some questions that we have for a potential user and for other research teams. These questions are aimed to fine tune our design to the user's needs and improve our understanding of the problem scope. The answers to these question will be central to our solution.

User

• Is weed a big problem?
• How do you currently fight weed?
• How how many time costs it take to fight weed?
• How many people are needed to fight weed?
• What is the planning in removing weed? i.e. after how many time should you start again with the fields, how many times a season do you need to go over each field.
• How big is the area in which weed should be removed?
• Are all the locations reachable without using public roads?
• Are there many animals such as rabbits, birds in the fields?
• Which tools do you use currently?
• What is the cost of the tools and how long do they last?
• What it the cost of the people that are removing the weed?

Other researchers

• What are the main advantages of fighting weed with robots?
• What is the main problem of pesticides?
• What is the current solution, you have?
• What is the weight of that solution?
• How many energy consumes the current solution?
• Is the solution dependent on the type of crop and on other circumstances?

Beltech contact log

We called Beltech to ask if it was possible to interview them about the possibilities of a mechanical weeds removal tool for use on the farm fields. Ron van Dooren, head marketing answered and he told us that there were possibilities for us and that Richard Vialle knew all the details about their weeds removal machine and that he should help us. We should send a mail to info@beltech.nl containing what our expectations for them were and what we wanted to know. This mail was sent and then on Monday 04-03-2019 we got a confirmation that the mail was received and that it was forwarded to Richard Vialle. He would reply to our request and help us further.

As we did not receive any reply from our mail, we contacted Beltech again to ask if they had taken a look at our mail. On the phone, we were told that both Ron van Dooren and Richard Vialle were not available and that sending a mail would be the fastest way to contact the right person. After the call, we sent another mail asking if they had taken a look at our request and Ron van Dooren replied, stating that he would remind Richard Vialle to answer us. This mail was received on Thursday 14-03-2019. From there on, we have not received a reply from Richard Vialle. In the previous mail we sent, we also included all dates and times where we would be available for an appointment, for them to pick a right time. This was done to speed up the communication as we do not have much time left if we want to incorporate their information into our project.

Interview

Overview of situation

Twigs that should be removed

Obstruction by dead tree and dead weeds

Obstruction by tree trunks

Obstruction by broken guide wood

Interview with the Farmer

We went to a farmer and came to the conclusion that greenhouses are not the right location for our robot plans and ideas. However, for the outside fruit cultivation it could be very useful. We got the tip to look at fruit trees like apples, pear and cherries, instead of his strawberries. These are grown in the neighborhood as well. We found a fruit farmer who was willing to tell us something about his farming and he answered all our questions. After the interview we went to have a look in the fields. We made pictures of the situation and from a lot of the common obstructions. This gives us a good image for making a model of the environment we are facing.

• Is weed a big problem?

Weed is quite an issue as it takes away moisture and nutrients from the crops growing and thus having a negative impact on harvest rates as the crops cannot grow as good as they would without weeds.

• How do you currently fight weed?

We are currently using herbicides to fight the weeds. As the biggest problem we have is actually the weed couch-grass, which is found everywhere around the trees, removing this manually without the use of herbicides is almost impossible as it is a very time consuming task.

• How much time does it take to fight weed?

We can treat the complete farm with herbicides in a couple of hours. However, before we can do the treatment, the land has to be dry and the weather forecast should not give any rain for the coming day, in order for the herbicides to do their work.

• How many people are needed to fight weed?

To apply the herbicides to the farmland, we drive in between the tree rows with a small tractor, including a trailer which contains a tank with the herbicide mixture and a spraying device which sprays the herbicides just next to the trees.

• What is the planning in removing weed? i.e. after how many time should you start again with the fields, how many times a season do you need to go over each field.

The amount of times we need to spray depends on the type of herbicide used. When there are a lot of though weeds, we tend to use a stronger herbicide. This way, you do not have to spray it that often as it remains in the ground for longer, but using strong herbicides is not preferable as it is also more aggressive towards the fruit trees, but on average we have to spray three times a year.

• Are all the locations reachable without using public roads?

We have several different orchards separated by public roads. However, there is a patch of land at the ends of each tree row for the tractor or robot to turn without crossing the public road. This way, driving through the orchard is possible without crossing public roads.

• Are there many animals such as rabbits, birds in the fields?

Sometimes there are rabbits in the orchard, but they use it only to cross the land. They normally do not stay for long. Birds are a bigger problem as they normally come to eat the fruit. This way, we cannot sell it anymore and our yieldings decrease. Sometimes we do find mice in small holes, but this is not that much different of the animals that you find in an average garden.

• Which tools do you use currently?

For weed removal, we use a small tractor with a herbicides spraying device. This device is basically a container holding the herbicide mixture and a sprayer which delivers it at the right place. The tractor is used to move the spraying device along the trees in the orchard.

• What is the cost of the tools and how long do they last?

The tractor uses fuel to operate, but as we give it regular maintenance, it lasts for a lot of years. If something breaks, we try to repair it ourselves if possible or otherwise call a service company to take a look at it, but this happens very rarely. This also applies for the spraying device. Giving a specific number for the use costs is quite difficult as sometimes there are expensive parts breaking, which have to be replaced, but this does not happen every year.

• What it the cost of the people that are removing the weed?

We use herbicides to fight weeds and to apply them, this can be done by a single person driving the tractor, including container with herbicides which are then sprayed around the trees. This way, the amount of man-hours can be kept relatively low, but manual weed removal would cost a huge amount of labor as it is not only hard to get rid of the weeds in the ground, they also return faster without the use of herbicides.

Design

RPCs

Requirements

• The system recharges autonomously
• The system must be able to differentiate crops from weeds
• The system removes weed from the farm field and collects it for disposal
• The system moves itself around the farmfield, following a predefined pattern unique for each farmfield
• The system must not be harmful for the crops
• The system detects obstructions in its path
• The system can notify users on its status
• The system can carry weeds

Preferences

• The system can operate for a long time before having to recharge
• The system should make minimal errors in recognizing weeds
• The system should damage its surroundings as less as possible

Constraints

• The system is more cost-efficient than human workers
• The system is more cost-efficient that using pesticides
• The system traverses the field autonomously
• The system goes to recharge, before running out of battery charge
• The system does not use pesticides

Weed control

Good weed control is important for farmers in order to maximize their yields. Weeds can decrease the amount of space, light, water and nutrients available for the crops. Weeds can also act as a shelter for insects and other animals like rats and mice. It is necessary to remove these weeds and prevent them from growing. Weed control has become an important part in farming because it has a big impact on the amount which can be harvested from the plants.

Weeds can be divided into three groups:

• Annual weeds: These weeds spread by seed and have a lifespan of one year, but produce a lot of seeds for the next year.
• Biennial weeds: lifespan of two years. The grow only a cluster of leaves in the first year. The second year it produces flowers and seeds after which it will die.
• Perennial weeds: Those weeds have big roots underground, so they are able to survive multiple years. Even if the part above the ground is removed, it will grow again the next year. These weeds are therefore the hardest to control.

Examples of some of the most common weeds:

• Cleavers (annual weed)
• Thistles (biennial weed)
• Stingnig nettles (perennial weed)
• Couch grass (perennial weed)

Types of weed control:

• Cultural: Prevents the weeds from growing by reducing open spaces where weeds can grow by placing the desired plants close to each other. This method can be used in gardens but isn’t practical at farming because most crops need enough space between each other.

• Mechanical: Pulling out or damaging the weeds causing them to die. These methods are effective but often time consuming. Pulling out the weed including the roots is one of the most effective ways. However, this is very time consuming since it must be done by hand. Ploughing the ground uproots weeds and causing most of them to die. This can be done with a machine, but some weeds can still continue growing if their roots aren't damaged.

• Chemical: Using pesticides which kill the weeds but not the desired plants. This method is less time consuming but doesn’t work with all weeds and can be harmful for the environment. Farmers spray the chemicals mostly with a big machine on the weeds. This can only be done when there isn't too much wind.

Weed detection

In order to remove the weeds it is necessary for the robot to be able to recognize the weed so it can remove it. As described in the state of the art there have been multiple studies on weed detection. With good cameras and image processing it is possible to accurately recognize weeds even between the crops. Most existing weed detection systems use a normal camera to take pictures of the ground which then are analyzed by a computer. It is out of scope to design such a system. The hardware is for those systems is widely available. The software is the most important part of these systems. The cost and time needed for developing is heavily dependent on the accuracy and speed of the system.

When and how fast weeds grow depends on the type of weed, the weather, the soil and many other factors. There are many different kinds of weed and they all start growing on a different time. Most weeds start growing in the spring when it is getting warmer outside. There is no fixed interval between weed removals. It depends on the temperature, the weed and if there were any roots or seeds left behind since the last time the weed was removed.

Navigation

As described in the problem statement, the robot should be able to move autonomously. We will investigate:

1. How the robot autonomously maneuvers through orchards, covering it fully.
2. How the robot operates when it needs to charge, needs to empty its container or has finished working a field.

For the autonomous maneuvering across fields we consider the following approaches:

• Full autonomy: the robot will given a field, determined by GPS coordinates, autonomously decide a(n) (optimal) route to fully cover a field.
• Semi-autonomy: the robot will follow set route, obtained from GPS data generated by the user or provider of the robot.

For both approaches the robot will automatically detect obstacles including humans and animals and halt operation temporarily.

Full autonomy is highly desirable, as this improves the portability of the robot, reduces overhead for client and provider and potentially increases efficiency, in cases where a provided route is not optimal. However, full autonomy is hard to achieve, bears greater upfront costs and potentially is not cost-effective at all. Semi-autonomy, on the other hand, is cheaper and easier to implement, as following a set GPS route is near trivial. However it has the overhead that a GPS route has to be determined before operation can commence. Determining a (good) GPS is not hard, but costs time and effort, and has to be done for every individual field. As discussed in paper 14, determining a route for a tractor is not hard, it only requires the user to drive the desired route and then the autonomous system can replicate it, however in this use case, where the robot potentially is small, this would require a user to either use specialized equipment to determine a route, such as a remote drone, or to walk the route, which is less desirable. It is out of scope to fully design a system whichs creates a GPS route for the robot, however a solution similar to the method in paper 14 is suitable and technologically possible.

Additionally, for bigger fields, a group of robots working together can be more cost-effective. In the case of fully autonomous robots, this means the robots must communicate and delegate parts of the field to each other. Depending on the implementation this can be done efficiently. Each robot could for example mark the visited coordinates or sectors on a shared digital data structure. Other robots can then avoid this areas and avoid doing extra work. Alternatively, at the initialization of the job, the robots could negotiate a sector of the field to be assigned to them and basically divide and conquer the work, as if the field was actually multiple small fields, worked by a single robot. Either way, this cooperation seems only marginally, if at all, more difficult than creating a fully autonomous robot and as such does not play for or against the fully autonomous system for deployment on bigger fields. As far as the semi-autonomous system is concerned, a possible cooperation technique for multiple such robots operating on the same big field could be a equal division of the pre-set path among all the cooperating robots. In conclusion, the size of the field does not particularly favor either implementation. Nevertheless, for an initial implementation, a single robot will work alone.

Next to the size of the field, let us look at how the shape of the field impacts our design decision. First of all we have (near) rectangular fields. Arguably, such fields are easier to handle, both for (semi-) autonomous as well as non-autonomous systems, than irregular fields. One simple, but possibly suboptimal, manner of dealing with (near) rectangular fields is to drive the full length (or width) of the fields, turn around and repeat until the entire field has been covered. For a semi-autonomous system, such an approach can be created easily as discussed in paper 14. For fully autonomous system, such a field should also be easy to operate on, as in the worst case, it should be able to do exactly the same as a semi-autonomous system; a simple reflex agent, which turns around once detects the field border (via GPS or some other feature) and some termination conditions might even suffice. Let us then turn to irregular field shapes. Currently, farmers are already dealing with irregular fields manually, as such a semi-autonomous system can be given a route without any issues. For autonomous systems, however, irregular fields might prove to be more challenging. Of course, this depends on the quality of the AI, but performance might be lower in the worse case (an approach might for example be a semi-bruteforce of the field), than the performance of a semi-autonomous system, but in the best case this might be better. Either system is able to handle an irregular field, but the performance of the autonomous system is heavily reliant on the quality of the AI and should be investigated for a conclusive recommendation. For the fruit orchard use case, however, the structure of the field is somewhat easier: a set of lines of trees can be followed by the robot to operate, meaning the AI does not have to be very complex.

Of course, an automated solution should be able to deal with obstructions. A robot can for example encounter low hanging/broken off branches of the trees, humans and animals. The system uses proximity sensor to sense its surroundings: if an obstacle presents itself in the robots path, the robot will halt and notify the user. While it waits for the user's response, it will update its status every 5 seconds. If at any point, before user response, the obstacle is no longer present, the robot will notify the user and continue operation. If not, it will wait for the user, who can decide to halt operation until a later point (for example after the user has cleared the obstacle) or can order the robot to ignore the obstacle and continue operation: for example imagine a low hanging branch is obstructing the robot. In this case the user can decide if the robot should simple run into the branch, with the intention that the robot pushes the branch out of the way by running into it. This proximity sensor system will prevent run-ins with obstacles, unless instructed by the user, meaning the system is safe for animals and humans.

Overall, when only looking at the maneuvering of the robot on the field, and the safety of the robot, the system is able to autonomously deal with an entire field. A semi-autonomous system is guaranteed to fully cover the field, and is cheaper than a fully autonomous solution, however it has the additional overhead that a route has to be predefined. A fully autonomous robot has the potential to be more efficient in its route, but is more technically challenging. Since the size of the field, and the cooperation between robots, does not favor any particular solution and the shape of the field inconclusively favors a semi-autonomous system, our recommendation is to utilize a semi-autonomous, single robot system for now, until fully autonomous systems become more mainstream and less expensive.

Next we discuss how a system should behave when it needs to charge, needs to empty its container or has finished working the field. We can distinguish a couple of behaviours in such cases:

• The robot signals the user and waits for it to be picked up
  • The robot stops where it was at that moment
  • The robot moves to the edge of the field or even a designated point at the edge of the field
• The robot autonomously moves to a point, where it can charge, empty or wait
  • The point is at the field edge
  • The point is further away
  • The robot autonomously moves to work another field

Before we discuss the pros and cons of the above behaviors, we must point out that the desired behavior likely depends on the user. Some farms have all its fields close to or adjacent to each other, whereas other farms have their fields scattered around the vicinity. This heavily impacts whether a user is willing to pick up the robots or wants the robots to come to some “waiting place” or continue working another field. Also the wealth of the user determines whether each field has a charging point, or not. Finally the implementation of the charging point impacts the above: is it a fixed-in-place charging point, or more like a power bank that you place where and when needed? On closed farms, meaning farms where you can move between fields without encountering random people or vehicles anywhere, a more automated approach is viable: The robot could move from a field to a charge, empty or wait point somewhere else on the farm easily by just following a set route. It would then be the user’s responsibility that no objects or people get in the way, which is very manageable. Alternatively, again, a fully autonomous system could be used for the routing and maneuvering to the point. We can present similar pros and cons for a fully autonomous versus a semi-autonomous system again as above, but for this particular use case (closed farm) a semi-autonomous system would do just fine.

Concept gripper

The gripper should grab all kinds of weeds and remove them efficiently. The first concept uses two long parallel rods, which extend under the weed so the stem of the weed will be in between the two rods. When the gripper is at its place, the rod, mounted to a carriage, will be moved towards the stationary rod and the stem will be clamped in between the two rods. Now the weed is hold tightly at a relatively strong point, it can be pulled out of the soil, together with its roots. The gripper can then move to a weed container and dropping the weed in there. Below, a drawing of this gripper is shown, together with a carriage that will move one of the rods. The movement of the carriage is done by a rack and pinion actuator. The drawing is just a concept: manufacturing margins, motors, bearing seatings etc. still have been left out of the picture.

The second gripper concept is radically different from the first one. After conducting research into weeds and analyzing a real life situation at a potential user, we concluded that only grabbing and pulling weeds might not be enough. That is why the second concept incorporates curved blades to obtain a scissor-like functionality. The blades can be used to cut stronger weeds that are too strong to be pulled out. We have chosen a curved blade in order to make sure that the weeds are being pushed to the center of the gripper, instead of being pushed out at the end. This is especially important for tough weeds that tend to move to the end of the gripper, the furthest away of the rotational axis. In addition, it removes unnecessary tension from the grabber when the weeds are being sliced off and it allows for a more evenly distributed cutting force along the radius of the weeds. Operation would be as follows: After a weed has been detected, the arm moves the grippers into position. The blades will be as low as possible to the ground and may even dig into the ground ever so slightly, the gripper grabs hold of the weed tightly enough to be able to pick it up, but not too tight to where it might crush the weed. After weeds have been grabbed by the gripper, the blades can snip the weed's stem or roots, depending on the weeds, and the gripper can then deposit the weed into the container of the robot. This concept is more complicated and more dangerous than the first one, but offers greater functionality and adaptability. Being able to snip the weeds' roots reduces the chance of the weeds returning, which is something the first concept could not do, therefore we suggest using the second concept for a first prototype.

Isometric view of the concept gripper

Alternative gripper design

Battery

The robot needs to have energy to operate. The easiest way to provide the robot with energy is using a battery. Putting solar cells on the robot will not be enough, because the robot will operate underneath and between trees. The trees are blocking the sun, so the solar panels will not generate that much energy. Another option is providing energy by cables. A very long cable is needed in this situation, because the scope in which the robot works is very big. The cable can also be stuck behind some tree or other obstacle. Stationary cables providing energy (like is used by electric trains) is also not convenient. It is dangerous for human that enter the area, because they can get an electric shock. Also lots of cables are needed and it can be damaged easy. A battery doesn't have those disadvantages, but there is another disadvantage. A battery can run out of energy.

A battery can be recharged, but that takes time. If the battery is in the robot, the robot has to wait till the battery is charged and cannot do the tasks for which it is intended. To solve this problem, there is an existing solution, namely a battery change system. In such a system, the battery can be easily removed from the robot and another battery can be placed. Currently the battery change is mostly done by humans, however the process is not difficult. We found a patent on a battery change system. The battery itself has contacts two opposite sides. At one side the contacts are used for charging and the contacts at the other side are used when the battery is in use. To remove the battery is something like a conveyor belt used. When the battery is removed, then a full battery needs to be placed in front of the battery space in the robot. This can be done with a linear movable battery rack. When the full battery is in front of the gap the conveyor belt is applied again to place the full battery in the robot. To see a picture of this, see [1].

After measuring the battery usage the robot knows how long it can operate before getting the next battery. In the navigation, this can be taken into account. If this is done correctly, the robot will in practice never run out of battery.

Weed container

If you remove the weed and let it fall on the place where it was is not preferable. The removed weed can make that even more weed is growing on that place. If the robot makes the field weed free the next time, then the robot may detect the removed weed as weed that has to be removed. The speed of the robot decreases and the energy consumption increases. The removed weed can also be an obstacle, that the robot has to avoid the next time. Because of this, the robot has to take the weed and bring it to another place. It is not desirable that the robot per piece of weed drives to the place where the weed has to go. To collect the weed, the robot needs a weed container.

If you throw the weed just in a container, then the container will be rapidly full, because most weed has a stiff structure. To solve this problem, a press can be installed in the container that makes the weed more compact. When the press has worked, more weed can be put in the container. The press does not have to work after each weed that is thrown into the container. This will impact the operation speed of the robot. If the top of the weed pile is to a certain level in the container, the press can operate. If the press does not go further than a set position, then the weed container is full and has to be emptied.

Garbage trucks have the same functions as the weed container that we need in our robot. A size of a garbage truck is too large, so we should scale the solution a bit down, to fit it into our robot. For the weed container a scaled down version of a garbage truck can be used.

The robot has to dump the weed also somewhere. This can be realized with an underground container that is only opened when the robot has to dump the weed in there. This is also a safety issue, such that human cannot fall in the gap. It is of course possible to open the container for human. The top of the container does not have to be electric. If it is realized as a sliding cover, then the robot can open and close it with the arm on which the tool to remove weed is.

Communication

Mockup of application for owner

Communication schematic

The robot has to communicate with its owner, such that the owner has control over the robot. Furthermore, if the robot works together with other robots, it has to communicate to the other robots too. If collaborating robots do not communicate, they can come in a situation, in which they are waiting on each other.

With multiple robots, it is the most efficient when there is one central control unit for all the robots, a server. The robot communicates, like the other robots, with the server. The owner of the robots communicates to the server too and the server forwards the commands of the user to the right robot.

The contact between the robot and the server is via an internet connection. The robot is connected to the internet via a mobile network, like 4G. The robot has also the possibility to connect to the internet over a WiFi connection. To setup those connections and to do other things in the software of the robot, the robot has also an USB connection, over which the technicians can send commands.

For safety, the robot has one or more physical emergency stop buttons, such that they are reachable without walking around the robot. If an emergency stop is pressed, the power supply of the robot will be switched off immediately.

The owner of the robot can control the robot via a smartphone app. We will give a mock-up for such an app, but we will first give the functionality of the app:

• Showing battery level
• Showing fill level of weed container
• Showing what the camera on the robot sees
• Warning in case help of a human is needed (for example when the robot is stuck)
• Showing the progress of the robot in the field
• Pause and resume the operation of the robot
• Start the operation of the robot
• Stop the operation of the robot
• Support for multiple robots

Starting the operation can be done in multiple fields. The robot determines based on location data in which field it starts the operation. This means that the robot should be in the field where it should start operating.

Safety

Safety is an important aspect of every machine. The robot will drive autonomously and has many moving parts and electrical components. These could cause injuries to people. To prevent this the robot has been designed to minimize those risks.

• The robot has no sharp edges or parts, excluding the gripper, which could potentially hurt someone. The gripper will have sharp knives which can cause serious injuries if it touches someone. Therefore the gripper should only move if there are no obstacles around it. If the robot arm touches something wich it didn't expect it will stop moving to prevent prevent further damage.
• Cameras and sensors on the robot can detect if something or someone is in front of the robot. This is necessary to prevent injuries to people and damage to the robot. In a situation where there is an obstacle in front of the robot the user will get a notification in the app and can let the robot continue driving if it is something small like a plastic bag or let the robot drive around the obstacle.
• There is an emergency stop button on the robot and in the app.
• The robot has a big battery and many electrical components. To prevent electric shocks the metal parts industrial robots and machines are earthed. In a driving robot which has rubber tires this is not possible. To minimize the risk of wires making contact with metal parts all electronic components must have double electrical insulation (class II).

Notes on design

The robot has 4 wheels, as 4 wheeled vehicles have proved to be effective and efficient in the agricultural sector there is no need to literally reinvent the wheel. The robot is small enough to fit between rows of trees in an orchard, but big enough to provide enough stability to maneuver through an orchard without issues. As not each orchard is the same, the size of the robot is not optimized to perfectly fit between a row of trees and cover either side, hence it only has one weeding arm. Future developments and iterations can change this property.

Operation

This section will describe the robot's operation in a real life environment. Before a robot can begin removing weeds in an orchard a number of steps have to be taken. First of all, the base station of the robot has to be installed, so that the robot can charge, empty its weed container and wait when it has finished operation. After this, a GPS route through the orchard has to be obtained and uploaded to the robot, and also a route from the base station to the orchard, which is preferably as small as possible. Now the robot can be started by the user and the robot will drive to the orchard and start removing weeds along the specified GPS route. With its cameras it will detect weeds, stop driving, remove the wees using the gripper mounted on the arm and place the weed in the robot's container, and continue driving until it finds more weeds. It will continue to do so until the entire route has been covered and then return to the base station, notifying the user that it has finished. It keeps track of its battery level, it makes sure to keep enough energy saved to return to the base station. If it needs to recharge during operation, it will notify the user and return to the base station and charge. After charging it notifies the user and returns to where it left off and continue operation. This process is identical for the emptying of the weed container. If the robot encounters situations it cannot deal with it notifies the user and halts operation until user response, unless the situation resolves itself and no longer obstructs the robot. Then it notifies the user that it has continued operation.

The user is able to view the robot's location, battery level, how full the weed container is and view the robot's camera vision through an app. Via this app the robot can be started, paused and stopped. The user can schedule the robot's operation to for example have the robot remove weeds from field 1 every two weeks on Monday, and remove weeds from field 2 every week Tuesday etc. and can tell the robot to continue with another field after finishing one.

A visualization of the robot model can be seen here:

Final Design

File:G5 movies.zip

Validation

In order to test our gripper we have conducted some experiments using our 3D printed gripper. First, we went to a place which had a lot of different weeds and then we tried to pull out some weeds from the ground, using our gripper. This turned out to go very well. Our main concern for the gripper was that an optimal gripping force had to be determined because if the force is too low, the gripper would not be able to pull out the weeds and if it is too big, it would crush the weeds and it would cut them off instead of pulling them out altogether. From our experiments, we saw that the gripper did never cut off any weeds, even when we applied very high forces. This way, the maximum gripping force is only limited to mechanical properties of the gripper and that there is no cutting effect caused by applying too much force. A cutting effect could still be seen when gripping wood-like things like branches, but our gripper is not designed for this purpose.

Something we did see was that sometimes when pulling weeds, we had to apply a lot of force pulling the gripper with the weed out of the ground and then we ended up breaking the weed because the roots were actually stronger than the stems. This confirms the need of our cutting blade, as it can cut off the weeds easily, without applying too much force. This process does not remove the weeds entirely, but it weakens them, including the roots. This way, they can be pulled out completely the next time our robot goes around the field.

In order to determine the torque needed for our servo’s controlling the gripper, we took kitchen scales and put them in between our hands and the gripper in order to measure the force we applied. The force necessary to pull out the weeds varied for different kind of weeds, but in our experiments, we saw that a force of 50N, applied to the tip of the gripper turned out to be enough to pull out all weeds, without destroying them. This force would be equivalent to a servo torque of 7.5 Nm, which is acceptable.

In addition to our experiments, we also went back to the farmer we had first spoken with, to discuss our design and to show how the robot would be operated. This included a demonstration of our app mock-up. The farmer liked the simplicity found in the app and that it did not contain any unnecessary or difficult to understand options. He also liked the design of the gripper and thought that it would function properly. However, he was concerned that it would take ages for the robot to go over the field as it has to grab every individual weed instead of just spraying over it altogether. We have not taken a look at speed optimizations of the complete system as this is dependent on the complete setup, all it’s different components and the way they work together. This is something that could be done in a follow-up study about our project. On top of that he also pointed out that it creates a big dependence on other companies, like IT for maintenance and setup, which conventional (mostly stubborn) farmers do not like.

Verification measurements

Weeds pulled out with roots

Cost Analysis

Cost Analysis

Cost Analysis

Cost Analysis

The cost analysis consists of estimates of part costs, and assembly costs. The estimates are very rough, and most likely are too low for a real product. The estimates are based on a hand made robot, not an assembly or factory with contracts with suppliers etc. Also some links are included with parts which could potentially be used. As visible in the table, part costs are approximately €6000. This is an estimate of the very minimal costs, and does not consider licensing fees, delivery costs etc. Neither does this include testing for functionality and quality and most likely, after an initial build, new or additional parts are required. The table also includes assembly and programming. It is hard to estimate a cost for those attributes. For a single robot/prototype, assembly and programming can be done in-house and would contain numerous ad-hoc solutions and inefficiencies. The values in the table represent a such in-house built and programmed robot. In a mass production environment the values would likely increase drastically. In conclusion, if we try to estimate the cost of a one-off in-house robot this would approach €20.000.

Without going into detail about sell prices, taxes and all other aspects of running a company, and only looking at the estimated amount of €20.000, we can investigate how this compares to the already in place, pesticide solutions. We will analyse multiple scenarios and make recommendations based on those. Assumptions: pesticides cost €100 per acre, pesticides have to be sprayed every month on average, weeding using the robot has to be done every 2 weeks. The costs for the user we will consider for this case are only the costs of the pesticides needed to remove the weeds. Thus, per acre, per year, the user pays €1200 to use pesticides. The cost of the robot is €20.000. So if a user has 1 acre of land, the estimated time to earn back the money spent on the robot is about 17 years. However if the user has 3 acres, the costs of pesticides per year are €3600, and then it only take about 6 years to earn back the money spent on a robot. The idea is clear: the more land the user has, the more profitable a switch to robots is. The only consideration is that the robot is able to remove the weeds from all of the orchards within two weeks, otherwise multiple robots must be purchased and the payback time increases drastically.

Without testing and prototypes we cannot accurately estimate the maintenance costs or life-span of our product. It is highly likely that the robot requires yearly maintenance and that such maintenance will cost money, therefore its return on investment time might be lower than depicted in the graphs, however we assumed a yearly cost of €200 for the maintenance of the robot to give a more realistic prediction than no maintenance at all. Also, for the robot to communicate with the user, the user potentially needs to pay for a 3G/4G subscription with a service provider. This will likely add another €100-€200 per year to the upkeep of the robot.

Next to the purchasing of farming equipment, farmers often lease equipment. For this robotic system, however, leasing is not as simple. Due to the fact that the robot needs a route, a user would need to obtain a route before leasing a robot and the user would have to have the infrastructure to support the robot and it's base station. More then likely, this is not the case. Moreover, the weeding of the fields has to be done very regularly and leasing a system every 2-4 weeks is likely more expensive in the long run than purchasing a system. Hence we do not view leasing as a viable option for this robot.

Conclusion

This initial overview of robot for automated, semi-autonomous removal of weeds in the fruit orchard sector shows there is a future for such products. However, there is a lot of work and research left to do beforehand. For this particular design the following things remain to be done:

• Proper part selection and testing: The robot must be build to last multiple years and minimize its costs and upkeep in order to become a solid competitor for pesticides, to this extent solid parts must be used and these must be tested for functionality, performance and life expectancy. The effects of rain, UV light etc. on the robot must be analysed.
• Testing weed detection implementation: The performance of the weed detection must be measured and improved as much as possible.
• Testing weed removal gripper: The gripper we designed must undergo intensive testing and the design must be improved accordingly.
• Determine and improve performance of the entire package: Answer question such as: How long will the robot be able to work for on one battery charge? What is its operation speed?
• Adjust cost analysis based on the above

This is of course an iterative process.

After the creation of a working system, it can be further optimized by for example adjusting its size to the use case, and add additional weed removal arms.

When fully autonomous technology becomes more wide-spread and affordable, the robot can be updated or upgraded to incorporate such technology, improving its user-friendliness and potentially performance.

A working system in the fruit orchard sector opens doors to many other sectors, both in agriculture and outside. Based on our findings and on those who continue with this base, systems can be designed for other sectors and ultimately reduce the worldwide usage of pesticides to create a better future.

Based on the cost analysis we can recommend a robotic weed removal system which does not utilize pesticides to any fruit orchard farmer, to replace current pesticide solutions as a cost effective alternative.

References

Links to sources

1. https://www.aaai.org/ojs/index.php/aimagazine/article/view/1132
2. https://www.aaai.org/ojs/index.php/aimagazine/article/view/1130
3. https://patents.google.com/patent/US5943733A/en
4. https://patents.google.com/patent/US5204814A/en
5. https://www.sciencedirect.com/science/article/pii/S0168169907001688
6. https://patents.google.com/patent/US2941223A/en
7. https://www.sciencedirect.com/science/article/pii/S092188909600053X
8. https://s3.amazonaws.com/academia.edu.documents/37918498/06850799.pdf?AWSAccessKeyId=AKIAIWOWYYGZ2Y53UL3A&Expires=1549803092&Signature=8GCjA4uM%2FhDytKf1RFWUmk6m0t4%3D&response-content-disposition=inline%3B%20filename%3DPath_Planning_Algorithm_Development_for.pdf
9. https://www.osti.gov/etdeweb/servlets/purl/895225
10. https://pdfs.semanticscholar.org/6b2f/19d3bd58c12071129ba6adba16a87c229aaa.pdf
11. https://ac.els-cdn.com/S0301479715304412/1-s2.0-S0301479715304412-main.pdf?_tid=bd717970-3888-4fd2-8cfd-4f89ef37f34a&acdnat=1549800079_288b461fdbddadb9ecad9edaef4d7786
12. https://ac.els-cdn.com/S0921889002002415/1-s2.0-S0921889002002415-main.pdf?_tid=b448d981-30f4-41b9-b527-6e13619c967b&acdnat=1549800321_10552114fd41bada89923042bbd7034d
13. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.418.3615&rep=rep1&type=pdf
14. https://www.researchgate.net/profile/Carl_Wellington/publication/239932742_A_System_for_SemiAutonomous_Tractor_Operations/links/559de2af08aec72001828a7e.pdf
15. https://onlinelibrary-wiley-com.dianus.libr.tue.nl/doi/epdf/10.1002/rob.20301
16. https://www.mdpi.com/1424-8220/8/2/1278/htm
17. https://www-sciencedirect-com.dianus.libr.tue.nl/science/article/pii/S1474667016449207
18. https://ieeexplore-ieee-org.dianus.libr.tue.nl/document/7438821
19. https://www.mdpi.com/1424-8220/13/2/2645/htm
20. https://www-tandfonline-com.dianus.libr.tue.nl/doi/abs/10.1163/016918609X12586141083777
21. http://edepot.wur.nl/398419
22. https://patents.google.com/patent/US20150202770A1/en
23. https://patents.google.com/patent/US9373149B2/en
24. https://patents.google.com/patent/US8364334B2/en
25. https://dspace.mit.edu/handle/1721.1/111698#files-area
26. http://dspace.ewubd.edu/handle/123456789/2501
27. https://ieeexplore.ieee.org/abstract/document/8336636
28. https://tue.on.worldcat.org/oclc/5872746903
29. https://tue.on.worldcat.org/oclc/5154827494
30. https://tue.on.worldcat.org/oclc/5831032581
31. https://link.springer.com/article/10.1007/s12369-009-0011-9
32. https://tue.on.worldcat.org/oclc/4934432761
33. https://tue.on.worldcat.org/oclc/5387876416
34. https://ieeexplore.ieee.org/abstract/document/5169487
35. https://acadpubl.eu/jsi/2018-118-7-9/articles/8/55.pdf
36. https://doi.org/10.1016/B978-0-12-809881-3.00008-5
37. https://doi.org/10.1016/j.cropro.2015.02.017
38. https://link.springer.com/article/10.1023/A:1015674004201
39. https://library.wur.nl/WebQuery/wurpubs/376454
40. https://doi.org/10.1002/rob.21727
41. http://dx.doi.org/10.4081/jae.2017.583
42. http://edepot.wur.nl/1099
43. https://doi.org/10.1016/j.compag.2013.08.008
44. https://doi.org/10.1016/j.compag.2011.10.006