User interface and communication model

Introduction

Now armed with the best mechanism for seeding from the Designing the robot section and the lessons learned from the Case studies we can talk real robot operation. However, we still need to specify how the robots will know what to do before they can be deployed. It is already clear how they will go about their job, as it was concluded a drill mechanism would be best for the planting of seeds. This page will be dedicated to placing the ground works for a dedicated interface in which the park rangers can specify the details of the reforestation mission to the robot and how a fleet robots would be able to communicate progress with each other.
General information about the project can be found at PRE2017 4 Groep6.

Envisioned working principles of the interface

After a forest fire has raged through a national park it will have left an a priori known area devastated which requires reforestation. The parameters defining this area, primarily its location, shape, and edges can be inquired from the observations of the firefighters or using satellite images of the area from during the forest fire. Park rangers will know the history of the national park and hence also the composition of the vegetation species in the burnt areas. The goal of the robot is not only to reforest the area but also to restore the biodiversity which was previously present in the burnt area, with the preference that the newly reforested area resembles the lost one as closely as possible.

In order for the robot, or fleet of robots, to operate in such a manner that the above goal can be accomplished, the park rangers will have to communicate the appropriate parameters such as area size and shape, type of seeds, desired species composition, etc.. The easiest way to implement the biodiversity requirement would be to employ a fleet or swarm of robots, each with a reservoir filled with only a specific type of seeds. The park ranger would then only have to specify to each robot what type of seed they carry so that the robots could infer from an internal database the properties belonging to those seeds (e.g. diameter, seeding depth, etc.) in order to know exactly how to plant the seeds.

Now one non-trivial case still remains, the robot needs to know where to plant the seeds in order for the reforestation operation to be a success. We cannot just let the robots run amok and spread the seeds with complete randomness, as this will most likely result in segments of the new forest which are underseeded and only partly recover or segments which are overseeded and lead to a high degree of competition between the tree species which also reduces total turnover. These two effects do not yet include the mutual interference the robots will have: it is quite possible that a robot will run over a previously seeded area, possibly disturbing the already sown seeds, or even worse drilling through a previously planted seed and completely destroying it. Also, the robots could possibly damage each other by crashing into each other if no path planning is taken into account, further delaying the operation. The short error-analysis above reveals two crucial components which are required to solve the where question. Each robot needs to be given precise orders as to which locations it must plant its seeds, and each robot needs to able to communicate with the rest of the collective, to prevent mutual obstruction.

The problem of knowing which seeds to sow where requires human intervention, as the target area, the relative occupation of the tree species, and the formation of tree colonies need to be specified. Luckily two of these parameters can be obtained from the park rangers, using thermal satellite images or mapping of the fire done by fire departments the park rangers can outline the area on a map in need of reforestation. The most user-friendly way of providing this information to the fleet of robots would be a graphical user interface (GUI) where a park ranger could click on points of the map to define the edges of the area in need of reforestation. Then the relative percentage of seeding per area has to be defined, which has to mimic the previously existing biodiversity. If the new area would have to be seeded non-uniformly or with a variable tree occupation, a next step in the GUI would be to provide the option to the park ranger to define partitions in the target area and define it into sub-areas, or grid elements as we’ll call them from now on. This element will be polygon shaped as to get the best fit to near the edges of the perimeter of the total area. Then for each element, the park ranger can specify the vegetation composition more in detail, by deselecting the plants present in the overall area selection. Then once the desired level of refinement in the map by division into elements has been reached, a computer algorithm suited for optimisation can calculate the geometrical distribution of tree species within each element, to create a point cloud representation of the elements, color-coded for each tree species such that the park ranger can confirm that this computed distribution is indeed the desired one.
However, for this method to work this requires the robots to know their exact position on the map, seeds will in general not be larger than a few centimeters, hence high planting densities could arise for some species (e.g. grasses or flowers) which would then require a high degree of precision and exhibit a great need for control. Intuitively one would say to use GPS. However standard civilian equipment GPS, that is GPS equipment which is within the price range of the budget National Parks have to acquire a fleet or reforestation robots, is only accurate up to a few meters. This is mostly because of the blocking and backscattering of GPS signals, which cannot be filtered out by most GPS software. Some GPS software exists which can filter out these effects, however, they require a sophisticated antenna costing a few thousand USD (Patel, 2015) . Especially in a forest area, a lot of backscattering of the signal will happen because of the vegetation, which does usually does not influence precision, but accuracy. An alternative solution to this, which would also immediately tackle the problem for how communication between robots of the collective would have to occur, is to deploy beacons. A set of special beacons can be deployed at the edges of the problem in order to effectively let the robots know they should not venture beyond these beacons and allow for the operation to be contained in an area. If then another set of beacons is placed with a fixed distance between them, in the inside of this perimeter defined by the set of special beacons, they can be used to accurately triangulate the position of each robot if the beacon density (#beacons/m^2) is adequate. However, the installation of beacons is then again cost and time intensive. The most promising GPS alternative is BeiDou, which has a public accuracy of 10cm in the Asian-Pacific region. It is currently in its last developmental stage and will start releasing in 2020. As it most likely that it will take some years beyond the release, say 2025, until the technology is widely available and accurate all over the world. Given our technology would still need some time to develop, the beacons can be used during the developing and testing stage, such that it can later be updated to the new GPS technology.

Next if these beacons and not only equipped with a radio receiver/transmitter for position triangulation, but also provided with the appropriate communication protocols and antennas, they can be used for the robot collective to provide updates on the progress of individual robots. Suppose robot A has finished its job in element E4, it will then send a message to the beacon to inquire the other robots in the vicinity, which still have to do their seeding operation in element E4, that the area is currently vacant. Robot B which has been idle and waiting to plant its seeds can then enter element E4 and proceed with its seeding plan, whilst avoiding to move through the area previously seeded by robot A to not distort the seeds which were only just planted. Because of the position triangulation with the beacons, this can be done by means of path planning algorithm within the element E4. Of course, this example is a simplification, where it is assumed that only 1 robot is operating in 1 element at a time to prevent the robots from interfering with each other’s task, which will most likely not happen in a real planting situation, but which will suffice for building a prototype of the GUI.

The above example describes a fairly sophisticated level of autonomy of the robots which can intercommunicate to make decisions on which element to seed next. It is, however, most likely that unanticipated obstacles will be present during the planting operation. For example, a burnt tree could have fallen down and impede the robot from planting its seeds. If such a situation would occur the robot would have to obtain new orders about what to do; in the simplest case it could either continue its current planting operation in an element to the best extent possible, by avoiding the obstacle or it could completely abandon it and cease the planting operation. In such a case it is probably best if the park ranger made this decision, as they will have to arrange for the obstacle to be removed anyway, so it is best if they know about the obstacle’s existence as soon as possible so that they can start the appropriate countermeasures. Therefore is such a situation were to arise, the robot should send a message to the park ranger which will pop up in the GUI, asking the park ranger to select from a list of options (again in the simplest case, the options are to abort or continue the seeding operation in a particular element). If the obstacle does not require immediate action, the robot can be allowed to continue, but if the obstacle does require immediate action (e.g. in the case of a small remaining fire seat, which might still be smoldering after the forest fire) the instruction could be given to abort the operation and let the robot leave the current element. In the latter case, once the option for abandoning has been chosen, other robots which still have to seed in the same element will most likely also come across this obstacle, creating a potential clutter of incoming requests from the robots to the park rangers. To prevent this, again, the beacon system can be used to propagate this information and telling the robots that the particular cell with the obstacle is now off-limits.

Development of the interface

From the envisioned workings of the interface the problem description, the user group and a list of design requirements for the interface are made. This is done taken into account the limited time left for the project course (2 weeks) and the desire to leave something behind which can be picked up and further developed by someone else.

Problem Description

To quickly restore the forest after a forest fire, the design of a robot was discussed to plant the seeds. The robot however needs to know where to plant what seed. This should be decided by the foresters since the foresters know what ratios of species was originally present in what area. It can not be expected that the forester is able to directly communicate with the robot due to their lack of knowledge. The goal of this interface is to bridge the communication between the foresters and the robots.

Target user group

The target group for the interface is foresters in national parks where the reforesting robots will be used. They will be in charge of the reforestation after a forest fire, which means that they should be able to fully control the robots as necessary. Since they don’t need to be adept in using technology and robotics, the interface needs to be intuitive and take care of most technical aspects below the surface. In this case the foresters only have to enter the biological and planning aspect of the reforestation. After the forester has entered what species need to be planted in what specific area. The program behind the interface will calculate how many robots are needed for certain species and how long it will take to finish the reforestation. Via the interface the forester can check the progress of reseeding the area.

Functional Requirements

• The interface must display an area that can be selected by the user
• The interface must provide the user a way to select an area within the displayed area as the planting area
• The interface must provide a way for the user to define the ratios of plants in the aforementioned area
• The interface must provide the user with an overview of the robot division on the tasks
• The interface should provide the user a way to redefine the robot's divisions.
• The interface should provide the user a way to select subdivisions in the area to edit on a smaller level
• The interface should provide a confirmation screen showcasing the areas and their plans
• The interface should provide a way to edit the existing area
• The interface should provide a way to edit the existing subdivisions

Overview of the interface

Since a ranger or other user is often not very adept at controlling professional and specialized systems, a simple user interface was designed to allow intuitive control over the setup of the robots. A mockup of a simple 4 step interface has been designed to allow the user to easily define the area that has to be reforested. Each step allows for in-depth customization, but is not mandatory. Time constraints did not allow development of a fully fledged application, but since this is preliminary research and not a prototype a working interface would not provide any additional advantages. An interactive version of the mockup can be found at the following link.

Step 1: Selecting the area to be reforested

The first step is extremely easy and straightforward. The user only has to define the area the robots should work in by dragging lines across the screen to form a polygon representing the total area. Satellite image suffices for this since the assumption has been made that the user has in-depth knowledge about the total area.

Step 2: Selecting the plants

The second step is also very straightforward. The user will have to select all plant that generally occur in the area and its overall ratio. Based on 3 types of plants, lists are shown where the user can select plants. A search option is als available to speed up the process of finding specific plants.

Step 3:Adding subdivisions

The third step allows for a more in-depth customization. Subdivisions can be created within the total area where the species and ration differ from the rest of the area. By simple creating another boundary the subdivision is easily added. Each subdivision also allows to add and remove plants and adjust their ration more specific. Overlapping subdivisions are automatically generated as subdivision of their own, where the mean is taken from all subdivisions involved.

Step 4: Map generation

After defining all subdivision needed, the program will generate a map based on the input of the user. An overview of the area shows where each plant will occur throughout the area. Note that an actual application will show more precise data and allow for zooming through the map. A recommendation of beacon placement will also be given based on the geography of the area.

Final overview

The final overview shows various statistics and estimated time until completion. A recommendation is given for the allocation of robot based on plant species. However, it is possible to manually define the distribution of the robots.

Time estimation model

During one of the final steps of the GUI, an estimated time for completion is given. This time is obtained by a calculation based on a couple of assumptions and model parameters which will be listed below.

List of assumptions:

• The robot has a constant drilling speed
• The robot has a constant traveling speed
• The time to retract drill from the hole is 20% of the time needed to make the hole. This difference is caused because drill has to exert a great force to make the hole, but is free to move upwards once the hole is made.
• The battery charge provides a constant work time.
• There is an average time (average taken over all positions of the grid) required for the robot to drive back to the station for refilling the seeds and swapping a new battery.
• There are sufficient reserve batteries to switch in between, such that robots are not taken out of the process for charging, and batteries do not deplete before a reserve battery is charged.
• The robot can move in a straight line between planting positions.
• The robot will detect when its battery depletes and return automatically, such that no time recovering a dead robot from the site by park rangers is lost.
• The refilling of seed reservoir, swapping of a battery pack and the event of dropping a seed happen instantaneously.

List of parameters for time calculation model:

• Travel velocity of the robot; [math]\displaystyle{ v }[/math] [ms^-1].
• Drilling rate of the robot; [math]\displaystyle{ \gamma }[/math] [mms^-1].
• Seeding depth for species [math]\displaystyle{ j }[/math]; [math]\displaystyle{ h_j }[/math] [cm].
• Average distance between planting sites of species [math]\displaystyle{ j }[/math]; [math]\displaystyle{ r_j }[/math] [m].
• Total number of robots [math]\displaystyle{ N }[/math] [-].
• Number of robots planting species [math]\displaystyle{ j }[/math]; [math]\displaystyle{ N_j }[/math] [-].
• Number of seeds of species [math]\displaystyle{ j }[/math] to be planted; [math]\displaystyle{ \sigma_j }[/math] [-].
• Battery life time; [math]\displaystyle{ T_{bat} }[/math] [h].
• Seed reservoir capacity for species [math]\displaystyle{ j }[/math]; [math]\displaystyle{ C_j }[/math] [-].
• Average travel time back to station; [math]\displaystyle{ \tau }[/math] [min]

The method of calculating the time is based on calculating the time contributions of each singular task for a species [math]\displaystyle{ j }[/math], as they are carried out independently, and summing over them to obtain the total time for species [math]\displaystyle{ j }[/math]. Next this total time is divided by the number of robots assigned to seeding species [math]\displaystyle{ j }[/math], such that the real operating time is found. Lastly, the longest of these times will determine the total time for the seeding operation. Or in formulae;

1. Number of required refills for species [math]\displaystyle{ j }[/math]; [math]\displaystyle{ \lceil \frac{\sigma_j}{C_j} \rceil }[/math], where [math]\displaystyle{ \lceil x \rceil }[/math] is the ceiling function applied to [math]\displaystyle{ x }[/math], which rounds [math]\displaystyle{ x }[/math] up to the next integer, since only an integer number of refills can be made, and rounding down would result in too few seeds planted. The total time spent refilling for species [math]\displaystyle{ j }[/math] is then: [math]\displaystyle{ t_{ref,j} = 2 \tau \lceil \frac{\sigma_j}{C_j} \rceil }[/math], where the factor 2 arises from the trip from the target reforestation area to the station and back again.
2. Time to drill one hole for species [math]\displaystyle{ j }[/math]; [math]\displaystyle{ 1.2 \frac{h_j}{\lambda} }[/math]. Such that the total time spent planting the seeds of species [math]\displaystyle{ j }[/math] is; [math]\displaystyle{ t_{plant,j} = 1.2 \sigma_j \frac{h_j}{\lambda} }[/math].
3. The time spent travelling between consecutive seeding sites of species [math]\displaystyle{ j }[/math]; [math]\displaystyle{ \frac{r_j}{v} }[/math], such that the total time spent travelling for species [math]\displaystyle{ j }[/math] is; [math]\displaystyle{ t_{trav,j} = \left (\sigma_j - 1 \right ) \frac{r_j}{v} }[/math], where the -1 factor comes from an additional assumption that a robot will always be able to start at some edge of the map where a seed needs to be planted, which considering our biodiversity requirement is fairly reasonable.
4. The total projected time for species [math]\displaystyle{ j }[/math], that is the time if battery life would be taken into account is then; [math]\displaystyle{ \textstyle t_{proj,j} = \sum_k t_{k,j} }[/math] where [math]\displaystyle{ k= ref,plant,trav }[/math].
5. The total number of required battery recharges for species [math]\displaystyle{ j }[/math] is then given by; [math]\displaystyle{ \lceil \frac{t_{proj,j}}{T_{bat}} \rceil }[/math], where the ceiling function again takes into account only an integer number of recharges can be done. The total time spent recharging is then; [math]\displaystyle{ t_{ch,j} = 2 \tau \lceil \frac{t_{proj,j}}{T_{bat}} \rceil }[/math].
6. The total accumulated time for the seeding of species [math]\displaystyle{ j }[/math] is then given by; [math]\displaystyle{ t_j = \textstyle \sum_k t_{k,j} }[/math] where [math]\displaystyle{ k=ref,plant,trav,ch }[/math].
7. The actual time required for the seeding of species [math]\displaystyle{ j }[/math], taking into account multiple robots assigned to one species is then; [math]\displaystyle{ T_j = \frac{t_j}{N_j} }[/math].
8. For the entire reforestation operation this gives a set of actual times [math]\displaystyle{ \big\{ T_j \big\} }[/math], where the time for the complete reforestation operation will be the maximum of this set; [math]\displaystyle{ T_{real} = \max \big\{ T_j \big\} }[/math].

An example Matlab script, showing database information of 6 species, and calculating the required time for a reforestation operation for 4 of these species is made available at: File:Matlabscripttimecalculation.pdf

Of course the resulting time should not be interpreted as the physical real time for the planting operation. The time calculation model is based on some assumptions to simplify the calculation and make the algorithm generalizable for any situation. Some limiting factors of this model include the assumed flat geometry of the seeding area, the constant travel and drilling speeds, the constant battery life and the constant seed capacity. For a real-life forest, the geometry will most definitively not be flat, but include bumps, inclinations, curved paths due to obstacles, etc. The drilling and traveling speed will not be constant, but rather in- and decrease gradually once of these functions starts/ends, because accelerations and decelerations are needed for this, travel and drilling times will be somewhat longer. Furthermore, the constant drilling speed is only true for a homogenous soil type, if the soil consists of multiple layers with each their own material properties, drilling speeds will vary between these layers. The constant battery life assumption should be interpreted as an average battery life, as battery life will be dependent on the function the robot is performing. In general, it can be stated that drilling will require significantly more power than driving, as many large forces are required to overcome the resistance of the soil to stay in its current state. Therefore robots which spent a lot of time drilling will have their batteries depleted sooner than robots which spent a lot of time driving. Perhaps the least influential assumption is that of the constant seed capacity, although seed dispensers will be designed for a certain capacity, because the diameters of most seeds are extremely small and refilling will be done by the park rangers, it is very plausible that a refilled container will have slightly less or slightly more seeds than described. Overall these deviations from the complex situation in reality indicate that the calculated time for reforestation should be interpreted as an absolute minimum time which will be achieved only in an optimal situation.